Adding a dynamic SiteMap for search engine optimization using LINQ
A couple months ago I added a feature to this site to build a Site Map for this site dynamically based on the information from the database for posts and files for the downloads. If your not familiar with how Sitemap files can help your site searchability, Google has a good documentation about Sitemaps in their Webmaster tools.
The SiteMap Protocal is a rather simple XML document consisting of a set of url nodes that consist of the following:
- loc - URL for the page link
- lastmod - Date the page was last modified
- changefreq - How often the page is changed
- priority - How high you think the page should be ranked relative to other pages on your site.
For this site, I decided to index the main (default.aspx) File, about and contact pages. In addition, I indexed each post as a separate url node. If you want to view the resulting data, browse to http://www.thinqlinq.com/sitemap.aspx. To do this, I used LINQ to XML with VB XML Literals. To begin, we need to add the XML Namespaces. At the top of our file, we enter the following imports:
Imports <xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
Imports <xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
We continue by adding the root urlset node and one child node representing the main page:
Dim map = _
<?xml version='1.0' encoding='UTF-8'?>
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.ThinqLinq.com/default.aspx</loc>
<lastmod>
<%= (From p In dc.PostItems _
Order By p.PublicationDate Descending) _
.First.PublicationDate.ToString("yyyy-MM-dd") %>
</lastmod>
<changefreq>daily</changefreq>
<priority>1.0</priority>
</url>
</urlset>
Most of this is standard XML. The main difference is the use of a LINQ query to show the last modification date based on the most recent post from our database. In this case we just want the First date when the dates are ordered descending. We do need to format it properly so that our search engine (Google) will be able to recognize it.
Next up, we need to add the link for the Downloads page. We'll do this much the same way that we added the url for the default page. However, in this case the modification date won't come from the database, but rather use a LINQ to Objects query to get the most recent file in the downloads directory on this site.
<url>
<loc>http://www.ThinqLinq.com/Files.aspx</loc>
<lastmod>
<%= (From f In New System.IO.DirectoryInfo( _
Server.MapPath("~/Downloads")).GetFiles _
Order By f.LastWriteTime Descending) _
.FirstOrDefault.LastWriteTime.ToString("yyyy-MM-dd") %>
</lastmod>
<changefreq>weekly</changefreq>
<priority>1.0</priority>
</url>
The About and Contact pages are relatively straight forward. The remaining url nodes are generated based on the records in the PostItems from our database. To populate them, we'll create a LINQ query pulling the data from the database using LINQ to SQL and projecting (Select) out individual url nodes for each row in the database:
<%= From p In dc.PostItems.ToList _
Select <url>
<loc>http://www.ThinqLinq.com/default/<%= p.TitleUrlRewrite %>.aspx</loc>
<lastmod><%= p.PublicationDate.ToString("yyyy-MM-dd") %></lastmod>
<changefreq>daily</changefreq>
<priority>0.3</priority>
</url> %>
As you can see, there isn't much here that is overly complex. It's just a series of LINQ queries filling the data from various sources. For reference purposes, Here's the complete code:
Imports <xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
Imports <xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
Partial Class SiteMap
Inherits System.Web.UI.Page
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
Response.Buffer = True
Response.Clear()
Response.Cache.SetCacheability(HttpCacheability.NoCache)
Response.ContentType = "text/xml"
Response.AddHeader("Content-Disposition", "inline;filename=blog.rss")
WriteRss()
Response.End()
End Sub
Private Sub WriteRss()
Try
Using dc As New LinqBlog.BO.LinqBlogDataContext
Dim map = _
<?xml version='1.0' encoding='UTF-8'?>
<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.ThinqLinq.com/default.aspx</loc>
<lastmod>
<%= (From p In dc.PostItems _
Order By p.PublicationDate Descending) _
.First.PublicationDate.ToString("yyyy-MM-dd") %>
</lastmod>
<changefreq>daily</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>http://www.ThinqLinq.com/Files.aspx</loc>
<lastmod>
<%= (From f In New System.IO.DirectoryInfo( _
Server.MapPath("~/Downloads")).GetFiles _
Order By f.LastWriteTime Descending) _
.FirstOrDefault.LastWriteTime.ToString("yyyy-MM-dd") %>
</lastmod>
<changefreq>weekly</changefreq>
<priority>1.0</priority>
</url>
<url>
<loc>http://www.ThinqLinq.com/about.aspx</loc>
<lastmod>
<%= System.IO.File.GetLastWriteTime( _
Server.MapPath("About.aspx")).ToString("yyyy-MM-dd") %>
</lastmod>
<changefreq>monthly</changefreq>
<priority>1.0</priority>
</url>
<%= From p In dc.PostItems.ToList _
Select <url>
<loc>http://www.ThinqLinq.com/default/<%= p.TitleUrlRewrite %>.aspx</loc>
<lastmod><%= p.PublicationDate.ToString("yyyy-MM-dd") %></lastmod>
<changefreq>daily</changefreq>
<priority>0.3</priority>
</url> %>
<url>
<loc>http://www.ThinqLinq.com/Contact.aspx</loc>
<lastmod>2008-02-28</lastmod>
<changefreq>never</changefreq>
<priority>0.1</priority>
</url>
</urlset>
Response.Write(map)
End Using
Catch ex As Exception
Response.Write(<error><%= ex.ToString %></error>)
End Try
End Sub
End Class