Google Sitemaps (Pt3) Structuring Large Sitemaps

Step 1) Index-Sitemaps:

The purpose of an index sitemap is to give search engines the destinations to all of your internal sitemaps. You'll want to sit down and try to map out your web site entirely.

main.xml or main.php

This specific sitemap should contain all of your main pages within your site, for example:

• Home
• About us
• Contact us
• Services
• Categories
• RSS Page
• etc.

Remember that you can detail up to 50,000 links within one sitemap. Make it count. I will use an example of what I’ve come up with while using one of my sites that has over 450 main categories. Within another file, call your categories with php, or anything else while parsing your sitemap(s) code with the category destination.

Parsing Categories:

You have two options when parsing categories for your web site. 1) You can simply use php to consistently parse the categories over and over again, or 2) Output the categories within a text file and paste them into your main.xml sitemap(s) page so that you don't use up vital bandwidth resources every time a search engine calls your information.

These options greatly depend on whether or not you change your categories all the time, add more to the list, or never update them.

Here's a simple script to parse the categories either way. Change it as needed.

<?php
mysql_connect("localhost", "username", "password") or die(mysql_error());

$mainCats = mysql_query("SELECT linkURL, date FROM YourCategories ORDER BY categoryID DESC LIMIT 50000");
while($myLink = mysql_fetch_array($mainCats)){

$dateValue = $myLink['date'];
$Date = date("Y-m-d", strtotime($dateValue));
$link = $myLink['linkURL'];

?>
   <url>
      <loc><?php echo $link; ?></loc>
      <lastmod><?php echo $Date; ?></lastmod>
      <changefreq>monthly</changefreq>
      <priority>0.5</priority>
   </url><?php } mysql_close(); ?>

Once you've parsed this within your browser, right click > view source and save the file on your computer to use later on. Alternatively, you can copy this script within your main.php sitemap and keep parsing your categories, again and again.

Remember, if you have more than 50,000 categories, you'll need to break them up into multiple sitemaps and add them to your index-sitemaps file.

Parsing Members:

Some sites have a large number of members within their database who hold their own personal members page. The best way to list your members is to add a unique sitemap for each letter of the alphabet.

Within your index-sitemap, add something to resemble the following:

   <sitemap>
      <loc>http://www.bogaboga.net/members/sitemapMembers-a.php</loc>
   </sitemap>
   <sitemap>
      <loc>http://www.bogaboga.net/members/sitemapMembers-b.php</loc>
   </sitemap>

And so forth. If your web site is consistently changing and is forced to add new members all the time, you'll want to use php to parse them each time a search engine crawls your site. Otherwise you can parse them once and save it as .xml as a static file.

The key is to somehow get all members and to make sure you don't go over the 50,000 link limit within one sitemap at a time.

Parsing Articles:

Just like I mentioned above, you can use the same strategy as your "categories" to separate your articles into groups. Add a unique sitemap to your index-sitemaps for each category of articles and parse the link to your articles within each separate file.

Is the Priority Really Important?

Another note I should make is that the optional "priority" statement within your sitemaps is not as important as listing every page you have. Google will take the priority statement into consideration, but they will still use their own collective knowledge to rank your pages according to link popularity, key word relevance, etc.

In Conclusion:

When structuring your tree for your sitemap(s), use an index-sitemap to give search engines the destination to multiple sitemap(s) that break into different categories, including, but not limited to:

• main pages
• categories
• members
• listings
• articles
• rss feeds
• other

Your index-sitemap may contain 100's of links to various little sitemaps all over your entire site. Some sites may need this much. Nevertheless, do not worry; you are not alone. I was forced to create one index with over 500 different sitemaps for inclusion. Once completed, it was a great tool which ultimately found another 65,000 new pages not indexed within search engines. Do not be surprised if Google finds more links than what you have listed within your sitemap(s); it happens more often than not.

In the next issue, I will go into detail on "Google Sitemaps (Pt4) Submitting Your Sitemaps Online".

About The Author:

Martin Lemieux, president of the Adcidia network, has over 16 years of experience online while maintaining a network of over 35 web sites. For more tips like these, please visit:

Martin's Blog:
http://www.MartinLemieux.ca/

Internet Marketing Blog:
http://www.MartinLemieux.ca/internet-marketing/

Martin's RSS Feed:
http://www.MartinLemieux.ca/xml/

© Copyright, Martin Lemieux - All Rights Reserved. Reprints Accepted.