Search engine ranking is perhaps the most important element that determines a website’s success. The key to getting your website to appear in the search engine results starts with getting your website’s pages indexed. By indexed, I mean allowing the Google robot to visit and scan each page on your website to determine its relevance. However, many search engines have a difficult time finding pages that are buried deep in your site’s navigation structure. Below I will discuss some techniques that will help the search engines find those hidden pages, as well as assist human visitors to navigate your website. I will be focusing exclusively on Google because, although there are many search engines, most websites get 50x more traffic from Google than Yahoo & MSN combined.
Depth of the Index
The frequency which Google visits your site to index pages is largely determined by the frequency of website updates. If your website is updated daily, Google is likely to visit everyday. The “depth” of Google’s indexing refers to the tiers of your website’s navigation tree that will get indexed. For example, your homepage would be the first tier and any pages linked from the homepage, such as your “Company Info” page would be 2nd tier pages. Any pages that are only accessible from these second tier pages would be considered 3rd tier pages.
The depth of Google’s index largely depends on each page’s Page Rank (that addicting little green bar on the Google Toolbar). Generally, as you move from the 1st to 2nd and from 2nd to 3rd tier of your website’s navigation structure, Page Rank will fall. If your homepage has a Page Rank of at least one, your 2nd tier pages should get indexed. If your second tier pages have low or no Page Rank, then Google probably hasn’t indexed any pages past this tier and isn’t aware that other pages exist.
What is a SiteMap?
A sitemap lists all your website’s pages in a structured, single page format. By viewing your sitemap, a website visitor can quickly and easily be aware of all your website’s pages and navigate to the area they are interested in. Although this type of sitemap is geared towards human viewers, it is also beneficial for search engines because it will allow the robots to see pages that may be 4 or 5 tiers deep in your website. Given that the Page Rank on you sitemap page is ample enough (make sure it is linked from your homepage), Google should crawl and index your whole website.
New Type of SiteMap
There is a new type of sitemap available to use – The Google SiteMap. It is a little different from the traditional sitemap – it is an XML file located on your website’s server that lists all your website’s pages and associated attributes. Google will read this file and will be aware of all your website’s pages, regardless of the depth of the navigation structure. In addition to listing each page of your website, you can also specify attributes such as the frequency of updates and priority of indexing for each page.
How Does It Work?
As I said before, the Google Sitemap is created in XML (Extensible Markup Language). It is similar to HTML, and often used these days in RSS and blog feeds. Here is an example of an entry that would be written for each page in a website:
The “url” tag specifies that you are entering information for a new URL (website page). The “lastmod” tag specifies when the page was last updated. The “changefreq” tag specifies how often the page is updated. The “priority” tag specifies the relative priority for indexing of that page.
This new type of sitemap is great for letting Google know about your whole website. The bad news is that this XML file needs to edited in Notepad each time you make a change or addition that effects your website’s navigation structure. If your website changes frequently or you have a lot of pages, coding the file by hand can be a real pain. Fortunately, there are many sitemap generators available for free on the Internet. Just do a search for “Google Sitemap Generator” and you will find dozens of free tools that will “crawl” your website and create the sitemap for you.
Google states that using their sitemap program will not increase your search engine rankings. But, it will get more of your pages indexed, and Google in turn should deem your website larger and more important. In the long run this should lead to increased search engine traffic. Don’t forget to update your sitemap when your website changes. By keeping both types of sitemaps updated and accessible, your website will quickly become a visitor and search engine favorite.
Word count: 805