By: Dezmon Landers

Are you tracking how Google indexes your site? If you rely on Google as much as most web companies, it is important to see how you are being indexed. You always hear about the dreaded “Supplemental Index” (whether it exist or not) which is a fear for any startup. The supplemental index (SI) is a segment of Google’s index where the pages are in a search engine purgatory. The problem is, there is no way to tell if you have web pages in the SI and worse yet, there is not 100% proof that it exist.

There is hope however. With the proper tracking you can get a good idea of when Google views your site as a positive additional to its index. If you submitted an XML site map in Google’s Webmaster Tools (how-to), you will have an idea of how many URL’s have been submitted to Google. This is not needed for the tracking but will give you an idea of the percentage of URL’s Google is giving you credit for.

Set up a spreadsheet with 3 columns; “Google Index”, “Included Index”, and “Visitors” (you can use Google Analytics for the visitors data). I recommend tracking this daily.

The next step will involve Google.com. Type site:www.yoursite.com into the search box and hit enter. Note the total number of results, that is the number of pages Google is “verifying” in their index from your site. This is the number you put into the “Google Index” column. This number will change daily.

Next, type site:www.yoursite.com/* into Google. The addition of /* to the search query will be your “Included Index” number on the spreadsheet. Mark the total number of results. This number will be smaller than your “Google Index” result. This number represents the pages that are in good standing and will be picked to show up in the SERP’s first. Subtracting the “Included Index” number from the “Google Index” number will not give you the number of pages in the supplemental index unfortunately. This number will also change daily.

Finally, add the number of visitors you had into the corresponding date on your spreadsheet. The visitors number is always going to be a day behind because GA is not in real time. Just make sure that you put the same indexing numbers with the proper visitor count date.

Finish by creating a graph to see the changes in all three data ranges. As you make changes to your site, and Google changes their algorithm, you will see changes in the pages indexed. Overtime you can see trends and better figure out how to make the index gods happy. It is best if you can have new pages added to your site frequently (which is a major plus with Google anyway).

If you get the chance to measure multiple sites I highly recommend it. This will help you to notice which trends are from your changes and which are coming from Google (Google changes will affect all sites).