By: Tyler

I have been tracking Google’s indexing for a few sites I work on (this is how I track Google’s indexing). The index for one site has dropped hard while the other sites saw no large movements. While this is a weak conclusion, I assumed a change on the site had caused the shift, not a change on Google’s side.

As I was researching possible causes for the shift downward, I remembered that our server (we have complete control of our server, it is not hosted by an external company) had issues around the same time. The site traffic and search engine crawling was working our servers hard. We started to experience server timeouts and slow server response times. Could this be the cause?

Google puts emphasis on server timeouts and slow server response time, as they see it to be a reliability issue for the website it is crawling. Google determines that if a site is not reliable enough to show a page, it will hurt the search engine results pages (SERP’s) relevancy and ultimately hurt Google.

So in Google’s view, our site became unreliable because they had issues indexing it. Almost every day Google experienced slow response times and the occasional timeouts. Since we get spidered often (general a good thing) it hurt us quite a bit. You can use Google’s Webmaster Tools to verify if Google is having issues crawling your site. Under “Web Crawl Errors” on the overview page, Google will show you which pages they had trouble with.

Remember that finding the exact cause of a shift in Google’s index is very difficult. While the cause for our drop in the index was probably several issues, anything that can be isolated and fixed is a move in the right direction.