Most people don’t realize the importance of web hosting for SEO until they get in trouble. That’s pretty much my story.

In order to save some money on hosting bills I’ve moved my whole website network to a single VPS server (with dedicated IPs for each website) when something really weard happened. My website traffic started to decrease sligthly but in a constant rate. I’ve tracked down that the number of indexed pages on Google had decreated proportionally to my traffic which made it clear that there was some sort of indexing problem. I quickly checked if my firewall accidently banned google, but that was not the case. What happened was that my crawl rate had decreased.

As you can see on the above screencap taken from my Google Webmasters account, you can clearly see that my initial time spent on downloading pages is about 100 milliseconds while the account was on my shared web hosting account and then increased to 300 milliseconds when I moved it to my VPS server. It seems that my VPS server is 3 times slower than regular shared hosting account and my plan to save some money is going to cost me more than what I would have renewed my shared hosting account.

Not long ago, Matt Cutts (Google’s Search Engine spokesman) said that Page Rank is primarily used for determining crawl rate — the frequency of having your website re-indexed. Based on my own experiments with new websites, I have found out that the larger one website is, the more PR you need in order to get it indexed completely. These two things combined pretty much lead to the theory that PR is highly important for determining the time search engines spend on indexing or re-indexing your website. This pretty much concludes that the slower your server is, the more PR you need to get indexed, and vise-versa, the faster your Web Server is, the less PR you need to get it indexed.

I’m not 100% sure if this theory is true, but I will be able to solve this problem next week when I move my websites to brand new 12 GHz server — should be much faster than my old shared hosting account. If my theory is correct I will be observing increased crawl rate followed by increase of search engine traffic.

I would also like to state that some while ago, when I moved one of my largest sites to a much faster server I was observing highly increased search engine bot activity and my website got completely re-indexed every week or so.

I will update this post when I have more data next month.