How I Blocked Googlebot By Accident

OK, things have not been going too great since the a site restructure, but I have been ploughing on all the same. Yesterday was the worse Monday in several years, a massive step back.

I took a look at the SERPs I noticed that my pages had not been updated in Google for a few days, generally updates happen every day. Also I have lost a lot of very old keywords, where I was no.1 for years, now nowhere at all.

I took a loot at Webmaster Tools, and did a Fetch as Googlebot. Got a 403 Error. Huh?

Now, when I get firewall reports I generally block the IP. Usually such things come from Brazil, Poland, far eastern places etc.

On 2nd October I got this one:

Web Page: /news.php?page=../../../../../../../../../../../../../../../../../../../proc/self/environ
Warning: URL may contain dangerous content!
Offending IP: 66.249.66.137 [ Get IP location ] Offending Parameter: page = ../../../../../../../../../../../../../../../../../../../proc/self/environ

My belief was than anything that sends such as request is sniffing around for weaknesses to hack. I admit I usually check the IP before blocking but obviously did not on this day. That IP is Google, Mountain View.

This morning I deleted out all the IPs that I had blocked in the last week and started replacing them and trying to fetch as googlebot until it broke again. That IP broke it.

Why did Google look for a page that does not exist?

Hopefully things will improve again, but I seem to have managed to upset Google yet again.

If any Googlers could check things their end and tell me if it is s minor set back or a major upset, that would be lovely.

This morning I started to think that it was time to pack it all in and go back to Banking. Not happy days!

Leave a Reply

Your email address will not be published. Required fields are marked *