[ic] Googlebot Getting 500 Errors ... but he's the only one

Jonathon Sim jonathonsim at zeald.com
Tue May 17 18:27:36 EDT 2005


On 8:23:13 am 05/18/05 Bryan Gmyrek <bryangmyrek at yahoo.com> wrote:
>
> Update:  For now it seems that for whatever reason switching to high
> traffic mode and restarting interchange has fixed it (googlebot
> getting 200 codes now).
> But I'm sure it'll be back.
> Any ideas as to what's really causing this or how I could track that
> down would help a lot.
>
Hrm.  We once had something similar to this happening on our servers : it
turned out to be a bug in mod_interchange caused by the "if-modified-since"
headers that google sends (this bug also affected safari).  Since google
only sends it when the last attempt it made succeded, google would
alternately succeed and fail each time it  tries to access a page.

You can see if thats the problem your seeing by running this perl script
with:
perl test_google_bug.pl http://yoursite.com

---test_google_bug.pl---
use WWW::Mechanize;
use Test::More tests => 2;

my $url = $ARGV[0];  
my $ua = WWW::Mechanize->new;

$ua->get($url);
ok($ua->status == 200, 'check http status is 200 without if-modified-since
header');

$ua->add_header('IF_MODIFIED_SINCE' => 'Wed, 08 Sep 2004 11:09:13 GMT');
$ua->get($url);
ok($ua->status != 500, "check http status is not 500 with if-modified-since
header (status is ".$ua->status.")");

--
if it returns :
ok
failed
then you know thats the problem.  If so I have a patch I can send you (I
think its been integrated in the mod_interchange)

--
Jonathon Sim <jonathonsim at zeald.com>
Senior Developer @ Zeald.com
Jabber:sim at jabber.org.nz ICQ:62562604 MSN:sim at zeald.com
Zeald.com - 'Websites that Work!' http://www.zeald.com
Ph: +64 9 415 7575, Fax: +64 9 443 9794



More information about the interchange-users mailing list