[ic] Googlebot Getting 500 Errors ... but he's the only one

tquark at gmail.com tquark at gmail.com
Fri Oct 15 14:05:26 EDT 2004


Hi,

I'm seeing a strange error.  When the actual Googlebot visits my
site right now he is getting '500 Internal Server' errors on
almost every page access.  However, when I use wget or switch my useragent in
Firefox to be Googlebot I don't get any errors at all.  I have
RobotLimt 0 and there are no entries in the catalog or
interchange error.log corresponding to all of these errors (over
4,000 so far today :( ).  In the apache access_log there are of
course the logs showing the 500 errors and in the apache
error_log are entries like:
[Fri Oct 15 10:24:54 2004] [error] [client 66.249.64.33]
Malformed header return by Interchange:

Thinking that maybe it had something to do with RobotIP I removed
that from interchange.cfg and restarted.  No luck.  Then thinking
it might have to do with the mode of interchange I tried rpc
mode, no luck (I had it in high mode with MaxServers 20) then
tried low mode.  In low mode Googlebot fortunately gets 200 codes
meaning the pages are actually served to him.  This is very good
news to me but causes a problem.  I had to have it in RPC mode to
get AuthorizeNet to always work properly.  So what to do now?
Can anyone suggest what might be causing this?  Or maybe where I
could uncomment/put a logDebug to track down what is going on?

By the way, the strange thing is Googlebot has been visiting
every day at least 700 times and not getting these errors.  Also,
msnbot has visited over 1000 times today and gotten no 500
errors.  Seems to me that if Googlebot were getting errors then
msnbot would too...

Thanks,
Bryan


More information about the interchange-users mailing list