[ic] New Robot UAs

Grant emailgrant at gmail.com
Sun Jan 16 01:38:42 UTC 2011

>> Everyone,
>> I've been going through Apache data lately and have discovered at
>> least two new robots
>> that ought to be added to robots.cfg:
>> 1) 'bingbot'
>> http://www.bing.com/toolbox/blogs/webmaster/archive/2010/06/28/bing-crawler-bingbot-on-the-horizon.aspx
>> Looks like Microsoft has changed the UA for MSNBot, but Interchange is
>> no longer recognizing it.
> It should still recognize it, as the simple term "bot" is in the RobotUA
> list.
> Neverless I added it.
> Thanks
>        Racke

SurveyBot is also not triggering the spider detection.  Maybe the
"bot" entry only works if it appears after a delimiter in the UA?

- Grant

More information about the interchange-users mailing list