[ic] New Robot UAs

[Grant]

>> Everyone,
>>
>> I've been going through Apache data lately and have discovered at
>> least two new robots
>> that ought to be added to robots.cfg:
>>
>> 1) 'bingbot'
>>
>> http://www.bing.com/toolbox/blogs/webmaster/archive/2010/06/28/bing-crawler-bingbot-on-the-horizon.aspx
>>
>> Looks like Microsoft has changed the UA for MSNBot, but Interchange is
>> no longer recognizing it.
>>
>
> It should still recognize it, as the simple term "bot" is in the RobotUA
> list.
> Neverless I added it.
>
> Thanks
>        Racke

SurveyBot is also not triggering the spider detection.  Maybe the
"bot" entry only works if it appears after a delimiter in the UA?

- Grant