<html><body><div style="color:#000; background-color:#fff; font-family:times new roman, new york, times, serif;font-size:12pt"><div><URL: http://rt.icdevgroup.org/Ticket/Display.html?id=344 ><br><br>On 03/02/2011 03:15 PM, David Christensen wrote:<br>>> 80 legs webcrawler identifies itself as:<br>>><br>>> Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620<br>>><br>>> Because of the NotRobotUA entry 'Gecko' this crawler is not identified as such.<br>>><br>>> Blocking via RobotIP will not work as it works via a distributed network of IP's ... So it will crawl creating a bunch of session id's with all different IP numbers.<br>><br>> Yeah, I've been reconsidering the NotRobotUA change. I like it in principle, but then you end up with cases like this. Short of a JustKiddingThisIsReallyARobotUA directive, I'm not sure how to do this generally—it starts to feel
like an arms race. I think in the general case, we'd rather users always be able to have a session/checkout, so basically we'd run into cases like this as the exception to handle.<br>><br>> Perhaps a suitable negative lookahead/behind pattern would help in this specific case. I'm also open to ideas/other thoughts.</div><div><br></div><div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Kristen Eisenberg</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Billige Flüge</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Marketing GmbH</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Emanuelstr. 3,</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">10317 Berlin</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Deutschland</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Telefon: +49 (33)</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">5310967</span></div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">Email:</span></div>
<div class="MsoNormal">utebachmeier at</div>
<div class="MsoNormal">gmail.com</div>
<div class="MsoNormal">Site:</div>
<div class="MsoNormal"><span lang="DE" style="mso-ansi-language:DE">http://flug.airego.de-
Billige Flüge vergleichen</span></div></div></div></body></html>