[ic] Spiders and affiliate tracking

[Jeff]

On Tue, Jan 06, 2004 at 04:57:39PM +0000, Jamie Neil wrote:

> I'm currently trying out a possible solution, which I thought I would 
> post here in case anyone is having the same problem. Please feel free to 
> shoot it full of holes:

	I use the directives below in my apache httpd.conf .  I found it 
on webmasterworld.com .  It looks for spiders and redirects them to the 
same page minus any parameters.  To use it, replace 
"shop.sophee.com" with the name of your own site.

# Redirect search engine spider requests which include a query string to same URL with blank query string
RewriteCond %{HTTP_USER_AGENT} ^FAST(-(Real)?WebCrawler/|\ FirstPage\ retriever) [OR]
RewriteCond %{HTTP_USER_AGENT} ^Gigabot/ [OR]
RewriteCond %{HTTP_USER_AGENT} ^Googlebot(-Image)?/[0-9]\.[0-9]{1,2} [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mediapartners-Google/[0-9]\.[0-9]{1,2} [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/.*(Ask\ Jeeves|Slurp/|ZealBot|Zyborg/) [OR]
RewriteCond %{HTTP_USER_AGENT} ^msnbot/ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Overture-WebCrawler/ [OR]
RewriteCond %{HTTP_USER_AGENT} ^Robozilla/ [OR]
RewriteCond %{HTTP_USER_AGENT} ^(Scooter/|Scrubby/) [OR]
RewriteCond %{HTTP_USER_AGENT} ^YahooSeeker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teoma
RewriteCond %{QUERY_STRING} .
RewriteRule (.*) http://shop.sophee.com/$1? [R=301,L]