[ic] Spiders and affiliate tracking
jamie at versado.net
Tue Jan 6 19:32:38 EST 2004
Rene Hertell wrote:
> Why don't you use the robots.txt file and redirect them to a page the
> generates a complete index of your products and a url to each product?
I do have a page like you suggest, and I don't have a problem getting my
product database crawled, in fact I think every single product we sell
is indexed by Google at the moment.
The problem is that in order to track affiliates and PPC advertising, we
use URLs like this:
but these links are somehow getting crawled by certain search engines
and we are ending up with affiliate links in Google (and others).
I can't stop the spiders following affiliate links, so I'm just trying
to stop them from indexing the page they land on without affecting
normal crawling behavoir.
Jamie Neil | <jamie at versado.net> | 0870 7777 454
Versado I.T. Services Ltd. | http://versado.net/ | 0845 450 1254
More information about the interchange-users