[ic] HELP - Inktomisearch stuck on ord/basket.html page
andrew at mapsdownunder.com.au
Mon Oct 25 20:28:16 EDT 2004
> Jamie Neil wrote:
> > Andrew Rich wrote:
> >> I initially stopped it by adding an entry to robots.txt for
> >> /cgi-bin/cartname/ord/basket.html and this pushed it off the page.
> >> I then thought orders were reduced and removed the entry from
> >> robots.txt. Inktomisearch came back and became stuck on the page
> > I don't see any reason why adding pages (or directories) to your
> > robots.txt file would affect orders - this is the preferred way of
> > limiting spider activity on your site.
> > Use something like:
> > User-agent: *
> > Disallow: /cgi-bin/cartname/ord/
> > Disallow: /cgi-bin/cartname/scan/
> > Disallow: /images/
> > Disallow: /cgi-bin/cartname/login.html
> > As well as "ord" we also ban robots from our images directory (to
> > our graphics turning up in image searches), and "scan" to stop them
> > indexing "more" links.
> I should add that banning robots from "scan" will prevent them
> _any_ category/search links on a stock foundation system, which is
> obviously undesirable. We use action maps to do most of our links,
> and so this is not a problem for us.
> We also feed spiders slightly different search results to normal
> normal users get 10 results per page while spiders get one page with
> the results on it. This allows the search engines to index all the
> products in a category without having to follow any "more" links.
Thank you Jamie for your feedback.
As we receive about 75% of orders via search engines we do not want to
do anything to stop them indexing our site.
Thia would obviously be a good start.
I am not familiar with action maps so I will have to do some reading
I have tried to move from the Foundation site to the new Standard site
structure (perhaps only new name) so not sure if this makes any
More information about the interchange-users