[ic] Meta Tags

Jonathan Clark jonc@webmaint.net
Fri, 6 Oct 2000 13:33:26 +0100

>Does anyone know if meta tags setup on the product pages will be indexed by
>most robots in search engines?  Looking for the most effective way to
>meta tags with interchange.

I suppose the question really is: Will robots be able to access the product

For the broadest possible audience of robots you should consider the

support for frames and image maps is patchy.

most robots will ignore any arguments after the page name. So a page
would be accessed as /cgi-bin/foo/page.html

some extensions may be ignored (.cgi, .pl, etc)

some robots will avoid a path with cgi-bin in it.

Also bear in mind that meta tags are _not_ the be all and end all of getting
good rankings, or even indexing.
- in fact they are ignored by some search engines (Excite, FAST, Google,
Lycos, NLight) (Excite - keywords only ignored)
imho page titles and the content are far more important

The way ic accesses pages within its space is good from a search engine
point of view (/cgi-bin/foo/page.html as apposed to

The other thing, though, is that there is a robot limit set in ic (default
50 I think). How many of your product pages are actually going to be

Our strategy is to alias the cgi-bin as /sales/ and make part of the site
static html outside of ic with product overviews which are content rich (and
may include meta tags). The ic part would run to maybe a thousand flypages
and so only a small number will ever get spidered.

hope this helps


PS. for information on the workings of search engines I recommend
Some of the info is subscription but this is not expensive and worth it