[interchange-cvs] interchange - kwalsh modified dist/robots.cfg
interchange-cvs at icdevgroup.org
interchange-cvs at icdevgroup.org
Wed Jul 11 15:34:42 EDT 2007
User: kwalsh
Date: 2007-07-11 19:34:41 GMT
Modified: dist robots.cfg
Log:
* Periodic insertion of search engine robots, and other non-browsers,
found on the www.interchange.rtfm.info website.
Revision Changes Path
2.5 +6 -0 interchange/dist/robots.cfg
rev 2.5, prev_rev 2.4
Index: robots.cfg
===================================================================
RCS file: /var/cvs/interchange/dist/robots.cfg,v
retrieving revision 2.4
retrieving revision 2.5
diff -u -r2.4 -r2.5
--- robots.cfg 11 May 2007 09:53:08 -0000 2.4
+++ robots.cfg 11 Jul 2007 19:34:41 -0000 2.5
@@ -52,6 +52,7 @@
Gromit,
Gulliver,
H?m?h?kki,
+ HTTrack,
Harvest,
holmes,
HTMLDOC,
@@ -75,6 +76,7 @@
Mercator,
Microsoft Data Access,
Microsoft Office,
+ Microsoft URL Control,
Microsoft-WebDAV,
MimeLive,
mirago,
@@ -90,6 +92,7 @@
ozelot,
pagebull,
ParaSite,
+ pavuk,
Pokey,
Pompos,
Refiner,
@@ -102,6 +105,7 @@
Scooter,
search,
seek,
+ shelob,
silk,
Slurp,
sna,
@@ -117,8 +121,10 @@
tarantula,
topiclink,
Toutatis,
+ TurnitinBot,
Tv*Merc,
urllib,
+ VB Project,
Valkyrie,
Voyager,
W3C_Validator,
More information about the interchange-cvs
mailing list