[interchange-cvs] interchange - kwalsh modified dist/robots.cfg

interchange-cvs at icdevgroup.org interchange-cvs at icdevgroup.org
Wed Jul 11 15:34:42 EDT 2007


User:      kwalsh
Date:      2007-07-11 19:34:41 GMT
Modified:  dist     robots.cfg
Log:
    * Periodic insertion of search engine robots, and other non-browsers,
      found on the www.interchange.rtfm.info website.

Revision  Changes    Path
2.5       +6 -0      interchange/dist/robots.cfg


rev 2.5, prev_rev 2.4
Index: robots.cfg
===================================================================
RCS file: /var/cvs/interchange/dist/robots.cfg,v
retrieving revision 2.4
retrieving revision 2.5
diff -u -r2.4 -r2.5
--- robots.cfg	11 May 2007 09:53:08 -0000	2.4
+++ robots.cfg	11 Jul 2007 19:34:41 -0000	2.5
@@ -52,6 +52,7 @@
     Gromit,
     Gulliver,
     H?m?h?kki,
+    HTTrack,
     Harvest,
     holmes,
     HTMLDOC,
@@ -75,6 +76,7 @@
     Mercator,
     Microsoft Data Access,
     Microsoft Office,
+    Microsoft URL Control,
     Microsoft-WebDAV,
     MimeLive,
     mirago,
@@ -90,6 +92,7 @@
     ozelot,
     pagebull,
     ParaSite,
+    pavuk,
     Pokey,
     Pompos,
     Refiner,
@@ -102,6 +105,7 @@
     Scooter,
     search,
     seek,
+    shelob,
     silk,
     Slurp,
     sna,
@@ -117,8 +121,10 @@
     tarantula,
     topiclink,
     Toutatis,
+    TurnitinBot,
     Tv*Merc,
     urllib,
+    VB Project,
     Valkyrie,
     Voyager,
     W3C_Validator,








More information about the interchange-cvs mailing list