Re: [mv] m3.14 - Couldn't spawn - bad robots - firewall

["kyle@invisio.com" <kyle@invisio.com>]

******    message to minivend-users from "kyle@invisio.com" <kyle@invisio.com>     ******

Ryan,

I set up Minivend (3.12) to use ipfw ( on FreeBSD ) to block
according to Minivend default ( >50 accesses with no 30 sec
delay between these could be adjusted though) and it has
been working great.

The only trick, was that I had to install sudo to give Minivend
permission to issue the ipfw command (a root suid script for
some reason would not do it)

It has not (so far) blocked any "normal" bots, I have checked
the blocks via nslookup and compared them to access logs
to find the browser information and found most blocks are
being put up when someone uses one of those
automated-offline-web-page-caching-server-crashing
pieces of software or one of those "projects" which claims
they are indexing the entire Internet.

If you want further details on how I set it up, you can contact me off list.

Latter,
Kyle (KC)


At 06:28 PM 10/24/00 , you wrote:
>******    message to minivend-users from Ryan Hertz <rhertz@gyb.baits.com>   
>  ******
>
>Dear List,
>
>         I've encountered a new one -- most definitely linked to those 
>pesky robots.  They must have really nailed me, since the server was 
>apparently unable to spawn the CGI.
>
>         So, my question is, does anyone have any success with blocking bad 
>robots?  Could you give suggestions as to how to firewall them?  I've 
>recently allowed (via robots.txt) bots to enter the store so I could get 
>some better search engine saturation.  Although I could re-instate the 
>blockage, many of these bad bots are ignoring the robots.txt directives
anyhow.
>
>         Any tips would be appreciated!
>
>Thanks,
>
>-Ryan
>
>
>
>
>>[Sun Oct 22 17:02:42 2000] [error] [client 152.163.207.59] (11)Try again: 
>>couldn
>>'t spawn child process: /var/lib/httpd/cgi-bin/order
>>
>>- - - [22/October/2000:11:45:38 -0700] - - START server (16835) (INET and 
>>UNIX)
>>coosk25ds62.osk.mesh.ad.jp scFv6jXp:coosk25ds62.osk.mesh.ad.jp - 
>>[24/October/200
>>0:07:07:18 -0700] order /cgi-bin/order/192 WARNING: POSSIBLE BAD 
>>ROBOT.  51 acce
>>sses with
>> > no 30 second pause.
>>
>>>
>>>Internal Server Error
>>>
>>>
>>>
>>>The server encountered an internal error or misconfiguration and was 
>>>unable to complete your request.
>>>
>>>Please contact the server administrator, rhertz@gyb.baits.com and inform 
>>>them of the time the error occurred, and anything you might have done 
>>>that may have caused the error.
>>>
>>>More information about this error may be available in the server error log.
>
>
>Ryan Hertz                                              tel  800-645-BAIT
>Webmaster                                               fax  520-645-2588
>Advertising Director                            http://yamamoto.baits.com
>Gary Yamamoto Custom Baits, Inc.                http://www.insideline.net
>
>-
>To unsubscribe from the list, DO NOT REPLY to this message.  Instead, send
>email with 'UNSUBSCRIBE minivend-users' in the body to Majordomo@minivend.com.
>Archive of past messages: http://www.minivend.com/minivend/minivend-list

-
To unsubscribe from the list, DO NOT REPLY to this message.  Instead, send
email with 'UNSUBSCRIBE minivend-users' in the body to Majordomo@minivend.com.
Archive of past messages: http://www.minivend.com/minivend/minivend-list