[ic] Upgrade to Interchange-5.7.4 performance issues

[Bill Carr]

On Feb 12, 2010, at 9:19 AM, Gert van der Spoel wrote:

>> -----Original Message-----
>> From: interchange-users-bounces at icdevgroup.org [mailto:interchange-
>> users-bounces at icdevgroup.org] On Behalf Of Bill Carr
>> Sent: Friday, February 12, 2010 4:19 PM
>> To: interchange-users at icdevgroup.org
>> Subject: Re: [ic] Upgrade to Interchange-5.7.4 performance issues
>> 
>> 
>> On Feb 12, 2010, at 2:03 AM, Gert van der Spoel wrote:
>> 
>>>> -----Original Message-----
>>>> From: interchange-users-bounces at icdevgroup.org [mailto:interchange-
>>>> users-bounces at icdevgroup.org] On Behalf Of Bill Carr
>>>> Sent: Friday, February 12, 2010 1:00 AM
>>>> To: interchange-users at icdevgroup.org
>>>> Subject: Re: [ic] Upgrade to Interchange-5.7.4 performance issues
>>>> 
>>>> 
>>>> On Feb 11, 2010, at 10:50 AM, Bill Carr wrote:
>>>> 
>>>>> 
>>>>> On Feb 11, 2010, at 10:34 AM, David Christensen wrote:
>>>>> 
>>>>>> 
>>>>>> On Feb 11, 2010, at 1:10 AM, Jon Jensen wrote:
>>>>>> 
>>>>>>> On Wed, 10 Feb 2010, Bill Carr wrote:
>>>>>>> 
>>>>>>>> However, 5.7.4 runs much slower. Pages that get served on in
>> under
>>>> 5 seconds are taking 10, 20, 30 seconds or longer. The load average
>> on
>>>> each host running 5.7.4 goes from under 2 to near 10. I'm running
>> the
>>>> rpc mode from interchange.cfg.dist. Configuration is the same for
>> 5.4.0
>>>> and 5.7.4 (besides small changes with actionmap api). I know 5.7.4
>> is
>>>> not a production release. I'm just trying to bring my setup up to
>> date.
>>>> Been running a hacked up 5.4.0 for a while.
>>>>>>>> 
>>>>>>>> Any ideas on why this would be or how I should debug it?
>>>>>>> 
>>>>>>> I haven't seen any slowdowns like that. My best guess would be
>> that
>>>> it's related to the new character set encoding stuff. What UTF8-
>> related
>>>> directives or variables do you have?
>>>>>> 
>>>>>> 
>>>>>> There have been reports of slower regexes in some cases when using
>>>> UTF8 strings; in particular unanchored regexes like /.*foo/ will
>>>> sometime take a much longer time to process.  (The IC parser is made
>>>> almost exclusively of regexes, many of which fall into this
>> category.)
>>>> I believe this is a perl bug, but I'm not sure if this has been
>> fixed
>>>> in later perl versions or not.  What version of Perl/Encode are you
>>>> using?
>>>>> perl v5.10.0 built for x86_64-linux
>>>>> 
>>>>> Encode 2.39
>>>>> 
>>>>>> 
>>>>>> As a workaround, you can disable the MV_UTF8 as mentioned
>> upthread,
>>>> or try a newer Perl version to see if it has been solved there.
>>>>> I had this disabled initially and it still went slow.
>>>>> 
>>>>> I'll try perl 5.10.1 now.
>>>> I upgraded perl to 5.10.1 and running IC 5.7.4 seemed a little
>> better
>>>> but still problematic. Several pages would be served up very quickly
>>>> but then there would be a page that took 10, 20, 30 seconds or more.
>>>> Then it would go quickly again.
>>>> 
>>>> My load balancer (LVS/ldirecrtor) calls a test page every 10 seconds
>>>> and it was getting timeouts on the test page and removing the real
>>>> server from the virtual server pool bringing down our application.
>> This
>>>> does not happen with 5.4.0. The test page is very basic and I call
>> it
>>>> with mv_tmp_session=1. When I called the test page from the LVS box
>>>> using wget it would take 30 seconds or so to return. Subsequent
>> calls
>>>> of the test page came back almost instantly. Then  I would get a
>> slow
>>>> one again.
>>> 
>>> Are you using timed-build  for some items on your pages?
>>> That could explain the once in a while slow - when it rebuilds the
>> part
>>> created with [timed-build].
>>> 
>>> It sounds like a very specific problem and I can only suggest to take
>> that
>>> test page and start reducing it into smaller bits and pieces and then
>> run
>>> the test on that and see if there are any calls that take longer. If
>> it
>>> stays the same then you quite likely removed the part that caused the
>>> problem.
>> This is the entire test page:
>> 
>> [tag op=header]Content-Type: text/plain[/tag]
>> testpage
>> [var HOST_NAME]
> 
> Not much more left that taking 'test' out of 'testpage' hehe .. 
> 
> Your test is done on the production system? Ie while you run this test it
> could actually be someone else who is also trying to load something on your
> site that creates the slowdown? 
I tried removing all UTF-8 flags and directives and I've tried with 5.7.5 nightly and I still get big slow downs compared to running 5.4.0. During the slow downs there are several interchange processes consuming 100% CPU. What else can I try?

I have some other issues and I would like to upgrade to see if they go away. Also I don't want to be stuck on 5.4.0 while the interchange world passes me by.

 
Bill Carr 
(413) 584-0400 
http://www.bottlenose-wine.com