[ic] Too very slow pages on ic 5.10.0

[David Christensen]

> On Mar 11, 2018, at 11:01 PM, Mike Heins <mike at heins.net> wrote:
> 
> Quoting Peter (peter at pajamian.dhs.org <mailto:peter at pajamian.dhs.org>):
>> On 11/03/18 01:44, marco wrote:
>>> after 3 months of installation and light use my old production catalogue
>>> with:
>> 
>>> I notice that for the pages with heavy query became very slow and
>>> sometimes bring to "Gateway Timeout"
>>> "top" utility display cpu to 100% during page wait sometimes for mysql
>>> and sometimes for interchange
>> 
>> You need to benchmark the code on the problematic pages and find out
>> exactly which lines or blocks of code are causing the issue.
>> 
>> This is most likely going to be the result of lots of DB hits from some
>> very inefficient code.  Look for calls to [data] or [field], or similar
>> tags that waste db hits inside of a [loop] or similar.  Rewrite your
>> code so that it does a single query to the db to get all the data you
>> need instead and it will end up being way more efficient.
> 
> Don't discount the possibility that the I/O on this virtual instance
> is slower than on the other one. Are there demanding applications which
> show the same speed on the new container? Try doing some large DB queries
> on both the old and the new and comparing.


With newer perls, my personal experience is that parsing slowdowns are caused by using regexes against Unicode strings.  If you’re using PostgreSQL with DBD::Pg >= 3, this can also materialize itself *even without* explicit configuration on the interchange side (i.e., MV_UTF8 and friends).  If this matches your situation: Newer perl, PostgreSQL, DBD::Pg >= 3 (*no* MV_UTF8 configuration) you can turn off the automatic unicodization of database data via this in your catalog.cfg:

DatabaseDefault PG_ENABLE_UTF8 0

The issue is that DBD::Pg >= 3 by default will automatically decode any data it pulls from the server if the client_encoding is UTF8, unless you explicitly disable that functionality, which means that any page where IC interpolates data that came from the database will have the page string upgraded to Unicode, which will take the slower regex path.  This is particularly noticeable in perls ~ 5.14 - 5.20, but is still substantially slower than the non-unicode path in 5.24.

Not sure if that might help explain things, but this is something we’ve run into with recent perls.  Additionally, the normal database optimization approaches mentioned elsewhere in this thread will also always be good too.

Best,

David
--
David Christensen
End Point Corporation
david at endpoint.com
785-727-1171



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.icdevgroup.org/pipermail/interchange-users/attachments/20180312/aeaa86a3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://www.icdevgroup.org/pipermail/interchange-users/attachments/20180312/aeaa86a3/attachment.pgp>