[ic] Iterations slow with mv_matchlimit incrementations

[Grant]

> I do not have a categories table to test this ... But 1) your categories
> table probably has round about 100-1000 max results, so you can put
> ml=999999999999999 and it won't be making any difference. Then you feed that
> to the innerloop, where again you probably have 100-5000 results per
> category so again the 9999999 match limit does not really get reached anyway
> ...
>
> So your fast workaround is eventually returning all products, but it breaks
> the returns up in pieces ... Less data to handle at once ...
>
> Anyway in case you have a huge speed difference with 10 or 10000 then it
> could be your IC version (I've tested on 5.7.1) , but if 10 and 10000 are
> similar in speed and the problem really is with the 999999 then perhaps you
> want to monitor you environment, check what happens when you do the query
> (swap etc).

Is this informative?  It looks like the process which is running the
job is using quite a bit of memory, or maybe this much is normal?

Mem:   1028780k total,   973860k used,    54920k free,    91364k buffers
Swap:  2008116k total,    34188k used,  1973928k free,   290136k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
31789 interc   20   0  261m 252m 6324 R 97.2 25.1   2:39.70
interchange
31754 interc   20   0 69632  57m 3968 S  0.0  5.7   0:01.77 interchange

- Grant


> I also still do not understand that it is apparently for you working as:
> processing <long break> processing <long break> processing <long break>
>
> For me it 'thinks' and then put a processing blob all at once on screen.
>
>
>> So it seems like IC is getting bogged down when there are too many
>> matches in a loop search.  Should that happen?  Does it indicate a
>> problem somewhere in my system?
>>
>> I tried many times to narrow the problem down to a certain section of
>> my "processing" code but I always got nowhere.  I have the problem in
>> two separate loop searches of two different tables.
>>
>> - Grant