[ic] timed-build breaks more list - help needed

[Dan Browning]

* DB <DB at M-and-D.com> [2004-01-26 05:57]:
> I'm running 4.8.8 using a modified foundation and MySQL. I have about 
> 150,000 items in the catalog. Instead of the stock category list along 
> the left side of the page, I have links to custom pages which contain 
> searches.
> 
> When customers click these links, the search is performed as expected. 
> But because the database is getting large, these searches are slowing 
> things down. Since the same search is performed each time these links 
> are clicked, the timed-build tag should really help reduce the server load.
> 
> On one of these custom pages, I put [timed-build auto=1 force=1 
> minutes=0] at the top of the page and [/timed-build] at the bottom. As 
> expected the page loaded much faster.. almost instantly. The only 
> problem is that the more list no longer functions. When I click "Next" 
> or any other link from the more list, I just get the first page again. 
> There is a big performance incentive for me to fix this so any hints are 
> welcomed. I tried moving the [timed-biuld] tage to various places on the 
> custom page but unless I wrapped almost the entire page, things broke 
> (raw ITL tags were shown on the page). Below is an example of one of 
> these custom pages' CONTENT section. If I wrap this whole section with 
> [timed-build...], I get the speedup but a broken more list.

Your question is important to me, too.  Right now, I don't think it is possible 
to cache search results when [more] is used, but it would be a very nice feature 
if it were possible.

In their current state, the [more] and [timed-build] conflict with eachother.  
[timed-build] trumps [more] because it saves the results of the first page.  No 
matter what options are passed to [more] (via the scan/ actionmap), they 
get ignored because [timed-build] is just calling up the cached HTML as soon as 
IC loads the page.

I would really enjoy hearing someone else's opinion on this matter.  The only 
solutions I can think of involve hacking the core.  Of those, I think the most 
elegant way is to modify the core to meld [timed-build] and [more] together in a 
compatible way:

For searches that have a [more]:

 * Generate a unique search key.  It could represent the search as a whole.  For 
   example, it could concatenate the values of the search params in a certain 
   order, optionally md5 them.  Note that this key would be unique enough that a 
   search with "se=bob" would not be the same as "se=jane".  However, if there 
   were two searches for "se=bob", the second one should generate the same key   
   as the first.
   
 * Save all of the search results (in the regular format) to one file (like 
   currently, except the key will be less-unique than standard IC).
 
 * Save the search results of the current page, as rendered, to a timed-build 
   file.
 
So you end up with:

	tmp/<search key> (same search results file you get now)
	timed/<search key>/1 (has the rendered search results for page 1)	
	timed/<search key>/2 (has the rendered search results for page 2)	
	timed/<search key>/N (has the rendered search results for page N)

Or, for alphabetical more lists:

	tmp/<search key> (same search results file you get now)
	timed/<search key>/A (has the rendered search results for page A)	
	timed/<search key>/B (has the rendered search results for page B)	
	timed/<search key>/N (has the rendered search results for page N)
	
You could pass standard [timed-build] parameters for specifying time limits, 
etc.  Extra parameters would be useful, like "these are the keys that I consider 
unique for the purpose of generating a search key: _____".

A different method would be to modify [more] so that it generated a URL with two 
things:

 * Generate the same unique key as above.
 * Pass the unique key and a "page" parameter (page=1, page=2, etc.) to 
   the scan/ Action.
 * In the HTML, parse the REQUEST_URI to pick out the search key and page.
 * Call [timed-build file="timed/<search_key>/<page>"
  
Like I said earlier, I welcome opinions on this matter.  I'm considering the 
thought of hacking this into the core myself.

-- 
Dan Browning, Kavod Technologies, <db at kavod.com> 360.843.4074x217
6700 NE 162nd Ave, Ste 611-210, Vancouver, WA.    Random Fortune:
You have a strong appeal for members of the opposite sex.