CVS log for interchange/lib/Vend/Dispatch.pm

Up to [Local Repository] / interchange / lib / Vend

Request diff between arbitrary revisions

Keyword substitution: kv
Default branch: MAIN

* Currently the only order cleanup -- i.e. functions to run after all data in the
  session has been used for order functions -- available is the bottom of the receipt
  page, if you happen to know that is the place to do it.

  This provides an authoritative place to perform such actions, in the same
  vein as AutoLoad and AutoEnd.

  Implmentation possibilites included profiles to be run at the end or a SpecialSub.

refined error message for missing actions.

Avoid crash on non-hashes in Filter directive. Logging isn't enough
to catch bad data.

Change AutoloadEarliest directive name to Preload.

Make global ActionMap input consistent with catalog ActionMaps (RT #48).

This will break all global ActionMaps that expect the old calling
parameter of a URL with the action removed from the beginning.

I've manually reviewed all of the built-in global ActionMaps, and tested
some of them, and found no problems. But I won't be surprised if some
problems are found. More testing is warranted.

Add new AutoloadEarliest catalog directive.

This is just like Autoload but runs at the earliest possible stage of
page processing, which allows tweaking the session, path, robot status,
cookie handling, authorization, cookie handling, etc.

Note UserTrack default change.

Clean up some whitespace.

Change UserTrack behavior to better match expectations.

* "UserTrack no" formerly also disabled TrackFile, because the whole
  Vend::Track module was disabled. This was not expected behavior. People
  are apparently using TrackFile fairly commonly, so this would make
  "UserTrack no" pretty unattractive.

* Make "UserTrack no" only disable sending the X-Track HTTP response header.

* As before, leaving TrackFile undefined will stop logging to a track file.

* Make UserTrack default to false now, which is an incompatible change, but
  one that I don't expect to adversely affect anyone, as the X-Track response
  header doesn't seem to get used. Adding "UserTrack yes" to catalog.cfg
  brings it back.

In short, most people upgrading will stop having an X-Track response header
sent, and otherwise will notice no difference.

we can now force global jobs into catalogs, e.g. for maintenance jobs

Renamed catalog_init SpecialSub to request_init to accommodate for a hook
called after catalog configuration (#160).

Job server displays catalog and job name in $0.

Force email notification of errors, can be prevented with ignore_errors
setting.

    * Created a new CountrySubdomains directive.  A predefined block
      of country subdomains is listed in a new subdomains.cfg file, which
      should be "included" into the interchange.cfg file.

      This method is faster and easier to maintain than my previous effort.

      Note: The subdomains.cfg file is incomplete; Several countries either
      have an incomplete subdomain list, or are not represented at all.
      This situation will correct itself over time.

    * Fixed a bug in the new DomainTail regex.

    * More DomainTail changes:

	-- Correction: .ae -> .af
	-- Additions: .ar and .co

    * Added some more countries to the new DomainTail list.  That's enough
      for now, I think.

    * Altered the DomainTail directive to take subdomains of various ccTLDs
      into account.  For example, if "foo.example.co.uk" pays a visit then
      "example.co.uk" will be used as the domain tail instead of just
      "co.uk".

      The list is not exhaustive;  It's based upon a list taken from the
      www.interchange.rtfm.info logs, with extra subdomains filled in
      as I saw fit.  My favourite ccTLD subdomain is "geek.nz". :-)

      Missing country-specific subdomains can be filled in as they are
      spotted, with the old behaviour (one level) remaining as the
      default.

New Vend::Server::set_process_name sub which is used to change the status of
the process name indicator.  This respects the MV_DOLLAR_ZERO settings.

Applied patch from Mark Johnson <mark@endpoint.com> to fix problem with RPC
mode preforking too many children on server startup due to race condition.

Allow parameters passed to jobs, acknowledges --email commandline option
now (#103).

* Add SpecialSub configurability for admin_init. Added at point just
  after initialization of embedded perl, so catalog subroutines can
  handle it and they will be assured of having a proper setup of
  @{$Session}{qw/ superuser admin username groups /}.

* This allows you to simplify access based on groups or usernames, by
  changing configuration. For instance, consider this:

SpecialSub  admin_init  admin_functions
Sub admin_functions <<EOR
sub {
    if(! $Session->{username} eq 'foundation') {
        $Variable->{MV_MENU_DIRECTORY} = 'include/foundation/menus';
    }
}
EOR
u

    * Typo in the last commit.  I used if_yes() instead of is_yes() in
      one of the changes.  This was tested on another server, and then the
      changes copied manually before committing, instead of copying the
      entire file over.  Oops.

    * The "add_session" and "use_global" Jobs parameters now accept
      Yes/No and True/False as well as simply 1 and 0.

If you have a lockout specialsub that returns a true value on a session
per IP lockout the lockout will still occur and not be overridden by the
return value of the specialsub. This is because there is a separate
lockout for session per IP triggers that is done outside of the
do_lockout sub and it does not know about the lockout specialsub or its
return value.

The attached patch passes the return value for the lockout specialsub
back as the return value for do_lockout and then uses the do_lockout
return value to determine whether to perform the secondary lockout for
sessions per IP.

This patch makes the following changes to sessions per IP limit protection:

* There are now seperate settings for the amount of time allowed to trigger a
	lockout and the amount of time a lockout lasts for.

* Lockouts are now triggered based on new sessions per time limit, which is
	different from the old critera of new sessions between pauses of length
	time.  This means that if RobotLimit is set to 100 and the other
	settings were left at thier defaults then a 24 hour lockout would be
	triggered if a given IP address had 100 new sessions in any given 60
	minute time period.

Settings used by this patch:

RobotLimit: Used to determine the number of new sessions required to trigger a
	lockout.  Default is 0 which disables this feature alltogether.

Limit robot_expire: Used to determine the amount of time a lockout will last
	in days once triggered.  Can be less than 1, for example 0.04 is
	slightly less than an hour.  Default is 1.

Limit ip_session_expire: Used to determine the length of time in minutes for
	RobotLimit sessions to build up in the counter file and trigger a
	lockout.  Default is 60 (1 hour).  This can also be set to fractional
	numbers, for example 0.5 will allow 30 seconds.


Also make note of the following:

* When first implementing you should delete all the old counters with:
	rm -rf catroot/tmp/addr_ctr/*
...be careful with the above command, if mistyped it can seriously mess up
your filesystem.

* Shell command to view the contents of one of the binary new counter files:

	perl -e 'binmode STDIN;' -e '$/=undef;' -e '$_ = <STDIN>;' \
	-e 's/(.{4})/localtime(unpack("N",$1))."\n"/seg;' \
	-e 'print;' < 0_0_0_0

...where 0_0_0_0 is the filename of the binary counter.  The command will
show you a list of timestamps in human readable form.

Switch to 301 redirect, which may be better, or may not matter ...

Add new BounceReferrals catalog directive, a boolean option.

When disabled (the default), there is no behavioral change.

When enabled, GET requests to URLs with mv_pc or mv_source set to an
affiliate code are redirected to the same URL minus the affiliate code.

This keeps search engines that respect redirects from storing the affiliate
code-salted URL in their indexes, and helps them focus on the real resource
with a single URL instead of a multitude of salted links.

It has the side-effect of causing regular users following an affiliate
link to land on a URL with no affiliate code but instead with a session
ID in it, if they didn't have a session cookie already. There's no
easy way around this, and I consider it a separate issue from the
BounceReferrals concept that either happens one the first page with
BounceReferrals, or the second page without it. Either way, if session
IDs in URLs are a concern, it'll need a separate solution.

(Historically, many application servers always bounce the first request
to check for cookie support. Nowadays, many simply require cookies for
anything that needs a session. Interchange is different on both counts.)

Add new catalog directive UserTrack, defaulting to Yes for backward
compatibility. New No option prevents sending the X-Track HTTP response
header.

New Free Software Foundation Address in headers of various files

New Free Software Foundation Address in headers of various files

Work around apparent Perl bug that allowed code called by DispatchRoutines
to overwrite the routines arrays themselves.

Found and fixed by Frederic Steinfels <fredo@dvdupgrades.ch>. Backported
from trunk lib/Vend/Dispatch.pm version 1.63.

Minor XHTML compatibility changes.

Add a space before XHTML self-closing /, as per some spec somewhere. :)

log error before removing flag to preserve error text

enclose job(s) execution in an eval to catch any fatal errors

Fix behavior of timed-build for logged-in users, as discussed in this
interchange-users thread:

http://www.icdevgroup.org/pipermail/interchange-users/2006-July/045795.html

* Add hook for catalog_init subroutine that gives you access before the
  session assignment is done. In this case, it was used to modify the
  name of session files so you can assign a session number based on the
  HTTP_SERVER_HOST coming in -- but it could be used for a number of
  purposes.

  Uses the usual SpecialSub method:

  	Sub my_init sub { $CGI::user = $CGI::server_host }
	SpecialSub catalog_init my_init

Avoid cluttering global log file with job run messages.

Added process ID to "Run jobs" and "Finished jobs" log messages for better
troubleshooting.

* Add lockout to list of SpecialSub routines allowed.

* If user-configured lockout routine returns true, it replaces the
  current routine completely. If it returns false, the normal one
  is run as well.

* Move the logging out of Vend::Dispatch to the do_lockout routine, so
  that you can avoid the log entry if your user-configured routine handles
  the lockout.

* Make the number of seconds for robot reset adjustable from its
  current hardcoded 30:

  	Limit   lockout_reset_seconds

  Maybe that should be robot_reset_seconds, I don't know.

* Change use of $Vend::Cfg->{Limit}{member} to $::Limit->{member}. As
  Limit is used in iterative routines like chain_cost, this should
  improve performance.

Move page extension assignment back to regex from which it's drawn.
Should not affect behavior.

log message about empty job at catalog level

    * If cookies are switched off in the user's browser and the user
      is just switching from a HTTP page to a HTTPS page then there's
      no reason to re-initialise the session.

Fix bug that allowed code called by DispatchRoutines or CleanupRoutines
to overwrite the routines arrays themselves.

Found and fixed by Frederic Steinfels <fredo@dvdupgrades.ch>.

Remove unused lexical variables.

* Change AutoEnd to use same methodology as Autoload for cleanup routines.

* Add ability to add Cleanup_code just like we add Dispatch_code now, allows
  extensions to do things at end of a page session.

* Safe enough to commit to stable at next point release (or sooner if people
  want to unbend).

* Make robot errors going to the log less verbose, and make it possible to route
  them to a separate log with:

	ErrorDestination  <<EOF
	'WARNING: POSSIBLE BAD ROBOT. %s accesses with no 30 second pause.' logs/robots.log
	'Too many IDs, %d hour wait enforced.'   logs/robots.log
	EOF

* Allow checkboxes for mv_order_* and other options in a "stacked" array.
  Mechanism is to name the variable with mv[0-9][0-9]?_*, which is changed
  into an array of values.

  <input type=hidden name="mv1_order_item" value="os28004">
  <input type=hidden name="mv3_order_item" value="os28008">
  <input type=hidden name="mv9_order_item" value="os29000">

  Is the equivalent of:

  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="os28004">
  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="os28008">
  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="">
  <input type=hidden name="mv_order_item" value="os29000">

* One potential problem is that the numbering starts from zero while [loop-
  increment] starts from one.

trackdb option added to keep track of your jobs

run autoend macro at the end of each job to complement autoload like for
ordinary page requests

- XHTML-compliance:

  - lowercased some HTML tags (all containers, so no ">" vs. "/>" issues)

  - fixed an omission (the "/" in ending tag </b> was missing):
     <b>Authorization Required</b>
     <b>Not Found</b>

* Add new "Feature" facility which allows easy installation of new
  capabilities to Interchange.

* We have the convention of "extensions" which allows us to put together
  features to add to Interchange. But the installation is manual, and
  requires good docs to make it easily installable for end-users. Also,
  many features require access to the global configuration.

* We also have the problem of feature creep, where we tend to add everything
  to the "standard" catalog.

* New Feature directive is simple in execution:

	Feature quickpoll

* There is also a global FeatureDir directive, with a default of
  "features". Features are placed in a subdirectory based on
  the feature name -- I am including a "quickpoll" feature in
  this.

* Basic mechanism is pretty simple. There are two special kinds
  of files called by the extensions .global and .init.

  (In the included "quickpoll" feature, these are named quickpoll.global
  and quickpoll.init)

  If a file has a .global extension, it is added to the global
  configuration. The included quickpoll feature adds the ActionMap
  quickpoll, and the UserTags [poll-answer] and [ascii-graph].

  If a file has a .init extension, it is run once the first time
  the target catalog is accessed. In the example, it is used to add
  mv_metadata entries and a couple of sample polls.

  All other files in the directory are catalog configuration,
  in this example "quickpoll.catalog.cfg". It could have been
  broken up into "quickpoll.sql" and "quickpoll_answer.sql".

  All subdirectories contain files which are copied to the
  catalog directory with the same relative path. In this
  case, ICDIR/features/quickpoll/templates/components/quickpoll
  goes to CATDIR/templates/components/quickpoll.

* The .init file, when run, sends its output to

	 ConfDir/init/<feature>/<feature>.init

  (etc/init/quickpoll/quickpoll.init in the example.)

  Once it is run, the existence of the file prevents it being run again.
  When the server is restarted, the file test is no longer needed.

* You can add documentation simply by creating a doc/<feature>/README file
  or anything else you want.

* TODO: Uninstall procedure.

* Add new cron-style facility for determining HouseKeeping jobs.

* Default is no change, i.e. no cron.

* The recommended method to add the file is:

	HouseKeepingCron  <crontab

  That will use the file etc/lib/crontab by default in the tarball,
  or /etc/interchangec/crontab in an LSB configuration.

* Requires the Set::Crontab module, which has been added to
  Bundle::Interchange.

* Structure of the crontab file is just like crontab(5) in UNIX
  except that a seconds column is added.

  The targets are GlobalSub or anything which you can make run
  with Vend::Dispatch::run_macro. Bear in mind there is no
  catalog context.

  Two special targets exist, :reconfig and :jobs. They allow calling
  of the catalog reconfig routines and jobs routines, respectively.
  The etc/reconfig and etc/jobsqueue files will be ignored if these
  targets are not present -- a warning will be issued at startup
  (and crontab change) if they are not there.

  A target prepended with > runs *after* the reconfig/restart/jobs/pid
  mgmt cycle. Normal specifications run before.

  The basic entry to implement "HouseKeeping 5" would be:

     HouseKeeping 1
	 HouseKeepingCron <<EOC
	 */5 * * * * * :restart
	 */5 * * * * * :jobs
	 EOC

  (Note that would normally be in etc/lib/crontab or /etc/interchange/crontab.)

  To only check the jobs queue every five minutes (on the minute), you
  do:

	 */5 * * * * * :restart
	 0 */5 * * * * :jobs

  If you want to run the GlobalSub "checkit" once a day at 4am, you would
  do:

	 0 0 4 * * * checkit

* If you set HouseKeeping to a granularity besides 1 (or if for some
  reason Interchange skips a second), it does the cron check for
  every intervening second. This ensures a job will not be skipped.
  The :restart and :jobs entries will only run once, but if you have
  a frequent GlobalSub job that pushes the granularity of HouseKeeping
  it can be run twice in succession.

* WARNING: You should not put long-running jobs in a GlobalSub! You have
  been warned. Use the Jobs facility for that.

* Probably should implement the ability to call out jobs, but not quite
  sure how to specify and do. Can we just call run_jobs() directly?
  If so, then maybe an = sign introduces a job:

  	0 0 * * * * =standard_cat hourly
  	0 0 4 * * * =standard_cat daily
  	0 0 2 * * 7 =standard_cat weekly

* Include bin/crontab script to edit the crontab and submit to the
  running IC daemon. BUG: Cannot run as root.

* Implement new AccumulateCode and TagRepository directives. The rationale
  is:

    -- There is a huge base of Interchange code, much of which is not
       needed in even the standard catalog with full UI. This causes a
       larger memory profile than necessary.

    -- It is difficult to determine from the page code what code is
       needed, especially when a [tag] can call a $Tag can call
       a filter can call some sort of Action.

    -- A feature is needed to allow building catalogs with a more
       nearly optimal set of code than just "everything".

  If AccumulateCode is no, operation is exactly as before. There have
  been some code initialization changes and routine calling changes,
  but the data structures are identical and no difference in operation
  should be seen.

  If you set AccumulateCode to "Yes" and specify a TagRepository that
  contains all known UserTag, ActionMap, Filter, Widget, etc. etc.
  code, Interchange starts accumulating and compiling these as
  needed.

  The code is sent to the master process for compilation and
  incorporation, so that the next iteration of a page after HouseKeeping
  seconds will find the code already compiled and ready to go.

  It also copies the code file to the "code" (actually $Global::TagDir)
  directory in the "Accumulated" subdirectory tree. When you restart
  Interchange, these tags/filters/widgets/checks are read normally
  and need not be recompiled on the fly.

  Over time, as you access pages and routines, a full set of tags
  will be developed and you can turn AccumulateCode to "No".

* There can be failures due to calling a $Tag from within embedded
  Perl for the first time, particularly when it uses a MapRoutine or
  calls another $Tag within. This is due to Safe, and there is probably
  not much to be done about it. The good news is that the error should
  go away after HouseKeeping seconds when the tag gets compiled by the
  master.

  This could be avoided in the case of an AllowGlobal catalog, and it
  might be possible to make a directive that turns on AllowGlobal only
  when in AccumulateCode mode.

  The area, tmp, tmpn, and image tags are known to fail in this
  way in the standard catalog. Tags that are frequently called
  in this fashion should probably be placed in a "code/Vital"
  directory and not be accumulated.

* This is only recommended for development -- it might
  be possible to remove a tag/filter/etc. from the master
  and recompile these on the fly, but I haven't looked at that
  yet.

  Another nice feature is that you can easily add a tag simply
  by adding its code to the TagRepository and having it
  compiled.

* WARNING: Nice features are often dangerous! Don't run this in
  production -- you have been warned!

* WARNING: OrderCheck is not yet implemented, and a full audit has
  not been done on all compiled code directives.

* WARNING: Not fully tested in Prefork mode, and really not intended for
  that mode.

* WARNING: Including multiple tags in a file may have unpredictable
  behavior. You should try to keep related Alias and tag things in
  the same file.

* This feature only applies to Global code -- Catalog-based code
  shows no change.

* Passes the regression tests 100% when called with an empty "code"
  directory, compiling every tested tag and executing without error.

* We had been running without warnings so long, we had gotten a bit sloppy
  in our code in some places.

  Because we do lots of uninitialized hash member stuff with arrays
  of fields and $opt, it is basically unreasonable for us to run with
  the "use of uninitialized variable" warnings. Because we do a fair
  amount of numerical comparisons on empty string values, it seemed
  unreasonable to run with "string value in numeric gt" enabled.

  Also, because Safe, HTML::Entities, and some other modules have
  these checks enabled, page code is bound to have warnings issued
  no matter what you do.

  It seems better to run with some warnings enabled so that we can
  catch bad code practice.

  So I have made the code run without warnings prior to and after
  interpolate_html(), and turned off warnings by default in the
  page code.

  If it is desirable to turn on warnings for pages, you can use
  [pragma perl_warnings_in_page] and see warnings for that page
  (or by default if you put in catalog.cfg).

* In general these changes are:

	 no warnings qw(uninitialized numeric);

  in most pages.

* Add $::Pragma->{perl_warnings_in_page}.

* Remove defined checks when simple set tests will do.

* Streamline some code.

* Various variable initializations and tests to avoid warnings.

* Add ability to specify multiple Autoload and AutoEnd routines
  with separate lines.

  Prior to this, if you wanted to run multiple autoload routines
  you had to specify them all on one line, and you couldn't mix
  ITL code in.

  Now you can (for both Autoload and AutoEnd) specify one routine at
  the top of the configuration file, then another on a later line.

  This *could* have the effect of changing catalog operation if
  someone currently has multiple lines -- in previous versions this
  would cause the last line to take precedence.

* Add ability for multiple AutoEnd routines by having it use
  run_macro() as it *should* have before.

* Make the start of XHTML support for Interchange.

* Add global XHTML and catalog XHTML options. Global is false/off by
  default, and catalog option is defaulted from global option.

* The only effect is that Dispatch.pm sets $Vend::Xtrailer to the
  appropriate value of '/' or ''.

* Use $Vend::Xtrailer to set any <br/> or <br> we use.

* Change some generated HTML to:

	-- lowercase tags
	-- quote parms
	-- use $Vend::Xtrailer

* Supply [br] tag which returns the proper <br/> or <br>.

* Add new DispatchRoutines code.

* Make the code for each DispatchRoutine in one common place, so that
  when multiple catalogs are using the code we don't have many copies
  of the identical subroutine.

* Make %Defaults, %Dispatch_priority, %Dispatch_code non-local variables
  so that AddDirective has full access to them.

* Added DispatchRoutines for:

	CookieLogin     first so Locale can be set as part of login
	Locale          next to account for settings for DiscountSpaces and Autoload
	DiscountSpaces  next
	Autoload        last so that catalog has complete environment set

* DiscountSpaces code works with latests tests.asc 000156 -- Ethan
  may wish to tweak.

* To add a new DispatchRoutine you just:

	-- Put in $Vend::Config::Defaults sub for set_defaults, with
	   line somewhere:

	   		push @Dispatches, 'YourDirective';

	-- $Vend::Config::Dispatch_code{YourDirecive} = $sub;
	-- $Vend::Config::Dispatch_priority{YourDirecive} = $n;

New discount-space functionality by Ethan Rowe <ethan@endpoint.com>.
His notes follow.

Implements discount namespaces, in a manner consistent with the
values-space functionality. The [discount-space] usertag can be used
to change the current discount namespace, causing all subsequent
discount-related tags and calculations to operate from that particular
namespace. Usage:

[discount-space name=<space>]
Changes the current discount namespace to <space>.

[discount-space current=1]
Returns the name of the current namespace.

[discount-space name=<space> clear=1]
Clears all discount information from the specified namespace.

The discount space is initialized, per page process, in a manner
similar to values-space; if a CGI value exists under the variable name
"mv_discount_space", the discount space will be initialized to that
value. Furthermore, a catalog-level variable MV_DISCOUNT_SPACE can be
used to specify the name of an additional CGI variable to check in this
manner. When MV_DISCOUNT_SPACE is defined, it has higher precedence than
mv_discount_space for determining the initial namespace.

The discount-space logic is designed to integrate well with the cart. The
default discount spacename is "main". Setting the catalog variable
MV_DISCOUNT_SPACE to 'mv_cartname' allows the current discount namespace
to match the current cart when the current cart is set via CGI variables.

The discount namespaces are stored in the session at
$Vend::Session->{discount_space}{<spacename>}. Note that, for backwards
compatibility, the default space is always at $Vend::Session->{discount},
and that this will always serve as the master for he "main"
namespace such that $Vend::Session->{discount_space}{main}
= $Vend::Session->{discount}. Reassigning the hashref
for $Vend::Session->{discount_space}{<spacename>}
or $Vend::Session->{discount} can take Interchange internal
variables out of sync, and is thus discouraged (just as performing
a $Session->{scratch} = {} mid-page could cause odd behavior. Use
[discount-space name=<space> clear=1] instead, or assign to the
dereferenced hash (%{$Vend::Session->{discount}} = ()).

For further support, a discount_space parameter has been added to each
of the following tags; these tags all can be affected by discounts,
either in the calculation of a discount, a taxable sum, or an item
price (passing a discount namespace to these tags via the discount_space
parameter will only affect the current discount space within the context
of the tag itself):

[discount]
[fly-tax] (Also added named cart support to this tag in the process)
[item-list] (namespace specified will be used throughout the tag and its subtags)
[salestax]
[subtotal]
[total_cost]

EXAMPLES:

Set the current discount space to the default (which is 'main')
[discount-space]
OR
[discount-space main]

Set the current discount space to space "alt_discount"
[discount name=alt_discount]

Clear discount namespace "garbage"
[discount-space name=garbage clear=1]

Show the current discount namespace
[discount-space current=1]

Use the discount namespace "other" for this loop through the cart
[item-list discount_space=other]
...
[/item-list]

	Variable MV_ROBOT_EXPIRE is obsolete.
	All tests are done based on $Vend::Cfg->{Limit}{robot_expire}.
	So, bring this in sync with routine count_ip located in Session.pm

Correct name of unknown sub in error message.

Simplify logic as per Paul Vinciguerra's observation.

Removed unneeded code.

	* Fixed a namespace error in one of the calls to
	  get_locale_message().  Reported by Carl Bailey.

* Correct bogus call to get_locale_message.

Use Tie::ShadowHash instead of copying %$Variable because it's much
faster for larger strings in Variables. Keep copying %$Pragma because it's
faster to copy than shadow when it's small.

Only protect %$Variable and %$Pragma when in PreFork or mod_perl mode;
it's unnecessary overhead in fork-on-demand mode. (Thanks, Mike.)

Fix IMO a major problem with thread-safety in PreFork or mod_perl modes:
Any change to %$Variable persists between page views for the life of that
child process, because the hash access happens outside the control of
Tie::ShadowHash (which controls $Vend::Cfg).

Apparently this was already noticed and fixed for %$Pragma, so if there's
a reason this shouldn't be done with %$Variable, we'll have to figure out
some other fix, because the current behavior is very dangerous.

* Merge ReadOnlyCfg security patch from trunk.

* Remove security hole where by a non-admin user with write permission
  to files for ITL could elevate their login status to admin.

	logout=[userdb logout]
	[calc]
		$Config->{AdminUserDB}{default} = 1;
	[/calc]

	login=[userdb function=login username=mike password=pass]

  This would cause setting of $Vend::admin.

* Create a new %Global::ReadOnlyCfg hash with the pristine
  values from the initial configuration. At catalog configuration
  time, the values from AdminUserDB and UserDB_repository are
  copied over.

* The UserDB login function now references the read-only config
  to determine admin status.

* TODO: Make all UserDB.pm functions reference this read-only config.

* Not from an exploit, from a code read.

* TODO: Determine if 5.0 and/or 5.2 should be patched.

Flesh out alternate values spaces functionality Mike added on 2002-11-18.

In addition to his mv_values_space CGI setting, there is now a
[values-space] tag for setting the values space for the lifetime of the
current page (its various options are documented in the tag itself), and
[value] and [value-extended] both accept the values_space option to pull
a value out of an alternate space without changing the current one. The
name of the current values space is now stored in $Vend::ValuesSpace,
which is used when optionally copying values from the old to the new
values space in [values-space].

Tests included.

Also added enable_itl option to [value] to make it feature compatible with
[value-extended].

Add handling for new special CGI parameter mv_force_session, a boolean.
When set, it reverses the precedence of cookie over CGI (the default
and historically the only option) to CGI over cookie when determining
which session ID to use.

This is useful when sharing sessions between catalogs on different
domains but the user has already got a valid cookie for a session
on the domain you're sending them to and you want to override it with
mv_session_id.

* Add PostURL, SecurePostURL, and ProcessPage directives. This allows
  a different path for GET and POST requests (presuming you only post
  with the [process] tag for the <form> action).

  This allows Interchange to handle internal redirects and DirectoryIndex
  requests via Apache transparently via reading the REDIRECT_* environment.
  It should also allow internal redirects via other modules such as mod_perl.

  The default for the [process ..] tag is the same as now, to use VendURL
  or SecureVendURL as the default URL.

  If you set up in Apache:

  	DirectoryIndex index.html /cgi-bin/foundation
	<LocationMatch *\.html>
		ErrorDocument 404 /cgi-bin/foundation
	</LocationMatch>

  In interchange.cfg:

  	AcceptRedirect Yes

  In catalog.cfg:

  	VendURL        http://www.foo.com/
  	SecureURL      https://www.foo.com/
  	PostURL        http://www.foo.com/cgi-bin/foundation
  	SecurePostURL  https://www.foo.com/cgi-bin/foundation

	ImageDir
	DirectoryIndex  index.html
	DeliverImage    yes

  then a properly set up catalog (using [process href="[whatever]"]
  on any form action that is a POST) will look just like a static
  HTML site, and will deliver relative images and simple links
  properly for a GET.

  There should be no difference to normal operation of Interchange if
  these changes are not made.

* Delivering images properly for a POST is a different story. There
  needs to be some thought on this -- it could be that:

	<head>
	[calc]
		my $method = $Tag->env('REQUEST_METHOD');
		return unless $method =~ /post/i;
		my $path = '@@MV_PAGE@@';
		return unless $path =~ s,(.*)/.*,$1,;
		return qq{<BASE HREF="$path">};
	[/calc]
	</head>

  will handle most anything.

* Improve handling of

	 [page href="http://www.foo.com/bar.html"
	 	   form="
		   		buz=baz
		   "]

   It will now deliver the link you would expect:

   		http://www.foo.com/bar.html?buz=baz

* Add Status: and Content-Type: headers if we are the recipient of
  an internal redirect.

* Remove references and tests on $Vend::InternalHTTP and
  $Vend::OnlyInternalHTTP, which are no longer wanted with the
  removal of the internal HTTP server.

* When no PATH_INFO is specified, normally we go to find_special_page('catalog').

  This change checks the REQUEST_URI when that condition occurs, and if
  the REQUEST_URI doesn't begin with SCRIPT_PATH we assume the web server
  has used the Interchange SCRIPT_PATH as the index entry in DirectoryIndex.

  This allows in (at least Apache's) httpd.conf:

	DirectoryIndex  index.html /cgi-bin/foundation

  When the index.html page is not found, /cgi-bin/foundation is called.
  If the URI is a subdirectory as in the request /foo/, then the REQUEST_URI
  will be /foo/. We then use /foo/ as the Interchange path, allowing
  transparent flowthrough of non-existent entries to Interchange.

  In other words, you can create an empty directory /var/www/html/foo,
  and when /foo/ comes in as a request it will automatically go to
  /cgi-bin/foundation/foo/ while still appearing to be /foo/ on the
  browser.

  If you combine this with the following in catalog.cfg:

  	DirectoryIndex  index.html
	DeliverImage    yes

  And the following in interchange.cfg:

  	AcceptRedirect Yes

  And finally:

  	<LocationMatch "^/(.*)/.*\.html">
		ErrorDocument 404 /cgi-bin/foundation
	</LocationMatch>
  you can run a complete set of

* Merge login scrub security fix from devel.

* Last security fix for login scrub was in wrong place -- should be
  in the init_session() routine.

* Back out change to Dispatch.pm and properly do it in Session.pm.

* Security fix, merge to stable.

* Merge security fix from head.

* Login information was getting re-saved on a session cancel.

* Security fix, backport.

* Allow individual profile checks defined in metadata.

  It is defined in the "Help and Check" section of meta_editor.

  IMPORTANT: To enable these checks, the ui_profile part of the overall
  table definition has to be non-blank. You can just put a comment like

    # Enable individual form profile checks

  To enforce a profile on a field, set it's check value to a valid
  IC profile check. For example, to make sure the length of a field
  is between 4 and 10 characters, do:

  	  length 4-10

  Or to ensure it looks like a US or Canada postal code

  	  postcode

  This results in the following being set in the extended field
  of mv_metadata:

  	{ check => 'postcode' }

  If the check has an "=" sign anywhere, the field name is not
  prepended. So to do multiple checks, you need to do

  SECURITY:

  Since the filter or lookup check type can do data lookups and potentially
  call routines, we need to have some security. The table-editor tag
  sets the scratch value "mv_individual_profile" when a ui_profile
  is defined.

  WITHOUT TABLE EDITOR:

  To use this without the table editor, just set an mv_profile as
  normal. Then for each individual check, define in the form:

  	<input type=text name=foo value="">
  	<input type=hidden name=mv_individual_profile value="foo=length 1">

  That would check the form variable "foo" for a length of 1.

  This is automatically done by the [display ...] tag, so you can
  do:

  	[display type=text name=foo check="length 1"]

  to do the same thing.

  You will need to set the scratch value mv_individual_profile yourself
  to enable these.

  POSITIONING WITHIN PROFILE:

  The facility works by modifying the first mv_form_profile run and
  inserts its values either just before the first &fatal=yes or
  *after* the first &update=yes, whichever comes first. If neither
  are in the profile, they are put first in the profile.

  MULTIPLE CHECKS:

  If you have multiple mv_form_profile settings, the individual checks
  are only run as a part of the first one and are not repeated.

* Add blank_default metadata setting, which is the value a field should
  be initialized to (on display) if the current value has a length of 0.

* Remove extended.js_check from meta editor, as that was never implemented.

* Add some labels and help to meta editor.

* Allow standard handler for PUT operations. To enable, do:

	SpecialPage  put_handler  some_action

  The some_action action (could be a page) will be prepended to
  any path sent with the PUT.

* Merge security fix from devel. (@Global::HideCGI)

* Define a set of CGI keys that we don't want to save to disk, as
  @Global::HideCGI.

* Allow [dump no-cgi=1 no-session=1 no-env=1] to finetune dump.

* Don't show sensitive (i.e. @Global::HideCGI) CGI variables in a dump.
  This allows saving a session to disk for diagnositic purposes in case
  of order failure.

* This will be backpatched.

* Add [traffic-report] tag and modified admin/reports/traffic/ByAffiliate
  page which calls it. Now reports on large files without crashing the
  system. Probably can handle up to 500MB files with on any kind
  of a reasonable server.

* Don't track admin pages.

* Implement Paul Vinciguerra's suggestion to not save mv_password in
  the History hash.

  You can also set up a different NoHistory set via

  	@Vend::Dispatch::NoHistory = qw/ foo bar mv_credit_card_number /;

* Remove conditional for caching history on Pragma: no-cache -- this is
  all wrong. The directive is to this page transaction of Interchange,
  i.e. timed-build and such. It should not be used to prevent building
  of history, partly because browsers may send it for their own reason.

* [history-scan] -- there should never be a leading / on page names sent
  to $Tag->area(), and they are removed all the time now.

* Prevent autovivification of $CGI::values{mv_username}, and make CookieLogin
  slightly more efficient when already logged in.

* Add some code to prepare for new site-building feature used in combination
  with AcceptRedirect. Will be committing Config.pm and Server.pm code that
  works with this.

* Add DeliverImage directive that enables fast IC delivery of
  images requested from it.

  To enable, do in catalog.cfg:

  	DeliverImage  Yes

  If the file extension is present and the MimeType for that extension
  begins with "image/", the path will be adjusted to add ImageDir
  or ImageDirSecure, and a 302 issued.

  This happens before database or session opens, and is quite fast.

  Sets $Vend::tmp_session so no cookie is issued.

* TODO: Add processing routine or option to deliver different path
  from ImageSecure, possibly for at DBI-based image CGI.

filter option added for Job output

Jobs: Set $CGI->{mv_tmp_session} to signal that a "robot" is in charge.
capture_page: Call substitute_image and add expiry parameter.

Call tracking functions only if Vend::Track object exists.
E.g. in jobs tracking isn't enabled.
Disabling tracking with a configuration option might follow
as well.

* Send Content-Size header with downloads.