[ic] Corruption of products file (double tabs after SKU?); problem, temp workaround, seeking solution

TMN Interchange interchange@welcometomyuniverse.com
Mon, 30 Apr 2001 06:24:29 +1200


Using the TAB delimited file for a large number of products (over 1000).
Having problems
with corruption of the file. I will covert over to SQL at some stage, but in
the meantime
wondering if anyone else is having same problem, heres a solution (quick &
nasty), and
seeking a better solution :o)

contact: interchange@welcometomyuniverse.com




Large number of items in a TAB delimited file, having modified the construct
demo pages but
keeping the original data structure. Over time, the file is being corrupted
(products just entered
keep their SKU, but lose their item information, pricing shows up as 0,

I've already tried upgrades of "stable" Interchange. I don't want to run the
Alpha versions.

Visible cause of problem:

There seems to be a double tab after the SKU, so the information is all
there (description, pricing
etc) but since we are using TAB delimited files, suprisingly enough its not


We are entering in items normally, have already considered cut-and-paste
issues along with tabbing
across fields for entry. Typing up an entry and switching between fields
using mouse clicks still
comes up with same kind of errors.

Ugly workaround:

Running a cron job each minute that sucks in the product database, does
replacements of double
tabs after the SKU, writes it out to a temporary file, quick check to see
nothing changed in the
meantime then renames the temp file to products.txt

Source code included at the end of this email. Please, do not flame me! I
know that its Ugly, its
brute force, it doesn't do a lot of nice things that it could/should do, and
if you show it to your
friends that they will ridicule you! However, it does work, and I'm Lazy (as
a virtue)  :o)

Requested solution:

Any thoughts/hints much appreciated!!!


I am running this as a cron job each minute (as Root - Yes I am aware of

#!/usr/bin/perl -w

use File::stat;

my $now = time();
my $product_path = '/path/to/catalog/directory/product/directory/';
my @contents = `cat $product_path/products.txt`;

my $desired_username = 'interch';        # depends on your setup
my $desired_usergroup = 'interch';    # depends on your setup

foreach (@contents) {
        s/^(\w+)\t\t/$1\t/;    # this bit changes a SKU|tab|tab| with
        print FILE;

my $statinfo = stat("$product_path/products.txt");

if($statinfo->mtime < ($now - 5)) {    # not modified after our start time
plus margin....

        my(@diff) = `diff $product_path/products.txt
$product_path/products-new.txt`;    # did we even make changes?!

        if(scalar @diff && $diff[0] !~ m|^\s*$|) {    # something did
change, so lets modify...

# aargh... just spotted race condition between the diff/renaming, and a
modification to file... locking would be good idea :o(


                (undef,undef,$uid,undef) = getpwnam($desired_username);
                (undef,undef,undef,$gid) = getpwnam($desired_usergroup);
                chown $uid, $gid, "$product_path/products.txt";

                my $mode = 0644;
                chmod $mode, "$product_path/products.txt";