[ic] ic-utf8 readfile/writefile patch

Mike Heins mike at perusion.com
Sat Mar 21 19:44:49 UTC 2009

Quoting David Christensen (david at endpoint.com):
> On Mar 21, 2009, at 7:33 AM, Stefan Hornburg (Racke) wrote:
> > One problem that might be related to this issue is the delivery of  
> > "binary"
> > content stored in a UTF8 database.
> >
> > Currently the files produced are corrupted, and the data in the db is
> > definitely correct. And it works with UTF8 inactive (as per bug #259).
> >
> > The code is as follows:
> >
> > my $data = $Db{transaction_documents}->field($td_code, 'content');
> > $data = $Tag->filter({op => 'decode_base64', body => $data});
> > $Tag->deliver({type => 'application/pdf', 						body => $data});
> >
> > Decode_base64 is a simple filter (content is stored base64 encoded):
> >
> > CodeDef decode_base64 Filter
> > CodeDef decode_base64 Routine <<EOR
> >
> > use MIME::Base64;
> > sub {
> > 	return MIME::Base64::decode_base64(shift);
> > }
> > EOR
> Without going into it yet, I'm going to guess that this has more to do  
> with the [deliver] tag than the database itself.

What does that have to do with it? There is no encoding or decoding
in the deliver tag, at least as I wrote it. It just takes what it is given
and spits it out to Vend::Dispatch::response. That routine accounts for
the output mode and then sends it to Vend::Server::respond.

It might set the content type, but again that is available to the
response routines which are much better positioned to know what
is going on overall. Introducing more Encode into more places isn't
the answer. 

> What I think is  
> happening is that the 8-bit character data returned from MIME::Base64  
> encode function is being reencoded to utf8 on output, something which  
> is only supposed to be happening if the content is text.

Output of what?

> I think the delivery tag should be special-cased here, or at least
> have the content-type examined in order to see if we are returning raw
> data or text; aka "text/*" -> output as utf8, anything else -> goes
> out raw. I thought the logic was in place for this already, so I will
> need to revisit; thanks for the detailed scenario.

It isn't the deliver tag as I see it. I am not sure we even know what
we are doing here.

Mike Heins
Perusion -- Expert Interchange Consulting    http://www.perusion.com/
phone +1.765.328.4479  <mike at perusion.com>

How far can you open your mind before your brains fall out?

More information about the interchange-users mailing list