[ic] Really strange [calc] evaluation
arogerso at admmail.uwaterloo.ca
Wed Jan 27 04:36:43 UTC 2010
On 26-Jan-10, at 9:44 PM, Grant wrote:
>>> The following should evaluate to zero:
>>> [calc]197.7 - (32.95 * 6)[/calc]
>>> but instead I get:
>>> Everything else is functioning normally. Does anyone know what's
>>> going on here? I'm on 5.6.1.
>> Standard floating point arithmetic. Same thing occurs in every
>> language that uses FP. Use rounding (printf) or CPAN modules to
>> handle it.
> OK, I'd like to change the code so that it produces the correct
> result. I tried:
> [calc]sprintf("%.0f",(197.7 - (32.95 * 6)))[/calc]
> but I get -0. This situation seems pretty strange to me. How can I
> get [calc] to calculate reliably?
> - Grant
Assuming 197.7 is $197.70, how about converting to integers and
specifying everything as cents, then convert it back to dollars.cents
at the end.
[calc]sprintf("%.2f", (int (197.70 * 100) - (int(32.95*100) * 6) )/100
Hmm. Is there something in sprintf that you can tell it to print an
integer like 3295 as 32.95? That would reduce the risk of the /100
causing floating point problems. Or is that pick or cobol or something
else buried deep in the back of my brain?
Retail Services, University of Waterloo
More information about the interchange-users