[ic] Really strange [calc] evaluation

Angus Rogerson arogerso at admmail.uwaterloo.ca
Wed Jan 27 04:36:43 UTC 2010


On 26-Jan-10, at 9:44 PM, Grant wrote:

>>> The following should evaluate to zero:
>>>
>>> [calc]197.7 - (32.95 * 6)[/calc]
>>>
>>> but instead I get:
>>>
>>> -2.8421709430404e-14
>>>
>>> Everything else is functioning normally.  Does anyone know what's
>>> going on here?  I'm on 5.6.1.
>>
>> Standard floating point arithmetic. Same thing occurs in every 
>> programming
>> language that uses FP. Use rounding (printf) or CPAN modules to 
>> handle it.
>
> OK, I'd like to change the code so that it produces the correct
> result.  I tried:
>
> [calc]sprintf("%.0f",(197.7 - (32.95 * 6)))[/calc]
>
> but I get -0.  This situation seems pretty strange to me.  How can I
> get [calc] to calculate reliably?
>
> - Grant
>

Assuming 197.7 is $197.70, how about converting to integers and 
specifying everything as cents, then convert it back to dollars.cents 
at the end.

[calc]sprintf("%.2f", (int (197.70 * 100) - (int(32.95*100) * 6) )/100 
)[/calc]

Hmm. Is there something in sprintf  that you can tell it to print an 
integer like 3295 as 32.95? That would reduce the risk of the /100 
causing floating point problems. Or is that pick or cobol or something 
else buried deep in the back of my brain?

Angus

Angus Rogerson
Retail Services, University of Waterloo



More information about the interchange-users mailing list