[ic] Really strange [calc] evaluation
Angus Rogerson
arogerso at admmail.uwaterloo.ca
Wed Jan 27 04:36:43 UTC 2010
On 26-Jan-10, at 9:44 PM, Grant wrote:
>>> The following should evaluate to zero:
>>>
>>> [calc]197.7 - (32.95 * 6)[/calc]
>>>
>>> but instead I get:
>>>
>>> -2.8421709430404e-14
>>>
>>> Everything else is functioning normally. Does anyone know what's
>>> going on here? I'm on 5.6.1.
>>
>> Standard floating point arithmetic. Same thing occurs in every
>> programming
>> language that uses FP. Use rounding (printf) or CPAN modules to
>> handle it.
>
> OK, I'd like to change the code so that it produces the correct
> result. I tried:
>
> [calc]sprintf("%.0f",(197.7 - (32.95 * 6)))[/calc]
>
> but I get -0. This situation seems pretty strange to me. How can I
> get [calc] to calculate reliably?
>
> - Grant
>
Assuming 197.7 is $197.70, how about converting to integers and
specifying everything as cents, then convert it back to dollars.cents
at the end.
[calc]sprintf("%.2f", (int (197.70 * 100) - (int(32.95*100) * 6) )/100
)[/calc]
Hmm. Is there something in sprintf that you can tell it to print an
integer like 3295 as 32.95? That would reduce the risk of the /100
causing floating point problems. Or is that pick or cobol or something
else buried deep in the back of my brain?
Angus
Angus Rogerson
Retail Services, University of Waterloo
More information about the interchange-users
mailing list