[ic] Really strange [calc] evaluation
Grant
emailgrant at gmail.com
Wed Jan 27 18:12:24 UTC 2010
>>>> The following should evaluate to zero:
>>>>
>>>> [calc]197.7 - (32.95 * 6)[/calc]
>>>>
>>>> but instead I get:
>>>>
>>>> -2.8421709430404e-14
>>>>
>>>> Everything else is functioning normally. Does anyone know what's
>>>> going on here? I'm on 5.6.1.
>>>
>>> Standard floating point arithmetic. Same thing occurs in every
>>> programming
>>> language that uses FP. Use rounding (printf) or CPAN modules to handle
>>> it.
>>
>> OK, I'd like to change the code so that it produces the correct
>> result. I tried:
>>
>> [calc]sprintf("%.0f",(197.7 - (32.95 * 6)))[/calc]
>>
>> but I get -0. This situation seems pretty strange to me. How can I
>> get [calc] to calculate reliably?
>>
>> - Grant
>>
>
> Assuming 197.7 is $197.70, how about converting to integers and specifying
> everything as cents, then convert it back to dollars.cents at the end.
>
> [calc]sprintf("%.2f", (int (197.70 * 100) - (int(32.95*100) * 6) )/100
> )[/calc]
>
> Hmm. Is there something in sprintf that you can tell it to print an integer
> like 3295 as 32.95? That would reduce the risk of the /100 causing floating
> point problems. Or is that pick or cobol or something else buried deep in
> the back of my brain?
>
> Angus
Can anyone tell me what it is about this calculation that is causing
the floating point problem? I use [calc] extensively and now I'm
wondering if this could pop up anywhere. Wrapping all of my
calculations in sprintf and int seems strange.
- Grant
More information about the interchange-users
mailing list