Re: Sum of doubles giving incorrect value
Hmmm... I see what you mean...
In C:
printf("%.60f\n",805.01);
printf("%.60f\n",41.99);
printf("%.60f\n",-847.00);
gives 3 values
805.009999999999990905052982270717620849609375000000000000000000
41.990000000000001989519660128280520439147949218750000000000000
-847.000000000000000000000000000000000000000000000000000000000000
The only one that is completely correct is the one with no decimal value - all others are just approximate.
So is there any way of doing more accurate floating operations using hardware? Or will I have to rely DECIMAL and 100% software?
Subject
Written By
Posted
Re: Sum of doubles giving incorrect value
January 05, 2011 10:43AM
Sorry, you can't reply to this topic. It has been closed.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion
of Oracle or any other party.