MySQL Forums
Forum List  »  General

rounding problem/error ?
Posted by: Bruce Leister
Date: November 30, 2004 04:09AM

Hi,

I am running 4.1.7 using MyISAM tables and I have the following test table:

create table test_table (hrs decimal(7,1), rate decimal(7,2), amt decimal(15,3))

I populate it with hrs & rate values from my production table, then I calculate the amt:

update test_table set amt = hrs*rate

But when I run this query I get results, WHY ?

select * from test_table where hrs*rate != amt

this is some of the output I get from the above query, and the hrs*rate == amt ??:

hrs,rate,amt
17.2,145.6,2504.32
49.3,145.6,7178.08
47.6,145.6,6930.56
50.5,145.6,7352.8
12.7,137,1739.9
1.1,137,150.7
2.7,137,369.9

yet this query returns zero results:

select * from test_table where format(hrs*rate,8) != format(amt,8)

BUT this query returns the same resultset as the first query:

select * from test_table where format(hrs*rate,15) != format(amt,15)

Help ?????

Options: ReplyQuote


Subject
Written By
Posted
rounding problem/error ?
November 30, 2004 04:09AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.