MySQL Forums
Forum List  »  General

Re: What storage requirement has FLOAT(24) ? Doc is ambigous
Posted by: Rick James
Date: February 05, 2010 10:54AM

I would say that 24 is 4 bytes:

mysql> show variables like 'version';
+---------------+------------------+
| Variable_name | Value            |
+---------------+------------------+
| version       | 5.1.30-community |
+---------------+------------------+

mysql> create table fltn(a float, b float(23), c float(24), d float(25),
                         e double);
mysql> insert into fltn values (1e7/3,1e7/3,1e7/3,1e7/3,1e7/3);
Query OK, 1 row affected (0.06 sec)

mysql> select * from fltn;
a: 3.33333e+006
b: 3.33333e+006
c: 3.33333e+006
d: 3333333.33333333
e: 3333333.33333333

mysql> select c-3333333, d-3333333 from fltn;
+------------------+------------------+
| c-3333333        | d-3333333        |
+------------------+------------------+
|             0.25 | 0.33333333348855 |
+------------------+------------------+

Technically, the IEEE 754 "single precision" contains 24 bits of precision, encoded as 23 bits, plus a "hidden bit". There is also 8 bits for the exponent (hence a range of about 10**+/-38) and a sign bit. Total is 32 bits, or 4 bytes.

Options: ReplyQuote


Subject
Written By
Posted
Re: What storage requirement has FLOAT(24) ? Doc is ambigous
February 05, 2010 10:54AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.