MySQL Forums
Forum List  »  Newbie

Re: Math function logarithm
Posted by: Rick James
Date: March 17, 2011 07:59PM

Gratuitous precision. I say this snide remark because I suspect you do not need nearly all the digits you are getting (or getting inconsistently). Why does it matter?

Anyway, I will explain...

* Virtually all "floating point" arithmetic in computers for the last three decades has been based on the IEEE 754 Floating Point Standard. It calls for two representations, one has 24 bits of precision, the other has 53 bits. (They also have different ranges.) Virtually every CPU in existence today (outside museums) has arithmetic for these implemented on the CPU chip. MySQL's FLOAT and DOUBLE datatypes use those formats, and depend on the CPU to do arithmetic with them.

* LN() (and other 'transcendentals' other than SQRT) goes beyond the 754 standard. It is implemented with some complicated formulas using add/sub/mul/div. The result has only ~53 bits of "precision".

* 53 _bits_ corresponds to _about_ 16 _decimal_ places ("significant digits").

* You are displaying output in decimal instead of binary. During the conversion of LN's output from binary to decimal, the base-conversion algorithm blindly assumes the 53 bits are exact. So, it has to keep going with digits that are essentially garbage.

* MySQL's DECIMAL uses a decimal library, thereby allowing it to go well beyond 16 digits of precision. (This is the long answer to the first part of this thread.)

Back to the question... MySQL and the Calculator are doing LN differently -- one is using the 53-bit hardware; the other is using a "decimal library" that is getting more precision. (Sorry, I am not sure which is doing which. All my uses of transcendentals don't really care.)

OK, you next question is "why do they differ in only the 14th significant digit?" That is an artifact of the derivative of the function. SQRT gains a bit of precision. LN loses a lot of precision near LN(1). Etc.

Options: ReplyQuote


Subject
Written By
Posted
March 15, 2011 10:06AM
March 15, 2011 12:03PM
Re: Math function logarithm
March 17, 2011 07:59PM
March 18, 2011 02:59AM


Sorry, you can't reply to this topic. It has been closed.

Content reproduced on this site is the property of the respective copyright holders. It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.