确定浮点数的十进制精度

Question

I have been searching the internet for days now and can't seem to find a generally accepted formula for determining the number of decimal digits that we can store in a floating point format without truncation error. The answer I came across the most often was:

Decimal precision = log₁₀(2ⁿ), where n = the number of bits in the mantissa.

Now, this lines up with everything I've seen with the single point and double point formats discussed in my CS book.

However, when I went to check this against the examples in my CS book, the numbers didn't line up.

The examples used an 8-bit format with a 4 bit mantissa. Given the formula above, I should only be able to store a decimal value with precision of up to 1.2 digits. However, according to the book (and confirmed by trying other decimal values) I can store a value such as 2.25 in a 4 bit mantissa with out truncation. 0.0625 can be stored as well without error. In both of these cases, the decimal digits exceed the 1.2 given from the equation above.

I can't help but wonder if there is something I am missing. Is there a way to determine decimal precision given a mantissa of any size? Does the formula above conflict with mantissa fields below a certain size?

Any clarity on the matter would be greatly appreciated.

Danny D.

Answer 1

a generally accepted formula for determining the number of decimal digits that we can store in a floating point format without truncation error.

Research FLT_DIG, DBL_DIG, LDBL_DIG . They are the number of significant decimal digits (as in text ) that can be rounded into a floating-point type and back again for float, double, long double .

With binary bases and to generalize for OP, use

floor((significant_bits - 1) log10(base))

With base == 2 and a significant_bits == 5 bit signicand (4 explicitly encoded, 1 implied), this is floor(1.204...) or 1 .

This means with decimal numbers of the form w.000 * 10 ^power , all combinations of w ( 1 - 9 ) and power (in a min/max range) would "round-trip" from decimal text to OP_s_Floating_point_type to decimal text successfully.

If one tried to round-trip decimal text of the form w.y00 * 10 ^power , for some combination of w,y , a failure would occur. It does not mean all combinations of that form would fail, just some.

Consider trying to take decimal text "9.0", "9.1", "9.2", ... "9.9" and convert those 10 to OP's FP format. In the 9 range, that format only supports encoding 9.0 and 9.5. So mapping those 10 numbers would result in 2 different FP numbers. Of course converting those 2 back to decimal form results in only 2 different answers. The round-trip failed for 8 of those 10.

if there is something I am missing.
Is there a way to determine decimal precision given a mantissa of any size?

The value of 1 derived from the above formula is the worst case and the worst case is the number usually reported for decimal precision .