Convert integer w/6 implied decimals to float, correcting rounding errors in Python

Question

Under RHEL, Python v 2.6.6

>>> from __future__ import division
>>> n = 5352800
>>> d = 1000000
>>> n/d
5.3528000000000002
>>> 

I understand the nature and reasons for this rounding error.

I need to convert an integer I receive to a float, and correct the rounding errors that result. What is the best way of doing this? "Best" in my case means Pythonic and fast.

The integer in question has 6 implied decimals.

Answer 1

If you want to store it as a string with 6 decimal places for display , then you can use string formatting:

>>> from __future__ import division
>>> n = 5352800
>>> d = 1000000
>>> n/d
5.3528
>>> '%.6f' % (n/d)
'5.352800'

If you want it stored as a number with 6 decimal places for further computation (though this probably isn't what you want), you should use the decimal module. Use the formatted string from above and use it as the argument to decimal.Decimal .

>>> from decimal import Decimal
>>> Decimal('%.6f' % (n/d))
Decimal('5.352800')

If your goal is further computation, it would probably be more performant to do everything in decimal from the beginning rather than messing around with string formatting, ie

>>> from decimal import Decimal
>>> n = Decimal(5352800)
>>> d = Decimal(1000000)
>>> n/d
Decimal('5.3528')

This prints differently (ie with fewer decimal places), but Decimal('5.352800') == Decimal('5.3528') , and the two forms should behave identically in all numerical operations. Note also that you don't need the __future__ import - division on decimals is automatically true division rather than floor division.