15.Floating-Point Arithmetic: Issues and Limitations¶

Floating-point numbers are represented in computer hardware as base 2 (binary)fractions. For example, thedecimal fraction0.625has value 6/10 + 2/100 + 5/1000, and in the same way thebinary fraction0.101has value 1/2 + 0/4 + 1/8. These two fractions have identical values, the onlyreal difference being that the first is written in base 10 fractional notation,and the second in base 2.

Unfortunately, most decimal fractions cannot be represented exactly as binaryfractions. A consequence is that, in general, the decimal floating-pointnumbers you enter are only approximated by the binary floating-point numbersactually stored in the machine.

The problem is easier to understand at first in base 10. Consider the fraction1/3. You can approximate that as a base 10 fraction:

0.3

or, better,

0.33

or, better,

0.333

and so on. No matter how many digits you’re willing to write down, the resultwill never be exactly 1/3, but will be an increasingly better approximation of1/3.

In the same way, no matter how many base 2 digits you’re willing to use, thedecimal value 0.1 cannot be represented exactly as a base 2 fraction. In base2, 1/10 is the infinitely repeating fraction

0.0001100110011001100110011001100110011001100110011...

Stop at any finite number of bits, and you get an approximation. On mostmachines today, floats are approximated using a binary fraction withthe numerator using the first 53 bits starting with the most significant bit andwith the denominator as a power of two. In the case of 1/10, the binary fractionis3602879701896397/2**55 which is close to but not exactlyequal to the true value of 1/10.

Many users are not aware of the approximation because of the way values aredisplayed. Python only prints a decimal approximation to the true decimalvalue of the binary approximation stored by the machine. On most machines, ifPython were to print the true decimal value of the binary approximation storedfor 0.1, it would have to display:

>>>0.10.1000000000000000055511151231257827021181583404541015625

That is more digits than most people find useful, so Python keeps the numberof digits manageable by displaying a rounded value instead:

>>>1/100.1

Just remember, even though the printed result looks like the exact valueof 1/10, the actual stored value is the nearest representable binary fraction.

Interestingly, there are many different decimal numbers that share the samenearest approximate binary fraction. For example, the numbers0.1 and0.10000000000000001 and0.1000000000000000055511151231257827021181583404541015625 are allapproximated by3602879701896397/2**55. Since all of these decimalvalues share the same approximation, any one of them could be displayedwhile still preserving the invarianteval(repr(x))==x.

Historically, the Python prompt and built-inrepr() function would choosethe one with 17 significant digits,0.10000000000000001. Starting withPython 3.1, Python (on most systems) is now able to choose the shortest ofthese and simply display0.1.

Note that this is in the very nature of binary floating point: this is not a bugin Python, and it is not a bug in your code either. You’ll see the same kind ofthing in all languages that support your hardware’s floating-point arithmetic(although some languages may notdisplay the difference by default, or in alloutput modes).

For more pleasant output, you may wish to use string formatting to produce alimited number of significant digits:

>>>format(math.pi,'.12g')# give 12 significant digits'3.14159265359'>>>format(math.pi,'.2f')# give 2 digits after the point'3.14'>>>repr(math.pi)'3.141592653589793'

It’s important to realize that this is, in a real sense, an illusion: you’resimply rounding thedisplay of the true machine value.

One illusion may beget another. For example, since 0.1 is not exactly 1/10,summing three values of 0.1 may not yield exactly 0.3, either:

>>>0.1+0.1+0.1==0.3False

Also, since the 0.1 cannot get any closer to the exact value of 1/10 and0.3 cannot get any closer to the exact value of 3/10, then pre-rounding withround() function cannot help:

>>>round(0.1,1)+round(0.1,1)+round(0.1,1)==round(0.3,1)False

Though the numbers cannot be made closer to their intended exact values,themath.isclose() function can be useful for comparing inexact values:

>>>math.isclose(0.1+0.1+0.1,0.3)True

Alternatively, theround() function can be used to compare roughapproximations:

>>>round(math.pi,ndigits=2)==round(22/7,ndigits=2)True

Binary floating-point arithmetic holds many surprises like this. The problemwith “0.1” is explained in precise detail below, in the “Representation Error”section. SeeExamples of Floating Point Problems fora pleasant summary of how binary floating point works and the kinds ofproblems commonly encountered in practice. Also seeThe Perils of Floating Pointfor a more complete account of other common surprises.

As that says near the end, “there are no easy answers.” Still, don’t be undulywary of floating point! The errors in Python float operations are inheritedfrom the floating-point hardware, and on most machines are on the order of nomore than 1 part in 2**53 per operation. That’s more than adequate for mosttasks, but you do need to keep in mind that it’s not decimal arithmetic andthat every float operation can suffer a new rounding error.

While pathological cases do exist, for most casual use of floating-pointarithmetic you’ll see the result you expect in the end if you simply round thedisplay of your final results to the number of decimal digits you expect.str() usually suffices, and for finer control see thestr.format()method’s format specifiers inFormat String Syntax.

For use cases which require exact decimal representation, try using thedecimal module which implements decimal arithmetic suitable foraccounting applications and high-precision applications.

Another form of exact arithmetic is supported by thefractions modulewhich implements arithmetic based on rational numbers (so the numbers like1/3 can be represented exactly).

If you are a heavy user of floating-point operations you should take a lookat the NumPy package and many other packages for mathematical andstatistical operations supplied by the SciPy project. See <https://scipy.org>.

Python provides tools that may help on those rare occasions when you reallydo want to know the exact value of a float. Thefloat.as_integer_ratio() method expresses the value of a float as afraction:

>>>x=3.14159>>>x.as_integer_ratio()(3537115888337719, 1125899906842624)

Since the ratio is exact, it can be used to losslessly recreate theoriginal value:

>>>x==3537115888337719/1125899906842624True

Thefloat.hex() method expresses a float in hexadecimal (base16), again giving the exact value stored by your computer:

>>>x.hex()'0x1.921f9f01b866ep+1'

This precise hexadecimal representation can be used to reconstructthe float value exactly:

>>>x==float.fromhex('0x1.921f9f01b866ep+1')True

Since the representation is exact, it is useful for reliably porting valuesacross different versions of Python (platform independence) and exchangingdata with other languages that support the same format (such as Java and C99).

Another helpful tool is thesum() function which helps mitigateloss-of-precision during summation. It uses extended precision forintermediate rounding steps as values are added onto a running total.That can make a difference in overall accuracy so that the errors do notaccumulate to the point where they affect the final total:

>>>0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1==1.0False>>>sum([0.1]*10)==1.0True

Themath.fsum() goes further and tracks all of the “lost digits”as values are added onto a running total so that the result has only asingle rounding. This is slower thansum() but will be moreaccurate in uncommon cases where large magnitude inputs mostly canceleach other out leaving a final sum near zero:

>>>arr=[-0.10430216751806065,-266310978.67179024,143401161448607.16,...-143401161400469.7,266262841.31058735,-0.003244936839808227]>>>float(sum(map(Fraction,arr)))# Exact summation with single rounding8.042173697819788e-13>>>math.fsum(arr)# Single rounding8.042173697819788e-13>>>sum(arr)# Multiple roundings in extended precision8.042178034628478e-13>>>total=0.0>>>forxinarr:...total+=x# Multiple roundings in standard precision...>>>total# Straight addition has no correct digits!-0.0051575902860057365

1/10~=J/(2**N)

J~=2**N/10

>>>2**52<=2**56//10<2**53True

>>>q,r=divmod(2**56,10)>>>r6

>>>q+17205759403792794

7205759403792794/2**56

3602879701896397/2**55

>>>0.1*2**553602879701896397.0

>>>3602879701896397*10**55//2**551000000000000000055511151231257827021181583404541015625

>>>format(0.1,'.17f')'0.10000000000000001'

>>>fromdecimalimportDecimal>>>fromfractionsimportFraction>>>Fraction.from_float(0.1)Fraction(3602879701896397, 36028797018963968)>>>(0.1).as_integer_ratio()(3602879701896397, 36028797018963968)>>>Decimal.from_float(0.1)Decimal('0.1000000000000000055511151231257827021181583404541015625')>>>format(Decimal.from_float(0.1),'.17')'0.10000000000000001'