Abstract

The current division (/) operator has an ambiguous meaning for numericalarguments: it returns the floor of the mathematical result of division if thearguments are ints or longs, but it returns a reasonable approximation of thedivision result if the arguments are floats or complex. This makesexpressions expecting float or complex results error-prone when integers arenot expected but possible as inputs.

We propose to fix this by introducing different operators for differentoperations:x/y to return a reasonable approximation of the mathematicalresult of the division (“true division”),x//y to return the floor(“floor division”). We call the current, mixed meaning of x/y“classic division”.

Because of severe backwards compatibility issues, not to mention a majorflamewar on c.l.py, we propose the following transitional measures (startingwith Python 2.2):

• Classic division will remain the default in the Python 2.x series; truedivision will be standard in Python 3.0.
• The// operator will be available to request floor divisionunambiguously.
• The future division statement, spelledfrom__future__importdivision,will change the/ operator to mean true division throughout the module.
• A command line option will enable run-time warnings for classic divisionapplied to int or long arguments; another command line option will make truedivision the default.
• The standard library will use the future division statement and the//operator when appropriate, so as to completely avoid classic division.

Motivation

The classic division operator makes it hard to write numerical expressionsthat are supposed to give correct results from arbitrary numerical inputs.For all other operators, one can write down a formula such asx*y**2+z,and the calculated result will be close to the mathematical result (within thelimits of numerical accuracy, of course) for any numerical input type (int,long, float, or complex). But division poses a problem: if the expressionsfor both arguments happen to have an integral type, it implements floordivision rather than true division.

The problem is unique to dynamically typed languages: in a statically typedlanguage like C, the inputs, typically function arguments, would be declaredas double or float, and when a call passes an integer argument, it isconverted to double or float at the time of the call. Python doesn’t haveargument type declarations, so integer arguments can easily find their wayinto an expression.

The problem is particularly pernicious since ints are perfect substitutes forfloats in all other circumstances:math.sqrt(2) returns the same value asmath.sqrt(2.0),3.14*100 and3.14*100.0 return the same value, andso on. Thus, the author of a numerical routine may only use floating pointnumbers to test his code, and believe that it works correctly, and a user mayaccidentally pass in an integer input value and get incorrect results.

Another way to look at this is that classic division makes it difficult towrite polymorphic functions that work well with either float or int arguments;all other operators already do the right thing. No algorithm that works forboth ints and floats has a need for truncating division in one case and truedivision in the other.

The correct work-around is subtle: casting an argument to float() is wrong ifit could be a complex number; adding 0.0 to an argument doesn’t preserve thesign of the argument if it was minus zero. The only solution without eitherdownside is multiplying an argument (typically the first) by 1.0. This leavesthe value and sign unchanged for float and complex, and turns int and longinto a float with the corresponding value.

It is the opinion of the authors that this is a real design bug in Python, andthat it should be fixed sooner rather than later. Assuming Python usage willcontinue to grow, the cost of leaving this bug in the language will eventuallyoutweigh the cost of fixing old code – there is an upper bound to the amountof code to be fixed, but the amount of code that might be affected by the bugin the future is unbounded.

Another reason for this change is the desire to ultimately unify Python’snumeric model. This is the subject ofPEP 228 (which is currentlyincomplete). A unified numeric model removes most of the user’s need to beaware of different numerical types. This is good for beginners, but alsotakes away concerns about different numeric behavior for advanced programmers.(Of course, it won’t remove concerns about numerical stability and accuracy.)

In a unified numeric model, the different types (int, long, float, complex,and possibly others, such as a new rational type) serve mostly as storageoptimizations, and to some extent to indicate orthogonal properties such asinexactness or complexity. In a unified model, the integer 1 should beindistinguishable from the floating point number 1.0 (except for itsinexactness), and both should behave the same in all numeric contexts.Clearly, in a unified numeric model, ifa==b andc==d,a/c shouldequalb/d (taking some liberties due to rounding for inexact numbers), andsince everybody agrees that1.0/2.0 equals 0.5,1/2 should also equal0.5. Likewise, since1//2 equals zero,1.0//2.0 should also equalzero.

Variations

Aesthetically,x//y doesn’t please everyone, and hence several variationshave been proposed. They are addressed here:

• xdivy. This would introduce a new keyword. Sincediv is apopular identifier, this would break a fair amount of existing code, unlessthe new keyword was only recognized under a future division statement.Since it is expected that the majority of code that needs to be converted isdividing integers, this would greatly increase the need for the futuredivision statement. Even with a future statement, the general sentimentagainst adding new keywords unless absolutely necessary argues against this.
• div(x,y). This makes the conversion of old code much harder.Replacingx/y withx//y orxdivy can be done with a simplequery replace; in most cases the programmer can easily verify that aparticular module only works with integers so all occurrences ofx/y canbe replaced. (The query replace is still needed to weed out slashesoccurring in comments or string literals.) Replacingx/y withdiv(x,y) would require a much more intelligent tool, since the extentof the expressions to the left and right of the/ must be analyzedbefore the placement of thediv( and) part can be decided.
• x\y. The backslash is already a token, meaning line continuation, andin general it suggests anescape to Unix eyes. In addition (this due toTerry Reedy) this would make things likeeval("x\y") harder to getright.

Alternatives

In order to reduce the amount of old code that needs to be converted, severalalternative proposals have been put forth. Here is a brief discussion of eachproposal (or category of proposals). If you know of an alternative that wasdiscussed on c.l.py that isn’t mentioned here, please mail the second author.

• Let/ keep its classic semantics; introduce// for true division.This still leaves a broken operator in the language, and invites to use thebroken behavior. It also shuts off the road to a unified numeric model a laPEP 228.
• Let int division return a special “portmanteau” type that behaves as aninteger in integer context, but like a float in a float context. Theproblem with this is that after a few operations, the int and the floatvalue could be miles apart, it’s unclear which value should be used incomparisons, and of course many contexts (like conversion to string) don’thave a clear integer or float preference.
• Use a directive to use specific division semantics in a module, rather thana future statement. This retains classic division as a permanent wart inthe language, requiring future generations of Python programmers to beaware of the problem and the remedies.
• Usefrom__past__importdivision to use classic division semantics in amodule. This also retains the classic division as a permanent wart, or atleast for a long time (eventually the past division statement could raise anImportError).
• Use a directive (or some other way) to specify the Python version for whicha specific piece of code was developed. This requires future Pythoninterpreters to be able to emulateexactly several previous versions ofPython, and moreover to do so for multiple versions within the sameinterpreter. This is way too much work. A much simpler solution is to keepmultiple interpreters installed. Another argument against this is that theversion directive is almost always overspecified: most code written forPython X.Y, works for Python X.(Y-1) and X.(Y+1) as well, so specifying X.Yas a version is more constraining than it needs to be. At the same time,there’s no way to know at which future or past version the code will break.

API Changes

During the transitional phase, we have to supportthree division operatorswithin the same program: classic division (for/ in modules without afuture division statement), true division (for/ in modules with a futuredivision statement), and floor division (for//). Each operator comes intwo flavors: regular, and as an augmented assignment operator (/= or//=).

The names associated with these variations are:

• Overloaded operator methods:

  __div__(),__floordiv__(),__truediv__();__idiv__(),__ifloordiv__(),__itruediv__().

• Abstract API C functions:

  PyNumber_Divide(),PyNumber_FloorDivide(),PyNumber_TrueDivide();PyNumber_InPlaceDivide(),PyNumber_InPlaceFloorDivide(),PyNumber_InPlaceTrueDivide().

• Byte code opcodes:

  BINARY_DIVIDE,BINARY_FLOOR_DIVIDE,BINARY_TRUE_DIVIDE;INPLACE_DIVIDE,INPLACE_FLOOR_DIVIDE,INPLACE_TRUE_DIVIDE.

• PyNumberMethod slots:

  nb_divide,nb_floor_divide,nb_true_divide,nb_inplace_divide,nb_inplace_floor_divide,nb_inplace_true_divide.

The addedPyNumberMethod slots require an additional flag intp_flags;this flag will be namedPy_TPFLAGS_HAVE_NEWDIVIDE and will be included inPy_TPFLAGS_DEFAULT.

The true and floor division APIs will look for the corresponding slots andcall that; when that slot isNULL, they will raise an exception. There isno fallback to the classic divide slot.

In Python 3.0, the classic division semantics will be removed; the classicdivision APIs will become synonymous with true division.

Command Line Option

The-Q command line option takes a string argument that can take fourvalues:old,warn,warnall, ornew. The default isoldin Python 2.2 but will change towarn in later 2.x versions. Theoldvalue means the classic division operator acts as described. Thewarnvalue means the classic division operator issues a warning (aDeprecationWarning using the standard warning framework) when appliedto ints or longs. Thewarnall value also issues warnings for classicdivision when applied to floats or complex; this is for use by thefixdiv.py conversion script mentioned below. Thenew value changesthe default globally so that the/ operator is always interpreted astrue division. Thenew option is only intended for use in certaineducational environments, where true division is required, but asking thestudents to include the future division statement in all their code would be aproblem.

This option will not be supported in Python 3.0; Python 3.0 will alwaysinterpret/ as true division.

(This option was originally proposed as-D, but that turned out to be anexisting option for Jython, hence the Q – mnemonic for Quotient. Other nameshave been proposed, like-Qclassic,-Qclassic-warn,-Qtrue, or-Qold_division etc.; these seem more verbose to me without much advantage.After all the term classic division is not used in the language at all (onlyin the PEP), and the term true division is rarely used in the language – onlyin__truediv__.)

Semantics of Floor Division

Floor division will be implemented in all the Python numeric types, and willhave the semantics of:

a//b==floor(a/b)

except that the result type will be the common type into whicha andb arecoerced before the operation.

Specifically, ifa andb are of the same type,a//b will be of thattype too. If the inputs are of different types, they are first coerced to acommon type using the same rules used for all other arithmetic operators.

In particular, ifa andb are both ints or longs, the result has the sametype and value as for classic division on these types (including the case ofmixed input types;int//long andlong//int will both return a long).

For floating point inputs, the result is a float. For example:

3.5//2.0==1.0

For complex numbers,// raises an exception, sincefloor() of acomplex number is not allowed.

For user-defined classes and extension types, all semantics are up to theimplementation of the class or type.

Semantics of True Division

True division for ints and longs will convert the arguments to float and thenapply a float division. That is, even2/1 will return afloat(2.0),not an int. For floats and complex, it will be the same as classic division.

The 2.2 implementation of true division acts as if the float type hadunbounded range, so that overflow doesn’t occur unless the magnitude of themathematicalresult is too large to represent as a float. For example,afterx=1L<<40000,float(x) raisesOverflowError (note thatthis is also new in 2.2: previously the outcome was platform-dependent, mostcommonly a float infinity). Butx/x returns 1.0 without exception,whilex/1 raisesOverflowError.

Note that for int and long arguments, true division may lose information; thisis in the nature of true division (as long as rationals are not in thelanguage). Algorithms that consciously use longs should consider using//, as true division of longs retains no more than 53 bits of precision(on most platforms).

If and when a rational type is added to Python (seePEP 239), truedivision for ints and longs should probably return a rational. This avoidsthe problem with true division of ints and longs losing information. Butuntil then, for consistency, float is the only choice for true division.

The Future Division Statement

Iffrom__future__importdivision is present in a module, or if-Qnew is used, the/ and/= operators are translated to truedivision opcodes; otherwise they are translated to classic division (untilPython 3.0 comes along, where they are always translated to true division).

The future division statement has no effect on the recognition or translationof// and//=.

SeePEP 236 for the general rules for future statements.

(It has been proposed to use a longer phrase, liketrue_division ormodern_division. These don’t seem to add much information.)

Open Issues

We expect that these issues will be resolved over time, as more feedback isreceived or we gather more experience with the initial implementation.

• It has been proposed to call// the quotient operator, and the/operator the ratio operator. I’m not sure about this – for some peoplequotient is just a synonym for division, and ratio suggests rationalnumbers, which is wrong. I prefer the terminology to be slightly awkwardif that avoids unambiguity. Also, for some folksquotient suggeststruncation towards zero, not towards infinity asfloor divisionsays explicitly.
• It has been argued that a command line option to change the default isevil. It can certainly be dangerous in the wrong hands: for example, itwould be impossible to combine a 3rd party library package that requires-Qnew with another one that requires-Qold. But I believe that theVPython folks need a way to enable true division by default, and othereducators might need the same. These usually have enough control over thelibrary packages available in their environment.
• For classes to have to support all three of__div__(),__floordiv__() and__truediv__() seems painful; and what to do in3.0? Maybe we only need__div__() and__floordiv__(), or maybe atleast true division should try__truediv__() first and__div__()second.

Resolved Issues

• Issue: For very large long integers, the definition of true division asreturning a float causes problems, since the range of Python longs is muchlarger than that of Python floats. This problem will disappear if and whenrational numbers are supported.

  Resolution: For long true division, Python uses an internal float type withnative double precision but unbounded range, so that OverflowError doesn’toccur unless the quotient is too large to represent as a native double.

• Issue: In the interim, maybe the long-to-float conversion could be made toraiseOverflowError if the long is out of range.

  Resolution: This has been implemented, but, as above, the magnitude of theinputs to long true division doesn’t matter; only the magnitude of thequotient matters.

• Issue: Tim Peters will make sure that whenever an in-range float isreturned, decent precision is guaranteed.

  Resolution: Provided the quotient of long true division is representable asa float, it suffers no more than 3 rounding errors: one each for convertingthe inputs to an internal float type with native double precision butunbounded range, and one more for the division. However, note that if themagnitude of the quotient is toosmall to represent as a native double,0.0 is returned without exception (“silent underflow”).

FAQ

Implementation

Essentially everything mentioned here is implemented in CVS and will bereleased with Python 2.2a3; most of it was already released with Python 2.2a2.

Copyright

This document has been placed in the public domain.