Float type limited to 104 days

The clocks resolution of desktop and laptop computers is getting closerto nanosecond resolution. More and more clocks have a frequency in MHz,up to GHz for the CPU TSC clock.

The Pythontime.time() function returns the current time as afloating-point number which is usually a 64-bit binary floating-pointnumber (in the IEEE 754 format).

The problem is that thefloat type starts to lose nanoseconds after 104days. Converting from nanoseconds (int) to seconds (float) andthen back to nanoseconds (int) to check if conversions loseprecision:

# no precision loss>>>x=2**52+1;int(float(x*1e-9)*1e9)-x0# precision loss! (1 nanosecond)>>>x=2**53+1;int(float(x*1e-9)*1e9)-x-1>>>print(datetime.timedelta(seconds=2**53/1e9))104days,5:59:59.254741

time.time() returns seconds elapsed since the UNIX epoch: January1st, 1970. This function hasn’t had nanosecond precision since May 1970(47 years ago):

>>>importdatetime>>>unix_epoch=datetime.datetime(1970,1,1)>>>print(unix_epoch+datetime.timedelta(seconds=2**53/1e9))1970-04-15 05:59:59.254741

Previous rejected PEP

Five years ago, thePEP 410 proposed a large and complex change in allPython functions returning time to support nanosecond resolution usingthedecimal.Decimal type.

The PEP was rejected for different reasons:

• The idea of adding a new optional parameter to change the result typewas rejected. It’s an uncommon (and bad?) programming practice inPython.
• It was not clear if hardware clocks really had a resolution of 1nanosecond, or if that made sense at the Python level.
• Thedecimal.Decimal type is uncommon in Python and so requiresto adapt code to handle it.

Issues caused by precision loss

CPython enhancements of the last 5 years

Since thePEP 410 was rejected:

• Theos.stat_result structure got 3 new fields for timestamps asnanoseconds (Pythonint):st_atime_ns,st_ctime_nsandst_mtime_ns.
• ThePEP 418 was accepted, Python 3.3 got 3 new clocks:time.monotonic(),time.perf_counter() andtime.process_time().
• The CPython private “pytime” C API handling time now uses a new_PyTime_t type: simple 64-bit signed integer (Cint64_t).The_PyTime_t unit is an implementation detail and not part of theAPI. The unit is currently1nanosecond.

Existing Python APIs using nanoseconds as int

Theos.stat_result structure has 3 fields for timestamps asnanoseconds (int):st_atime_ns,st_ctime_ns andst_mtime_ns.

Thens parameter of theos.utime() function accepts a(atime_ns:int,mtime_ns:int) tuple: nanoseconds.

New functions

This PEP adds six new functions to thetime module:

• time.clock_gettime_ns(clock_id)
• time.clock_settime_ns(clock_id,time:int)
• time.monotonic_ns()
• time.perf_counter_ns()
• time.process_time_ns()
• time.time_ns()

These functions are similar to the version without the_ns suffix,but return a number of nanoseconds as a Pythonint.

For example,time.monotonic_ns()==int(time.monotonic()*1e9) ifmonotonic() value is small enough to not lose precision.

These functions are needed because they may return “large” timestamps,liketime.time() which uses the UNIX epoch as reference, and so theirfloat-returning variants are likely to lose precision at the nanosecondresolution.

Unchanged functions

Since thetime.clock() function was deprecated in Python 3.3, notime.clock_ns() is added.

Python has other time-returning functions. No nanosecond variant isproposed for these other functions, either because their internalresolution is greater or equal to 1 us, or because their maximum valueis small enough to not lose precision. For example, the maximum value oftime.clock_getres() should be 1 second.

Examples of unchanged functions:

• os module:sched_rr_get_interval(),times(),wait3()andwait4()
• resource module:ru_utime andru_stime fields ofgetrusage()
• signal module:getitimer(),setitimer()
• time module:clock_getres()

See also theAnnex: Clocks Resolution in Python.

A new nanosecond-returning flavor of these functions may be added laterif an operating system exposes new functions providing better resolution.

Sub-nanosecond resolution

time.time_ns() API is not theoretically future-proof: if clockresolutions continue to increase below the nanosecond level, new Pythonfunctions may be needed.

In practice, the 1 nanosecond resolution is currently enough for allstructures returned by all common operating systems functions.

Hardware clocks with a resolution better than 1 nanosecond alreadyexist. For example, the frequency of a CPU TSC clock is the CPU basefrequency: the resolution is around 0.3 ns for a CPU running at 3GHz. Users who have access to such hardware and really needsub-nanosecond resolution can however extend Python for their needs.Such a rare use case doesn’t justify to design the Python standard libraryto support sub-nanosecond resolution.

For the CPython implementation, nanosecond resolution is convenient: thestandard and well supportedint64_t type can be used to store ananosecond-precise timestamp. It supports a timespan of -292 yearsto +292 years. Using the UNIX epoch as reference, it therefore supportsrepresenting times since year 1677 to year 2262:

>>>1970-2**63/(10**9*3600*24*365.25)1677.728976954687>>>1970+2**63/(10**9*3600*24*365.25)2262.271023045313

Modifying time.time() result type

It was proposed to modifytime.time() to return a different numbertype with better precision.

ThePEP 410 proposed to returndecimal.Decimal which already exists andsupports arbitrary precision, but it was rejected. Apart fromdecimal.Decimal, no portable real number type with better precisionis currently available in Python.

Changing the built-in Pythonfloat type is out of the scope of thisPEP.

Moreover, changing existing functions to return a new type introduces arisk of breaking the backward compatibility even if the new type isdesigned carefully.

Different types

Many ideas of new types were proposed to support larger or arbitraryprecision: fractions, structures or 2-tuple using integers,fixed-point number, etc.

See also thePEP 410 for a previous long discussion on other types.

Adding a new type requires more effort to support it, than reusingthe existingint type. The standard library, third party code andapplications would have to be modified to support it.

The Pythonint type is well known, well supported, easy tomanipulate, and supports all arithmetic operations such asdt=t2-t1.

Moreover, taking/returning an integer number of nanoseconds is not anew concept in Python, as witnessed byos.stat_result andos.utime(ns=(atime_ns,mtime_ns)).

Different API

Thetime.time(ns=False) API was proposed to avoid adding newfunctions. It’s an uncommon (and bad?) programming practice in Python tochange the result type depending on a parameter.

Different options were proposed to allow the user to choose the timeresolution. If each Python module uses a different resolution, it canbecome difficult to handle different resolutions, instead of justseconds (time.time() returningfloat) and nanoseconds(time.time_ns() returningint). Moreover, as written above,there is no need for resolution better than 1 nanosecond in practice inthe Python standard library.

A new module

It was proposed to add a newtime_ns module containing the followingfunctions:

• time_ns.clock_gettime(clock_id)
• time_ns.clock_settime(clock_id,time:int)
• time_ns.monotonic()
• time_ns.perf_counter()
• time_ns.process_time()
• time_ns.time()

The first question is whether thetime_ns module should expose exactlythe same API (constants, functions, etc.) as thetime module. It can bepainful to maintain two flavors of thetime module. How are users usesupposed to make a choice between these two modules?

If tomorrow, other nanosecond variants are needed in theos module,will we have to add a newos_ns module as well? There are functionsrelated to time in many modules:time,os,signal,resource,select, etc.

Another idea is to add atime.ns submodule or a nested-namespace toget thetime.ns.time() syntax, but it suffers from the same issues.

Script

Example of script to measure the smallest difference between twotime.time() andtime.time_ns() reads ignoring differences of zero:

importmathimporttimeLOOPS=10**6print("time.time_ns():%s"%time.time_ns())print("time.time():%s"%time.time())min_dt=[abs(time.time_ns()-time.time_ns())for_inrange(LOOPS)]min_dt=min(filter(bool,min_dt))print("min time_ns() delta:%s ns"%min_dt)min_dt=[abs(time.time()-time.time())for_inrange(LOOPS)]min_dt=min(filter(bool,min_dt))print("min time() delta:%s ns"%math.ceil(min_dt*1e9))

Linux

Clocks resolution measured in Python on Fedora 26 (kernel 4.12):

Notes on resolutions:

• clock() frequency isCLOCKS_PER_SECOND which is 1,000,000 Hz(1 MHz): resolution of 1 us.
• times() frequency isos.sysconf("SC_CLK_TCK") (or theHZconstant) which is equal to 100 Hz: resolution of 10 ms.
• resource.getrusage(),os.wait3() andos.wait4() use theru_usage structure. The type of theru_usage.ru_utime andru_usage.ru_stime fields is thetimeval structure which has aresolution of 1 us.

Windows

Clocks resolution measured in Python on Windows 8.1:

The frequency ofperf_counter() andperf_counter_ns() comes fromQueryPerformanceFrequency(). The frequency is usually 10 MHz: resolution of100 ns. In old Windows versions, the frequency was sometimes 3,579,545 Hz (3.6MHz): resolution of 279 ns.

Analysis

The resolution oftime.time_ns() is much better thantime.time():84 ns (2.8x better) vs 239 ns on Linux and 318 us(2.8x better) vs 894 us on Windows. Thetime.time() resolution willonly become larger (worse) as years pass since every day adds86,400,000,000,000 nanoseconds to the system clock, which increases theprecision loss.

The difference betweentime.perf_counter(),time.monotonic(),time.process_time() and their respective nanosecond variants isnot visible in this quick script since the script runs for less than 1minute, and the uptime of the computer used to run the script wassmaller than 1 week. A significant difference may be seen if uptimereaches 104 days or more.

resource.getrusage() andtimes() have a resolution greater orequal to 1 microsecond, and so don’t need a variant with nanosecondresolution.