What is deterministic profiling?¶

Deterministic profiling captures every function call, function return,and exception event during program execution. The profiler measures the precisetime intervals between these events, providing exact statistics about how theprogram behaves.

In contrast tostatistical profiling, which samplesthe call stack periodically to estimate where time is spent, deterministicprofiling records every event. This means you get exact call counts rather thanstatistical approximations. The trade-off is that instrumenting every eventintroduces overhead that can slow down program execution.

Python’s interpreted nature makes deterministic profiling practical. Theinterpreter already dispatches events for function calls and returns, so theprofiler can hook into this mechanism without requiring code modification. Theoverhead tends to be moderate relative to the inherent cost of interpretation,making deterministic profiling suitable for most development workflows.

Deterministic profiling helps answer questions like:

• How many times was this function called?

• What is the complete call graph of my program?

• Which functions are called by a particular function?

• Are there unexpected function calls happening?

Call count statistics can identify bugs (surprising counts) and inlineexpansion opportunities (high call counts). Internal time statistics reveal“hot loops” that warrant optimization. Cumulative time statistics help identifyalgorithmic inefficiencies. The handling of cumulative times in this profilerallows direct comparison of recursive and iterative implementations.

Command-line interface¶

Theprofiling.tracing module can be invoked as a script to profileanother script or module:

python -m profiling.tracing [-o output_file] [-s sort_order] (-m module | script.py)

This runs the specified script or module under the profiler and prints theresults to standard output (or saves them to a file).

-o<output_file>¶

Write the profile results to a file instead of standard output. The outputfile can be read by thepstats module for later analysis.

-s<sort_order>¶

Sort the output by the specified key. This accepts any of the sort keysrecognized bypstats.Stats.sort_stats(), such ascumulative,time,calls, orname. This option only applies when-o is not specified.

-m<module>¶

Profile a module instead of a script. The module is located using thestandard import mechanism.

Added in version 3.7:The-m option forcProfile.

Added in version 3.8:The-m option forprofile.

Programmatic usage examples¶

For more control over profiling, use the module’s functions and classesdirectly.

importprofiling.tracingprofiling.tracing.run('my_function()')

profiling.tracing.run('my_function()','output.prof')

importprofiling.tracingimportpstatsfromioimportStringIOpr=profiling.tracing.Profile()pr.enable()# ... code to profile ...pr.disable()# Print resultss=StringIO()ps=pstats.Stats(pr,stream=s).sort_stats(pstats.SortKey.CUMULATIVE)ps.print_stats()print(s.getvalue())

importprofiling.tracingwithprofiling.tracing.Profile()aspr:# ... code to profile ...pr.print_stats()

Module reference¶

profiling.tracing.run(command,filename=None,sort=-1)¶

Profile execution of a command and print or save the results.

This function executes thecommand string usingexec() in the__main__ module’s namespace:

exec(command,__main__.__dict__,__main__.__dict__)

Iffilename is not provided, the function creates apstats.Statsinstance and prints a summary to standard output. Iffilename isprovided, the raw profile data is saved to that file for later analysiswithpstats.

Thesort argument specifies the sort order for printed output, acceptingany value recognized bypstats.Stats.sort_stats().

profiling.tracing.runctx(command,globals,locals,filename=None,sort=-1)¶

Profile execution of a command with explicit namespaces.

Likerun(), but executes the command with the specifiedglobalsandlocals mappings instead of using the__main__ module’s namespace:

exec(command,globals,locals)

classprofiling.tracing.Profile(timer=None,timeunit=0.0,subcalls=True,builtins=True)¶

A profiler object that collects execution statistics.

The optionaltimer argument specifies a custom timing function. If notprovided, the profiler uses a platform-appropriate default timer. Whensupplying a custom timer, it must return a single number representing thecurrent time. If the timer returns integers, usetimeunit to specify theduration of one time unit (for example,0.001 for milliseconds).

Thesubcalls argument controls whether the profiler tracks callrelationships between functions. Thebuiltins argument controls whetherbuilt-in functions are profiled.

Changed in version 3.8:Added context manager support.

enable()¶

Start collecting profiling data.

disable()¶

Stop collecting profiling data.

create_stats()¶

Stop collecting data and record the results internally as the currentprofile.

print_stats(sort=-1)¶

Create apstats.Stats object from the current profile and printthe results to standard output.

Thesort argument specifies the sorting order. It accepts a singlekey or a tuple of keys for multi-level sorting, using the same valuesaspstats.Stats.sort_stats().

Added in version 3.13:Support for a tuple of sort keys.

dump_stats(filename)¶

Write the current profile data tofilename. The file can be read bypstats.Stats for later analysis.

run(cmd)¶

Profile the command string viaexec().

runctx(cmd,globals,locals)¶

Profile the command string viaexec() with the specifiednamespaces.

runcall(func,/,*args,**kwargs)¶

Profile a function call. Returns whateverfunc returns:

result=pr.runcall(my_function,arg1,arg2,keyword=value)

Using a custom timer¶

TheProfile class accepts a custom timing function, allowing you tomeasure different aspects of execution such as wall-clock time or CPU time.Pass the timing function to the constructor:

pr=profiling.tracing.Profile(my_timer_function)

The timer function must return a single number representing the current time.If it returns integers, also specifytimeunit to indicate the duration ofone unit:

# Timer returns time in millisecondspr=profiling.tracing.Profile(my_ms_timer,0.001)

For best performance, the timer function should be as fast as possible. Theprofiler calls it frequently, so timer overhead directly affects profilingoverhead.

Thetime module provides several functions suitable for use as customtimers:

• time.perf_counter() for high-resolution wall-clock time

• time.process_time() for CPU time (excluding sleep)

• time.monotonic() for monotonic clock time

Limitations¶

Deterministic profiling has inherent limitations related to timing accuracy.

The underlying timer typically has a resolution of about one millisecond.Measurements cannot be more accurate than this resolution. With enoughmeasurements, timing errors tend to average out, but individual measurementsmay be imprecise.

There is also latency between when an event occurs and when the profilercaptures the timestamp. Similarly, there is latency after reading thetimestamp before user code resumes. Functions called frequently accumulatethis latency, which can make them appear slower than they actually are. Thiserror is typically less than one clock tick per call but can becomesignificant for functions called many times.

Theprofiling.tracing module (and itscProfile alias) isimplemented as a C extension with low overhead, so these timing issues areless pronounced than with the deprecated pure Pythonprofile module.