Effects of the Python Development Mode¶

Enabling the Python Development Mode is similar to the following command, butwith additional effects described below:

PYTHONMALLOC=debugPYTHONASYNCIODEBUG=1python-Wdefault-Xfaulthandler

Effects of the Python Development Mode:

• Adddefaultwarning filter. Thefollowing warnings are shown:

  • DeprecationWarning

  • ImportWarning

  • PendingDeprecationWarning

  • ResourceWarning

  Normally, the above warnings are filtered by the defaultwarningfilters.

  It behaves as if the-Wdefault command line option is used.

  Use the-Werror command line option or set thePYTHONWARNINGS environment variable toerror to treat warningsas errors.

• Install debug hooks on memory allocators to check for:

  • Buffer underflow

  • Buffer overflow

  • Memory allocator API violation

  • Unsafe usage of the GIL

  See thePyMem_SetupDebugHooks() C function.

  It behaves as if thePYTHONMALLOC environment variable is set todebug.

  To enable the Python Development Mode without installing debug hooks onmemory allocators, set thePYTHONMALLOC environment variable todefault.

• Callfaulthandler.enable() at Python startup to install handlers fortheSIGSEGV,SIGFPE,SIGABRT,SIGBUS andSIGILL signals to dump the Python traceback on a crash.

  It behaves as if the-Xfaulthandler command line option isused or if thePYTHONFAULTHANDLER environment variable is set to1.

• Enableasyncio debug mode. For example,asyncio checks for coroutines that were not awaited and logs them.

  It behaves as if thePYTHONASYNCIODEBUG environment variable is setto1.

• Check theencoding anderrors arguments for string encoding and decodingoperations. Examples:open(),str.encode() andbytes.decode().

  By default, for best performance, theerrors argument is only checked atthe first encoding/decoding error and theencoding argument is sometimesignored for empty strings.

• Theio.IOBase destructor logsclose() exceptions.

• Set thedev_mode attribute ofsys.flags toTrue.

The Python Development Mode does not enable thetracemalloc module bydefault, because the overhead cost (to performance and memory) would be toolarge. Enabling thetracemalloc module provides additional informationon the origin of some errors. For example,ResourceWarning logs thetraceback where the resource was allocated, and a buffer overflow error logsthe traceback where the memory block was allocated.

The Python Development Mode does not prevent the-O command lineoption from removingassert statements nor from setting__debug__ toFalse.

The Python Development Mode can only be enabled at the Python startup. Itsvalue can be read fromsys.flags.dev_mode.

ResourceWarning Example¶

Example of a script counting the number of lines of the text file specified inthe command line:

importsysdefmain():fp=open(sys.argv[1])nlines=len(fp.readlines())print(nlines)# The file is closed implicitlyif__name__=="__main__":main()

The script does not close the file explicitly. By default, Python does not emitany warning. Example using README.txt, which has 269 lines:

$pythonscript.pyREADME.txt269

Enabling the Python Development Mode displays aResourceWarning warning:

$python-Xdevscript.pyREADME.txt269script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>  main()ResourceWarning: Enable tracemalloc to get the object allocation traceback

In addition, enablingtracemalloc shows the line where the file wasopened:

$python-Xdev-Xtracemalloc=5script.pyREADME.rst269script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'>  main()Object allocated at (most recent call last):  File "script.py", lineno 10    main()  File "script.py", lineno 4    fp = open(sys.argv[1])

The fix is to close explicitly the file. Example using a context manager:

defmain():# Close the file explicitly when exiting the with blockwithopen(sys.argv[1])asfp:nlines=len(fp.readlines())print(nlines)

Not closing a resource explicitly can leave a resource open for way longer thanexpected; it can cause severe issues upon exiting Python. It is bad inCPython, but it is even worse in PyPy. Closing resources explicitly makes anapplication more deterministic and more reliable.

Bad file descriptor error example¶

Script displaying the first line of itself:

importosdefmain():fp=open(__file__)firstline=fp.readline()print(firstline.rstrip())os.close(fp.fileno())# The file is closed implicitlymain()

By default, Python does not emit any warning:

$pythonscript.pyimport os

The Python Development Mode shows aResourceWarning and logs a “Bad filedescriptor” error when finalizing the file object:

$python-Xdevscript.pyimport osscript.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>  main()ResourceWarning: Enable tracemalloc to get the object allocation tracebackException ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'>Traceback (most recent call last):  File "script.py", line 10, in <module>    main()OSError: [Errno 9] Bad file descriptor

os.close(fp.fileno()) closes the file descriptor. When the file objectfinalizer tries to close the file descriptor again, it fails with theBadfiledescriptor error. A file descriptor must be closed only once. In theworst case scenario, closing it twice can lead to a crash (seebpo-18748for an example).

The fix is to remove theos.close(fp.fileno()) line, or open the file withclosefd=False.