Get the runtime configuration

PEP 587 has no API toget thecurrent runtime configuration,only toconfigure the Pythoninitialization.

For example, the global configuration variablePy_UnbufferedStdioFlag was deprecated in Python 3.12 and usingPyConfig.buffered_stdio is recommended instead. It only works toconfigure Python, there is no public API to getPyConfig.buffered_stdio.

Users of the limited C API are asking for a public API to get thecurrent runtime configuration.

Cython needs to get theoptimization_level configuration option:issue.

When global configuration variables were deprecated in 2022,Marc-AndréLemburg requesteda C API to access these configuration variables at runtime (not onlyduring Python initialization).

Security fix

To fixCVE-2020-10735,a denial-of-service when converting a very large string to an integer (in base10), it was discussed to add a newPyConfig member to stablebranches which affects the ABI.

Gregory P. Smith proposed a different API using text based configurationfile to not be limited byPyConfig members:FR: Allow privateruntime config to enable extending without breaking the PyConfig ABI(August 2022).

In the end, it was decided to not add a newPyConfig member tostable branches, but only add a newPyConfig.int_max_str_digitsmember to the development branch (which became Python 3.12). A dedicatedprivate global variable (unrelated toPyConfig) is used in stablebranches.

Redundancy between PyPreConfig and PyConfig

The Python preinitialization uses thePyPreConfig structure and thePython initialization uses thePyConfig structure. Both structureshave four duplicated members:dev_mode,parse_argv,isolatedanduse_environment.

The redundancy is caused by the fact that the two structures areseparated, whereas somePyConfig members are needed by thepreinitialization.

Embedding Python

Set the runtime configuration

Marc-André Lemburg requesteda C API toset the value of some configuration options at runtime:

• optimization_level
• verbose
• parser_debug
• inspect
• write_bytecode

Previously, it was possible to set directly global configurationvariables:

• Py_OptimizeFlag
• Py_VerboseFlag
• Py_DebugFlag
• Py_InspectFlag
• Py_DontWriteBytecodeFlag

But these configuration flags were deprecated in Python 3.12 and arescheduled for removal in Python 3.14.

PyInitConfig structure

ThePyInitConfig structure is implemented by combining the threestructures of thePyConfig API and has aninittab member aswell:

• PyPreConfigpreconfig
• PyConfigconfig
• PyStatusstatus
• struct_inittab*inittab forPyInitConfig_AddModule()

ThePyStatus status is no longer separated, but part of the unifiedPyInitConfig structure, which makes the API easier to use.

Configuration Options

Configuration options are named afterPyPreConfig andPyConfigstructure members. See thePyPreConfig documentation andthePyConfig documentation.

Deprecating and removing configuration options is out of the scope ofthe PEP and should be discussed on a case by case basis.

Create Config

PyInitConfig structure:

Opaque structure to configure the Python preinitialization and thePython initialization.

PyInitConfig*PyInitConfig_Create(void):

Create a new initialization configuration using default valuesof theIsolated Configuration.

It must be freed withPyInitConfig_Free().

ReturnNULL on memory allocation failure.

voidPyInitConfig_Free(PyInitConfig*config):

Free memory of an initialization configuration.

Get Options

The configuration optionname parameter must be a non-NULLnull-terminated UTF-8 encoded string.

intPyInitConfig_HasOption(PyInitConfig*config,constchar*name):

Test if the configuration has an option calledname.

Return1 if the option exists, or return0 otherwise.

intPyInitConfig_GetInt(PyInitConfig*config,constchar*name,int64_t*value):

Get an integer configuration option.

• Set*value, and return0 on success.
• Set an error inconfig and return-1 on error.

intPyInitConfig_GetStr(PyInitConfig*config,constchar*name,char**value):

Get a string configuration option as a null-terminated UTF-8encoded string.

• Set*value, and return0 on success.
• Set an error inconfig and return-1 on error.

On success, the string must be released withfree(value).

intPyInitConfig_GetStrList(PyInitConfig*config,constchar*name,size_t*length,char***items):

Get a string list configuration option as an array ofnull-terminated UTF-8 encoded strings.

• Set*length and*value, and return0 on success.
• Set an error inconfig and return-1 on error.

On success, the string list must be released withPyInitConfig_FreeStrList(length,items).

voidPyInitConfig_FreeStrList(size_tlength,char**items):

Free memory of a string list created byPyInitConfig_GetStrList().

Set Options

The configuration optionname parameter must be a non-NULLnull-terminated UTF-8 encoded string.

Some configuration options have side effects on other options. Thislogic is only implemented whenPy_InitializeFromInitConfig() iscalled, not by the “Set” functions below. For example, settingdev_mode to1 does not setfaulthandler to1.

intPyInitConfig_SetInt(PyInitConfig*config,constchar*name,int64_tvalue):

Set an integer configuration option.

• Return0 on success.
• Set an error inconfig and return-1 on error.

intPyInitConfig_SetStr(PyInitConfig*config,constchar*name,constchar*value):

Set a string configuration option from a null-terminated UTF-8encoded string. The string is copied.

• Return0 on success.
• Set an error inconfig and return-1 on error.

intPyInitConfig_SetStrList(PyInitConfig*config,constchar*name,size_tlength,char*const*items):

Set a string list configuration option from an array ofnull-terminated UTF-8 encoded strings. The string list is copied.

• Return0 on success.
• Set an error inconfig and return-1 on error.

intPyInitConfig_AddModule(PyInitConfig*config,constchar*name,PyObject*(*initfunc)(void)):

Add a built-in extension module to the table of built-in modules.

The new module can be imported by the namename, and uses thefunctioninitfunc as the initialization function called on thefirst attempted import.

• Return0 on success.
• Set an error inconfig and return-1 on error.

If Python is initialized multiple times,PyInitConfig_AddModule() must be called at each Pythoninitialization.

Similar to thePyImport_AppendInittab() function.

Initialize Python

intPy_InitializeFromInitConfig(PyInitConfig*config):

Initialize Python from the initialization configuration.

• Return0 on success.
• Set an error inconfig and return-1 on error.
• Set an exit code inconfig and return-1 if Python wants toexit.

SeePyInitConfig_GetExitcode() for the exitcode case.

Error Handling

intPyInitConfig_GetError(PyInitConfig*config,constchar**err_msg):

Get theconfig error message.

• Set*err_msg and return1 if an error is set.
• Set*err_msg toNULL and return0 otherwise.

An error message is an UTF-8 encoded string.

Ifconfig has an exit code, format the exit code as an errormessage.

The error message remains valid until anotherPyInitConfigfunction is called withconfig. The caller doesn’t have to free theerror message.

intPyInitConfig_GetExitcode(PyInitConfig*config,int*exitcode):

Get theconfig exit code.

• Set*exitcode and return1 if Python wants to exit.
• Return0 ifconfig has no exit code set.

Only thePy_InitializeFromInitConfig() function can set an exitcode if theparse_argv option is non-zero.

An exit code can be set when parsing the command line failed (exitcode 2) or when a command line option asks to display the commandline help (exit code 0).

Get and Set the Runtime Configuration

The configuration optionname parameter must be a non-NULLnull-terminated UTF-8 encoded string.

PyObject*PyConfig_Get(constchar*name):

Get the current runtime value of a configuration option as a Pythonobject.

• Return a new reference on success.
• Set an exception and returnNULL on error.

The object type depends on the option: seeConfiguration Optionstables.

Other options are get from internalPyPreConfig andPyConfig structures.

The caller must hold the GIL. The function cannot be called beforePython initialization nor after Python finalization.

intPyConfig_GetInt(constchar*name,int*value):

Similar toPyConfig_Get(), but get the value as an integer.

• Set*value and return0 success.
• Set an exception and return-1 on error.

PyObject*PyConfig_Names(void):

Get all configuration option names as afrozenset.

Set an exception and returnNULL on error.

The caller must hold the GIL.

PyObject*PyConfig_Set(constchar*name,PyObject*value):

Set the current runtime value of a configuration option.

• Raise aValueError if there is no optionname.
• Raise aValueError ifvalue is an invalid value.
• Raise aValueError if the option is read-only: cannot be set.
• Raise aTypeError ifvalue has not the proper type.

Read-only configuration options cannot be set.

The caller must hold the GIL. The function cannot be called beforePython initialization nor after Python finalization.

Stability

The behavior of options, the default option values, and the Pythonbehavior can change at each Python version: they are not “stable”.

Moreover, configuration options can be added, deprecated and removedfollowing the usualPEP 387 deprecation process.

Interaction with the PyPreConfig and PyConfig APIs

The lower levelPEP 587PyPreConfig andPyConfig APIs remainavailable and fully supported. As noted in the Abstract, they remain thepreferred approach for embedding use cases that are aiming to closelyemulate the behaviour of the full CPython CLI, rather than just making aPython runtime available as part of a larger application.

ThePyPreConfig APIs may be used in combination with theinitialization API in this PEP. In such cases, the read-only vsread/write restrictions for preconfiguration settings apply toPyInitConfig_SetInt in addition toPyConfig_Set once theinterpreter has been preconfigured (specifically, onlyuse_environment may be updated, attempting to update any of theother preconfiguration variables will report an error).

Initialize Python

Example initializing Python, set configuration options of various types,return-1 on error:

intinit_python(void){PyInitConfig*config=PyInitConfig_Create();if(config==NULL){printf("PYTHON INIT ERROR: memory allocation failed\n");return-1;}// Set an integer (dev mode)if(PyInitConfig_SetInt(config,"dev_mode",1)<0){gotoerror;}// Set a list of UTF-8 strings (argv)char*argv[]={"my_program","-c","pass"};if(PyInitConfig_SetStrList(config,"argv",Py_ARRAY_LENGTH(argv),argv)<0){gotoerror;}// Set a UTF-8 string (program name)if(PyInitConfig_SetStr(config,"program_name",L"my_program")<0){gotoerror;}// Initialize Python with the configurationif(Py_InitializeFromInitConfig(config)<0){gotoerror;}PyInitConfig_Free(config);return0;error:// Display the error messageconstchar*err_msg;(void)PyInitConfig_GetError(config,&err_msg);printf("PYTHON INIT ERROR: %s\n",err_msg);PyInitConfig_Free(config);return-1;}

Increase initialization bytes_warning option

Example increasing thebytes_warning option of an initializationconfiguration:

intconfig_bytes_warning(PyInitConfig*config){int64_tbytes_warning;if(PyInitConfig_GetInt(config,"bytes_warning",&bytes_warning)){return-1;}bytes_warning+=1;if(PyInitConfig_SetInt(config,"bytes_warning",bytes_warning)){return-1;}return0;}

Get the runtime verbose option

Example getting the current runtime value of the configuration optionverbose:

intget_verbose(void){intverbose;if(PyConfig_GetInt("verbose",&verbose)<0){// Silently ignore the errorPyErr_Clear();return-1;}returnverbose;}

On error, the function silently ignores the error and returns-1. Inpractice, getting theverbose option cannot fail, unless a futurePython version removes the option.

Configuration as text

It was proposed to provide the configuration as text to make the APIcompatible with the stable ABI and to allow custom options.

Example:

# integerbytes_warning=2# stringfilesystem_encoding="utf8"# comment# list of stringsargv=['python','-c','code']

The API would take the configuration as a string, not as a file. Examplewith a hypotheticalPyInit_SetConfig() function:

voidstable_abi_init_demo(intset_path){PyInit_SetConfig("isolated = 1\n""argv = ['python', '-c', 'code']\n""filesystem_encoding = 'utf-8'\n");if(set_path){PyInit_SetConfig("pythonpath = '/my/path'");}}

The example ignores error handling to make it easier to read.

The problem is that generating such configuration text requires addingquotes to strings and to escape quotes in strings. Formatting an arrayof strings becomes non-trivial.

Providing an API to format a string or an array of strings is not reallyworth it, whereas Python can provide directly an API to set aconfiguration option where the value is passed directly as a string oran array of strings. It avoids giving special meaning to somecharacters, such as newline characters, which would have to be escaped.

Refer to an option with an integer

Using strings to refer to a configuration option requires comparingstrings which can be slower than comparing integers.

Use integers, similar to type “slots” such asPy_tp_doc, to refer toa configuration option. Theconstchar*name parameter is replacedwithintoption.

Accepting custom options is more likely to cause conflicts when usingintegers, since it’s harder to maintain “namespaces” (ranges) forinteger options. Using strings, a simple prefix with a colon separatorcan be used.

Integers also requires maintaining a list of integer constants and somake the C API and the Python API larger.

Python 3.13 only has around 62 configuration options, and so performanceis not really a blocker issue. If better performance is needed later, ahash table can be used to get an option by its name.

If getting a configuration option is used in hot code, the value can beread once and cached. By the way, most configuration options cannot bechanged at runtime.

Multi-phase initialization (similar to PEP 432)

Eric Snow expressed concernsthat this proposal might reinforce with embedders the idea thatinitialization is a single monolithic step. He argued that initializationinvolves 5 distinct phases and even suggested that the API shouldreflect this explicitly. Eric proposed that, at the very least, theimplementation of initialization should reflect the phases, in partfor improved code health. Overall, his explanation has somesimilarities withPEP 432 andPEP 587.

Another of Eric’s key points relevant to this PEP was that, ideally,the config passed toPy_InitializeFromConfig() should be completebefore that function is called, whereas currently initializationactually modifies the config.

While Eric wasn’t necessarily suggesting an alternative to PEP 741,any proposal to add a granular initialization API around phases iseffectively the opposite of what this PEP is trying to accomplish.Such API is more complicated, it requires adding new public structuresand new public functions. It makes the Python initialization morecomplicated, rather than this PEP tries to unify existing APIs and makethem simpler (the opposite). Having multiple structures for similarpurpose can lead to duplicate members, similar issue than duplicatedmembers between existingPyPreConfig andPyConfig structures.

Locale encoding and wide strings

Accepting strings encoded to the locale encoding and accepting widestrings (wchar_t*) in thePyInitConfig API was deferred to keepthePyInitConfig API simple and avoid the complexity of the Pythonpreinitialization. These features are also mostly needed when emulatingthe full CPython CLI behaviour, and hence better served by the lowerlevelPEP 587 API.