Abstract

PyArg_ParseTuple is confronted with difficult memory management ifan argument converter creates new memory. To deal with thesecases, a specialized argument type is proposed.

PEP Deferral

Further exploration of the concepts covered in this PEP has been deferredfor lack of a current champion interested in promoting the goals of thePEP and collecting and incorporating feedback, and with sufficientavailable time to do so effectively.

The resolution of this PEP may also be affected by the resolution ofPEP 426, which proposes the use of a preprocessing step to generatesome aspects of C API interface code.

Problem description

Today, argument tuples keep references to the function arguments,which are guaranteed to live as long as the argument tuple existswhich is at least as long as the function call is being executed.

In some cases, parsing an argument will allocate new memory, whichis then to be released by the caller. This has two problems:

1. In case of failure, the application cannot know what memory torelease; most callers don’t even know that they have theresponsibility to release that memory. Example for this aretheN converter (bug #416288[1]) and thees# converter (bug#501716[2]).
2. Even for successful argument parsing, it is still inconvenientfor the caller to be responsible for releasing the memory. Insome cases, this is unnecessarily inefficient. For example,thees converter copies the conversion result into memory, eventhough there already is a string object that has the rightcontents.

Proposed solution

A new type ‘argument tuple’ is introduced. This type derives fromtuple, adding an__dict__ member (attp_dictoffset -4). Instancesof this type might get the following attributes:

• ‘failobjects’, a list of objects which need to be deallocatedin case of success
• ‘okobjects’, a list of object which will be released when theargument tuple is released

To manage this type, the following functions will be added, andused appropriately inceval.c andgetargs.c:

• PyArgTuple_New(int);
• PyArgTuple_AddFailObject(PyObject*,PyObject*);
• PyArgTuple_AddFailMemory(PyObject*,void*);
• PyArgTuple_AddOkObject(PyObject*,PyObject*);
• PyArgTuple_AddOkMemory(PyObject*,void*);
• PyArgTuple_ClearFailed(PyObject*);

When argument parsing fails, all fail objects will be releasedthroughPy_DECREF, and all fail memory will be released throughPyMem_Free. If parsing succeeds, the references to the failobjects and fail memory are dropped, without releasing anything.

When the argument tuple is released, all ok objects and memorywill be released.

If those functions are called with an object of a different type,a warning is issued and no further action is taken; usage of theaffected converters without using argument tuples is deprecated.

Affected converters

The following converters will add fail memory and fail objects:N,es,et,es#,et# (unless memory is passed into the converter)

New converters

To simplify Unicode conversion, thee* converters are duplicatedasE* converters (Es,Et,Es#,Et#). The usage of theE*converters is identical to that of thee* converters, except thatthe application will not need to manage the resulting memory.This will be implemented through registration of Ok objects withthe argument tuple. Thee* converters are deprecated.

References

Copyright

This document has been placed in the public domain.