PEP 286 – Enhanced Argument Tuples
- martin at v.loewis.de (Martin von Löwis)
- Standards Track
PyArg_ParseTuple is confronted with difficult memory management if
an argument converter creates new memory. To deal with these
cases, a specialized argument type is proposed.
Further exploration of the concepts covered in this PEP has been deferred for lack of a current champion interested in promoting the goals of the PEP and collecting and incorporating feedback, and with sufficient available time to do so effectively.
The resolution of this PEP may also be affected by the resolution of PEP 426, which proposes the use of a preprocessing step to generate some aspects of C API interface code.
Today, argument tuples keep references to the function arguments, which are guaranteed to live as long as the argument tuple exists which is at least as long as the function call is being executed.
In some cases, parsing an argument will allocate new memory, which is then to be released by the caller. This has two problems:
- In case of failure, the application cannot know what memory to
release; most callers don’t even know that they have the
responsibility to release that memory. Example for this are
Nconverter (bug #416288 ) and the
es#converter (bug #501716 ).
- Even for successful argument parsing, it is still inconvenient
for the caller to be responsible for releasing the memory. In
some cases, this is unnecessarily inefficient. For example,
esconverter copies the conversion result into memory, even though there already is a string object that has the right contents.
A new type ‘argument tuple’ is introduced. This type derives from
tuple, adding an
__dict__ member (at
tp_dictoffset -4). Instances
of this type might get the following attributes:
- ‘failobjects’, a list of objects which need to be deallocated in case of success
- ‘okobjects’, a list of object which will be released when the argument tuple is released
To manage this type, the following functions will be added, and
used appropriately in
When argument parsing fails, all fail objects will be released
Py_DECREF, and all fail memory will be released through
PyMem_Free. If parsing succeeds, the references to the fail
objects and fail memory are dropped, without releasing anything.
When the argument tuple is released, all ok objects and memory will be released.
If those functions are called with an object of a different type, a warning is issued and no further action is taken; usage of the affected converters without using argument tuples is deprecated.
The following converters will add fail memory and fail objects:
et# (unless memory is passed into the converter)
To simplify Unicode conversion, the
e* converters are duplicated
E* converters (
Et#). The usage of the
converters is identical to that of the
e* converters, except that
the application will not need to manage the resulting memory.
This will be implemented through registration of Ok objects with
the argument tuple. The
e* converters are deprecated.
- infrequent memory leak in pyexpat (http://bugs.python.org/issue416288)
- “es#” parser marker leaks memory (http://bugs.python.org/issue501716)
This document has been placed in the public domain.
Last modified: 2022-01-21 11:03:51 GMT