I propose to investigate an incompatible C API change: make the PyTypeObject and PyHeapTypeObject structures opaque, remove their members from the public C API (move them to the internal C API). We have to investigate how it's outside in 3rd party C extensions (ex: Cython, pybind11, etc.), and design a smooth migration plan.
The PyTypeObject structure is exposed as part of the public Python C API. For example, Py_TYPE(obj)->tp_name directly gets a type name (as a UTF-8 encoded byte string, char*).
The PyTypeObject members are NOT part of the limited C API (PEP 384).
In Python 3.9 (2020), I reworked the C API to avoid accessing directly PyTypeObject members at the ABI level: issue #84351. For example, Python 3.8 implements PyObject_IS_GC() as a macro:
/* Test if an object has a GC head */
#define PyObject_IS_GC(o) \
(PyType_IS_GC(Py_TYPE(o)) \
&& (Py_TYPE(o)->tp_is_gc == NULL || Py_TYPE(o)->tp_is_gc(o)))whereas Python 3.9 only provides an opaque function call:
/* Test if an object implements the garbage collector protocol */
PyAPI_FUNC(int) PyObject_IS_GC(PyObject *obj);
At the ABI level, the direct access to the PyTypeObject.tp_is_gc member became an opaque function call.
Changing PyTypeObject API and ABI caused a lot of troubles in the past. Example:
For many years, there is a work-in-progress to convert all Python built-in types and types of stdlib extensions from static types to heap types. See for example issue #84258 and PEP 630.
The API and ABI for heap type was also enhanced over the years. Examples:
In the past, other structure members were removed:
- PyInterpreterState: Python 3.8
- PyGC_Head: Python 3.9
- PyFrameObject: Python 3.11
The work was also prepared for:
I propose to investigate an incompatible C API change: make the
PyTypeObjectandPyHeapTypeObjectstructures opaque, remove their members from the public C API (move them to the internal C API). We have to investigate how it's outside in 3rd party C extensions (ex: Cython, pybind11, etc.), and design a smooth migration plan.The PyTypeObject structure is exposed as part of the public Python C API. For example,
Py_TYPE(obj)->tp_namedirectly gets a type name (as a UTF-8 encoded byte string,char*).The PyTypeObject members are NOT part of the limited C API (PEP 384).
In Python 3.9 (2020), I reworked the C API to avoid accessing directly PyTypeObject members at the ABI level: issue #84351. For example, Python 3.8 implements
PyObject_IS_GC()as a macro:whereas Python 3.9 only provides an opaque function call:
At the ABI level, the direct access to the
PyTypeObject.tp_is_gcmember became an opaque function call.Changing PyTypeObject API and ABI caused a lot of troubles in the past. Example:
PyTypeObject.tp_printremoval -- related toPyTypeObject.tp_vectorcall_offsetadditionPyTypeObject.tp_finalize(PEP 442). IsPy_TPFLAGS_HAVE_FINALIZEneeded for ABI compatibility? See also the python-dev thread.Py_TPFLAGS_HAVE_VERSION_TAGflag (ABI compatibility)For many years, there is a work-in-progress to convert all Python built-in types and types of stdlib extensions from static types to heap types. See for example issue #84258 and PEP 630.
The API and ABI for heap type was also enhanced over the years. Examples:
PyType_FromMetaclass()In the past, other structure members were removed:
The work was also prepared for: