Object Life Cycle

This section explains how a type’s slots relate to each other throughout thelife of an object. It is not intended to be a complete canonical reference forthe slots; instead, refer to the slot-specific documentation inType Object Structures for details about a particular slot.

Life Events

The figure below illustrates the order of events that can occur throughout anobject’s life. An arrow fromA toB indicates that eventB can occurafter eventA has occurred, with the arrow’s label indicating the conditionthat must be true forB to occur afterA.

Life Eventstp_newtp_newstart->tp_new    type call  tp_alloctp_alloctp_new->tp_alloc  direct call  tp_inittp_inittp_new->tp_initreachablereachabletp_init->reachablereachable->tp_inittp_traversetp_traversereachable->tp_traverse  not in a    cyclic    isolate  reachable->tp_traverse  periodic    cyclic isolate     detection  finalized?marked asfinalized?reachable->finalized?  no refs  tp_finalizetp_finalizereachable->tp_finalize  resurrected    (maybe remove    finalized mark)  uncollectableuncollectable(leaked)reachable->uncollectable  cyclic    isolate    (no GC    support)  tp_dealloctp_deallocreachable->tp_dealloc  no refstp_traverse->finalized?  cyclic    isolate  finalized?->tp_finalize  no (mark    as finalized)  tp_cleartp_clearfinalized?->tp_clear  yes  tp_finalize->tp_clear  no refs or     cyclic isolate  tp_finalize->tp_dealloc  recommended  call (see  explanation)tp_finalize->tp_dealloc   no refs  tp_clear->uncollectable  cyclic    isolate  tp_clear->tp_dealloc  no refs  tp_freetp_freetp_dealloc->tp_free    direct call  

Explanation:

  • When a new object is constructed by calling its type:

    1. tp_new is called to create a new object.

    2. tp_alloc is directly called bytp_new to allocate the memory for the newobject.

    3. tp_init initializes the newly created object.tp_init can be called again to re-initialize an object, ifdesired. Thetp_init call can also be skipped entirely,for example by Python code calling__new__().

  • Aftertp_init completes, the object is ready to use.

  • Some time after the last reference to an object is removed:

    1. If an object is not marked asfinalized, it might be finalized bymarking it asfinalized and calling itstp_finalize function. Python doesnot finalize an object when the last reference to it is deleted; usePyObject_CallFinalizerFromDealloc() to ensure thattp_finalize is always called.

    2. If the object is marked as finalized,tp_clear might be called by the garbage collectorto clear references held by the object. It isnot called when theobject’s reference count reaches zero.

    3. tp_dealloc is called to destroy the object.To avoid code duplication,tp_dealloc typicallycalls intotp_clear to free up the object’sreferences.

    4. Whentp_dealloc finishes object destruction,it directly callstp_free (usually set toPyObject_Free() orPyObject_GC_Del() automatically asappropriate for the type) to deallocate the memory.

  • Thetp_finalize function is permitted to add areference to the object if desired. If it does, the object isresurrected, preventing its pending destruction. (Onlytp_finalize is allowed to resurrect an object;tp_clear andtp_dealloc cannot without calling intotp_finalize.) Resurrecting an object mayor may not cause the object’sfinalized mark to be removed. Currently,Python does not remove thefinalized mark from a resurrected object ifit supports garbage collection (i.e., thePy_TPFLAGS_HAVE_GCflag is set) but does remove the mark if the object does not supportgarbage collection; either or both of these behaviors may change in thefuture.

  • tp_dealloc can optionally calltp_finalize viaPyObject_CallFinalizerFromDealloc() if it wishes to reuse thatcode to help with object destruction. This is recommended because itguarantees thattp_finalize is always called beforedestruction. See thetp_dealloc documentationfor example code.

  • If the object is a member of acyclic isolate and eithertp_clear fails to break the reference cycle orthe cyclic isolate is not detected (perhapsgc.disable() was called,or thePy_TPFLAGS_HAVE_GC flag was erroneously omitted in oneof the involved types), the objects remain indefinitely uncollectable(they “leak”). Seegc.garbage.

If the object is marked as supporting garbage collection (thePy_TPFLAGS_HAVE_GC flag is set intp_flags), the following events are also possible:

  • The garbage collector occasionally callstp_traverse to identifycyclic isolates.

  • When the garbage collector discovers acyclic isolate, itfinalizes one of the objects in the group by marking it asfinalized andcalling itstp_finalize function, if it has one.This repeats until the cyclic isolate doesn’t exist or all of the objectshave been finalized.

  • tp_finalize is permitted to resurrect the objectby adding a reference from outside thecyclic isolate. The newreference causes the group of objects to no longer form a cyclic isolate(the reference cycle may still exist, but if it does the objects are nolonger isolated).

  • When the garbage collector discovers acyclic isolate and all ofthe objects in the group have already been marked asfinalized, thegarbage collector clears one or more of the uncleared objects in the group(possibly concurrently) by calling each’stp_clear function. This repeats as long as thecyclic isolate still exists and not all of the objects have been cleared.

Cyclic Isolate Destruction

Listed below are the stages of life of a hypotheticalcyclic isolatethat continues to exist after each member object is finalized or cleared. Itis a memory leak if a cyclic isolate progresses through all of these stages; it shouldvanish once all objects are cleared, if not sooner. A cyclic isolate canvanish either because the reference cycle is broken or because the objects areno longer isolated due to finalizer resurrection (seetp_finalize).

  1. Reachable (not yet a cyclic isolate): All objects are in their normal,reachable state. A reference cycle could exist, but an external referencemeans the objects are not yet isolated.

  2. Unreachable but consistent: The final reference from outside the cyclicgroup of objects has been removed, causing the objects to become isolated(thus a cyclic isolate is born). None of the group’s objects have beenfinalized or cleared yet. The cyclic isolate remains at this stage untilsome future run of the garbage collector (not necessarily the next runbecause the next run might not scan every object).

  3. Mix of finalized and not finalized: Objects in a cyclic isolate arefinalized one at a time, which means that there is a period of time when thecyclic isolate is composed of a mix of finalized and non-finalized objects.Finalization order is unspecified, so it can appear random. A finalizedobject must behave in a sane manner when non-finalized objects interact withit, and a non-finalized object must be able to tolerate the finalization ofan arbitrary subset of its referents.

  4. All finalized: All objects in a cyclic isolate are finalized before anyof them are cleared.

  5. Mix of finalized and cleared: The objects can be cleared serially orconcurrently (but with theGIL held); either way, some will finishbefore others. A finalized object must be able to tolerate the clearing ofa subset of its referents.PEP 442 calls this stage “cyclic trash”.

  6. Leaked: If a cyclic isolate still exists after all objects in the grouphave been finalized and cleared, then the objects remain indefinitelyuncollectable (seegc.garbage). It is a bug if a cyclic isolatereaches this stage—it means thetp_clear methodsof the participating objects have failed to break the reference cycle asrequired.

Iftp_clear did not exist, then Python would have noway to safely break a reference cycle. Simply destroying an object in a cyclicisolate would result in a dangling pointer, triggering undefined behavior whenan object referencing the destroyed object is itself destroyed. The clearingstep makes object destruction a two-phase process: firsttp_clear is called to partially destroy the objectsenough to detangle them from each other, thentp_dealloc is called to complete the destruction.

Unlike clearing, finalization is not a phase of destruction. A finalizedobject must still behave properly by continuing to fulfill its designcontracts. An object’s finalizer is allowed to execute arbitrary Python code,and is even allowed to prevent the impending destruction by adding a reference.The finalizer is only related to destruction by call order—if it runs, it runsbefore destruction, which starts withtp_clear (ifcalled) and concludes withtp_dealloc.

The finalization step is not necessary to safely reclaim the objects in acyclic isolate, but its existence makes it easier to design types that behavein a sane manner when objects are cleared. Clearing an object mightnecessarily leave it in a broken, partially destroyed state—it might beunsafe to call any of the cleared object’s methods or access any of itsattributes. With finalization, only finalized objects can possibly interactwith cleared objects; non-finalized objects are guaranteed to interact withonly non-cleared (but potentially finalized) objects.

To summarize the possible interactions:

  • A non-finalized object might have references to or from non-finalized andfinalized objects, but not to or from cleared objects.

  • A finalized object might have references to or from non-finalized, finalized,and cleared objects.

  • A cleared object might have references to or from finalized and clearedobjects, but not to or from non-finalized objects.

Without any reference cycles, an object can be simply destroyed once its lastreference is deleted; the finalization and clearing steps are not necessary tosafely reclaim unused objects. However, it can be useful to automatically calltp_finalize andtp_clearbefore destruction anyway because type design is simplified when all objectsalways experience the same series of events regardless of whether theyparticipated in a cyclic isolate. Python currently only callstp_finalize andtp_clear asneeded to destroy a cyclic isolate; this may change in a future version.

Functions

To allocate and free memory, seeAllocating Objects on the Heap.

voidPyObject_CallFinalizer(PyObject*op)

Finalizes the object as described intp_finalize.Call this function (orPyObject_CallFinalizerFromDealloc()) insteadof callingtp_finalize directly because thisfunction may deduplicate multiple calls totp_finalize.Currently, calls are only deduplicated if the type supports garbagecollection (i.e., thePy_TPFLAGS_HAVE_GC flag is set); this maychange in the future.

intPyObject_CallFinalizerFromDealloc(PyObject*op)

Same asPyObject_CallFinalizer() but meant to be called at thebeginning of the object’s destructor (tp_dealloc).There must not be any references to the object. If the object’s finalizerresurrects the object, this function returns -1; no further destructionshould happen. Otherwise, this function returns 0 and destruction cancontinue normally.

See also

tp_dealloc for example code.