Skip to content

Commit

Permalink
GH-117759: Document incremental GC (GH-123266)
Browse files Browse the repository at this point in the history
* Update what's new

* Update gc module docs and fix inconsistency in gc.get_objects
  • Loading branch information
markshannon authored Aug 27, 2024
1 parent 460ee5b commit f49a916
Show file tree
Hide file tree
Showing 3 changed files with 69 additions and 26 deletions.
55 changes: 40 additions & 15 deletions Doc/library/gc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,18 @@ The :mod:`gc` module provides the following functions:

.. function:: collect(generation=2)

With no arguments, run a full collection. The optional argument *generation*
Perform a collection. The optional argument *generation*
may be an integer specifying which generation to collect (from 0 to 2). A
:exc:`ValueError` is raised if the generation number is invalid. The sum of
:exc:`ValueError` is raised if the generation number is invalid. The sum of
collected objects and uncollectable objects is returned.

Calling ``gc.collect(0)`` will perform a GC collection on the young generation.

Calling ``gc.collect(1)`` will perform a GC collection on the young generation
and an increment of the old generation.

Calling ``gc.collect(2)`` or ``gc.collect()`` performs a full collection

The free lists maintained for a number of built-in types are cleared
whenever a full collection or collection of the highest generation (2)
is run. Not all items in some free lists may be freed due to the
Expand All @@ -53,6 +60,9 @@ The :mod:`gc` module provides the following functions:
The effect of calling ``gc.collect()`` while the interpreter is already
performing a collection is undefined.

.. versionchanged:: 3.13
``generation=1`` performs an increment of collection.


.. function:: set_debug(flags)

Expand All @@ -68,13 +78,20 @@ The :mod:`gc` module provides the following functions:

.. function:: get_objects(generation=None)


Returns a list of all objects tracked by the collector, excluding the list
returned. If *generation* is not ``None``, return only the objects tracked by
the collector that are in that generation.
returned. If *generation* is not ``None``, return only the objects as follows:

* 0: All objects in the young generation
* 1: No objects, as there is no generation 1 (as of Python 3.13)
* 2: All objects in the old generation

.. versionchanged:: 3.8
New *generation* parameter.

.. versionchanged:: 3.13
Generation 1 is removed

.. audit-event:: gc.get_objects generation gc.get_objects

.. function:: get_stats()
Expand All @@ -101,19 +118,27 @@ The :mod:`gc` module provides the following functions:
Set the garbage collection thresholds (the collection frequency). Setting
*threshold0* to zero disables collection.

The GC classifies objects into three generations depending on how many
collection sweeps they have survived. New objects are placed in the youngest
generation (generation ``0``). If an object survives a collection it is moved
into the next older generation. Since generation ``2`` is the oldest
generation, objects in that generation remain there after a collection. In
order to decide when to run, the collector keeps track of the number object
The GC classifies objects into two generations depending on whether they have
survived a collection. New objects are placed in the young generation. If an
object survives a collection it is moved into the old generation.

In order to decide when to run, the collector keeps track of the number of object
allocations and deallocations since the last collection. When the number of
allocations minus the number of deallocations exceeds *threshold0*, collection
starts. Initially only generation ``0`` is examined. If generation ``0`` has
been examined more than *threshold1* times since generation ``1`` has been
examined, then generation ``1`` is examined as well.
With the third generation, things are a bit more complicated,
see `Collecting the oldest generation <https://devguide.python.org/garbage_collector/#collecting-the-oldest-generation>`_ for more information.
starts. For each collection, all the objects in the young generation and some
fraction of the old generation is collected.

The fraction of the old generation that is collected is **inversely** proportional
to *threshold1*. The larger *threshold1* is, the slower objects in the old generation
are collected.
For the default value of 10, 1% of the old generation is scanned during each collection.

*threshold2* is ignored.

See `Garbage collector design <https://devguide.python.org/garbage_collector>`_ for more information.

.. versionchanged:: 3.13
*threshold2* is ignored


.. function:: get_count()
Expand Down
13 changes: 13 additions & 0 deletions Doc/whatsnew/3.13.rst
Original file line number Diff line number Diff line change
Expand Up @@ -495,6 +495,19 @@ Incremental garbage collection
The cycle garbage collector is now incremental.
This means that maximum pause times are reduced
by an order of magnitude or more for larger heaps.

There are now only two generations: young and old.
When :func:`gc.collect` is not called directly, the
GC is invoked a little less frequently. When invoked, it
collects the young generation and an increment of the
old generation, instead of collecting one or more generations.

The behavior of :func:`!gc.collect` changes slightly:

* ``gc.collect(1)``: Performs an increment of GC,
rather than collecting generation 1.
* Other calls to :func:`!gc.collect` are unchanged.

(Contributed by Mark Shannon in :gh:`108362`.)


Expand Down
27 changes: 16 additions & 11 deletions Python/gc.c
Original file line number Diff line number Diff line change
Expand Up @@ -1712,20 +1712,25 @@ _PyGC_GetObjects(PyInterpreterState *interp, int generation)
GCState *gcstate = &interp->gc;

PyObject *result = PyList_New(0);
if (result == NULL) {
return NULL;
/* Generation:
* -1: Return all objects
* 0: All young objects
* 1: No objects
* 2: All old objects
*/
if (result == NULL || generation == 1) {
return result;
}

if (generation == -1) {
/* If generation is -1, get all objects from all generations */
for (int i = 0; i < NUM_GENERATIONS; i++) {
if (append_objects(result, GEN_HEAD(gcstate, i))) {
goto error;
}
if (generation <= 0) {
if (append_objects(result, &gcstate->young.head)) {
goto error;
}
}
else {
if (append_objects(result, GEN_HEAD(gcstate, generation))) {
if (generation != 0) {
if (append_objects(result, &gcstate->old[0].head)) {
goto error;
}
if (append_objects(result, &gcstate->old[1].head)) {
goto error;
}
}
Expand Down

0 comments on commit f49a916

Please sign in to comment.