GH-120804: add docs for removal for asyncio child watchers (#120895)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
This commit is contained in:
@@ -77,11 +77,6 @@ Subprocess Support on Windows
|
|||||||
On Windows, the default event loop :class:`ProactorEventLoop` supports
|
On Windows, the default event loop :class:`ProactorEventLoop` supports
|
||||||
subprocesses, whereas :class:`SelectorEventLoop` does not.
|
subprocesses, whereas :class:`SelectorEventLoop` does not.
|
||||||
|
|
||||||
The :meth:`policy.set_child_watcher()
|
|
||||||
<AbstractEventLoopPolicy.set_child_watcher>` function is also
|
|
||||||
not supported, as :class:`ProactorEventLoop` has a different mechanism
|
|
||||||
to watch child processes.
|
|
||||||
|
|
||||||
|
|
||||||
macOS
|
macOS
|
||||||
=====
|
=====
|
||||||
|
|||||||
@@ -79,25 +79,6 @@ The abstract event loop policy base class is defined as follows:
|
|||||||
|
|
||||||
This method should never return ``None``.
|
This method should never return ``None``.
|
||||||
|
|
||||||
.. method:: get_child_watcher()
|
|
||||||
|
|
||||||
Get a child process watcher object.
|
|
||||||
|
|
||||||
Return a watcher object implementing the
|
|
||||||
:class:`AbstractChildWatcher` interface.
|
|
||||||
|
|
||||||
This function is Unix specific.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
.. method:: set_child_watcher(watcher)
|
|
||||||
|
|
||||||
Set the current child process watcher to *watcher*.
|
|
||||||
|
|
||||||
This function is Unix specific.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
|
|
||||||
.. _asyncio-policy-builtin:
|
.. _asyncio-policy-builtin:
|
||||||
|
|
||||||
@@ -139,172 +120,6 @@ asyncio ships with the following built-in policies:
|
|||||||
.. availability:: Windows.
|
.. availability:: Windows.
|
||||||
|
|
||||||
|
|
||||||
.. _asyncio-watchers:
|
|
||||||
|
|
||||||
Process Watchers
|
|
||||||
================
|
|
||||||
|
|
||||||
A process watcher allows customization of how an event loop monitors
|
|
||||||
child processes on Unix. Specifically, the event loop needs to know
|
|
||||||
when a child process has exited.
|
|
||||||
|
|
||||||
In asyncio, child processes are created with
|
|
||||||
:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec`
|
|
||||||
functions.
|
|
||||||
|
|
||||||
asyncio defines the :class:`AbstractChildWatcher` abstract base class, which child
|
|
||||||
watchers should implement, and has four different implementations:
|
|
||||||
:class:`ThreadedChildWatcher` (configured to be used by default),
|
|
||||||
:class:`MultiLoopChildWatcher`, :class:`SafeChildWatcher`, and
|
|
||||||
:class:`FastChildWatcher`.
|
|
||||||
|
|
||||||
See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>`
|
|
||||||
section.
|
|
||||||
|
|
||||||
The following two functions can be used to customize the child process watcher
|
|
||||||
implementation used by the asyncio event loop:
|
|
||||||
|
|
||||||
.. function:: get_child_watcher()
|
|
||||||
|
|
||||||
Return the current child watcher for the current policy.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
.. function:: set_child_watcher(watcher)
|
|
||||||
|
|
||||||
Set the current child watcher to *watcher* for the current
|
|
||||||
policy. *watcher* must implement methods defined in the
|
|
||||||
:class:`AbstractChildWatcher` base class.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Third-party event loops implementations might not support
|
|
||||||
custom child watchers. For such event loops, using
|
|
||||||
:func:`set_child_watcher` might be prohibited or have no effect.
|
|
||||||
|
|
||||||
.. class:: AbstractChildWatcher
|
|
||||||
|
|
||||||
.. method:: add_child_handler(pid, callback, *args)
|
|
||||||
|
|
||||||
Register a new child handler.
|
|
||||||
|
|
||||||
Arrange for ``callback(pid, returncode, *args)`` to be called
|
|
||||||
when a process with PID equal to *pid* terminates. Specifying
|
|
||||||
another callback for the same process replaces the previous
|
|
||||||
handler.
|
|
||||||
|
|
||||||
The *callback* callable must be thread-safe.
|
|
||||||
|
|
||||||
.. method:: remove_child_handler(pid)
|
|
||||||
|
|
||||||
Removes the handler for process with PID equal to *pid*.
|
|
||||||
|
|
||||||
The function returns ``True`` if the handler was successfully
|
|
||||||
removed, ``False`` if there was nothing to remove.
|
|
||||||
|
|
||||||
.. method:: attach_loop(loop)
|
|
||||||
|
|
||||||
Attach the watcher to an event loop.
|
|
||||||
|
|
||||||
If the watcher was previously attached to an event loop, then
|
|
||||||
it is first detached before attaching to the new loop.
|
|
||||||
|
|
||||||
Note: loop may be ``None``.
|
|
||||||
|
|
||||||
.. method:: is_active()
|
|
||||||
|
|
||||||
Return ``True`` if the watcher is ready to use.
|
|
||||||
|
|
||||||
Spawning a subprocess with *inactive* current child watcher raises
|
|
||||||
:exc:`RuntimeError`.
|
|
||||||
|
|
||||||
.. versionadded:: 3.8
|
|
||||||
|
|
||||||
.. method:: close()
|
|
||||||
|
|
||||||
Close the watcher.
|
|
||||||
|
|
||||||
This method has to be called to ensure that underlying
|
|
||||||
resources are cleaned-up.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
|
|
||||||
.. class:: ThreadedChildWatcher
|
|
||||||
|
|
||||||
This implementation starts a new waiting thread for every subprocess spawn.
|
|
||||||
|
|
||||||
It works reliably even when the asyncio event loop is run in a non-main OS thread.
|
|
||||||
|
|
||||||
There is no noticeable overhead when handling a big number of children (*O*\ (1) each
|
|
||||||
time a child terminates), but starting a thread per process requires extra memory.
|
|
||||||
|
|
||||||
This watcher is used by default.
|
|
||||||
|
|
||||||
.. versionadded:: 3.8
|
|
||||||
|
|
||||||
.. class:: MultiLoopChildWatcher
|
|
||||||
|
|
||||||
This implementation registers a :py:data:`SIGCHLD` signal handler on
|
|
||||||
instantiation. That can break third-party code that installs a custom handler for
|
|
||||||
:py:data:`SIGCHLD` signal.
|
|
||||||
|
|
||||||
The watcher avoids disrupting other code spawning processes
|
|
||||||
by polling every process explicitly on a :py:data:`SIGCHLD` signal.
|
|
||||||
|
|
||||||
There is no limitation for running subprocesses from different threads once the
|
|
||||||
watcher is installed.
|
|
||||||
|
|
||||||
The solution is safe but it has a significant overhead when
|
|
||||||
handling a big number of processes (*O*\ (*n*) each time a
|
|
||||||
:py:data:`SIGCHLD` is received).
|
|
||||||
|
|
||||||
.. versionadded:: 3.8
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
.. class:: SafeChildWatcher
|
|
||||||
|
|
||||||
This implementation uses active event loop from the main thread to handle
|
|
||||||
:py:data:`SIGCHLD` signal. If the main thread has no running event loop another
|
|
||||||
thread cannot spawn a subprocess (:exc:`RuntimeError` is raised).
|
|
||||||
|
|
||||||
The watcher avoids disrupting other code spawning processes
|
|
||||||
by polling every process explicitly on a :py:data:`SIGCHLD` signal.
|
|
||||||
|
|
||||||
This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O*\ (*n*)
|
|
||||||
complexity but requires a running event loop in the main thread to work.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
.. class:: FastChildWatcher
|
|
||||||
|
|
||||||
This implementation reaps every terminated processes by calling
|
|
||||||
``os.waitpid(-1)`` directly, possibly breaking other code spawning
|
|
||||||
processes and waiting for their termination.
|
|
||||||
|
|
||||||
There is no noticeable overhead when handling a big number of
|
|
||||||
children (*O*\ (1) each time a child terminates).
|
|
||||||
|
|
||||||
This solution requires a running event loop in the main thread to work, as
|
|
||||||
:class:`SafeChildWatcher`.
|
|
||||||
|
|
||||||
.. deprecated:: 3.12
|
|
||||||
|
|
||||||
.. class:: PidfdChildWatcher
|
|
||||||
|
|
||||||
This implementation polls process file descriptors (pidfds) to await child
|
|
||||||
process termination. In some respects, :class:`PidfdChildWatcher` is a
|
|
||||||
"Goldilocks" child watcher implementation. It doesn't require signals or
|
|
||||||
threads, doesn't interfere with any processes launched outside the event
|
|
||||||
loop, and scales linearly with the number of subprocesses launched by the
|
|
||||||
event loop. The main disadvantage is that pidfds are specific to Linux, and
|
|
||||||
only work on recent (5.3+) kernels.
|
|
||||||
|
|
||||||
.. versionadded:: 3.9
|
|
||||||
|
|
||||||
|
|
||||||
.. _asyncio-custom-policies:
|
.. _asyncio-custom-policies:
|
||||||
|
|
||||||
Custom Policies
|
Custom Policies
|
||||||
|
|||||||
@@ -316,18 +316,6 @@ default.
|
|||||||
On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default),
|
On Windows subprocesses are provided by :class:`ProactorEventLoop` only (default),
|
||||||
:class:`SelectorEventLoop` has no subprocess support.
|
:class:`SelectorEventLoop` has no subprocess support.
|
||||||
|
|
||||||
On UNIX *child watchers* are used for subprocess finish waiting, see
|
|
||||||
:ref:`asyncio-watchers` for more info.
|
|
||||||
|
|
||||||
|
|
||||||
.. versionchanged:: 3.8
|
|
||||||
|
|
||||||
UNIX switched to use :class:`ThreadedChildWatcher` for spawning subprocesses from
|
|
||||||
different threads without any limitation.
|
|
||||||
|
|
||||||
Spawning a subprocess with *inactive* current child watcher raises
|
|
||||||
:exc:`RuntimeError`.
|
|
||||||
|
|
||||||
Note that alternative event loop implementations might have own limitations;
|
Note that alternative event loop implementations might have own limitations;
|
||||||
please refer to their documentation.
|
please refer to their documentation.
|
||||||
|
|
||||||
|
|||||||
@@ -17,7 +17,6 @@ Doc/howto/descriptor.rst
|
|||||||
Doc/howto/enum.rst
|
Doc/howto/enum.rst
|
||||||
Doc/library/ast.rst
|
Doc/library/ast.rst
|
||||||
Doc/library/asyncio-extending.rst
|
Doc/library/asyncio-extending.rst
|
||||||
Doc/library/asyncio-policy.rst
|
|
||||||
Doc/library/asyncio-subprocess.rst
|
Doc/library/asyncio-subprocess.rst
|
||||||
Doc/library/collections.rst
|
Doc/library/collections.rst
|
||||||
Doc/library/dbm.rst
|
Doc/library/dbm.rst
|
||||||
|
|||||||
@@ -652,14 +652,14 @@ asyncio
|
|||||||
making some use-cases 2x to 5x faster.
|
making some use-cases 2x to 5x faster.
|
||||||
(Contributed by Jacob Bower & Itamar Oren in :gh:`102853`, :gh:`104140`, and :gh:`104138`)
|
(Contributed by Jacob Bower & Itamar Oren in :gh:`102853`, :gh:`104140`, and :gh:`104138`)
|
||||||
|
|
||||||
* On Linux, :mod:`asyncio` uses :class:`asyncio.PidfdChildWatcher` by default
|
* On Linux, :mod:`asyncio` uses :class:`!asyncio.PidfdChildWatcher` by default
|
||||||
if :func:`os.pidfd_open` is available and functional instead of
|
if :func:`os.pidfd_open` is available and functional instead of
|
||||||
:class:`asyncio.ThreadedChildWatcher`.
|
:class:`!asyncio.ThreadedChildWatcher`.
|
||||||
(Contributed by Kumar Aditya in :gh:`98024`.)
|
(Contributed by Kumar Aditya in :gh:`98024`.)
|
||||||
|
|
||||||
* The event loop now uses the best available child watcher for each platform
|
* The event loop now uses the best available child watcher for each platform
|
||||||
(:class:`asyncio.PidfdChildWatcher` if supported and
|
(:class:`!asyncio.PidfdChildWatcher` if supported and
|
||||||
:class:`asyncio.ThreadedChildWatcher` otherwise), so manually
|
:class:`!asyncio.ThreadedChildWatcher` otherwise), so manually
|
||||||
configuring a child watcher is not recommended.
|
configuring a child watcher is not recommended.
|
||||||
(Contributed by Kumar Aditya in :gh:`94597`.)
|
(Contributed by Kumar Aditya in :gh:`94597`.)
|
||||||
|
|
||||||
@@ -1162,15 +1162,15 @@ Deprecated
|
|||||||
|
|
||||||
* :mod:`asyncio`:
|
* :mod:`asyncio`:
|
||||||
|
|
||||||
* The child watcher classes :class:`asyncio.MultiLoopChildWatcher`,
|
* The child watcher classes :class:`!asyncio.MultiLoopChildWatcher`,
|
||||||
:class:`asyncio.FastChildWatcher`, :class:`asyncio.AbstractChildWatcher`
|
:class:`!asyncio.FastChildWatcher`, :class:`!asyncio.AbstractChildWatcher`
|
||||||
and :class:`asyncio.SafeChildWatcher` are deprecated and
|
and :class:`!asyncio.SafeChildWatcher` are deprecated and
|
||||||
will be removed in Python 3.14.
|
will be removed in Python 3.14.
|
||||||
(Contributed by Kumar Aditya in :gh:`94597`.)
|
(Contributed by Kumar Aditya in :gh:`94597`.)
|
||||||
|
|
||||||
* :func:`asyncio.set_child_watcher`, :func:`asyncio.get_child_watcher`,
|
* :func:`!asyncio.set_child_watcher`, :func:`!asyncio.get_child_watcher`,
|
||||||
:meth:`asyncio.AbstractEventLoopPolicy.set_child_watcher` and
|
:meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher` and
|
||||||
:meth:`asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated
|
:meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated
|
||||||
and will be removed in Python 3.14.
|
and will be removed in Python 3.14.
|
||||||
(Contributed by Kumar Aditya in :gh:`94597`.)
|
(Contributed by Kumar Aditya in :gh:`94597`.)
|
||||||
|
|
||||||
|
|||||||
@@ -173,6 +173,26 @@ ast
|
|||||||
|
|
||||||
(Contributed by Alex Waygood in :gh:`119562`.)
|
(Contributed by Alex Waygood in :gh:`119562`.)
|
||||||
|
|
||||||
|
asyncio
|
||||||
|
-------
|
||||||
|
|
||||||
|
* Remove the following classes and functions. They were all deprecated and
|
||||||
|
emitted deprecation warnings since Python 3.12:
|
||||||
|
|
||||||
|
* :class:`!asyncio.AbstractChildWatcher`
|
||||||
|
* :class:`!asyncio.SafeChildWatcher`
|
||||||
|
* :class:`!asyncio.MultiLoopChildWatcher`
|
||||||
|
* :class:`!asyncio.FastChildWatcher`
|
||||||
|
* :class:`!asyncio.ThreadedChildWatcher`
|
||||||
|
* :class:`!asyncio.PidfdChildWatcher`
|
||||||
|
* :meth:`!asyncio.AbstractEventLoopPolicy.get_child_watcher`
|
||||||
|
* :meth:`!asyncio.AbstractEventLoopPolicy.set_child_watcher`
|
||||||
|
* :func:`!asyncio.get_child_watcher`
|
||||||
|
* :func:`!asyncio.set_child_watcher`
|
||||||
|
|
||||||
|
(Contributed by Kumar Aditya in :gh:`120804`.)
|
||||||
|
|
||||||
|
|
||||||
collections.abc
|
collections.abc
|
||||||
---------------
|
---------------
|
||||||
|
|
||||||
|
|||||||
@@ -352,7 +352,7 @@ that schedules a shutdown for the default executor that waits on the
|
|||||||
:func:`asyncio.run` has been updated to use the new :term:`coroutine`.
|
:func:`asyncio.run` has been updated to use the new :term:`coroutine`.
|
||||||
(Contributed by Kyle Stanley in :issue:`34037`.)
|
(Contributed by Kyle Stanley in :issue:`34037`.)
|
||||||
|
|
||||||
Added :class:`asyncio.PidfdChildWatcher`, a Linux-specific child watcher
|
Added :class:`!asyncio.PidfdChildWatcher`, a Linux-specific child watcher
|
||||||
implementation that polls process file descriptors. (:issue:`38692`)
|
implementation that polls process file descriptors. (:issue:`38692`)
|
||||||
|
|
||||||
Added a new :term:`coroutine` :func:`asyncio.to_thread`. It is mainly used for
|
Added a new :term:`coroutine` :func:`asyncio.to_thread`. It is mainly used for
|
||||||
|
|||||||
Reference in New Issue
Block a user