Oct 1, 2022 · Jul 25, 2022 · Jul 26, 2022 · Jul 27, 2022 · Jul 27, 2022 · Jul 28, 2022
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst
 is explicitly caught, it should generally be propagated when
 clean-up is complete. Most code can safely ignore :exc:`asyncio.CancelledError`.

 Important asyncio components, like :class:`asyncio.TaskGroup` and the
 :func:`asyncio.timeout` context manager, are implemented using cancellation
 internally and might misbehave if a coroutine swallows
 :exc:`asyncio.CancelledError`.
 The asyncio components that enable structured concurrency, like
 :class:`asyncio.TaskGroup` and the :func:`asyncio.timeout` context manager,
 are implemented using cancellation internally and might misbehave if
 a coroutine swallows :exc:`asyncio.CancelledError`. Similarly, user code
 should not attempt to :meth:`uncancel <asyncio.Task.uncancel>`

 .. _taskgroups:

 Task Groups
 ===========
      Deprecation warning is emitted if *loop* is not specified
      and there is no running event loop.

   .. method:: cancel(msg=None)

      Request the Task to be cancelled.

      This arranges for a :exc:`CancelledError` exception to be thrown
      into the wrapped coroutine on the next cycle of the event loop.

      The coroutine then has a chance to clean up or even deny the
      request by suppressing the exception with a :keyword:`try` ...
      ... ``except CancelledError`` ... :keyword:`finally` block.
      Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
      not guarantee that the Task will be cancelled, although
      suppressing cancellation completely is not common and is actively
      discouraged.

      .. versionchanged:: 3.9
         Added the *msg* parameter.

      .. deprecated-removed:: 3.11 3.14
         *msg* parameter is ambiguous when multiple :meth:`cancel`
         are called with different cancellation messages.
         The argument will be removed.

      .. _asyncio_example_task_cancel:

      The following example illustrates how coroutines can intercept
      the cancellation request::

          async def cancel_me():
              print('cancel_me(): before sleep')

              try:
                  # Wait for 1 hour
                  await asyncio.sleep(3600)
              except asyncio.CancelledError:
                  print('cancel_me(): cancel sleep')
                  raise
              finally:
                  print('cancel_me(): after sleep')

          async def main():
              # Create a "cancel_me" Task
              task = asyncio.create_task(cancel_me())

              # Wait for 1 second
              await asyncio.sleep(1)

              task.cancel()
              try:
                  await task
              except asyncio.CancelledError:
                  print("main(): cancel_me is cancelled now")

          asyncio.run(main())

          # Expected output:
          #
          #     cancel_me(): before sleep
          #     cancel_me(): cancel sleep
          #     cancel_me(): after sleep
          #     main(): cancel_me is cancelled now

   .. method:: cancelled()

      Return ``True`` if the Task is *cancelled*.

      The Task is *cancelled* when the cancellation was requested with
      :meth:`cancel` and the wrapped coroutine propagated the
      :exc:`CancelledError` exception thrown into it.

   .. method:: done()

      Return ``True`` if the Task is *done*.
      in the :func:`repr` output of a task object.

      .. versionadded:: 3.8

   .. method:: cancel(msg=None)

      Request the Task to be cancelled.

      This arranges for a :exc:`CancelledError` exception to be thrown
      into the wrapped coroutine on the next cycle of the event loop.

      The coroutine then has a chance to clean up or even deny the
      request by suppressing the exception with a :keyword:`try` ...
      ... ``except CancelledError`` ... :keyword:`finally` block.
      Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
      not guarantee that the Task will be cancelled, although
      suppressing cancellation completely is not common and is actively
      discouraged.

      .. versionchanged:: 3.9
         Added the *msg* parameter.

      .. deprecated-removed:: 3.11 3.14
         *msg* parameter is ambiguous when multiple :meth:`cancel`
         are called with different cancellation messages.
         The argument will be removed.

      .. _asyncio_example_task_cancel:

      The following example illustrates how coroutines can intercept
      the cancellation request::

          async def cancel_me():
              print('cancel_me(): before sleep')

              try:
                  # Wait for 1 hour
                  await asyncio.sleep(3600)
              except asyncio.CancelledError:
                  print('cancel_me(): cancel sleep')
                  raise
              finally:
                  print('cancel_me(): after sleep')

          async def main():
              # Create a "cancel_me" Task
              task = asyncio.create_task(cancel_me())

              # Wait for 1 second
              await asyncio.sleep(1)

              task.cancel()
              try:
                  await task
              except asyncio.CancelledError:
                  print("main(): cancel_me is cancelled now")

          asyncio.run(main())

          # Expected output:
          #
          #     cancel_me(): before sleep
          #     cancel_me(): cancel sleep
          #     cancel_me(): after sleep
          #     main(): cancel_me is cancelled now

   .. method:: cancelled()

      Return ``True`` if the Task is *cancelled*.

      The Task is *cancelled* when the cancellation was requested with
      :meth:`cancel` and the wrapped coroutine propagated the
      :exc:`CancelledError` exception thrown into it.

   .. method:: uncancel()

      Decrement the count of cancellation requests to this Task.

      Returns the remaining number of cancellation requests.

      Note that once execution of a cancelled task completed, further
      calls to :meth:`uncancel` are ineffective.

      .. versionadded:: 3.11

      This method is used by asyncio's internals and isn't expected to be
      used by end-user code.  In particular, if a Task gets successfully
      uncancelled, this allows for elements of structured concurrency like
      :ref:`taskgroups` and :func:`asyncio.timeout` to continue running,
      isolating cancellation to the respective structured block.
      For example::

        async def make_request_with_timeout():
            try:
                async with asyncio.timeout(1):
                    # Structured block affected by the timeout:
                    await make_request()
                    await make_another_request()
            except TimeoutError:
                log("There was a timeout")
            # Outer code not affected by the timeout:
            await unrelated_code()

      While the block with ``make_request()`` and ``make_another_request()``
      might get cancelled due to the timeout, ``unrelated_code()`` should
      continue running even in case of the timeout.  This is implemented
      with :meth:`uncancel`.  :class:`TaskGroup` context managers use
      :func:`uncancel` in a similar fashion.

   .. method:: cancelling()

      Return the number of cancellation requests to this Task, i.e.,
      the number of calls to :meth:`cancel`.

      Note that if this number is greater than zero but the Task is
      still executing, :meth:`cancelled` will still return ``False``.
      This is because this number can be lowered by calling :meth:`uncancel`,
      which can lead to the task not being cancelled after all if the
      cancellation requests go down to zero.

      This method is used by asyncio's internals and isn't expected to be
      used by end-user code.  See :meth:`uncancel` for more details.

      .. versionadded:: 3.11
diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py
    def uncancel(self):
        """Decrement the task's count of cancellation requests.

        This should beused bytasksthatcatch CancelledError
 and wish to continue indefinitely until they are cancelled again.
        This should becalled bythe partythatcalled `cancel()` on the task
 beforehand.

        Returns the remaining number of cancellation requests.
        """
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
        finally:
            loop.close()

    deftest_uncancel(self):
    deftest_uncancel_basic(self):
        loop = asyncio.new_event_loop()

        async def task():
        try:
            t = self.new_task(loop, task())
            loop.run_until_complete(asyncio.sleep(0.01))
            self.assertTrue(t.cancel())  # Cancel first sleep

            # Cancel first sleep
            self.assertTrue(t.cancel())
            self.assertIn(" cancelling ", repr(t))
            self.assertEqual(t.cancelling(), 1)
            self.assertFalse(t.cancelled())  # Task is still not complete
            loop.run_until_complete(asyncio.sleep(0.01))
            self.assertNotIn(" cancelling ", repr(t))  # after .uncancel()
            self.assertTrue(t.cancel())  # Cancel second sleep

            # after .uncancel()
            self.assertNotIn(" cancelling ", repr(t))
            self.assertEqual(t.cancelling(), 0)
            self.assertFalse(t.cancelled())  # Task is still not complete

            # Cancel second sleep
            self.assertTrue(t.cancel())
            self.assertEqual(t.cancelling(), 1)
            self.assertFalse(t.cancelled())  # Task is still not complete
            with self.assertRaises(asyncio.CancelledError):
                loop.run_until_complete(t)
            self.assertTrue(t.cancelled())  # Finally, task complete
            self.assertTrue(t.done())

            # uncancel is no longer effective after the task is complete
            t.uncancel()
            self.assertTrue(t.cancelled())
            self.assertTrue(t.done())
        finally:
            loop.close()

    def test_uncancel_structured_blocks(self):
        # This test recreates the following high-level structure using uncancel()::
        #
        #     async def make_request_with_timeout():
        #         try:
        #             async with asyncio.timeout(1):
        #                 # Structured block affected by the timeout:
        #                 await make_request()
        #                 await make_another_request()
        #         except TimeoutError:
        #             pass  # There was a timeout
        #         # Outer code not affected by the timeout:
        #         await unrelated_code()

        loop = asyncio.new_event_loop()

        async def make_request_with_timeout(*, sleep: float, timeout: float):
            task = asyncio.current_task()
            loop = task.get_loop()

            timed_out = False
            structured_block_finished = False
            outer_code_reached = False

            def on_timeout():
                nonlocal timed_out
                timed_out = True
                task.cancel()

            timeout_handle = loop.call_later(timeout, on_timeout)
            try:
                try:
                    # Structured block affected by the timeout
                    await asyncio.sleep(sleep)
                    structured_block_finished = True
                finally:
                    timeout_handle.cancel()
                    if (
                        timed_out
                        and task.uncancel() == 0
                        and sys.exc_info()[0] is asyncio.CancelledError
                    ):
                        # Note the five rules that are needed here to satisfy proper
                        # uncancellation:
                        #
                        # 1. handle uncancellation in a `finally:` block to allow for
                        #    plain returns;
                        # 2. our `timed_out` flag is set, meaning that it was our event
                        #    that triggered the need to uncancel the task, regardless of
                        #    what exception is raised;
                        # 3. we can call `uncancel()` because *we* called `cancel()`
                        #    before;
                        # 4. we call `uncancel()` but we only continue converting the
                        #    CancelledError to TimeoutError if `uncancel()` caused the
                        #    cancellation request count go down to 0.  We need to look
                        #    at the counter vs having a simple boolean flag because our
                        #    code might have been nested (think multiple timeouts). See
                        #    commit 7fce1063b6e5a366f8504e039a8ccdd6944625cd for
                        #    details.
                        # 5. we only convert CancelledError to TimeoutError; for other
                        #    exceptions raised due to the cancellation (like
                        #    a ConnectionLostError from a database client), simply
                        #    propagate them.
                        #
                        # Those checks need to take place in this exact order to make
                        # sure the `cancelling()` counter always stays in sync.
                        #
                        # Additionally, the original stimulus to `cancel()` the task
                        # needs to be unscheduled to avoid re-cancelling the task later.
                        # Here we do it by cancelling `timeout_handle` in the `finally:`
                        # block.
                        raise TimeoutError
            except TimeoutError:
                self.assertTrue(timed_out)

            # Outer code not affected by the timeout:
            outer_code_reached = True
            await asyncio.sleep(0)
            return timed_out, structured_block_finished, outer_code_reached

        # Test which timed out.
        t1 = self.new_task(loop, make_request_with_timeout(sleep=10.0, timeout=0.1))
        timed_out, structured_block_finished, outer_code_reached = (
            loop.run_until_complete(t1)
        )
        self.assertTrue(timed_out)
        self.assertFalse(structured_block_finished)  # it was cancelled
        self.assertTrue(outer_code_reached)  # task got uncancelled after leaving
                                             # the structured block and continued until
                                             # completion
        self.assertEqual(t1.cancelling(), 0) # no pending cancellation of the outer task

        # Test which did not time out.
        t2 = self.new_task(loop, make_request_with_timeout(sleep=0, timeout=10.0))
        timed_out, structured_block_finished, outer_code_reached = (
            loop.run_until_complete(t2)
        )
        self.assertFalse(timed_out)
        self.assertTrue(structured_block_finished)
        self.assertTrue(outer_code_reached)
        self.assertEqual(t2.cancelling(), 0)

    def test_cancel(self):

        def gen():
Original file line number	Diff line number	Diff line change
Expand Up		@@ -294,11 +294,13 @@ perform clean-up logic. In case :exc:`asyncio.CancelledError`
		is explicitly caught, it should generally be propagated when
		clean-up is complete. Most code can safely ignore :exc:`asyncio.CancelledError`.

		Important asyncio components, like :class:`asyncio.TaskGroup` and the
		:func:`asyncio.timeout` context manager, are implemented using cancellation
		internally and might misbehave if a coroutine swallows
		:exc:`asyncio.CancelledError`.
		The asyncio components that enable structured concurrency, like
		:class:`asyncio.TaskGroup` and the :func:`asyncio.timeout` context manager,
gvanrossum marked this conversation as resolved. Show resolvedHide resolved
		are implemented using cancellation internally and might misbehave if
		a coroutine swallows :exc:`asyncio.CancelledError`. Similarly, user code
		should not attempt to :meth:`uncancel <asyncio.Task.uncancel>`
gvanrossum marked this conversation as resolved. Show resolvedHide resolved

		.. _taskgroups:

		Task Groups
		===========
Expand DownExpand Up		@@ -994,76 +996,6 @@ Task Object
		Deprecation warning is emitted if loop is not specified
		and there is no running event loop.

		.. method:: cancel(msg=None)

		Request the Task to be cancelled.

		This arranges for a :exc:`CancelledError` exception to be thrown
		into the wrapped coroutine on the next cycle of the event loop.

		The coroutine then has a chance to clean up or even deny the
		request by suppressing the exception with a :keyword:`try` ...
		... ``except CancelledError`` ... :keyword:`finally` block.
		Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
		not guarantee that the Task will be cancelled, although
		suppressing cancellation completely is not common and is actively
		discouraged.

		.. versionchanged:: 3.9
		Added the msg parameter.

		.. deprecated-removed:: 3.11 3.14
		msg parameter is ambiguous when multiple :meth:`cancel`
		are called with different cancellation messages.
		The argument will be removed.

		.. _asyncio_example_task_cancel:

		The following example illustrates how coroutines can intercept
		the cancellation request::

		async def cancel_me():
		print('cancel_me(): before sleep')

		try:
		# Wait for 1 hour
		await asyncio.sleep(3600)
		except asyncio.CancelledError:
		print('cancel_me(): cancel sleep')
		raise
		finally:
		print('cancel_me(): after sleep')

		async def main():
		# Create a "cancel_me" Task
		task = asyncio.create_task(cancel_me())

		# Wait for 1 second
		await asyncio.sleep(1)

		task.cancel()
		try:
		await task
		except asyncio.CancelledError:
		print("main(): cancel_me is cancelled now")

		asyncio.run(main())

		# Expected output:
		#
		# cancel_me(): before sleep
		# cancel_me(): cancel sleep
		# cancel_me(): after sleep
		# main(): cancel_me is cancelled now

		.. method:: cancelled()

		Return ``True`` if the Task is cancelled.

		The Task is cancelled when the cancellation was requested with
		:meth:`cancel` and the wrapped coroutine propagated the
		:exc:`CancelledError` exception thrown into it.

		.. method:: done()

		Return ``True`` if the Task is done.
Expand DownExpand Up		@@ -1177,3 +1109,124 @@ Task Object
		in the :func:`repr` output of a task object.

		.. versionadded:: 3.8

		.. method:: cancel(msg=None)

		Request the Task to be cancelled.

		This arranges for a :exc:`CancelledError` exception to be thrown
		into the wrapped coroutine on the next cycle of the event loop.

		The coroutine then has a chance to clean up or even deny the
		request by suppressing the exception with a :keyword:`try` ...
		... ``except CancelledError`` ... :keyword:`finally` block.
		Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
		not guarantee that the Task will be cancelled, although
		suppressing cancellation completely is not common and is actively
		discouraged.

		.. versionchanged:: 3.9
		Added the msg parameter.

		.. deprecated-removed:: 3.11 3.14
		msg parameter is ambiguous when multiple :meth:`cancel`
		are called with different cancellation messages.
		The argument will be removed.

		.. _asyncio_example_task_cancel:

		The following example illustrates how coroutines can intercept
		the cancellation request::

		async def cancel_me():
		print('cancel_me(): before sleep')

		try:
		# Wait for 1 hour
		await asyncio.sleep(3600)
		except asyncio.CancelledError:
		print('cancel_me(): cancel sleep')
		raise
		finally:
		print('cancel_me(): after sleep')

		async def main():
		# Create a "cancel_me" Task
		task = asyncio.create_task(cancel_me())

		# Wait for 1 second
		await asyncio.sleep(1)

		task.cancel()
		try:
		await task
		except asyncio.CancelledError:
		print("main(): cancel_me is cancelled now")

		asyncio.run(main())

		# Expected output:
		#
		# cancel_me(): before sleep
		# cancel_me(): cancel sleep
		# cancel_me(): after sleep
		# main(): cancel_me is cancelled now

		.. method:: cancelled()

		Return ``True`` if the Task is cancelled.

		The Task is cancelled when the cancellation was requested with
		:meth:`cancel` and the wrapped coroutine propagated the
		:exc:`CancelledError` exception thrown into it.

Comment on lines +1122 to +1191 Copy link ContributorAuthor ambvJul 26, 2022 Choose a reason for hiding this comment The reason will be displayed to describe this comment to others.Learn more. This part is unchanged, only moved down. IMO cancellation isn't as important as getting other things out of a task. Plus this move allows us to keep cancel-specific methods next to each other,`uncancel` in particular being pretty low-level. gvanrossum reacted with thumbs up emoji
		.. method:: uncancel()

		Decrement the count of cancellation requests to this Task.

		Returns the remaining number of cancellation requests.

		Note that once execution of a cancelled task completed, further
		calls to :meth:`uncancel` are ineffective.

		.. versionadded:: 3.11

		This method is used by asyncio's internals and isn't expected to be
		used by end-user code. In particular, if a Task gets successfully
ambv marked this conversation as resolved. Show resolvedHide resolved
		uncancelled, this allows for elements of structured concurrency like
		:ref:`taskgroups` and :func:`asyncio.timeout` to continue running,
		isolating cancellation to the respective structured block.
		For example::

		async def make_request_with_timeout():
		try:
		async with asyncio.timeout(1):
		# Structured block affected by the timeout:
		await make_request()
		await make_another_request()
		except TimeoutError:
		log("There was a timeout")
		# Outer code not affected by the timeout:
		await unrelated_code()

		While the block with ``make_request()`` and ``make_another_request()``
		might get cancelled due to the timeout, ``unrelated_code()`` should
		continue running even in case of the timeout. This is implemented
		with :meth:`uncancel`. :class:`TaskGroup` context managers use
		:func:`uncancel` in a similar fashion.
Comment on lines +1221 to +1225 Copy link Member gvanrossumAug 1, 2022 Choose a reason for hiding this comment The reason will be displayed to describe this comment to others.Learn more. This seems to imply that without`uncancel()`,`await unrelated_code()` would be cancelled. But even with the most naive implementation possible using only primitives existing in 3.10, if`except TimeoutError:` is triggered, the following`await` will executed just fine. (In Trio this would not be the case, since cancellation there is level-triggered, i.e. once a task is cancelled it stays cancelled and specifically any further`await` will immediately be cancelled; but in`asyncio` cancellation is edge-triggered, and once the task has regained control, which the context manager can do, it is no longer cancelled.) So why does`uncancel()` exist and need to be called? Because when structured concurrency primitives are nested, they may cancel the same task. The inner primitive(s) of these must let the`CancellationError` bubble out, while the outermost one must take the action it promises its caller (e.g. raise`TimeoutError` or raise an`ExceptionGroup` bundling the error(s) experienced by failed but not cancelled subtasks). I think this means that to make the example meaningful, you'd have to nest two timeout blocks whose timers go off simultaneously. Then the inner one will raise`CancellationError` (so an`except TimeoutError` will not trigger!) and the outer one will raise`TimeoutError`. It is not a coincidence that`uncancel()` is called in`__aexit__()` by both timeout and TaskGroup. The pattern is pretty much some async event calls`t.cancel()` on some task,and sets an internal flag indicating it did so; later, typically in`__aexit__()`, if the internal flag is set,`t.uncancel()` is called, and if it returns a value greater than zero,`CancelledError` is (re)raised (by returning`None` from`__aexit__()`!), else some other action is taken.

		.. method:: cancelling()

		Return the number of cancellation requests to this Task, i.e.,
		the number of calls to :meth:`cancel`.
gvanrossum marked this conversation as resolved. Show resolvedHide resolved

		Note that if this number is greater than zero but the Task is
		still executing, :meth:`cancelled` will still return ``False``.
		This is because this number can be lowered by calling :meth:`uncancel`,
		which can lead to the task not being cancelled after all if the
		cancellation requests go down to zero.
Comment on lines +1233 to +1237 Copy link Member gvanrossumAug 1, 2022 Choose a reason for hiding this comment The reason will be displayed to describe this comment to others.Learn more. This seems to imply that the effect of`cancel()` +`uncancel()` is a no-op, but I'm not sure that's always the case. Honestly after several attempts I'm still not sure how`cancel()` and`__step()` interacted even in 3.10. :-( But it seems at least possible that`.cancel()` sets`_must_cancel` which causes`__step()` to throw a`CancelledError` into the coroutine even if`uncancel()` is called immediately after`cancel()`. See also the comment added to`cancel()` starting with "These two lines are controversial." (Also`7d611b4`)

		This method is used by asyncio's internals and isn't expected to be
		used by end-user code. See :meth:`uncancel` for more details.

		.. versionadded:: 3.11
Original file line number	Diff line number	Diff line change
Expand Up		@@ -243,8 +243,8 @@ def cancelling(self):
		def uncancel(self):
		"""Decrement the task's count of cancellation requests.

		This should beused bytasksthatcatch CancelledError
		and wish to continue indefinitely until they are cancelled again.
		This should becalled bythe partythatcalled `cancel()` on the task
		beforehand.
Comment on lines -246 to +247 Copy link ContributorAuthor ambvJul 28, 2022 Choose a reason for hiding this comment The reason will be displayed to describe this comment to others.Learn more. The previous docstring was invalid, we activelydon't want for user code to call`uncancel()` when catching CancelledError.

		Returns the remaining number of cancellation requests.
		"""
Expand Down