Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Dropping concurrent.futures.Executor.map result cancels pending futures #136578

Open
Labels
@mirober

Description

@mirober

Bug report

Bug description:

Code:

fromconcurrent.futuresimportThreadPoolExecutordata= [list(range(0,5)),list(range(5,10)),]def_f(x):print(f"Processing{x}")returnTrueprint("=== 1 - No consumption from the iterator ===")executor=ThreadPoolExecutor(max_workers=1)forintsindata:executor.map(_f,ints)executor.shutdown(wait=True)print("=== 2 - Consume all values from the iterator ===")executor=ThreadPoolExecutor(max_workers=1)forintsindata:futures=executor.map(_f,ints)results=list(futures)executor.shutdown(wait=True)print("=== 3 - Consume one value from the iterator ===")executor=ThreadPoolExecutor(max_workers=1)forintsindata:futures=executor.map(_f,ints)first=next(futures)executor.shutdown(wait=True)print("=== 4 - Dropping iterator cancels remaining futures ===")executor=ThreadPoolExecutor(max_workers=1)futures=executor.map(_f,range(0,5))first=next(futures)delfuturesexecutor.shutdown(wait=True)

Result:

=== 1 - No consumption from the iterator ===Processing 0Processing 1Processing 2Processing 3Processing 4Processing 5Processing 6Processing 7Processing 8Processing 9=== 2 - Consume all values from the iterator ===Processing 0Processing 1Processing 2Processing 3Processing 4Processing 5Processing 6Processing 7Processing 8Processing 9=== 3 - Consume one value from the iterator ===Processing 0Processing 1Processing 5Processing 6Processing 7Processing 8Processing 9=== 4 - Dropping iterator cancels remaining futures ===Processing 0Processing 1

The behaviour seems to be:

  • If the iterator returned frommap is never used (case 1), futures are not cancelled
  • If the iterator returned frommap is exhausted (case 2), futures are not cancelled
  • If the iterator returned frommap ispartially consumed and then dropped (cases 3 & 4), the remaining futures are cancelled

We hit this doing a version of case 3, callingany on the iterator, which short-circuited, causing the remaining futures to not execute. This tripped us up and seems like quite a confusing behaviour that is not flagged in the docs.

It looks like this is caused by this code:https://github.com/python/cpython/blob/main/Lib/concurrent/futures/_base.py#L669-L671

Possibly related to#108518

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions


      [8]ページ先頭

      ©2009-2025 Movatter.jp