Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-114763: Protect lazy loading modules from attribute access race#114781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
brettcannon merged 12 commits intopython:mainfromeffigies:fix-issue-114763
Feb 24, 2024

Conversation

@effigies
Copy link
Contributor

@effigieseffigies commentedJan 31, 2024•
edited by bedevere-appbot
Loading

As described in#114763, setting the__class__ attribute of a lazy-loading module toModuleType enables other threads to attempt to access attributes before the loading is complete. This PR movesself.__class__ = types.ModuleType to be the final act.

This requires two additional pieces to work:

  1. A lock, followed by a__class__ check to prevent threads that arrive while the module is being loaded from attempting to double-load the module.
  2. AnEvent to indicate that the load is in progress to the loading thread, so it can permit dunder attribute access to theexec_module() call. This path also requires that the lock be reentrant.

TheEvent needs to be tied to the specific module, so I also tied the lock to the module, as opposed to a lock scoped to theimportlib.util module. I usedobject.__getattribute__() for__spec__ and__dict__, which were previously accessed directly afterself.__class__ was reset.

Otherwise, I tried to keep things as close as possible to the original. I will try to write a unit test fitting with the module style, but the minimal reproduction in the issue is resolved by these changes.

Closes#114763

stefanv reacted with hooray emoji
@effigies
Copy link
ContributorAuthor

@brettcannon Just checking in if you're waiting on me for anything. (No rush from my end.)

@brettcannon
Copy link
Member

@effigies Nope, I'm just swamped right now, so I haven't had time to do another review yet.

effigies reacted with thumbs up emoji

@brettcannonbrettcannon self-requested a reviewFebruary 13, 2024 00:29
@effigies
Copy link
ContributorAuthor

No worries! I appreciate you taking the time, whenever you get it.

Copy link
Member

@brettcannonbrettcannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Some minor tweaks, but otherwise LGTM!

@bedevere-app
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@effigies
Copy link
ContributorAuthor

effigies commentedFeb 17, 2024•
edited
Loading

@bedevere-bot I have made the requested changes; please review again.

loader_state['__dict__']=module.__dict__.copy()
loader_state['__class__']=module.__class__
loader_state['lock']=threading.RLock()
loader_state['is_loading']=threading.Event()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hi, can we use abool flag instead ofthreading.Event? I see that access and modification toloader_state['is_loading'] is protected by theloader_state['lock'], so there is no thread safety issue.

Using an additionalthreading.Event would introduce unnecessary resource costs.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I agree, good catch. I will switch to a bool and push later today.

aisk reacted with thumbs up emoji
@brettcannon
Copy link
Member

!buildbot wasi

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@brettcannon for commit023d65d 🤖

The command will test the builders whose names match following regular expression:wasi

The builders matched are:

  • wasm32-wasi Non-Debug PR
  • wasm32 WASI 8Core PR
  • wasm32-wasi PR

@brettcannonbrettcannon added needs backport to 3.11only security fixes needs backport to 3.12only security fixes type-bugAn unexpected behavior, bug, or error topic-importlib labelsFeb 23, 2024
@brettcannonbrettcannon merged commit200271c intopython:mainFeb 24, 2024
@miss-islington-app
Copy link

Thanks@effigies for the PR, and@brettcannon for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestFeb 24, 2024
…aces (pythonGH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.(cherry picked from commit200271c)Co-authored-by: Chris Markiewicz <effigies@gmail.com>
@bedevere-app
Copy link

GH-115870 is a backport of this pull request to the3.12 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.12only security fixes labelFeb 24, 2024
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestFeb 24, 2024
…aces (pythonGH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.(cherry picked from commit200271c)Co-authored-by: Chris Markiewicz <effigies@gmail.com>
@bedevere-app
Copy link

GH-115871 is a backport of this pull request to the3.11 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.11only security fixes labelFeb 24, 2024
@effigieseffigies deleted the fix-issue-114763 branchFebruary 24, 2024 00:03
brettcannon pushed a commit that referenced this pull requestFeb 26, 2024
…races (GH-114781) (GH-115870)gh-114763: Protect lazy loading modules from attribute access races (GH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.(cherry picked from commit200271c)Co-authored-by: Chris Markiewicz <effigies@gmail.com>
brettcannon pushed a commit that referenced this pull requestFeb 26, 2024
…races (GH-114781) (GH-115871)gh-114763: Protect lazy loading modules from attribute access races (GH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.(cherry picked from commit200271c)Co-authored-by: Chris Markiewicz <effigies@gmail.com>
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull requestMar 4, 2024
…aces (pythonGH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.
@hugovk
Copy link
Member

Please see#117178 for a potential regression in 3.11 - 3.13.

effigies reacted with thumbs up emoji

diegorusso pushed a commit to diegorusso/cpython that referenced this pull requestApr 17, 2024
…aces (pythonGH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.
LukasWoodtli pushed a commit to LukasWoodtli/cpython that referenced this pull requestJan 22, 2025
…aces (pythonGH-114781)Setting the __class__ attribute of a lazy-loading module to ModuleType enables other threads to attempt to access attributes before the loading is complete. Now that is protected by a lock.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@aiskaiskaisk left review comments

@brettcannonbrettcannonbrettcannon approved these changes

@ericsnowcurrentlyericsnowcurrentlyAwaiting requested review from ericsnowcurrentlyericsnowcurrently is a code owner

@ncoghlanncoghlanAwaiting requested review from ncoghlanncoghlan is a code owner

@warsawwarsawAwaiting requested review from warsawwarsaw is a code owner

Assignees

No one assigned

Labels

topic-importlibtype-bugAn unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Accessing attributes of a lazily-loaded module is not thread-safe

5 participants

@effigies@brettcannon@bedevere-bot@hugovk@aisk
[8]ページ先頭
©2009-2025 Movatter.jp