Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-142518: Document thread-safety guarantees of list operations#142519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
lysnikolaou merged 9 commits intopython:mainfromlysnikolaou:list-thread-safety-docs
Jan 12, 2026

Conversation

@lysnikolaou
Copy link
Member

@lysnikolaoulysnikolaou commentedDec 10, 2025
edited by hugovk
Loading

I'm opening this to start a discussion on what we want the structure of such documentation to be, as well as how to balance detail with succinctness. Feedback very welcome!


📚 Documentation preview 📚:https://cpython-previews--142519.org.readthedocs.build/en/142519/library/stdtypes.html#lists

@emmatyping
Copy link
Member

Merged main to fix CI

@emmatyping
Copy link
Member

I think any note about the atomicity of list operations should probably go after the documentation about the type. I can imagine a beginner to Python going to the documentation to read about lists and being confused about what free-threading semantics are talking about.

I also wonder if this should not be a note. Perhaps it would be better to describe as part of the type description what operations are atomic and which are not?

encukou and lysnikolaou reacted with thumbs up emoji

Copy link
Member

@encukouencukou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This looks like a good start.

@encukou
Copy link
Member

encukou commentedDec 11, 2025
edited
Loading

My general opinion: reference documentation needs to be precise first; complete second. Succinctness comes after that.

Edit: Also, IMO, it's fine to add information first, then reorganize when a better structure becomes more obvious.

@lysnikolaou
Copy link
MemberAuthor

I've addressed most of the feedback and added a lot more details.

I also inadvertently force-pushed. Sorry about that.

The following operations/methods are not fully atomic:

.. code-block::
:class: maybe
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe this could say something like “Operations/methods that involve iteration are generally not atomic, except when used with specific built-in types”, and iteration itself can be moved here?
Mentioning iteration might help people make sense of this, i.e. it's no longer two arbitrary lists of operations/methods.

Then thebad section below would be left only with examples of “manually” combining multiple operations.

Copy link
MemberAuthor

@lysnikolaoulysnikolaouDec 11, 2025
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't know how I feel about this. "Operations/methods that involve iteration are generally not atomic" is probably not a mnemonic we want people to use, because there are methods that are atomic but traverse the list. Granted most of those are ones that also mutate it, but e.g.list.copy doesn't.

But the idea of separating iteration from manually combining multiple operations is good. Maybe we should do just that?

encukou reacted with thumbs up emoji
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Sounds good.
(I guess it's “operations that involve arbitrary iterators and/or comparison functions”, but that's too long; readers whom that would help can figure it out from the list.)

As for iteration, it sounds like the guarantees are the same for single-threaded code: iteration of a list that is being modified may skip elements or yield repeated elements, but will not crash or produce elements that were never part of the list. Is that right?


Is this the place to document list iterators -- i.e. what happens if you use a sharediterator in several threads?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I've completely reworded the docs to account for lock-free operations. Is this better?


I think documenting iterators should be done in theIterator docs, not really individually for each iterator type.

process(item) # another thread may modify lst

Consider external synchronization when sharing :class:`list` instances
across threads. See :ref:`freethreading-python-howto` for more information.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is getting long enough that maybe it deserves to be in its own page in the reference docs, along with thread-safety notes for all the builtin types. Then we can link to those reference docs from here.

I think our hope when we originally wanted to include these notes directly in the docs for the builtins was that these notes would be pretty short. But it turns out there are a decent number of caveats and that hope might not be realistic.

lysnikolaou reacted with thumbs up emoji
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I agree with this. Splitting everything out in a new page in the reference docs and then cross-linking sounds like the better approach, especially sincedict docs are going to be even longer.@encukou What do you think?

encukou reacted with thumbs up emoji
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hi! Sorry I missed the notification.
Either way works for me. We can reorganize the docs after the content is in -- it'll be easier to get the right structure then.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Ok, let's get this in as-is and once we have more built-in types (I havedict andset PRs ready to go), we'll have a better picture of how the new document should look like.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Sounds good to me then. Maybe let's wait to backport until we're done with the standard library types? But do whatever makes the most sense to you.


All of the above methods/operations are also lock-free. They do not block
concurrent modifications. Other operations that hold a lock will not block
these from observing intermediate states.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is "intermediate states" correct, or is it just that readers might observe the state either before or after concurrent modification? The way it's written now a very literal-minded reader might conclude that a reader might observe a torn read or other C data race.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It's not about data races, it's about intermediate states when the lock-free operations race with operations that touch multiple elements. For example, iflist.index andlist.reverse race with one another (I know, bad example, becauselist.reverse does have a data race, but let's assume that gets fixed),list.index might return the reversed index, even though some elements might not have been reversed yet.

list appears empty for the duration of the sort.

The following operations may allow lock-free operations to observe
intermediate states since they modify multiple elements in place:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It's not clear to me what "intermediate states" means here. See my comment above. Would help maybe to clarify globally (i.e. somewhere not in the list docs) what observing an object in an intermediate state means. Is any possible layout of the list while it's being processed possible?

@lysnikolaou
Copy link
MemberAuthor

I think we're all okay to merge this. Once it's in, I'll open follow-up PRs fordict andset and we can talk about moving stuff over to a new document then.

@ngoldbaum Are you okay with this?

@lysnikolaoulysnikolaou added the needs backport to 3.14bugs and security fixes labelJan 12, 2026
@lysnikolaoulysnikolaou merged commit1de4671 intopython:mainJan 12, 2026
30 checks passed
@miss-islington-app
Copy link

Thanks@lysnikolaou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@github-project-automationgithub-project-automationbot moved this fromTodo toDone inDocs PRsJan 12, 2026
@bedevere-app
Copy link

GH-143736 is a backport of this pull request to the3.14 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.14bugs and security fixes labelJan 12, 2026
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJan 12, 2026
…pythonGH-142519)* Add everything to code blocks* Improve wording around atomicity; specify exact types* Better explain lock-free and atomicity(cherry picked from commit1de4671)Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
lysnikolaou added a commit that referenced this pull requestJan 12, 2026
GH-142519) (#143736)* Add everything to code blocks* Improve wording around atomicity; specify exact types* Better explain lock-free and atomicity(cherry picked from commit1de4671)Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Copy link
Contributor

@dpdanidpdani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Just a couple minor comments. Good work Lys!

concurrent modifications. Other operations that hold a lock will not block
these from observing intermediate states.

All other operations from here on block using the per-object lock.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe a glossary reference is useful here? Something like:

per-object lock: In free-threading builds ... Whereas in GIL-enabled builds ...

Comment on lines +1466 to +1467
Writing a single item via ``lst[i] = x`` is safe to call from multiple
threads and will not corrupt the list.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It is safe in the sense that it won't corrupt the list, but should we also mention that it results in a race condition?

@dpdani
Copy link
Contributor

A little late with posting a review, sorry.

I had the diff opened and it was still showing the PR as open 🫠

reidenong pushed a commit to reidenong/cpython that referenced this pull requestJan 12, 2026
…python#142519)* Add everything to code blocks* Improve wording around atomicity; specify exact types* Better explain lock-free and atomicity
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@encukouencukouencukou left review comments

@hugovkhugovkhugovk left review comments

+2 more reviewers

@ngoldbaumngoldbaumngoldbaum left review comments

@dpdanidpdanidpdani left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

docsDocumentation in the Doc dirskip newstopic-free-threading

Projects

Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@lysnikolaou@emmatyping@encukou@dpdani@hugovk@ngoldbaum@kumaraditya303

[8]ページ先頭

©2009-2026 Movatter.jp