Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Support for pickling sentinel objects as singletons#617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
HexDecimal wants to merge1 commit intopython:main
base:main
Choose a base branch
Loading
fromHexDecimal:conforming-sentinel

Conversation

@HexDecimal
Copy link
Contributor

@HexDecimalHexDecimal commentedJun 4, 2025
edited
Loading

My attempt at implementingPEP 661 since I was unhappy with#594. I'm hoping that this is a PR of decent quality and not just me desperately wanting pickle support.

Changes made to Sentinel:

  • repr parameter removed, explicit repr tests removed,repr was rejected by the PEP, reverted,repr is kept
  • __repr__ modified to match PEP implementation (removed angle brackets)
  • Addedmodule_name parameter following PEP implementation and tweaking that to use the local_caller helper function.
  • name required support for qualified names to follow PEP and ensure forward compatibility, this is tested. Unless sentinels in class scopes are forbidden then qualified names are not optional.
  • Added__reduce__ to track Sentinel by name and module_name.
  • Added a Sentinel registry to preserve Sentinel identity across multiple calls to the class. Added tests for this.
  • Added an import step to allow forward compatibility with other sentinel libraries. Import step is tested.
    This is not from the PEP but it is required for typing_extensions to be forward compatible with any official sentinel type.
    , reverted, not necessary when pickle handles singletons.
  • Added copy and pickle tests. There were several other use-cases for pickle from the PEP 661 discussion. The PEP requires that the sentinel identity is preserved during these operations.
  • Updated documentation for Sentinel.

For a while I still supported therepr parameter and after following thePEP 661 discussion I ended up implementing the truthiness tweaks mentioned there, but then I ended up scrapping all of that so that I could follow the PEP more closely, but they would be easy to reintroduce later if desired.

I'm unsure of how to mention version changes in the docs. Replacingrepr withmodule_name is technically a breaking change.

_sentinel_registry.setdefault is potentially supposed to be protected with a lock but I'm unsure of if I can import thethreading module without causing platform issues.

_sentinel_registry could be replaced with a weak values dictionary but I'm unsure if that's necessary due to how permanent most sentinels are.

@python-cla-bot
Copy link

python-cla-botbot commentedJun 4, 2025
edited
Loading

All commit authors signed the Contributor License Agreement.

CLA signed

@Viicos
Copy link
Contributor

Replacingrepr withmodule_name is technically a breaking change.

I think breaking changes are expected for draft PEPs, so imo no deprecation process should be used. Users should be aware that changes are expected. However, I think we can:

  • Improvethe documentation about draft PEPs, stating that implementation can change.
  • Explicitly state that Sentinels are from a draft PEP in theSentinel class documentation, and isn't stable yet.

@Viicos
Copy link
Contributor

cc@taleinat

@JelleZijlstra
Copy link
Member

I'm not willing to remove therepr parameter fromtyping_extensions.Sentinel without a deprecation. However, I'm OK to re-export the builtin Sentinel class in 3.15 (if it makes it in), even if it doesn't have this parameter.

@HexDecimal
Copy link
ContributorAuthor

I'm not willing to remove therepr parameter fromtyping_extensions.Sentinel without a deprecation. However, I'm OK to re-export the builtin Sentinel class in 3.15 (if it makes it in), even if it doesn't have this parameter.

Then I will changerepr into a keyword parameter.

If compatibility is important enough then I can also have the code assumemodule_name is actuallyrepr if it does not refer to a existing module. Would this be necessary?

Configuration likerepr would be defined only with the initial Sentinel object. It wouldn't get stored in the reduce function. This allows changes inrepr to be reflected in unpickled sentinels. This means that removingrepr later will not break serialization.

@HexDecimalHexDecimalforce-pushed theconforming-sentinel branch 2 times, most recently from2c509de toe29210aCompareJuly 1, 2025 23:49
@HexDecimal
Copy link
ContributorAuthor

HexDecimal commentedJul 2, 2025
edited
Loading

I've replaced the__reduce__ method with a version using pickle's singleton support. This is the most conservative and inoffensive option for a new reduce function since it doesn't add a custom unpickle function, is forward compatible with any future method of pickling, is the standard method of handling singletons via pickle, and is generally strict. This does not support pickling anonymous sentinels which will now raisepickle.PicklingError instead ofTypeError. I can revert this if the previous behavior was more desired, but I personally only need to pickle sentinels which have a top-level definition, and the more I work with them the more that anonymous sentinels feel counter intuitive.

I've attempted to add to the discussion of PEP 661. Anonymous sentinels bring a lot of issues which need to be addressed.
https://discuss.python.org/t/pep-661-sentinel-values/9126/251

Before this is merged I'd like to add astrict keyword parameter to Sentinel which prevents sentinels from being accidentally named after existing top-level objects which are not Sentinel as well as ensuring thatrepr is never given conflicting values. I'll need feedback on if this is appropriate.

@Viicos
Copy link
Contributor

Thanks@HexDecimal. I just tested your implementation in Pydantic to implement anUNSET sentinel, andpickling seems to work as expected.

HexDecimal reacted with thumbs up emoji

@HexDecimal
Copy link
ContributorAuthor

HexDecimal commentedJul 4, 2025
edited
Loading

Edit: all of this is no longer relevant for this PR

Note on behavior regarding the sentinel registry and imported sentinels:

classMISSING:passassertSentinel("MISSING")isMISSING# Sentinel named after an existing non-Sentinel object

Which of these makes the most sense for the above code?

  1. Register and return the existing class as the sentinel identity, the assert will pass (current PR implementation)
  2. Return a new Sentinel identity, the assert will fail (old implementation)
  3. Raise a warning, then do 1
  4. RaiseTypeError becauseMISSING is not aSentinel (what I recommend, the most strict option)
  5. Add astrict parameter toSentinel to decide between 1 and 4

Edit: currently going with 2 to simplify the implementation.


2nd behavior involving registered sentinel parameters:

MISSING=Sentinel("MISSING",repr="foo")assertMISSINGisSentinel("MISSING")# Implicit 'repr' conflicts with previous 'repr'assertMISSINGisSentinel("MISSING",repr="bar")# Explicit 'repr' conflicts with previous 'repr'

Which of these makes the most sense for the above code?

  1. repr="foo" is locked in for all subsequent sentinels, subsequentrepr is always ignored, asserts pass (current implementation)
  2. Both asserts raiseValueError as long asrepr="foo" is not given again (my recommendation, the most strict option)
  3. repr only checked if explicitly given, 1st assert passes, 2nd assert raisesValueError
  4. Checkrepr like 2 but raise warnings instead of errors
  5. Addstrict parameter to decide between 1 and 2

Edit: currently going with 4 to not break anything.


The truthiness of sentinels is still being discussed. One thing which is unclear is if conversions to bool are common enough to be an issue in the first place (because sentinels are normally checked using identity comparison and little else). It'd be interesting to experiment with the most strict option and disable conversions to bool entirely:

def__bool__(self)->Never:raiseTypeError(...)

@JelleZijlstra
Copy link
Member

Which of these makes the most sense for the above code?

I feel strongly that 2 is the right behavior. Constructing a class shouldn't look around in the globals for other stuff that might be using the same name.

@HexDecimal
Copy link
ContributorAuthor

Constructing a class shouldn't look around in the globals for other stuff that might be using the same name.

That was necessary to handle unpickling until I finally implemented the correct reduce function, but at this point I can remove that code now and revert to the old behavior. I'd accept that the other options are unnecessarily handholdy.

@HexDecimal
Copy link
ContributorAuthor

HexDecimal commentedJul 24, 2025
edited
Loading

Unless my removal of sentinel truthiness causes new issues then this PR is finished. I can rebase this if needed.

Nevermind, anything related to truthiness can be a separate PR. It's outside the scope of this one.

@HexDecimalHexDecimal requested a review fromViicosJuly 24, 2025 18:28
@HexDecimalHexDecimal marked this pull request as draftDecember 2, 2025 20:43
@HexDecimalHexDecimal changed the titleRefactor Sentinel to conform to PEP 661Support for pickling sentinel objects as singletonsDec 2, 2025
@codecov
Copy link

codecovbot commentedDec 2, 2025
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.38%. Comparing base (19cc3bc) to head (678b73b).

@@           Coverage Diff           @@##             main     #617   +/-   ##=======================================  Coverage   97.38%   97.38%           =======================================  Files           3        3             Lines        7689     7701   +12     =======================================+ Hits         7488     7500   +12  Misses        201      201
FlagCoverage Δ
3.1089.00% <100.00%> (+0.01%)⬆️
3.10.489.00% <100.00%> (+0.01%)⬆️
3.1188.23% <100.00%> (+0.01%)⬆️
3.11.087.46% <100.00%> (+0.01%)⬆️
3.1288.18% <100.00%> (+0.01%)⬆️
3.12.088.16% <100.00%> (+0.01%)⬆️
3.1381.68% <100.00%> (+0.02%)⬆️
3.13.082.41% <100.00%> (+0.02%)⬆️
3.1478.86% <100.00%> (+0.03%)⬆️
3.989.71% <100.00%> (+0.01%)⬆️
3.9.1289.71% <100.00%> (+0.01%)⬆️
pypy3.1088.83% <100.00%> (+0.01%)⬆️
pypy3.1188.09% <100.00%> (+0.01%)⬆️
pypy3.989.54% <100.00%> (+0.01%)⬆️

Flags with carried forward coverage won't be shown.Click here to find out more.

Files with missing linesCoverage Δ
src/test_typing_extensions.py98.39% <100.00%> (+<0.01%)⬆️
src/typing_extensions.py93.95% <100.00%> (+<0.01%)⬆️
🚀 New features to boost your workflow:
  • ❄️Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@HexDecimal
Copy link
ContributorAuthor

I've narrowed the scope of this PR down to only what's needed to support pickling sentinels. Unfortunately this still means thatrepr needs to be touched to allow for themodule_name parameter. Thankfully a lot of other changes were no longer needed after this latest rebase.

Has anyone mentioned that_marker is a terribly obfuscated name for a sentinel?

@HexDecimalHexDecimal marked this pull request as ready for reviewDecember 2, 2025 22:36
The `repr` parameters position was replaced by `module_name` to conform to PEP 661.Added copy and pickle tests.Updated documentation for Sentinel.`_marker` was defined before `caller` which causes minor issues,resolved by setting its module name manually.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@ViicosViicosAwaiting requested review from Viicos

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@HexDecimal@Viicos@JelleZijlstra

[8]ページ先頭

©2009-2025 Movatter.jp