Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Enforce local versions of objects#505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
jaraco wants to merge12 commits intomain
base:main
Choose a base branch
Loading
frombugfix/486-local-objects

Conversation

@jaraco
Copy link
Member

@jaracojaraco commentedSep 11, 2024
edited
Loading

If a provider is returning objects fromimportlib.metadata (Distribution,Message,PackagePath), replace them with local versions for predictable types for consumers and type checkers.

Closes#486; supersedes#487

  • Add a couple of functions from jaraco.functools.
  • Create 'localize_dist' function to convert stdlib to local versions of Distributions.
  • Create 'localize_metadata' function to normalize the outputs for .metadata.
  • Create 'localize_package_path' function to convert stdlib to local versions of PackagePath.

@jaracojaracoforce-pushed thebugfix/486-local-objects branch frombd4ba0f todba0127CompareSeptember 11, 2024 16:07
@jaraco
Copy link
MemberAuthor

@abravalheri What do you think about this approach compared to#487?

The nice thing about this approach is it provides simpler type expectations for downstream consumers while also not imposing any restrictions on providers.

There's still a failing mypy check that I need to resolve, but the tests pass otherwise.

Also, it's a little bit ugly that objects that can't be converted are cast instead, which is technically incorrect, but should provide a low-disruption signal to address those objects. I'll flag that in a code comment.

Comment on lines 78 to 81
warnings.warn(f"Unrecognized distribution subclass{dist.__class__}")
returncast(importlib_metadata.Distribution,dist)
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This block is perhaps the least respectable part of the proposed change. It means that any Distribution subclasses that aren't derived fromimportlib_metadata.Distribution orimportlib.metadata.PathDistribution (e.g. a custom subclass ofimportlib.metadata.Distribution) will get a warning and then will possibly not comply with the interface. It's no worse than the status quo, however, except that there's no indication that a type may not be complying with an interface. At least now, a warning will be issued.

The proper fix will be for providers to supplyimportlib_metadata.Distribution subclasses whenimportlib_metadata is present.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thank you very much for having a look on this. The solution looks good for me, although I share the feeling that the simple cast may be a little fragile.

The proper fix will be for providers to supply importlib_metadata.Distribution subclasses when importlib_metadata is present.

Should this be officially communicated somehow to the developers? I believe that there is some opposition against using atry..except ifimportlib_metadata is not included explicitly as a package dependency. We can see an example of that feeling in the thread starting withpypa/pyproject-hooks#195 (comment).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Not sure how the core devs would feel about adding this as a "tip" or recommendation in theimportlib.metadata docs...

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Should this be officially communicated somehow to the developers?

Yes, probably.

Not sure how the core devs would feel about adding this as a "tip" or recommendation in theimportlib.metadata docs...

It should go wherever the docs are that provide guidance for providers. I'll figure out where that is and include it here or link it.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Inpython/cpython#123976, I've filed an issue upstream to track the documentation, but as I think more about what the documentation says, I'm not confident in the approach. I'll continue the conversation in#486.

@jaracojaracoforce-pushed thebugfix/486-local-objects branch from9504147 to4b4b091CompareSeptember 11, 2024 16:52
@jaraco
Copy link
MemberAuthor

jaraco commentedSep 11, 2024
edited
Loading

Tests are passing now on all but Python 3.8 and 3.9. They're failing on theproject unit (pytest-checkdocs). It seems that the warning functionality is somehow getting triggered forpytest-checkdocs even though the class is of the intended type:

.::projecttests/data/sources/example2::project  /home/runner/work/importlib_metadata/importlib_metadata/importlib_metadata/_compat.py:80: UserWarning: Unrecognized distribution subclass <class 'importlib_metadata.PathDistribution'>    warnings.warn(f"Unrecognized distribution subclass {dist.__class__}")

Surelyimportlib_metadata.PathDistribution is an instance ofimportlib_metadata.Distribution. I need to figure out what's causing that not to be the case.

@jaraco
Copy link
MemberAuthor

I think there are two issues going on:

First, when running under pytest, some modules (namely plugins) get imported early, before assertion rewriting is turned on. That includespytest_checkdocs, which importsjaraco.packaging which importsbuild, whichimports importlib_metadata. When laterimportlib_metadata is re-imported with the assert-rewrite support, a new copy is imported such that the types aren't comparable using isinstance:

> /Users/jaraco/code/python/importlib_metadata/importlib_metadata/_compat.py(82)localize_dist()-> return cast(importlib_metadata.Distribution, dist)(Pdb) importlib_metadata<module 'importlib_metadata' from '/Users/jaraco/code/python/importlib_metadata/importlib_metadata/__init__.py'>(Pdb) build._compat.importlib.metadata<module 'importlib_metadata' from '/Users/jaraco/code/python/importlib_metadata/importlib_metadata/__init__.py'>(Pdb) importlib_metadata is build._compat.importlib.metadataFalse

A second, related issue, is that the same issue happens duringlocalize_metadata. Because the isinstance check fails, theimportlib_metadata._adapters.Message object gets re-constructed from an existing message (of the same class), causing theredent function to be invoked twice on the Description, mangling the reStructuredText (by adding' '*8 at the beginning).

@jaracojaracoforce-pushed thebugfix/486-local-objects branch fromaa9c5ce to9fae83bCompareSeptember 11, 2024 18:25
@jaraco
Copy link
MemberAuthor

I notice there is some small impact on performance:

exercises.py:cached distribution: 0:00:00.000302 (+0:00:00.000007, 2%)exercises.py:discovery: 0:00:00.000300 (+0:00:00.000003, 1%)exercises.py:entry_points(): 0:00:00.002260 (+0:00:00.000040, 2%)exercises.py:entrypoint_regexp_perf: 0:00:00.000059 (+0:00:00.000001, 2%)exercises.py:uncached distribution: 0:00:00.000452 (+0:00:00.000002, 0%)

Nothing too scary.

@jaracojaracoforce-pushed thebugfix/486-local-objects branch from9fae83b to4a350a5CompareSeptember 11, 2024 18:32
@jaraco
Copy link
MemberAuthor

I think there are two issues going on:

I confirmed that runningpytest --assert plain against1a30e01 (prior to the workarounds), the tests pass, confirming that the issue was indeed isolated to the pytest assert rewrite importer.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

1 more reviewer

@abravalheriabravalheriabravalheri left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

API incompatibility withimportlib.metadata (or at least the API is not type-safe?)

3 participants

@jaraco@abravalheri

[8]ページ先頭

©2009-2025 Movatter.jp