Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Remove pkg_resources#5007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
jaraco wants to merge7 commits intomain
base:main
Choose a base branch
Loading
fromfeature/remove-more-pkg_resources
Draft

Conversation

@jaraco
Copy link
Member

  • Remove pkg_resources
  • Remove assertion that 'import pkg_resources' works; it's no longer an invariant.
  • Mark pbr integration test as xfail.
  • Mark namespace tests as xfail. Support for pkg_resources-style namespaces is dropped.
  • Add news fragment.

Summary of changes

Closes#3085

Pull Request Checklist

@jaraco
Copy link
MemberAuthor

jaraco commentedMay 21, 2025
edited
Loading

To be sure, this change is a big deal. It's been decades in the making to provide replacements for this functionality. There are known open issues and efforts to remove the dependence on pkg_resources and probably other use-cases that haven't migrated. My hope is that these cases can pin to Setuptools<81.

I don't have any plans to rush this out or force it on the community, but I'd like to start thinking about what it would take to get committed.

@jaracojaracoforce-pushed thefeature/remove-more-pkg_resources branch from35e39e7 tob3f9277CompareMay 21, 2025 22:22
@jaracojaraco requested a review fromabravalheriMay 22, 2025 00:01
Copy link
Contributor

@abravalheriabravalheri left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thank you very much Jason for the effort in untangling the 2 code bases.
I agree that it is time to start thinking about the removal ofpkg_resources.

I propose we do it in 2 stages:

  1. Explicitly mention a "due date" to thewarning message
  2. Removal once the "due date" arrives.

I have 2 doubts:

  • Does thenamespace_packages (deprecated) keyword forsetup.py works withoutpkg_resources.declare_namespace?

    If I understood correctly previously these worked in tandem, but I don't know what is the situation now after the latest changes inhttps://github.com/pypa/setuptools/blob/v80.8.0/setuptools/namespaces.py#L47-L63.

  • Is the idea ofhttps://github.com/pypa/pkg_resources to also publish something to PyPI? Even if the package is born deprecated it may be a "last-resort" alternative for users that rely on packages that have not received updates for a long time, or that are still working on the migration...

    I know at least that folks fromzope/plone have been slowly trying to replacepkg_resources.declare_namespace for a while, but are not there quite yet (refs:#3434 (comment),zopefoundation/meta#194), so maybe that would be useful if they take can install a package to extend the time they can work on the migration. We could also consider addingpkg_resources insetuptools.build_meta:__legacy__.get_requires_for_build_sdist if we want to be very conservative about the transition.

    Update: I can see in#3085 (comment) that this is not straight forward.

@jaraco
Copy link
MemberAuthor

  • Does thenamespace_packages (deprecated) keyword forsetup.py works withoutpkg_resources.declare_namespace?

It's a mixed bag.

If I recall correctly, when pip-installed, a declared namespace gets its__init__.py omitted, so ends up acting like a PEP 420 namespace. I also seem to recall that there's a.pth file that gets installed under certain circumstances (although maybe not any more after the removal of easy_install).

Looking at zope.interface as an example:

 🐚 py -3.13 -m pip-run git+https://github.com/zopefoundation/zope.interface -- -c "import zope, pathlib; print('files in zope:', list(pathlib.Path(zope.__path__[0]).iterdir())); print('nspkg.pth file:', next(pathlib.Path(zope.__path__[0]).parent.glob('*-nspkg.pth')).read_text())"files in zope: [PosixPath('/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-run-r43zfapw/zope/interface')]nspkg.pth file: import sys, types, os;p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('zope',));importlib = __import__('importlib.util');__import__('importlib.machinery');m = sys.modules.setdefault('zope', importlib.util.module_from_spec(importlib.machinery.PathFinder.find_spec('zope', [os.path.dirname(p)])));m = m or sys.modules.setdefault('zope', types.ModuleType('zope'));mp = (m or []) and m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p)

The same istrue for the wheel.

These results confirm that, at least for the pip-installed packages, the use of pkg_resources is completely absent.

This confirmation gives me higher confidence that this removal is at least safer than I'd previously thought.

In fact, I wonder ifdeclare_namespace is in use anywhere at all.

  If I understood correctly previously these worked in tandem, but I don't know what is the situation now after the latest changes in https://github.com/pypa/setuptools/blob/v80.8.0/setuptools/namespaces.py#L47-L63.

There was a time where even the -nspkg.pth file did nothing and fell back to PEP 420 on Python 3.3+. In27b7d4e, however, I limited the PEP 420 fallback to Python 3.3 and 3.4. I didn't leave any good justification for making that change (no linked issue, no mention in changelog). Maybe it had something to do with#1214. I wonder if that change was a mistake and could have been allowed to simply fall back to PEP 420.

In fact, I wonder ifdeclare_namespace is in use anywhere at all.

I guess it has to still be used for the editable case... at least wheresetup.py develop was used.

Confirmed - it's still used even for the modern editable case:

 zope.interface master 🐚 py -3.13 -m pip-run -e .Python 3.13.3 (main, Apr  8 2025, 13:54:08) [Clang 16.0.0 (clang-1600.0.26.6)] on darwinType "help", "copyright", "credits" or "license" for more information.>>> import zope>>> zope.__file__'/Users/jaraco/draft/zope.interface/src/zope/__init__.py'>>> zope.__path__['/Users/jaraco/draft/zope.interface/src/zope']

I'm less worried about the editable case, because it's outside the typical integration and production workflows and the developer has a lot more control to implement workarounds, such as to pin setuptools.

  • Is the idea ofhttps://github.com/pypa/pkg_resources to also publish something to PyPI? Even if the package is born deprecated it may be a "last-resort" alternative for users that rely on packages that have not received updates for a long time, or that are still working on the migration...

As you've observed, I was considering this option, but I'm still not confident it's the right thing to do. It would require getting the name opened up for use, which I'm fairly confident the maintainers would do for this purpose.

But it would still require the users to install it. It wouldn't be installed implicitly as Setuptools is in some environments. And if they have the ability to install a custompkg_resources package, they probably also have the ability to pin to an older Setuptools... so maybe "pin setuptools" is the universal workaround.

I propose we do it in 2 stages:

1. Explicitly mention a "due date" to the [warning message](https://github.com/pypa/setuptools/blob/v80.8.0/pkg_resources/__init__.py#L98-L103)2. Removal once the "due date" arrives.

I'm good with that plan. I'll set the due date for 6 months and mark this PR as draft in the meantime.

abravalheri reacted with thumbs up emoji

@jaracojaraco marked this pull request as draftMay 25, 2025 13:56
@jaraco
Copy link
MemberAuthor

I propose we do it in 2 stages:

1. Explicitly mention a "due date" to the [warning message](https://github.com/pypa/setuptools/blob/v80.8.0/pkg_resources/__init__.py#L98-L103)2. Removal once the "due date" arrives.

I started looking into this plan, but it seems that pkg_resources doesn't have any of the due date functionality ofsetuptools.warnings, so I'm just going to apply some changes to the messaging.

@abravalheri
Copy link
Contributor

Thank you very much Jason for adding the due date and providing the review about the namespaces.

But it would still require the users to install it. It wouldn't be installed implicitly as Setuptools is in some environments. And if they have the ability to install a custom pkg_resources package, they probably also have the ability to pin to an older Setuptools... so maybe "pin setuptools" is the universal workaround.

As a brainstorm: there is one possibility to make it "installed implicitly" in build environments.

I mentioned in#3085 (comment) that we could returnpkg_resources insetuptools.build_meta:__legacy__.get_requires_for_build_sdist/wheel.

But I am not sure this direction is worth pursuing specially becausepip (and posiblyuv?) don't fully1 fallback by default yet tosetuptools.build_meta:__legacy__ whenpyproject.toml is absent (instead they runpython setup.py ...).

Footnotes

  1. This behaviour is meant to be the default, but thepip team has-postponed/never-manage-to-implement this change to avoid backwards incompatibility. So what we have right now is an heuristic that sometimes usespython setup.py and sometimessetuptools.build_meta:__legacy__.

@Avasam
Copy link
Contributor

Avasam commentedJun 1, 2025
edited
Loading

Incidentally, this would close mostpkg_resources-related issues:

As well as:

There are a lot more issues referencingpkg_resources, but I only listed those that seemed to come from obvious directpkg_resources usage, or were raising issues/feature requests specifically aboutpkg_resources's API.

(as a sidenote: while looking at these, I found a few issues that may be closable already, I commented on those, sohttps://github.com/pypa/setuptools/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen%20commenter%3AAvasam%20pkg_resources should have a handful of issues you can close already)

neutrinoceros and Avasam reacted with thumbs up emoji



@pytest.mark.uses_network
@pytest.mark.xfail(reason="https://bugs.launchpad.net/pbr/+bug/2111459")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks like this may have been resolved last month ?https://bugs.launchpad.net/pbr/+bug/2111459/comments/6

Comment on lines +55 to 56
@pytest.mark.xfail(reason="pkg_resources has been removed")
deftest_pkg_resources_import(self,tmpdir):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is there a point to keeping such tests ?

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@abravalheriabravalheriabravalheri left review comments

+1 more reviewer

@AvasamAvasamAvasam left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Setuptools should not rely on pkg_resources

4 participants

@jaraco@abravalheri@Avasam

[8]ページ先頭

©2009-2025 Movatter.jp