Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-132933: Allow zipapp target to overwrite filtered source file#132934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
pfmoore merged 7 commits intopython:mainfromJolmberg:zipapp-filtered-target
Apr 29, 2025

Conversation

Jolmberg
Copy link
Contributor

@JolmbergJolmberg commentedApr 25, 2025
edited by bedevere-appbot
Loading

@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

@picnixz
Copy link
Member

picnixz commentedApr 25, 2025
edited
Loading

I wrongly said "no NEWS" but since the fix has been backported to 3.13, we need a NEWS entry as 3.13 is a stable release (and because we already shipped a release of 3.14; actually everything that changes from something that has been shipped in another release should have a NEWS entry when possible)

Jolmberg reacted with thumbs up emoji

Copy link
Member

@pfmoorepfmoore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I'm not convinced we should be supporting the approach of putting the target in the same directory as the sources are coming from. If you can get the logic to work correctly and cleanly, I'll consider adding it, as I don't want to arbitrarily break people's workflows. But please understand, that there's a reason the existing check is simplistic, and it's clearly stated in the comments - it's to catch common errors,not to make the approach robust.

Lib/zipapp.py Outdated
raise ZipAppError(
f"The target archive {target} overwrites one of the source files.")
if filter:
arcname = target.relative_to(source)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is not a valid check.source andtarget could be on different drives (on Windows), in which caserelative_to will fail with aValueError.

The logic here seems backwards to me. Rather than trying to replicate the filtering done in the archive writing loop, we should be calculating files_to_add more precisely, based on the filter. That wouldn't fix the problem oftarget being on a different drive, though - there's a fundamental problem with the whole idea here, asfilter is defined to work on thearcname of the source file, andtarget may not even have a definedarcname, because logically it isn't a source file.

Ultimately, the correct answer here is what I've said elsewhere -don't put your output file in the directory you're creating the zipapp from. That's a much more straightforward approach than trying to exclude it using a filter.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for taking the time with this! You are right of course, that check is wrong and can fail even within a single drive I believe. I've rewritten the code to filter files_to_add right away and then just use your original target check.

I can appreciate your sentiment about not writing the output file to the source directory, but I would prefer not to mess with my build system and I still think this change is for the better. I use the filter as a whitelist, precisely because I know that I can't trust the source directory to always be clean. I think it's pretty risk free™.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hm, thinking some more about it, I'm not sure I see how the relative_to check could fail. It's only performed after we've confirmed that target is equal to one of the paths inside source. But there could be intricacies to the Path class that I'm missing, and I think the new code is a cleaner way of doing this either way.

@bedevere-app
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

Lib/zipapp.py Outdated
files_to_add = {f: f2 for f in files_to_add
if filter(f2 := f.relative_to(source))}
else:
files_to_add = {f: f.relative_to(source) for f in files_to_add}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The comprehension doesn't add to readability, IMO. I suggest something more like:

files_to_add = []for file in sorted(source.rglob("*")):    relative_name = file.relative_to(source)    if filter is None or filter(relative_name):        files_to_add.append((file, relative_name))

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I can do away with the comprehension, but building files_to_add as a list of tuples makes it more difficult to search for target in it. A simpletarget in files_to_add will no longer work.
One option is to again use a dictionary for files_to_add. Another is to do the check for target as part of this loop, like this:

files_to_add = []for file in sorted(source.rglob("*")):    relative_name = file.relative_to(source)    if filter is None or filter(relative_name):        if file == target:            raise ZipAppError(...)        files_to_add.append((file, relative_name))

I have no strong opinion. Doing the target check separately astarget in files_to_add is more idiomatic but doing it as part of the loop (as above) is probably a tiny bit more efficient.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Good point. I have no particular opinion between using a dictionary or checking in the loop - do whatever you prefer. But let's use an explicit loop, readability is important.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Ok, I went with the dictionary. It occurred to me that when files_to_add is a dictionary, thetarget in files_to_add check should be basically free, and more importantly, this means I don't have to split the exception message string over two lines. :)

pfmoore reacted with thumbs up emoji
@@ -0,0 +1 @@
The zipapp module now applies the provided filter to the list of files to be added before determining whether the target file overwrites a source file. This allows the target file to be written inside the source directory provided it is excluded by the filter.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I disagree strongly with this news item, as it suggests that writing the target into the source directory is intended usage.

Maybe

The zipapp module now applies the filter when creating the list of files to add, rather than waiting until the file is being added to the archive

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Sure. I wanted to get a bit of rationale in there, but I agree that it could come off as a recommendation, and it got long-winded too. I'll change it.

Lib/zipapp.py Outdated
@@ -154,15 +159,14 @@ def create_archive(source, target=None, interpreter=None, main=None,
raise ZipAppError(
f"The target archive {target} overwrites one of the source files.")


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Unintended blank line?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yes, will fix!

@@ -113,6 +113,18 @@ def test_target_overwrites_source_file(self):
with self.assertRaises(zipapp.ZipAppError):
zipapp.create_archive(source, target)

def test_target_overwrites_filtered_source_file(self):
# The target cannot be one of the files to add.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change
#The target cannot be one of thefiles to add.
#If there's a filter that excludes thetarget,
# the overwrite check shouldn't trigger.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks, copy/paste error!

@Jolmberg
Copy link
ContributorAuthor

I have made the requested changes; please review again

@bedevere-app
Copy link

Thanks for making the requested changes!

@pfmoore: please review the changes made to this pull request.

@bedevere-appbedevere-appbot requested a review frompfmooreApril 28, 2025 07:08
@pfmoore
Copy link
Member

Thanks for your patience. This now seems like a straightforward and sensible improvement, and the code is clean.

I still don't encourage putting the output file in the source directory, but I don't think that's important enough to block this quality of life improvement.

@Jolmberg
Copy link
ContributorAuthor

Cool, thanks for your time!

@pfmoorepfmoore merged commit698c6e3 intopython:mainApr 29, 2025
39 checks passed
@pfmoore
Copy link
Member

Thanks for the contribution!

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@pfmoorepfmoorepfmoore approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@Jolmberg@picnixz@pfmoore

[8]ページ先頭

©2009-2025 Movatter.jp