Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

In handle_process_output don't forward finalizer result#1788

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Byron merged 1 commit intogitpython-developers:mainfromEliahKagan:void-finalizer
Dec 26, 2023

Conversation

EliahKagan
Copy link
Member

@EliahKaganEliahKagan commentedDec 25, 2023•
edited
Loading

Thegit.cmd.handle_process_output function is non-public (git.cmd.__all__ only listsGit) but used throughout GitPython and referenced in thegit.util.RemoteProgress.new_message_handler docstring. Itsfinalizer argument is annotated to accept an optional callable that always returnsNone. It is always used this way in GitPython andRemoteProcess.new_message_handler is annotated accordingly. However, thehandle_process_output docstring and implementation had documented the return value as the result of the finalizer, and the implementation had forwarded that result if passed.

This modifies the docstring and implementation to disregard any result, in accordance with the everywhere-annotated assumption that the finalizer is conceptually void and the absence of any code in GitPython that uses the result of callinghandle_process_output.None is now implicitly returned, simplifying the implementation and bringing it and the docstring in line with annotations and usage.

This would be a breaking change if done on a public function, but becausehandle_process_output is nonpublic (and documentation for it is omitted by Sphinx, so users probably don't wrongly think it is public), I believe this is safe and non-breaking.

This builds on#1755, which mentioned this issue but did not fix it.

However, there is another way to fix it, which is what would have to be done instead of this if the function were public, and which I think is what should be done if, for any reason, it is intended that it continue to forward the result of a finalizer in the event that a finalizer ever returns one. The implementation could be put back to what it was before or something similar to it, and the annotations could be updated to allow the function to return a value.

It occurs to me that this may have been what was intended all along, and that the annotations might not have reflected it due to the absence of@overload support in some type checker that was in use at the time, or perhaps unawareness of it. (GitPython is using@overload elsewhere at present.) I was not sure which approach should be taken to fix the inconsistency inhandle_process_output, so I chose the simpler approach of not forwarding the result.

However, if you would prefer to retain the old behavior (even aspects of it that are not in use in GitPython) and instead fix the annotations, I'd be pleased to do so. In that case, either this pull request could be closed and a new one created, or I could add another commit to it for the change.

The git.cmd.handle_process_output function is non-public(git.cmd.__all__ only lists Git) but used throughout GitPython andreferenced in the git.util.RemoteProgress.new_message_handlerdocstring. Its finalizer argument is annotated to accept anoptional callable that always returns None. It is always used thisway in GitPython and RemoteProcess.new_message_handler is annotatedaccordingly. However, the handle_process_output docstring andimplementation had documented the return value as the result of thefinalizer, and the implementation had forwarded that result ifpassed.This modifies the docstring and implementation to disregard anyresult, in accordance with the everywhere-annotated assumption thatthe finalizer is conceptually void and the absence of any code inGitPython that uses the result of calling handle_process_output.None is now implicitly returned, simplifying the implementation andbringing it and the docstring in line with annotationd and usage.This would be a breaking change if done on a public function, butbecause handle_process_output is nonpublic (and documentation forit is omitted by Sphinx, so users probably don't wrongly think itis public), I believe this is safe and non-breaking.
Copy link
Member

@ByronByron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

That's a great catch, thanks for digging in!

I am sure you checked the observable history to see how this came to be without finding hints of this being intentional, maybe its even 'lost' in history of it ever was.

Let's go with this simple version of dealing with this rightfully assuming that nobody will be affected by this, but consider rolling it back (and make it official) if there is an outcry following a release.

@ByronByron merged commitb2c46f7 intogitpython-developers:mainDec 26, 2023
@EliahKaganEliahKagan deleted the void-finalizer branchDecember 26, 2023 15:17
@EliahKagan
Copy link
MemberAuthor

I am sure you checked the observable history to see how this came to be without finding hints of this being intentional, maybe its even 'lost' in history of it ever was.

I haven't looked at the history of all uses ever ofhandle_process_output; perhaps I should have. But as for the origin of forwarding thefinalizer's result, that goes back to whenhandle_process_output was first introduced in4914405. I presume this was done with the idea that it might eventually be useful to examine the forwarded result, but at that time, although the same commit added a few uses ofhandle_process_output, none of them examined the return value.

Another big moment in the history ofhandle_process_output was85f38a1 (#530), when the signature changed to permitNone in place of a callable forfinalizer. There were no uses ofhandle_process_output at that time that used the result either (at least so far asgit grep -EB1 '\bhandle_process_output\(' 85f38a1 could reveal).

When type annotations were later added to that code inf62c8d8 (#1234), there were likewise no uses ofhandle_process_output that used its return value (going by inspection with the analogousgit grep command). That may be part of why it was annotated the way it was. That may have been the first point at which thefinalizer always returningNone was really formalized.

Beyond looking at those few points in the history, though, I have not investigated further. A deeper investigation might well reveal more.

Let's go with this simple version of dealing with this rightfully assuming that nobody will be affected by this, but consider rolling it back (and make it official) if there is an outcry following a release.

Sounds good.

@Byron
Copy link
Member

I do remember that when pulling, the stderr stream is analysed line by line and vaguely recall that there 'was something' about how callbacks where handled. However, this finalizer doesn't seem to be used to analyse stderr to obtain a remotes progress messages. So probably it's something I miss-associate and thus what we are doing is indeed non-breaking.

@EliahKagan
Copy link
MemberAuthor

handle_process_output has another parameter,stderr_handler, which can also be a callable orNone, whichhandle_process_output calls when non-None, both directly and through itspump_stream local function. However, when it is a callable, it is also expected to returnNone, andhandle_process_output does not forward or otherwise use a result from it (and has not, at least not recently--I haven't looked into its history).

That parameter is used, including when pulling, for reporting progress:

GitPython/git/remote.py

Lines 828 to 855 in32c02d1

def_get_fetch_info_from_stderr(
self,
proc:"Git.AutoInterrupt",
progress:Union[Callable[...,Any],RemoteProgress,None],
kill_after_timeout:Union[None,float]=None,
)->IterableList["FetchInfo"]:
progress=to_progress_instance(progress)
# Skip first line as it is some remote info we are not interested in.
output:IterableList["FetchInfo"]=IterableList("name")
# Lines which are no progress are fetch info lines.
# This also waits for the command to finish.
# Skip some progress lines that don't provide relevant information.
fetch_info_lines= []
# Basically we want all fetch info lines which appear to be in regular form, and
# thus have a command character. Everything else we ignore.
cmds=set(FetchInfo._flag_map.keys())
progress_handler=progress.new_message_handler()
handle_process_output(
proc,
None,
progress_handler,
finalizer=None,
decode_streams=False,
kill_after_timeout=kill_after_timeout,
)

(The function doesn't end there, but that's the relevant part.)

to_progress_instance returns aRemoteProgress object (it is annotated with a return type of a union ofRemoteProgress andCallableRemoteProgress, but the latter inherits from the former). Callingnew_message_handler on that returns a function annotated always to returnNone... yet, its implementation innRemoteProgress (it is not overridden inCallableRemoteProgress) forwards a result from the_parse_progress_line method:

GitPython/git/util.py

Lines 674 to 686 in32c02d1

defnew_message_handler(self)->Callable[[str],None]:
"""
:return:
A progress handler suitable for handle_process_output(), passing lines on to
this Progress handler in a suitable format
"""
defhandler(line:AnyStr)->None:
returnself._parse_progress_line(line.rstrip())
# END handler
returnhandler

But I believe this is always just forwarding an implicitly returnedNone (often due to areturn statement, but never with an operand). The_parse_progress_line method has this signature, which it implementation appears completely true to, both in behavior and in style:

def_parse_progress_line(self,line:AnyStr)->None:

Rather than return a result or status from parsing to the caller, it mutates the state ofself.

It seems to me that thegit.util.RemoteProgress.new_message_handler method should therefore have itshandler local function, which it returns, returnNone implicitly rather than forwarding the implicitNone fromRemoteProgress._parse_progress_line. That change would be:

     def new_message_handler(self) -> Callable[[str], None]:         """         :return:             A progress handler suitable for handle_process_output(), passing lines on to             this Progress handler in a suitable format         """          def handler(line: AnyStr) -> None:-            return self._parse_progress_line(line.rstrip())+            self._parse_progress_line(line.rstrip())          # END handler          return handler

@Byron
Copy link
Member

Thanks for investigating!

The patch above looks like the part that was at some point added because someone had a need for it, and what I vaguely remember might not be without its merit. Whether that is true or not should be in the history of thatreturn self._parse_progress_line(…) though, as it would mean somebody has made this areturn earlier.
But if not, it should be safe to assure there is no return value.

lettuce-botbot referenced this pull request in lettuce-financial/github-bot-signed-commitJan 10, 2024
[![MendRenovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.40` -> `==3.1.41` |[![age](https://developer.mend.io/api/mc/badges/age/pypi/GitPython/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/GitPython/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/GitPython/3.1.40/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/GitPython/3.1.40/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.41`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.41):- fix Windows security issue[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41)The details about the Windows security issue [can be found in thisadvisory](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx).Special thanks go to[@&#8203;EliahKagan](https://togithub.com/EliahKagan) who reported theissue and fixed it in a single stroke, while being responsible for anincredible amount of improvements that he contributed over the lastcouple of months ❤️.#### What's Changed- Add `__all__` in git.exc by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1719](https://togithub.com/gitpython-developers/GitPython/pull/1719)- Set submodule update cadence to weekly by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1721](https://togithub.com/gitpython-developers/GitPython/pull/1721)- Never modify sys.path by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1720](https://togithub.com/gitpython-developers/GitPython/pull/1720)- Bump git/ext/gitdb from `8ec2390` to `ec58b7e` by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1722](https://togithub.com/gitpython-developers/GitPython/pull/1722)- Revise comments, docstrings, some messages, and a bit of code by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1725](https://togithub.com/gitpython-developers/GitPython/pull/1725)- Use zero-argument super() by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1726](https://togithub.com/gitpython-developers/GitPython/pull/1726)- Remove obsolete note in \_iter_packed_refs by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1727](https://togithub.com/gitpython-developers/GitPython/pull/1727)- Reorganize test_util and make xfail marks precise by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1729](https://togithub.com/gitpython-developers/GitPython/pull/1729)- Clarify license and make module top comments more consistent by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1730](https://togithub.com/gitpython-developers/GitPython/pull/1730)- Deprecate compat.is\_<platform>, rewriting all uses by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1732](https://togithub.com/gitpython-developers/GitPython/pull/1732)- Revise and restore some module docstrings by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1735](https://togithub.com/gitpython-developers/GitPython/pull/1735)- Make the rmtree callback Windows-only by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1739](https://togithub.com/gitpython-developers/GitPython/pull/1739)- List all non-passing tests in test summaries by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1740](https://togithub.com/gitpython-developers/GitPython/pull/1740)- Document some minor subtleties in test_util.py by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1749](https://togithub.com/gitpython-developers/GitPython/pull/1749)- Always read metadata files as UTF-8 in setup.py by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1748](https://togithub.com/gitpython-developers/GitPython/pull/1748)- Test native Windows on CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1745](https://togithub.com/gitpython-developers/GitPython/pull/1745)- Test macOS on CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1752](https://togithub.com/gitpython-developers/GitPython/pull/1752)- Let close_fds be True on all platforms by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1753](https://togithub.com/gitpython-developers/GitPython/pull/1753)- Fix IndexFile.from_tree on Windows by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1751](https://togithub.com/gitpython-developers/GitPython/pull/1751)- Remove unused TASKKILL fallback in AutoInterrupt by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1754](https://togithub.com/gitpython-developers/GitPython/pull/1754)- Don't return with operand when conceptually void by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1755](https://togithub.com/gitpython-developers/GitPython/pull/1755)- Group .gitignore entries by purpose by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1758](https://togithub.com/gitpython-developers/GitPython/pull/1758)- Adding dubious ownership handling by[@&#8203;marioaag](https://togithub.com/marioaag) in[https://github.com/gitpython-developers/GitPython/pull/1746](https://togithub.com/gitpython-developers/GitPython/pull/1746)- Avoid brittle assumptions about preexisting temporary files in testsby [@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1759](https://togithub.com/gitpython-developers/GitPython/pull/1759)- Overhaul noqa directives by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1760](https://togithub.com/gitpython-developers/GitPython/pull/1760)- Clarify some Git.execute kill_after_timeout limitations by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1761](https://togithub.com/gitpython-developers/GitPython/pull/1761)- Bump actions/setup-python from 4 to 5 by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1763](https://togithub.com/gitpython-developers/GitPython/pull/1763)- Don't install black on Cygwin by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1766](https://togithub.com/gitpython-developers/GitPython/pull/1766)- Extract all "import gc" to module level by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1765](https://togithub.com/gitpython-developers/GitPython/pull/1765)- Extract remaining local "import gc" to module level by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1768](https://togithub.com/gitpython-developers/GitPython/pull/1768)- Replace xfail with gc.collect in TestSubmodule.test_rename by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1767](https://togithub.com/gitpython-developers/GitPython/pull/1767)- Enable CodeQL by [@&#8203;EliahKagan](https://togithub.com/EliahKagan)in[https://github.com/gitpython-developers/GitPython/pull/1769](https://togithub.com/gitpython-developers/GitPython/pull/1769)- Replace some uses of the deprecated mktemp function by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1770](https://togithub.com/gitpython-developers/GitPython/pull/1770)- Bump github/codeql-action from 2 to 3 by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1773](https://togithub.com/gitpython-developers/GitPython/pull/1773)- Run some Windows environment variable tests only on Windows by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1774](https://togithub.com/gitpython-developers/GitPython/pull/1774)- Fix TemporaryFileSwap regression where file_path could not be Path by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1776](https://togithub.com/gitpython-developers/GitPython/pull/1776)- Improve hooks tests by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1777](https://togithub.com/gitpython-developers/GitPython/pull/1777)- Fix if items of Index is of type PathLike by[@&#8203;stegm](https://togithub.com/stegm) in[https://github.com/gitpython-developers/GitPython/pull/1778](https://togithub.com/gitpython-developers/GitPython/pull/1778)- Better document IterableObj.iter_items and improve some subclasses by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1780](https://togithub.com/gitpython-developers/GitPython/pull/1780)- Revert "Don't install black on Cygwin" by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1783](https://togithub.com/gitpython-developers/GitPython/pull/1783)- Add missing pip in $PATH on Cygwin CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1784](https://togithub.com/gitpython-developers/GitPython/pull/1784)- Shorten Iterable docstrings and put IterableObj first by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1785](https://togithub.com/gitpython-developers/GitPython/pull/1785)- Fix incompletely revised Iterable/IterableObj docstrings by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1786](https://togithub.com/gitpython-developers/GitPython/pull/1786)- Pre-deprecate setting Git.USE_SHELL by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1782](https://togithub.com/gitpython-developers/GitPython/pull/1782)- Deprecate Git.USE_SHELL by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1787](https://togithub.com/gitpython-developers/GitPython/pull/1787)- In handle_process_output don't forward finalizer result by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1788](https://togithub.com/gitpython-developers/GitPython/pull/1788)- Fix mypy warning "Missing return statement" by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1789](https://togithub.com/gitpython-developers/GitPython/pull/1789)- Fix two remaining Windows untrusted search path cases by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1792](https://togithub.com/gitpython-developers/GitPython/pull/1792)#### New Contributors- [@&#8203;marioaag](https://togithub.com/marioaag) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1746](https://togithub.com/gitpython-developers/GitPython/pull/1746)- [@&#8203;stegm](https://togithub.com/stegm) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1778](https://togithub.com/gitpython-developers/GitPython/pull/1778)**Full Changelog**:gitpython-developers/GitPython@3.1.40...3.1.41</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Disabled by config. Please merge this manually once youare satisfied.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about theseupdates again.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/lettuce-financial/github-bot-signed-commit).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
renovatebot referenced this pull request in allenporter/flux-localJan 11, 2024
[![MendRenovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.40` -> `==3.1.41` |[![age](https://developer.mend.io/api/mc/badges/age/pypi/GitPython/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/GitPython/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/GitPython/3.1.40/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/GitPython/3.1.40/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.41`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.41):- fix Windows security issue[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41)The details about the Windows security issue [can be found in thisadvisory](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx).Special thanks go to[@&#8203;EliahKagan](https://togithub.com/EliahKagan) who reported theissue and fixed it in a single stroke, while being responsible for anincredible amount of improvements that he contributed over the lastcouple of months ❤️.#### What's Changed- Add `__all__` in git.exc by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1719](https://togithub.com/gitpython-developers/GitPython/pull/1719)- Set submodule update cadence to weekly by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1721](https://togithub.com/gitpython-developers/GitPython/pull/1721)- Never modify sys.path by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1720](https://togithub.com/gitpython-developers/GitPython/pull/1720)- Bump git/ext/gitdb from `8ec2390` to `ec58b7e` by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1722](https://togithub.com/gitpython-developers/GitPython/pull/1722)- Revise comments, docstrings, some messages, and a bit of code by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1725](https://togithub.com/gitpython-developers/GitPython/pull/1725)- Use zero-argument super() by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1726](https://togithub.com/gitpython-developers/GitPython/pull/1726)- Remove obsolete note in \_iter_packed_refs by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1727](https://togithub.com/gitpython-developers/GitPython/pull/1727)- Reorganize test_util and make xfail marks precise by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1729](https://togithub.com/gitpython-developers/GitPython/pull/1729)- Clarify license and make module top comments more consistent by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1730](https://togithub.com/gitpython-developers/GitPython/pull/1730)- Deprecate compat.is\_<platform>, rewriting all uses by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1732](https://togithub.com/gitpython-developers/GitPython/pull/1732)- Revise and restore some module docstrings by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1735](https://togithub.com/gitpython-developers/GitPython/pull/1735)- Make the rmtree callback Windows-only by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1739](https://togithub.com/gitpython-developers/GitPython/pull/1739)- List all non-passing tests in test summaries by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1740](https://togithub.com/gitpython-developers/GitPython/pull/1740)- Document some minor subtleties in test_util.py by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1749](https://togithub.com/gitpython-developers/GitPython/pull/1749)- Always read metadata files as UTF-8 in setup.py by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1748](https://togithub.com/gitpython-developers/GitPython/pull/1748)- Test native Windows on CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1745](https://togithub.com/gitpython-developers/GitPython/pull/1745)- Test macOS on CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1752](https://togithub.com/gitpython-developers/GitPython/pull/1752)- Let close_fds be True on all platforms by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1753](https://togithub.com/gitpython-developers/GitPython/pull/1753)- Fix IndexFile.from_tree on Windows by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1751](https://togithub.com/gitpython-developers/GitPython/pull/1751)- Remove unused TASKKILL fallback in AutoInterrupt by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1754](https://togithub.com/gitpython-developers/GitPython/pull/1754)- Don't return with operand when conceptually void by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1755](https://togithub.com/gitpython-developers/GitPython/pull/1755)- Group .gitignore entries by purpose by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1758](https://togithub.com/gitpython-developers/GitPython/pull/1758)- Adding dubious ownership handling by[@&#8203;marioaag](https://togithub.com/marioaag) in[https://github.com/gitpython-developers/GitPython/pull/1746](https://togithub.com/gitpython-developers/GitPython/pull/1746)- Avoid brittle assumptions about preexisting temporary files in testsby [@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1759](https://togithub.com/gitpython-developers/GitPython/pull/1759)- Overhaul noqa directives by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1760](https://togithub.com/gitpython-developers/GitPython/pull/1760)- Clarify some Git.execute kill_after_timeout limitations by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1761](https://togithub.com/gitpython-developers/GitPython/pull/1761)- Bump actions/setup-python from 4 to 5 by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1763](https://togithub.com/gitpython-developers/GitPython/pull/1763)- Don't install black on Cygwin by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1766](https://togithub.com/gitpython-developers/GitPython/pull/1766)- Extract all "import gc" to module level by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1765](https://togithub.com/gitpython-developers/GitPython/pull/1765)- Extract remaining local "import gc" to module level by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1768](https://togithub.com/gitpython-developers/GitPython/pull/1768)- Replace xfail with gc.collect in TestSubmodule.test_rename by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1767](https://togithub.com/gitpython-developers/GitPython/pull/1767)- Enable CodeQL by [@&#8203;EliahKagan](https://togithub.com/EliahKagan)in[https://github.com/gitpython-developers/GitPython/pull/1769](https://togithub.com/gitpython-developers/GitPython/pull/1769)- Replace some uses of the deprecated mktemp function by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1770](https://togithub.com/gitpython-developers/GitPython/pull/1770)- Bump github/codeql-action from 2 to 3 by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1773](https://togithub.com/gitpython-developers/GitPython/pull/1773)- Run some Windows environment variable tests only on Windows by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1774](https://togithub.com/gitpython-developers/GitPython/pull/1774)- Fix TemporaryFileSwap regression where file_path could not be Path by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1776](https://togithub.com/gitpython-developers/GitPython/pull/1776)- Improve hooks tests by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1777](https://togithub.com/gitpython-developers/GitPython/pull/1777)- Fix if items of Index is of type PathLike by[@&#8203;stegm](https://togithub.com/stegm) in[https://github.com/gitpython-developers/GitPython/pull/1778](https://togithub.com/gitpython-developers/GitPython/pull/1778)- Better document IterableObj.iter_items and improve some subclasses by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1780](https://togithub.com/gitpython-developers/GitPython/pull/1780)- Revert "Don't install black on Cygwin" by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1783](https://togithub.com/gitpython-developers/GitPython/pull/1783)- Add missing pip in $PATH on Cygwin CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1784](https://togithub.com/gitpython-developers/GitPython/pull/1784)- Shorten Iterable docstrings and put IterableObj first by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1785](https://togithub.com/gitpython-developers/GitPython/pull/1785)- Fix incompletely revised Iterable/IterableObj docstrings by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1786](https://togithub.com/gitpython-developers/GitPython/pull/1786)- Pre-deprecate setting Git.USE_SHELL by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1782](https://togithub.com/gitpython-developers/GitPython/pull/1782)- Deprecate Git.USE_SHELL by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1787](https://togithub.com/gitpython-developers/GitPython/pull/1787)- In handle_process_output don't forward finalizer result by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1788](https://togithub.com/gitpython-developers/GitPython/pull/1788)- Fix mypy warning "Missing return statement" by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1789](https://togithub.com/gitpython-developers/GitPython/pull/1789)- Fix two remaining Windows untrusted search path cases by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1792](https://togithub.com/gitpython-developers/GitPython/pull/1792)#### New Contributors- [@&#8203;marioaag](https://togithub.com/marioaag) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1746](https://togithub.com/gitpython-developers/GitPython/pull/1746)- [@&#8203;stegm](https://togithub.com/stegm) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1778](https://togithub.com/gitpython-developers/GitPython/pull/1778)**Full Changelog**:gitpython-developers/GitPython@3.1.40...3.1.41</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Enabled.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about this updateagain.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/allenporter/flux-local).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xMjcuMCIsInVwZGF0ZWRJblZlciI6IjM3LjEyNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@EliahKagan
Copy link
MemberAuthor

The linereturn self._parse_progress_line(line.rstrip()) was introduced in the initial version of thegit.util.RemoteProgress.new_message_handler method in4914405.

Prior to that commit, there was agit.remote.digest_process_messages function:

defdigest_process_messages(fh,progress):
"""Read progress messages from file-like object fh, supplying the respective
progress messages to the progress instance.
:param fh: File handle to read from
:return: list(line, ...) list of lines without linebreaks that did
not contain progress information"""
line_so_far=b''
dropped_lines=list()
whileTrue:
char=fh.read(1)# reads individual single byte strings
ifnotchar:
break
ifcharin (b'\r',b'\n')andline_so_far:
dropped_lines.extend(progress._parse_progress_line(line_so_far.decode(defenc)))
line_so_far=b''
else:
line_so_far+=char
# END process parsed line
# END while file is not done reading
returndropped_lines

At that time,git.util.RemoteProgress._parse_progress_line did return a value--a list of any lines that failed to be process and were thus dropped while parsing:

GitPython/git/util.py

Lines 166 to 170 ind83f6e8

def_parse_progress_line(self,line):
"""Parse progress information from the given line as retrieved by git-push
or git-fetch
:return: list(line, ...) list of lines that could not be processed"""

git.remote.digest_process_messages had used this functionality to populatedropped_lines (which it returned):

dropped_lines.extend(progress._parse_progress_line(line_so_far.decode(defenc)))

Examining the current behavior, thegit.util.RemoteProgress.new_message_handler method defines and returns a localhandler function (discussed above) that itself calls theRemoteProgress instance's_parse_progress_line method. The implementation of_parse_progress_line inRemoteProgress stores lines in lists on the instance:

GitPython/git/util.py

Lines 582 to 589 in867305d

def_parse_progress_line(self,line:AnyStr)->None:
"""Parse progress information from the given line as retrieved by git-push
or git-fetch.
- Lines that do not contain progress info are stored in :attr:`other_lines`.
- Lines that seem to contain an error (i.e. start with ``error:`` or ``fatal:``)
are stored in :attr:`error_lines`.
"""

GitPython/git/util.py

Lines 601 to 603 in867305d

ifself._cur_line.startswith(("error:","fatal:")):
self.error_lines.append(self._cur_line)
return

GitPython/git/util.py

Lines 624 to 627 in867305d

ifnotmatch:
self.line_dropped(line_str)
self.other_lines.append(line_str)
return

RemoteProgress._parse_progress_line is annotated to returnNone, its implementation always implicitly returnsNone (either by returning without an operand or falling off the end), andCallableRemoteProgress does not override that method.

There is also a subclass in the test suite.test_remote.TestRemoteProgress overrides_parse_progress_line. It delegates most of its work to the superclass implementation and forwards its result:

def_parse_progress_line(self,line):
# We may remove the line later if it is dropped.
# Keep it for debugging.
self._seen_lines.append(line)
rval=super()._parse_progress_line(line)
returnrval

That result is alwaysNone these days, so this could and should be changed to:

def_parse_progress_line(self,line):# We may remove the line later if it is dropped.# Keep it for debugging.self._seen_lines.append(line)super()._parse_progress_line(line)

The tests don't do anything with that returned result. They do not themselves call_parse_progress_line. So I am inclined to say the handler created bynew_message_handler could also call_parse_progress_line with noreturn keyword, in accordance with the annotations and current behavior.

But it's possible that, based on code intest_remote.py, a project using the GitPython library may be subclassingRemoteProgress and defining_parse_progress_line to return something. So that should be considered.

@Byron
Copy link
Member

Thanks for the update!

But it's possible that, based on code intest_remote.py, a project using the GitPython library may be subclassingRemoteProgress and defining_parse_progress_line to return something. So that should be considered.

Yes, APIs that support inheritance would essentially be locked-in to certain behaviours, and it is definitely allowed (and maybe useful) to let subclasses override this method and return something where the base-implementation would only returnNone, implicitly or explicitly.

Due to the vague memory I have, I'd be hesitant to change thereturn statements outside of tests to keep supporting such sub-class implementations, as otherwise it also doesn't hurt the code to do as it does now.

Maybe this helps with clarifying documentation.

@EliahKagan
Copy link
MemberAuthor

Yes, I think that information will help.

If it is considered reasonable for new code to override_parse_progress_line with an implementation that returns a non-None value, then I think it may be worthwhile to broaden the type annotations.

If it is not considered reasonable, then maybe a docstring could caution that it is unexpected, or recommend against it, or describe it as deprecated.

@Byron
Copy link
Member

If it is considered reasonable for new code to override_parse_progress_line with an implementation that returns a non-None value, then I think it may be worthwhile to broaden the type annotations.

I'd definitely go with this and broaden the type-annotations specifically to represent the status quo, assuming that it is or at some point was reasonable. With that said, I also wouldn't specifically advertise it.

@EliahKagan
Copy link
MemberAuthor

EliahKagan commentedFeb 20, 2024•
edited
Loading

In hindsight, I should've opened an issue for this so it would not be forgotten about, because a substantial amount of useful information has accumulated here, and unfortunately I may not end up opening a PR for it for some time. I don't want to discourage anyone else from doing so, but I think my approach would be to wait until:

  1. Most of themypy errors are solved, so it is easier to make subjective judgments about clarity of annotations, as well as easier to notice if one has introduced an actual error or made it harder to solve one that already exists.

  2. All actual deprecations are clearly indicated, so it is easy to make clear the distinction between a deprecation and something that is just documented as not being recommended to use in most situations.

    (ThePEP 702@warnings.deprecated decorator may help with this, since it is meant to work with@overload, though I don't think it's necessary to wait for that to come in before fixing this. The way it would help is that if it is used for all actual deprecations where a function or some overloads are deprecated, then itsabsence here will help clarify the distinction.)

I can make an issue that describes this and links here for more information. Or you can remind me of this in the future when something related to it comes up, if you think of it. Or both.

@Byron
Copy link
Member

Thanks for bringing this up.

I can make an issue that describes this and links here for more information. Or you can remind me of this in the future when something related to it comes up, if you think of it. Or both.

I think it would be best to have a reminder-feature with the summary and back-link here for details, as I cannot vouch for the 'me of the future' - judging by my current track record, chances of me remembering would be way lower than yours 😅.

@EliahKagan
Copy link
MemberAuthor

I think it would be best to have a reminder

I've opened#1834 for this.

Byron reacted with thumbs up emoji

JoeWang1127 referenced this pull request in googleapis/sdk-platform-javaApr 6, 2024
[![MendRenovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.40` -> `==3.1.41` |[![age](https://developer.mend.io/api/mc/badges/age/pypi/GitPython/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/GitPython/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/GitPython/3.1.40/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/GitPython/3.1.40/3.1.41?slim=true)](https://docs.renovatebot.com/merge-confidence/)|---> [!WARNING]> Some dependencies could not be looked up. Check the DependencyDashboard for more information.### GitHub Vulnerability Alerts####[CVE-2024-22190](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx)### SummaryThis issue exists because of an incomplete fix forCVE-2023-40590. OnWindows, GitPython uses an untrusted search path if it uses a shell torun `git`, as well as when it runs `bash.exe` to interpret hooks. Ifeither of those features are used on Windows, a malicious `git.exe` or`bash.exe` may be run from an untrusted repository.### DetailsAlthough GitPython often avoids executing programs found in an untrustedsearch path since 3.1.33, two situations remain where this still occurs.Either can allow arbitrary code execution under some circumstances.#### When a shell is usedGitPython can be told to run `git` commands through a shell rather thanas direct subprocesses, by passing `shell=True` to any method thataccepts it, or by both setting `Git.USE_SHELL = True` and not passing`shell=False`. Then the Windows `cmd.exe` shell process performs thepath search, and GitPython does not prevent that shell from finding andrunning `git` in the current directory.When GitPython runs `git` directly rather than through a shell, theGitPython process performs the path search, and currently omits thecurrent directory by setting `NoDefaultCurrentDirectoryInExePath` in itsown environment during the `Popen` call. Although the `cmd.exe` shellwill honor this environment variable when present, GitPython does notcurrently pass it into the shell subprocess's environment.Furthermore, because GitPython sets the subprocess CWD to the root of arepository's working tree, using a shell will run a malicious `git.exe`in an untrusted repository even if GitPython itself is run from atrusted location.This also applies if `Git.execute` is called directly with `shell=True`(or after `Git.USE_SHELL = True`) to run any command.#### When hook scripts are runOn Windows, GitPython uses `bash.exe` to run hooks that appear to bescripts. However, unlike when running `git`, no steps are taken to avoidfinding and running `bash.exe` in the current directory.This allows the author of an untrusted fork or branch to cause amalicious `bash.exe` to be run in some otherwise safe workflows. Anexample of such a scenario is if the user installs a trusted hook whileon a trusted branch, then switches to an untrusted feature branch(possibly from a fork) to review proposed changes. If the untrustedfeature branch contains a malicious `bash.exe` and the user's currentworking directory is the working tree, and the user performs an actionthat runs the hook, then although the hook itself is uncorrupted, itruns with the malicious `bash.exe`.Note that, while `bash.exe` is a shell, this is a separate scenario fromwhen `git` is run using the unrelated Windows `cmd.exe` shell.### PoCOn Windows, create a `git.exe` file in a repository. Then create a`Repo` object, and call any method through it (directly or indirectly)that supports the `shell` keyword argument with `shell=True`:```powershellmkdir testrepogit init testrepocp ... testrepo git.exe # Replace "..." with any executable of choice.python -c "import git; print(git.Repo('testrepo').git.version(shell=True))"```The `git.exe` executable in the repository directory will be run.Or use no `Repo` object, but do it from the location with the `git.exe`:```powershellcd testrepopython -c "import git; print(git.Git().version(shell=True))"```The `git.exe` executable in the current directory will be run.For the scenario with hooks, install a hook in a repository, create a`bash.exe` file in the current directory, and perform an operation thatcauses GitPython to attempt to run the hook:```powershellmkdir testrepocd testrepogit initmv .git/hooks/pre-commit.sample .git/hooks/pre-commitcp ... bash.exe # Replace "..." with any executable of choice.echo "Some text" >file.txtgit add file.txtpython -c "import git; git.Repo().index.commit('Some message')"```The `bash.exe` executable in the current directory will be run.### ImpactThe greatest impact is probably in applications that set `Git.USE_SHELL= True` for historical reasons. (Undesired console windows had, in thepast, been created in some kinds of applications, when it was not used.)Such an application may be vulnerable to arbitrary code execution from amalicious repository, even with no other exacerbating conditions. Thisis to say that, if a shell is used to run `git`, the full effect ofCVE-2023-40590 is still present. Furthermore, as noted above, runningthe application itself from a trusted directory is not a sufficientmitigation.An application that does not direct GitPython to use a shell to run`git` subprocesses thus avoids most of the risk. However, there is nosuch straightforward way to prevent GitPython from running `bash.exe` tointerpret hooks. So while the conditions needed for that to be exploitedare more involved, it may be harder to mitigate decisively prior topatching.### Possible solutionsA straightforward approach would be to address each bug directly:- When a shell is used, pass `NoDefaultCurrentDirectoryInExePath` intothe subprocess environment, because in that scenario the subprocess isthe `cmd.exe` shell that itself performs the path search.- Set `NoDefaultCurrentDirectoryInExePath` in the GitPython processenvironment during the `Popen` call made to run hooks with a `bash.exe`subprocess.These need only be done on Windows.---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.41`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.41):- fix Windows security issue[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.40...3.1.41)The details about the Windows security issue [can be found in thisadvisory](https://togithub.com/gitpython-developers/GitPython/security/advisories/GHSA-2mqj-m65w-jghx).Special thanks go to[@&#8203;EliahKagan](https://togithub.com/EliahKagan) who reported theissue and fixed it in a single stroke, while being responsible for anincredible amount of improvements that he contributed over the lastcouple of months ❤️.#### What's Changed- Add `__all__` in git.exc by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1719](https://togithub.com/gitpython-developers/GitPython/pull/1719)- Set submodule update cadence to weekly by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1721](https://togithub.com/gitpython-developers/GitPython/pull/1721)- Never modify sys.path by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1720](https://togithub.com/gitpython-developers/GitPython/pull/1720)- Bump git/ext/gitdb from `8ec2390` to `ec58b7e` by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1722](https://togithub.com/gitpython-developers/GitPython/pull/1722)- Revise comments, docstrings, some messages, and a bit of code by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1725](https://togithub.com/gitpython-developers/GitPython/pull/1725)- Use zero-argument super() by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1726](https://togithub.com/gitpython-developers/GitPython/pull/1726)- Remove obsolete note in \_iter_packed_refs by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1727](https://togithub.com/gitpython-developers/GitPython/pull/1727)- Reorganize test_util and make xfail marks precise by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1729](https://togithub.com/gitpython-developers/GitPython/pull/1729)- Clarify license and make module top comments more consistent by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1730](https://togithub.com/gitpython-developers/GitPython/pull/1730)- Deprecate compat.is\_<platform>, rewriting all uses by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1732](https://togithub.com/gitpython-developers/GitPython/pull/1732)- Revise and restore some module docstrings by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1735](https://togithub.com/gitpython-developers/GitPython/pull/1735)- Make the rmtree callback Windows-only by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1739](https://togithub.com/gitpython-developers/GitPython/pull/1739)- List all non-passing tests in test summaries by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1740](https://togithub.com/gitpython-developers/GitPython/pull/1740)- Document some minor subtleties in test_util.py by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1749](https://togithub.com/gitpython-developers/GitPython/pull/1749)- Always read metadata files as UTF-8 in setup.py by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1748](https://togithub.com/gitpython-developers/GitPython/pull/1748)- Test native Windows on CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1745](https://togithub.com/gitpython-developers/GitPython/pull/1745)- Test macOS on CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1752](https://togithub.com/gitpython-developers/GitPython/pull/1752)- Let close_fds be True on all platforms by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1753](https://togithub.com/gitpython-developers/GitPython/pull/1753)- Fix IndexFile.from_tree on Windows by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1751](https://togithub.com/gitpython-developers/GitPython/pull/1751)- Remove unused TASKKILL fallback in AutoInterrupt by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1754](https://togithub.com/gitpython-developers/GitPython/pull/1754)- Don't return with operand when conceptually void by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1755](https://togithub.com/gitpython-developers/GitPython/pull/1755)- Group .gitignore entries by purpose by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1758](https://togithub.com/gitpython-developers/GitPython/pull/1758)- Adding dubious ownership handling by[@&#8203;marioaag](https://togithub.com/marioaag) in[https://github.com/gitpython-developers/GitPython/pull/1746](https://togithub.com/gitpython-developers/GitPython/pull/1746)- Avoid brittle assumptions about preexisting temporary files in testsby [@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1759](https://togithub.com/gitpython-developers/GitPython/pull/1759)- Overhaul noqa directives by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1760](https://togithub.com/gitpython-developers/GitPython/pull/1760)- Clarify some Git.execute kill_after_timeout limitations by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1761](https://togithub.com/gitpython-developers/GitPython/pull/1761)- Bump actions/setup-python from 4 to 5 by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1763](https://togithub.com/gitpython-developers/GitPython/pull/1763)- Don't install black on Cygwin by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1766](https://togithub.com/gitpython-developers/GitPython/pull/1766)- Extract all "import gc" to module level by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1765](https://togithub.com/gitpython-developers/GitPython/pull/1765)- Extract remaining local "import gc" to module level by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1768](https://togithub.com/gitpython-developers/GitPython/pull/1768)- Replace xfail with gc.collect in TestSubmodule.test_rename by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1767](https://togithub.com/gitpython-developers/GitPython/pull/1767)- Enable CodeQL by [@&#8203;EliahKagan](https://togithub.com/EliahKagan)in[https://github.com/gitpython-developers/GitPython/pull/1769](https://togithub.com/gitpython-developers/GitPython/pull/1769)- Replace some uses of the deprecated mktemp function by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1770](https://togithub.com/gitpython-developers/GitPython/pull/1770)- Bump github/codeql-action from 2 to 3 by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1773](https://togithub.com/gitpython-developers/GitPython/pull/1773)- Run some Windows environment variable tests only on Windows by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1774](https://togithub.com/gitpython-developers/GitPython/pull/1774)- Fix TemporaryFileSwap regression where file_path could not be Path by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1776](https://togithub.com/gitpython-developers/GitPython/pull/1776)- Improve hooks tests by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1777](https://togithub.com/gitpython-developers/GitPython/pull/1777)- Fix if items of Index is of type PathLike by[@&#8203;stegm](https://togithub.com/stegm) in[https://github.com/gitpython-developers/GitPython/pull/1778](https://togithub.com/gitpython-developers/GitPython/pull/1778)- Better document IterableObj.iter_items and improve some subclasses by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1780](https://togithub.com/gitpython-developers/GitPython/pull/1780)- Revert "Don't install black on Cygwin" by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1783](https://togithub.com/gitpython-developers/GitPython/pull/1783)- Add missing pip in $PATH on Cygwin CI by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1784](https://togithub.com/gitpython-developers/GitPython/pull/1784)- Shorten Iterable docstrings and put IterableObj first by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1785](https://togithub.com/gitpython-developers/GitPython/pull/1785)- Fix incompletely revised Iterable/IterableObj docstrings by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1786](https://togithub.com/gitpython-developers/GitPython/pull/1786)- Pre-deprecate setting Git.USE_SHELL by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1782](https://togithub.com/gitpython-developers/GitPython/pull/1782)- Deprecate Git.USE_SHELL by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1787](https://togithub.com/gitpython-developers/GitPython/pull/1787)- In handle_process_output don't forward finalizer result by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1788](https://togithub.com/gitpython-developers/GitPython/pull/1788)- Fix mypy warning "Missing return statement" by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1789](https://togithub.com/gitpython-developers/GitPython/pull/1789)- Fix two remaining Windows untrusted search path cases by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1792](https://togithub.com/gitpython-developers/GitPython/pull/1792)#### New Contributors- [@&#8203;marioaag](https://togithub.com/marioaag) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1746](https://togithub.com/gitpython-developers/GitPython/pull/1746)- [@&#8203;stegm](https://togithub.com/stegm) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1778](https://togithub.com/gitpython-developers/GitPython/pull/1778)**Full Changelog**:gitpython-developers/GitPython@3.1.40...3.1.41</details>---### Configuration📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (noschedule defined).🚦 **Automerge**: Disabled by config. Please merge this manually once youare satisfied.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about this updateagain.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/googleapis/sdk-platform-java).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ByronByronByron approved these changes

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@EliahKagan@Byron
[8]ページ先頭
©2009-2025 Movatter.jp