Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork947
Fix Git.execute shell use and reporting bugs#1687
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
That test is not testing whether or not a shell is used (nor doesit need to test that), but just whether the library actually runsgit, passes an argument to it, and returns text from its stdout.
In the Popen calls in Git.execute, for all combinations of allowedvalues for Git.USE_SHELL and the shell= keyword argument.
The log message shows "Popen(...)", not "execute(...)". So itshould faithfully report what is about to be passed to Popen incases where a user reading the log would otherwise be misled intothinking this is what has happened.Reporting the actual "shell=" argument passed to Popen is also moreuseful to log than the argument passed to Git.execute (at least ifonly one of them is to be logged).
Both new shell-related tests were causing the code under test tosplit "git" into ["g", "i", "t"] and thus causing the code undertest to attempt to execute a "g" command. This passes the commandas a one-element list of strings rather than as a string, whichavoids this on all operating systems regardless of whether the codeunder test has a bug being tested for.This would not occur on Windows, which does not iterate commands oftype str character-by-character even when the command is runwithout a shell. But it did happen on other systems.Most of the time, the benefit of using a command that actually runs"git" rather than "g" is the avoidance of confusion in the errormessage. But this is also important because it is possible for theuser who runs the tests to have a "g" command, in which case itcould be very inconvenient, or even unsafe, to run "g". This shouldbe avoided even when the code under test has a bug that causes ashell to be used when it shouldn't or not used when it should, sohaving separate commands (list and str) per test case parameterswould not be a sufficient solution: it would only guard againstrunning "g" when a bug in the code under test were absent.
This also helps mock Popen over a smaller scope, which may bebeneficial (especially if it is mocked in the subprocess module,rather than the git.cmd module, in the future).
Because mock.call.kwargs, i.e. the ability to examinem.call_args.kwargs where m is a Mock or MagicMock, was introducedin Python 3.8.Currently it is only in test/test_git.py that any use of mocksrequires this, so I've put the conditional import logic to importmock (the top-level package) rather than unittest.mock only there.The mock library is added as a development (testing) dependencyonly when the Python version is lower than 3.8, so it is notinstalled when not needed.This fixes a problem in the new tests of whether a shell is used,and reported as used, in the Popen call in Git.execute. Thosejust-introduced tests need this, to be able to usemock_popen.call_args.kwargs on Python 3.7.
This updates the shell variable itself, only when it is None, fromself.USE_SHELL.(That attribute is usually Git.USE_SHELL rather than an instanceattribute. But although people probably shouldn't set it oninstances, people will expect that this is permitted.)Now:- USE_SHELL is used as a fallback only. When shell=False is passed, USE_SHELL is no longer consuled. Thus shell=False always keeps a shell from being used, even in the non-default case where the USE_SHELL attribue has been set to True.- The debug message printed to the log shows the actual value that is being passed to Popen, because the updated shell variable is used both to produce that message and in the Popen call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks so much for the fix which brings the behaviour in line with the documentation, and for adding regression protection.
This PR is also a wonderful example on what's possible with theunittest
andmock
libraries and generally what's possible with python testing.
🙏
[](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.37` -> `==3.1.40` |[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.40`](https://togithub.com/gitpython-developers/GitPython/compare/3.1.38...3.1.40)[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.38...3.1.40)###[`v3.1.38`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.38)[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.37...3.1.38)#### What's Changed- Add missing assert keywords by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1678](https://togithub.com/gitpython-developers/GitPython/pull/1678)- Make clear every test's status in every CI run by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1679](https://togithub.com/gitpython-developers/GitPython/pull/1679)- Fix new link to license in readme by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1680](https://togithub.com/gitpython-developers/GitPython/pull/1680)- Drop unneeded flake8 suppressions by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1681](https://togithub.com/gitpython-developers/GitPython/pull/1681)- Update instructions and test helpers for git-daemon by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1684](https://togithub.com/gitpython-developers/GitPython/pull/1684)- Fix Git.execute shell use and reporting bugs by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1687](https://togithub.com/gitpython-developers/GitPython/pull/1687)- No longer allow CI to select a prerelease for 3.12 by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1689](https://togithub.com/gitpython-developers/GitPython/pull/1689)- Clarify Git.execute and Popen arguments by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1688](https://togithub.com/gitpython-developers/GitPython/pull/1688)- Ask git where its daemon is and use that by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1697](https://togithub.com/gitpython-developers/GitPython/pull/1697)- Fix bugs affecting exception wrapping in rmtree callback by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1700](https://togithub.com/gitpython-developers/GitPython/pull/1700)- Fix dynamically-set **all** variable by[@​DeflateAwning](https://togithub.com/DeflateAwning) in[https://github.com/gitpython-developers/GitPython/pull/1659](https://togithub.com/gitpython-developers/GitPython/pull/1659)- Fix small[#​1662](https://togithub.com/gitpython-developers/GitPython/issues/1662)regression due to[#​1659](https://togithub.com/gitpython-developers/GitPython/issues/1659)by [@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1701](https://togithub.com/gitpython-developers/GitPython/pull/1701)- Drop obsolete info on yanking from security policy by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1703](https://togithub.com/gitpython-developers/GitPython/pull/1703)- Have Dependabot offer submodule updates by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1702](https://togithub.com/gitpython-developers/GitPython/pull/1702)- Bump git/ext/gitdb from `49c3178` to `8ec2390` by[@​dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1704](https://togithub.com/gitpython-developers/GitPython/pull/1704)- Bump git/ext/gitdb from `8ec2390` to `6a22706` by[@​dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1705](https://togithub.com/gitpython-developers/GitPython/pull/1705)- Update readme for milestone-less releasing by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1707](https://togithub.com/gitpython-developers/GitPython/pull/1707)- Run Cygwin CI workflow commands in login shells by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1709](https://togithub.com/gitpython-developers/GitPython/pull/1709)#### New Contributors- [@​DeflateAwning](https://togithub.com/DeflateAwning) made theirfirst contribution in[https://github.com/gitpython-developers/GitPython/pull/1659](https://togithub.com/gitpython-developers/GitPython/pull/1659)**Full Changelog**:gitpython-developers/GitPython@3.1.37...3.1.38</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Enabled.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about this updateagain.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/allenporter/flux-local).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xOS4yIiwidXBkYXRlZEluVmVyIjoiMzcuMTkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Uh oh!
There was an error while loading.Please reload this page.
Fixes#1685
Fixes#1686
Changes to
git
module codeThis updates the
shell
variable itself, only when it isNone
, fromself.USE_SHELL
.(That attribute usually gets its value from
Git.USE_SHELL
rather than being separately an instance attribute. Butself.USE_SHELL
is an idiomatic way to access it. Furthermore, although people probably shouldn't set it on instances, people may expect that this is permitted.)Now:
USE_SHELL
is used as a fallback only. Whenshell=False
is passed,USE_SHELL
is no longer consulted. Thusshell=False
always keeps a shell from being used, even in the non-default case where theUSE_SHELL
attribute has been set toTrue
.Popen
, because the updated shell variable is used both to produce that message and in thePopen
call.New tests
This also (actually, first) adds tests, some to check that the correct value is always passed as
shell=
whenGit.execute
callsPopen
, and others to check that whatever it passes is the same as it claims to be passing in the log. I've made them two separate test methods rather than a test method with multiple assertions, because I think the results of these particular tests are easier to understand that way (even compared to the option of usingself.subTest
). But the major shared logic is extracted to a helper function, where its important subtleties are commented. I have verified that the tests fail, in the expected way, before the bugs are fixed, and then pass, and that these are both the case even on native Windows where we don't (yet) have CI jobs.The tests cover the six combinations of valid values of the
shell=
argument toGit.execute
(None
,False
, andTrue
) andGit.USE_SHELL
(False
andTrue
). I usedddt
for this, so there are only two test methods, even though there are six test cases. The tests are simple in principle, with the main source of complexity being the question of how to deal with how the command argument toPopen
typically differs, including bytype, based on theshell
argument, yet the tests must not misbehave, nor induce extraneous misbehavior, even when the code under test has a bug that would ordinarily prevent such agreement.On Python 3.7 only, this adds
mock
as a development dependency (in thetest
extra, not as a regular dependency).mock
is a backport ofunittest.mock
, and a couple of my assertions use a handy feature that was introduced to the mock library in Python 3.8. Besides being harder to write without it, I believe it would be less clearly expressed. (Details are commented with the import logic.)While adding the new tests, I also renamed
test_it_executes_git_to_shell_and_returns_result
totest_it_executes_git_and_returns_result
, because that test case does not test that a shell is used to rungit
(and a shell isnot used in that test).