Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Clarify Git.execute and Popen arguments#1688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Byron merged 11 commits intogitpython-developers:mainfromEliahKagan:execute-args
Oct 3, 2023

Conversation

EliahKagan
Copy link
Member

@EliahKaganEliahKagan commentedOct 3, 2023
edited
Loading

This is a sequel to#1687, improving how parameters toGit.execute are documented, and improving debug logging and tests of how they affect the arguments passed toPopen.

In theGit.execute docstring, the method is no longer described as using a shell, since this is typically (and by default) not the case; the items documenting each parameter are listed in the order of the parameters; and some copyediting is done, for clarity, consistency, spacing, and in a few cases other formatting. I copyedited some other docstrings in the module as well.

The order of names in theexecute_kwargs set is tweaked to also match the order in which they appear as parameters toGit.execute. (This is just for the purpose of code clarity, asset objects guarantee no particular iteration order.) Since not all unintentional mismatches between this set and the defined method parameters would cause existing tests to fail, and the failures that would occur would not always immediately show the cause of the problem, I also added a test thatexecute_kwargs has the exact expected relationship to the parameters ofGit.execute. (Though an alternative might be to generateexecute_kwargs programmatically fromGit.execute using a similar technique.)

One part of the test logic in#1687 was unnecessarily complicated, due to swallowing an exception produced by runninggit with no arguments. This changes that by passingwith_exceptions=False so that exception is never generated.

Another part of the the test logic in#1687 combined claims about the code under test with custom assertion logic in a way that made it hard to see what claims were being made by reading the test code. This fixes that by generalizing, and extracting out, an_assert_logged_for_popen test helper method. I also took this opportunity to remove its unstated incompatibility with value representations containing regular expression metacharacters. (Please note that the new code is still only robust enough for the special purpose for which it is intended; it does not actually parse the debug message rigorously.)

As requested in#1686 (comment), I changedistream= tostdin= in the debug logging message documenting thePopen call. I reused the test helper in writing a new test,test_it_logs_istream_summary_for_stdin,
which checks both that it has the new name and that it has the expected simplified value representation. I did not change any parameters or variables calledistream tostdin or any other name; rather, this changes the message to better reflect thePopen call it exists to document.

If I would only have changed the displayed parameter name, I would likely not have written a test, but I also wanted to change two other things, in which the test helped verify that correctness was maintained: I reordered the displayedname=value representations in the log message to match the relative order in which they are passed toPopen. And I eliminated theistream_ok variable (which was named like a boolean flag but was not one), because having the logic for producing the string in the logging call makes it better resemble the nonidentical but corresponding logic in thePopen call, so they can be compared.

Although this is peripheral to the core concept of the clarity of function arguments, I also renamed and fixed the docstring of a local function that appeared (including by claiming) to be a method.

- Reorder the items in the git.cmd.Git.execute docstring that  document its parameters, to be in the same order the parameters  are actually defined in.- Use consistent spacing, by having a blank line between successive  items that document parameters. Before, most of them were  separated this way, but some of them were not.- Reorder the elements of execute_kwargs (which list all those  parameters except the first parameter, command) to be listed in  that order as well. These were mostly in order, but a couple were  out of order. This is just about the order they appear in the  definition, since sets in Python (unlike dicts) have no key order  guarantees.
The top line of the Git.execute docstring said that it used ashell, which is not necessarily the case (and is not usually thecase, since the default is not to use one). This removes thatclaim while keeping the top-line wording otherwise the same.It also rephrases the description of the command parameter, in away that does not change its meaning but reflects the more commonpractice of passing a sequence of arguments (since portable callsthat do not use a shell must do that).
These are some small clarity and consistency revisions to thedocstring of git.cmd.Git.execute that didn't specifically fit inthe narrow topics of either of the preceding two commits.
(Not specific to git.cmd.Git.execute.)
The kill_process local function defined in the Git.execute methodis a local function and not a method, but it was easy to misread asa method for two reasons:- Its docstring described it as a method.- It was named with a leading underscore, as though it were a  nonpublic method. But this name is a local variable, and local  variables are always nonpublic (except when they are function  parameters, in which case they are in a sense public). A leading  underscore in a local variable name usually means the variable is  unused in the function.This fixes the docstring and drops the leading underscore from thename. If this is ever extracted from the Git.execute method andplaced at class or module scope, then the name can be changed back.
Instead of swallowing GitCommandError exceptions in the helper usedby test_it_uses_shell_or_not_as_specified andtest_it_logs_if_it_uses_a_shell, this modifies the helper so itprevents Git.execute from raising the exception in the first place.
This extracts the logic of searching log messages, and assertingthat (at least) one matches a pattern for the report of a Popencall with a given argument, from test_it_logs_if_it_uses_a_shellinto a new nonpublic test helper method _assert_logged_for_popen.The extracted version is modified to make it slightly more general,and slightly more robust. This is still not extremely robust: thenotation used to log Popen calls is informal, so it wouldn't makesense to really parse it as code. But this no longer assumes thatthe representation of a value ends at a word boundary, nor that thevalue is free of regular expression metacharacters.
This changes how the Popen call debug logging line shows theinformal summary of what kind of thing is being passed as the stdinargument to Popen, showing it with stdin= rather than istream=.The new test, with "istream" in place of "stdin", passed before thecode change in the git.cmd module, failed once "istream" waschanged to "stdin" in the test, and then, as expected, passed againonce "istream=" was changed to "stdin=" in the log.debug call ingit.cmd.Git.execute.
This is still not including all or even most of the arguments, norare all the logged arguments literal (nor should either of thosethings likely be changed). It is just to facilitate easiercomparison of what is logged to the Popen call in the code.
In Git.execute, the stdin argument to Popen is the only one where acompound expression (rather than a single term) is currentlypassed. So having that be the same in the log message makes iteasier to understand what is going on, as well as to see how theinformation shown in the log corresponds to what Popen receives.
@EliahKaganEliahKagan marked this pull request as ready for reviewOctober 3, 2023 15:34
Comment on lines +378 to +384
def test_execute_kwargs_set_agrees_with_method(self):
parameter_names = inspect.signature(cmd.Git.execute).parameters.keys()
self_param, command_param, *most_params, extra_kwargs_param = parameter_names
self.assertEqual(self_param, "self")
self.assertEqual(command_param, "command")
self.assertEqual(set(most_params), cmd.execute_kwargs) # Most important.
self.assertEqual(extra_kwargs_param, "subprocess_kwargs")
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Orgit.cmd.execute_kwargs could be generated in a manner similar to this, and this test, the need to manually update it, and the note in theGit.execute docstring about that need, could all be done away with. Maybe something like this:

execute_kwargs=inspect.signature(Git.execute).parameters.keys()- {"self","command","subprocess_kwargs"}

Right now it is defined very high up in the module, even higher than__all__, and this suggests that it may be intended to be available for static inspection. But I don't think any tools will statically inspect aset of strings like that (plus, static analysis toolscan examine the parameters ofGit.execute... though the incomplete lists of parameters in the@overload-decorated stubs that precede it confuse the situation somewhat).

git.cmd.__all__ contains only"Git" and I hope that means code that uses GitPython should not be usinggit.cmd.execute_kwargs or relying on its presence. If possible, perhaps it could be made more explicitly private (_execute_kwargs, or if it needs to remain statically defined, maybe_EXECUTE_KWARGS) or, even better, removed altogether if theGit.execute method's arguments can be inspected efficiently enough without it whereexecute_kwargs is currently used. On the other hand, people may have come to depend on it even though the presence of__all__ that omits it means no oneshould have depended on it.

Anyway, I do not think any of the changes I suggest in this comment need to be made in this pull request. But I wanted to mention the issue just in case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Removingexecute_kwargs seems like a reduction of complexity, which would always be a valuable reduction of maintenance costs should changes need to be made.

Asexecute_kwargs was never advertised in__all__ I'd think that it's fair to say that those who depend on it nonetheless new the risk. I think the same argument is valid knowing that nothing is ever truly private, everything can be introspected if one truly wants to, yet it's something one simply has to ignore in order to be able to make any changes to python software once released.

EliahKagan reacted with thumbs up emoji
Comment on lines -69 to +72
"stdout_as_string",
"output_stream",
"with_stdout",
"stdout_as_string",
"kill_after_timeout",
"with_stdout",
Copy link
MemberAuthor

@EliahKaganEliahKaganOct 3, 2023
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This changes only the order in which they are given, not the contents, and the set is equal to the same value as before.

However, I am wondering ifcommand should actually be added here. Although the@overload-decorated stubs forGit.execute define some parameters as keyword-only, in the actual definition no parameter is keyword-only and there is definitional symmetry betweencommand and the others. Should I worry about what happens if someone has agit-command script that they can usually run asgit command and tries to rung.command() whereg is agit.cmd.Git instance? Or related situations?

Another ramification of the parameters not being keyword-only is that, because they can be passed positionally, it is a breaking change to add a new one toGit.execute elsewhere than the end. Still, regarding#1686 (comment), if you think astdin parameter should be added as a synonym ofistream, this could still be done even withstdin at the end, with the docstring items that document the parameters referring to each other to overcome any confusion. I am inclined to say the added complexity is not worthwhile (for example, the function would have to handle the case where they are both passed and with inconsistent values). But a possible argument against this is that thetext synonym ofuniversal_newlines could also be added.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Should I worry about what happens if someone has agit-command script that they can usually run asgit command and tries to rung.command() whereg is agit.cmd.Git instance? Or related situations?

In these situations it would be great to know whycommand was put there in the first place, and I for one do not remember. I know there never was an issue related tocommand specifically, which seems to indicate it's a limitation worth ignoring or fixing in a non-breaking fashion (which seems possible).

Regardingistream, I'd think maintaining stability would let me sleep better at night especially since you seem to agree that it's used consistently here and ingitdb. Probably along with the documentation improvements, all is well and better than it was before, which even in the past didn't seem to have caused enough confusion to cause an issue to be opened.

Copy link
MemberAuthor

@EliahKaganEliahKaganOct 3, 2023
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

In these situations it would be great to know whycommand was put there in the first place, and I for one do not remember.

Do you mean why it wasomitted from theexecute_kwargs set? If so, my guess is that it was intended to be used positionally while the others were intended to be keyword-only. This is judging by how they are given as keyword-only arguments in the@overloads, where they are preceded by a*, item in the list of formal parameters, which is absent in the actual function definition (thus causing them to accept both positional and keyword arguments).

But it may be that I am misunderstanding what you mean here. It is also possible that their keyword-only status in the@overloads was intentionally different from the ultimate definition and intended to provide guidance. (I'm not sure. The@overloads are missing most of the arguments, which confuses tooling and seems unintentional.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I am happy to adopt your conclusion in that this is not intentional and since there are negative side-effects, like tooling not working correctly, it's certainly something that could be improved.
If against all odds such an action does create downstream breakage, it could also be undone with a patch release.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't think we should make the parameters in the real definition actually keyword-only, since that is a breaking change. Or, at least, I would not rush to that. It seems likely to me that, at least for the first few, people are passing them positionally.

I believe it is merely the absence of many parameters from the@overload-decorated stubs (which precede the real definition) that is causing VS Code not to show or autocomplete some arguments. Adding the missing parameters to the@overload-decorated stubs should be sufficient to fix that, and it shouldn't be a breaking change because(a) they are just type stubs, so it doesn't break runtime behavior,(b) the actual change seems compatible even relative to what the type stubs said before, and(c) GitPython doesn't have static typing stability currently anyway, because althoughpy.typed is included in the package,mypy checks don't pass (nor, per#1659, dopyright checks pass).

Byron reacted with thumbs up emoji
@ByronByron added this to thev3.1.38 - Bugfixes milestoneOct 3, 2023
Copy link
Member

@ByronByron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks a lot for this second round of improvements!

It's at the core of GitPython and any improvements is most certainly valued downstream as long as it's not subtly breaking, which you made sure of.

With that said, I remember that this module is also used to start two long-running python processes lazily when objects are accessed, along with various calls togit for doing pretty much anything else.

Also thanks to your work the idea of usinggitoxide (or rather its still to be created python-bindings) has been forming in my head, which could lead togit invocations to be reduced to zero while probably performing as well or better. Of course, anything like it is still years away, but it's something I look forward to.

Comment on lines +378 to +384
def test_execute_kwargs_set_agrees_with_method(self):
parameter_names = inspect.signature(cmd.Git.execute).parameters.keys()
self_param, command_param, *most_params, extra_kwargs_param = parameter_names
self.assertEqual(self_param, "self")
self.assertEqual(command_param, "command")
self.assertEqual(set(most_params), cmd.execute_kwargs) # Most important.
self.assertEqual(extra_kwargs_param, "subprocess_kwargs")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Removingexecute_kwargs seems like a reduction of complexity, which would always be a valuable reduction of maintenance costs should changes need to be made.

Asexecute_kwargs was never advertised in__all__ I'd think that it's fair to say that those who depend on it nonetheless new the risk. I think the same argument is valid knowing that nothing is ever truly private, everything can be introspected if one truly wants to, yet it's something one simply has to ignore in order to be able to make any changes to python software once released.

EliahKagan reacted with thumbs up emoji
@ByronByron merged commit91f63cd intogitpython-developers:mainOct 3, 2023
@EliahKaganEliahKagan deleted the execute-args branchOctober 3, 2023 17:54
renovatebot referenced this pull request in allenporter/flux-localOct 20, 2023
[![MendRenovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.37` -> `==3.1.40` |[![age](https://developer.mend.io/api/mc/badges/age/pypi/GitPython/3.1.40?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/GitPython/3.1.40?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/GitPython/3.1.37/3.1.40?slim=true)](https://docs.renovatebot.com/merge-confidence/)|[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/GitPython/3.1.37/3.1.40?slim=true)](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.40`](https://togithub.com/gitpython-developers/GitPython/compare/3.1.38...3.1.40)[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.38...3.1.40)###[`v3.1.38`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.38)[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.37...3.1.38)#### What's Changed- Add missing assert keywords by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1678](https://togithub.com/gitpython-developers/GitPython/pull/1678)- Make clear every test's status in every CI run by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1679](https://togithub.com/gitpython-developers/GitPython/pull/1679)- Fix new link to license in readme by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1680](https://togithub.com/gitpython-developers/GitPython/pull/1680)- Drop unneeded flake8 suppressions by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1681](https://togithub.com/gitpython-developers/GitPython/pull/1681)- Update instructions and test helpers for git-daemon by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1684](https://togithub.com/gitpython-developers/GitPython/pull/1684)- Fix Git.execute shell use and reporting bugs by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1687](https://togithub.com/gitpython-developers/GitPython/pull/1687)- No longer allow CI to select a prerelease for 3.12 by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1689](https://togithub.com/gitpython-developers/GitPython/pull/1689)- Clarify Git.execute and Popen arguments by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1688](https://togithub.com/gitpython-developers/GitPython/pull/1688)- Ask git where its daemon is and use that by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1697](https://togithub.com/gitpython-developers/GitPython/pull/1697)- Fix bugs affecting exception wrapping in rmtree callback by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1700](https://togithub.com/gitpython-developers/GitPython/pull/1700)- Fix dynamically-set **all** variable by[@&#8203;DeflateAwning](https://togithub.com/DeflateAwning) in[https://github.com/gitpython-developers/GitPython/pull/1659](https://togithub.com/gitpython-developers/GitPython/pull/1659)- Fix small[#&#8203;1662](https://togithub.com/gitpython-developers/GitPython/issues/1662)regression due to[#&#8203;1659](https://togithub.com/gitpython-developers/GitPython/issues/1659)by [@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1701](https://togithub.com/gitpython-developers/GitPython/pull/1701)- Drop obsolete info on yanking from security policy by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1703](https://togithub.com/gitpython-developers/GitPython/pull/1703)- Have Dependabot offer submodule updates by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1702](https://togithub.com/gitpython-developers/GitPython/pull/1702)- Bump git/ext/gitdb from `49c3178` to `8ec2390` by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1704](https://togithub.com/gitpython-developers/GitPython/pull/1704)- Bump git/ext/gitdb from `8ec2390` to `6a22706` by[@&#8203;dependabot](https://togithub.com/dependabot) in[https://github.com/gitpython-developers/GitPython/pull/1705](https://togithub.com/gitpython-developers/GitPython/pull/1705)- Update readme for milestone-less releasing by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1707](https://togithub.com/gitpython-developers/GitPython/pull/1707)- Run Cygwin CI workflow commands in login shells by[@&#8203;EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1709](https://togithub.com/gitpython-developers/GitPython/pull/1709)#### New Contributors- [@&#8203;DeflateAwning](https://togithub.com/DeflateAwning) made theirfirst contribution in[https://github.com/gitpython-developers/GitPython/pull/1659](https://togithub.com/gitpython-developers/GitPython/pull/1659)**Full Changelog**:gitpython-developers/GitPython@3.1.37...3.1.38</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Enabled.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about this updateagain.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/allenporter/flux-local).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xOS4yIiwidXBkYXRlZEluVmVyIjoiMzcuMTkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ByronByronByron approved these changes

Assignees
No one assigned
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
@EliahKagan@Byron

[8]ページ先頭

©2009-2025 Movatter.jp