Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Improve scripts and tool configurations#1693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Byron merged 32 commits intogitpython-developers:mainfromEliahKagan:sh
Oct 4, 2023
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
32 commits
Select commitHold shift + click to select a range
8c4df3c
Add pre-commit hook to run shellcheck
EliahKaganSep 27, 2023
f3be76f
Force color when running shellcheck in pre-commit
EliahKaganSep 27, 2023
7dd8add
Suppress SC2086 where word splitting is intended
EliahKaganSep 27, 2023
21875b5
Don't split and glob the interpreter name
EliahKaganSep 27, 2023
0920371
Extract suggest_venv out of the else block
EliahKaganSep 27, 2023
e973f52
Use some handy bash-isms in version check script
EliahKaganSep 27, 2023
be53823
Have init script treat master unambiguously as a branch
EliahKaganSep 27, 2023
e604b46
Use 4-space indentation in all shell scripts
EliahKaganSep 27, 2023
19dfbd8
Make the init script a portable POSIX shell script
EliahKaganSep 27, 2023
7110bf8
Move extra tag-fetching step into init script
EliahKaganSep 27, 2023
c7cdaf4
Reduce code duplication in version check script
EliahKaganSep 27, 2023
f6dbba2
A couple more script tweaks for clarity
EliahKaganSep 27, 2023
5060c9d
Explain what each step in the init script achieves
EliahKaganSep 27, 2023
d5479b2
Use set -u in init script
EliahKaganSep 28, 2023
52f9a68
Make the "all" Makefile target more robust
EliahKaganSep 28, 2023
b88d07e
Use a single awk instead of two greps and a cut
EliahKaganSep 28, 2023
d36818c
Add a black check to pre-commit
EliahKaganSep 28, 2023
4ba5ad1
Fix typo in comment
EliahKaganSep 28, 2023
5d8ddd9
Use two hooks for black: to check, and format
EliahKaganSep 28, 2023
a872d9c
Pass --all-files explicitly so it is retained
EliahKaganOct 3, 2023
9b9de11
Fix the formatting
EliahKaganOct 3, 2023
5d15063
Add "make lint" to lint without auto-formatting
EliahKaganSep 28, 2023
6de86a8
Update readme about most of the test/lint tools
EliahKaganSep 28, 2023
f094909
Add BUILDDIR var to doc/Makefile; have tox use it
EliahKaganSep 28, 2023
fc96980
Have init script check for GitHub Actions
EliahKaganSep 28, 2023
b98f15e
Get tags for tests from original repo as fallback
EliahKaganSep 29, 2023
7cca7d2
Don't print the exact same warning twice
EliahKaganSep 29, 2023
e4e009d
Reword comment to fix ambiguity
EliahKaganSep 29, 2023
e16e4c0
Format all YAML files in the same style
EliahKaganSep 29, 2023
62c024e
Let tox run lint, mypy, and html envs without 3.9
EliahKaganSep 29, 2023
9e245d0
Update readme: CI jobs not just for "main" branch
EliahKaganOct 1, 2023
c2472e9
Note that the init script can be run from Git Bash
EliahKaganOct 1, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion.github/dependabot.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -3,4 +3,4 @@ updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
interval: "weekly"
2 changes: 1 addition & 1 deletion.github/workflows/cygwin-test.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -40,7 +40,7 @@ jobs:

- name: Prepare this repo for tests
run: |
TRAVIS=yes./init-tests-after-clone.sh
./init-tests-after-clone.sh

- name: Set git user identity and command aliases for the tests
run: |
Expand Down
4 changes: 4 additions & 0 deletions.github/workflows/lint.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,3 +14,7 @@ jobs:
python-version: "3.x"

- uses: pre-commit/action@v3.0.0
with:
extra_args: --all-files --hook-stage manual
env:
SKIP: black-format
2 changes: 1 addition & 1 deletion.github/workflows/pythonpackage.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -37,7 +37,7 @@ jobs:

- name: Prepare this repo for tests
run: |
TRAVIS=yes./init-tests-after-clone.sh
./init-tests-after-clone.sh

- name: Set git user identity and command aliases for the tests
run: |
Expand Down
52 changes: 37 additions & 15 deletions.pre-commit-config.yaml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,39 @@
repos:
- repo: https://github.com/PyCQA/flake8
rev:6.1.0
hooks:
- id:flake8
additional_dependencies:
- flake8-bugbear==23.9.16
- flake8-comprehensions==3.14.0
- flake8-typing-imports==1.14.0
exclude: ^doc|^git/ext/
- repo: https://github.com/psf/black-pre-commit-mirror
rev:23.9.1
hooks:
- id:black
alias: black-check
name: black (check)
args: [--check, --diff]
exclude: ^git/ext/
stages: [manual]

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-merge-conflict
- id: check-toml
- id: check-yaml
- id: black
alias: black-format
name: black (format)
exclude: ^git/ext/

- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
hooks:
- id: flake8
additional_dependencies:
- flake8-bugbear==23.9.16
- flake8-comprehensions==3.14.0
- flake8-typing-imports==1.14.0
exclude: ^doc|^git/ext/

- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.9.0.5
hooks:
- id: shellcheck
args: [--color]
exclude: ^git/ext/

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-toml
- id: check-yaml
- id: check-merge-conflict
7 changes: 5 additions & 2 deletionsMakefile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
.PHONY: all clean release force_release
.PHONY: alllintclean release force_release

all:
@grep -Ee'^[a-z].*:' Makefile| cut -d: -f1| grep -vF all
@awk -F:'/^[[:alpha:]].*:/ && !/^all:/ {print $$1}' Makefile

lint:
SKIP=black-format pre-commit run --all-files --hook-stage manual

clean:
rm -rf build/ dist/ .eggs/ .tox/
Expand Down
42 changes: 28 additions & 14 deletionsREADME.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -76,17 +76,20 @@ To clone the [the GitHub repository](https://github.com/gitpython-developers/Git
```bash
git clone https://github.com/gitpython-developers/GitPython
cd GitPython
git fetch --tags
./init-tests-after-clone.sh
```

On Windows, `./init-tests-after-clone.sh` can be run in a Git Bash shell.

If you are cloning [your own fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/about-forks), then replace the above `git clone` command with one that gives the URL of your fork. Or use this [`gh`](https://cli.github.com/) command (assuming you have `gh` and your fork is called `GitPython`):

```bash
gh repo clone GitPython
```

Having cloned the repo, create and activate your [virtual environment](https://docs.python.org/3/tutorial/venv.html). Then make an [editable install](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs):
Having cloned the repo, create and activate your [virtual environment](https://docs.python.org/3/tutorial/venv.html).

Then make an [editable install](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs):

```bash
pip install -e ".[test]"
Expand DownExpand Up@@ -114,9 +117,9 @@ See [Issue #525](https://github.com/gitpython-developers/GitPython/issues/525).

### RUNNING TESTS

_Important_: Right after cloning this repository, please be sure to have
executed `git fetch --tags` followed bythe `./init-tests-after-clone.sh`
script in the repository root. Otherwiseyou will encounter test failures.
_Important_: Right after cloning this repository, please be sure to have executed
the `./init-tests-after-clone.sh` script in the repository root. Otherwise
you will encounter test failures.

On _Windows_, make sure you have `git-daemon` in your PATH. For MINGW-git, the `git-daemon.exe`
exists in `Git\mingw64\libexec\git-core\`.
Expand All@@ -143,30 +146,41 @@ To test, run:
pytest
```

To lint, run:
To lint,and apply automatic code formatting,run:

```bash
pre-commit run --all-files
```

- Linting without modifying code can be done with: `make lint`
- Auto-formatting without other lint checks can be done with: `black .`

To typecheck, run:

```bash
mypy -p git
```

For automatic code formatting, run:
#### CI (and tox)

```bash
black .
```
The same linting, and running tests on all the different supported Python versions, will be performed:

- Upon submitting a pull request.
- On each push, *if* you have a fork with GitHub Actions enabled.
- Locally, if you run [`tox`](https://tox.wiki/) (this skips any Python versions you don't have installed).

#### Configuration files

Specific tools:

Configuration for flake8 is in the `./.flake8` file.
- Configurations for `mypy`, `pytest`, `coverage.py`, and `black` are in `./pyproject.toml`.
- Configuration for `flake8` is in the `./.flake8` file.

Configurations for `mypy`, `pytest`, `coverage.py`, and `black` are in `./pyproject.toml`.
Orchestration tools:

The same linting and testing will also be performed against different supported python versions
upon submitting a pull request (or on each push if you have a fork with a "main" branch and actions enabled).
- Configuration for `pre-commit` is in the `./.pre-commit-config.yaml` file.
- Configuration for `tox` is in `./tox.ini`.
- Configuration for GitHub Actions (CI) is in files inside `./.github/workflows/`.

### Contributions

Expand Down
15 changes: 8 additions & 7 deletionsbuild-release.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -6,21 +6,22 @@
set -eEu

function release_with() {
$1 -m build --sdist --wheel
"$1" -m build --sdist --wheel
}

if test -n "${VIRTUAL_ENV:-}"; then
function suggest_venv() {
local venv_cmd='python -m venv env && source env/bin/activate'
printf "HELP: To avoid this error, use a virtual-env with '%s' instead.\n" "$venv_cmd"
}

if test -n "${VIRTUAL_ENV-}"; then
deps=(build twine) # Install twine along with build, as we need it later.
echo "Virtual environment detected. Adding packages: ${deps[*]}"
pip install --quiet --upgrade "${deps[@]}"
echo 'Starting the build.'
release_with python
else
function suggest_venv() {
venv_cmd='python -m venv env && source env/bin/activate'
printf "HELP: To avoid this error, use a virtual-env with '%s' instead.\n" "$venv_cmd"
}
trap suggest_venv ERR # This keeps the original exit (error) code.
echo 'Starting the build.'
release_with python3 # Outside a venv, use python3.
release_with python3# Outside a venv, use python3.
fi
26 changes: 18 additions & 8 deletionscheck-version.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,29 +10,39 @@ trap 'echo "$0: Check failed. Stopping." >&2' ERR
readonly version_path='VERSION'
readonly changes_path='doc/source/changes.rst'

function check_status() {
git status -s "$@"
test -z "$(git status -s "$@")"
}

function get_latest_tag() {
local config_opts
printf -v config_opts ' -c versionsort.suffix=-%s' alpha beta pre rc RC
# shellcheck disable=SC2086 # Deliberately word-splitting the arguments.
git $config_opts tag -l '[0-9]*' --sort=-v:refname | head -n1
}

echo 'Checking current directory.'
test "$(cd -- "$(dirname -- "$0")" && pwd)" = "$(pwd)" # Ugly, but portable.

echo "Checking that $version_path and $changes_path exist and have no uncommitted changes."
test -f "$version_path"
test -f "$changes_path"
git status -s -- "$version_path" "$changes_path"
test -z "$(git status -s -- "$version_path" "$changes_path")"
check_status -- "$version_path" "$changes_path"

# This section can be commented out, if absolutely necessary.
echo 'Checking that ALL changes are committed.'
git status -s --ignore-submodules
test -z "$(git status -s --ignore-submodules)"
check_status --ignore-submodules

version_version="$(cat"$version_path")"
version_version="$(<"$version_path")"
changes_version="$(awk '/^[0-9]/ {print $0; exit}' "$changes_path")"
config_opts="$(printf ' -c versionsort.suffix=-%s' alpha beta pre rc RC)"
latest_tag="$(git $config_opts tag -l '[0-9]*' --sort=-v:refname | head -n1)"
latest_tag="$(get_latest_tag)"
head_sha="$(git rev-parse HEAD)"
latest_tag_sha="$(git rev-parse "${latest_tag}^{commit}")"

# Display a table of all the current version, tag, and HEAD commit information.
echo $'\nThe VERSION must be the same in all locations, and so must the HEAD and tag SHA'
echo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This conveys spacing much better thanks to the separateecho line, even though I have a feeling this was done for portability as$'magic' might not be allowed everywhere.

Copy link
MemberAuthor

@EliahKaganEliahKaganOct 4, 2023
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I did it for spacing, and to to takefuller advantage ofecho to have fewer\n sequences. Although it's true that$'' is not POSIX and shouldn't be used in a#!/bin/sh script, this script is abash script and$'' is a available inbash on all systems. (ksh,zsh, and probably some other shells also have$'', but not all POSIX-compatible shells have it.)

Regarding spacing, the table is currently formatted usingprintf and this has the advantage that its own output spacing can be adjusted by, for example, changing14 to15. But I originally usedprintf for it because I was preferringprintf toecho, because originally I wasdoing this inline inMakefile, and aMakefile runs commands using/bin/sh (unlessSHELL is assigned in theMakefile itself). This was to follow the practice of avoidingecho in portablesh scripts,at least when displaying data read from files.

But that is not necessary in a#!/bin/bash script, where we know howecho works and whereecho doesn't expand escape sequences by default. So it would actually be okay, at this point, to go back to displaying the table this way:

echoecho'The VERSION must be the same in all locations, and so must the HEAD and tag SHA'echo"VERSION file   =$version_version"echo"changes.rst    =$changes_version"echo"Latest tag     =$latest_tag"echo"HEAD SHA       =$head_sha"echo"Latest tag SHA =$latest_tag_sha"

This is arguably more readable, and arguably conveys spacing the best. I'd be happy to change it to this if you like.

(Another option could be to go in the opposite direction and make thischeck-version.sh script, and maybe thebuild-release.sh script as well, a#!/bin/sh script, which is feasible.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is arguably more readable, and arguably conveys spacing the best. I'd be happy to change it to this if you like.
(Another option could be to go in the opposite direction and make thischeck-version.sh script, and maybe thebuild-release.sh script as well, a#!/bin/sh script, which is feasible.)

Sincetrap is supported in POSIX shells and thussh, making these scripts more portable seems valuable, even though I think that as the project currently is setup there isn't any need as it's only me using them.
Alternatives involve running these on CI which also supportsbash everywhere (at least so it seems).

If you think there is benefits to portability assuming that one day CI can perform trusted publishes, I think it's fair to adjust for portability next time you touch them. Otherwise, it would be just as fair to changeprintf withecho as shown above. It's perfectly optional as well.

Copy link
MemberAuthor

@EliahKaganEliahKaganOct 9, 2023
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The desired interaction oftrap andset -e may be achievable insh. Even if not, we don't have to usetrap; it can be replaced even without making the control-flow logic more complicated or error prone. For example, the script could accept-q to not append the failure message, and otherwise invoke itself with-q, check the result, print the message if it failed, and exit with the original failing exit code. (There is alsoan argument to be made thatset -e shouldn't be relied on at all.)

There are not many systems wherebash doesn't exist and can't be made available. However, due to the problem described in729372f--which only happens when MinGWmake is being used on a native Windows system that has also WSL installed, and does not happen outsidemake nor on other systems--the scripts'bash hashbangs are written as#!/bin/bash rather than#!/usr/bin/env bash, and thus assumebash is in/bin.

Making them#!/bin/sh scripts would also fix that. It is even safer to assumesh is inbin than to assumeenv is in/usr/bin, so#!/usr/bin/env sh is rarely used. (Anyway, there is no corresponding WSL-relatedsh.exe inSystem32.) So, when combined with the other minor reasons for using#!/bin/sh, that is probably worth doing.

As you've suggested, next time I work on those scripts I'll look into making them POSIXsh scripts. However, this would be unrelated to facilitating trusted publishing. CI does supportbash everywhere, at least on runners hosted by GitHub Actions.

Byron reacted with thumbs up emoji
echo 'The VERSION must be the same in all locations, and so must the HEAD and tag SHA'
printf '%-14s = %s\n' 'VERSION file' "$version_version" \
'changes.rst' "$changes_version" \
'Latest tag' "$latest_tag" \
Expand Down
43 changes: 22 additions & 21 deletionsdoc/Makefile
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,14 +2,15 @@
#

# You can set these variables from the command line.
BUILDDIR = build
SPHINXOPTS = -W
SPHINXBUILD = sphinx-build
PAPER =

# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -dbuild/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
ALLSPHINXOPTS = -d$(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source

.PHONY: help clean html web pickle htmlhelp latex changes linkcheck

Expand All@@ -24,52 +25,52 @@ help:
@echo " linkcheck to check all external links for integrity"

clean:
-rm -rfbuild/*
-rm -rf$(BUILDDIR)/*

html:
mkdir -pbuild/htmlbuild/doctrees
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS)build/html
mkdir -p$(BUILDDIR)/html$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS)$(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are inbuild/html."
@echo "Build finished. The HTML pages are in$(BUILDDIR)/html."

pickle:
mkdir -pbuild/picklebuild/doctrees
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS)build/pickle
mkdir -p$(BUILDDIR)/pickle$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS)$(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."

web: pickle

json:
mkdir -pbuild/jsonbuild/doctrees
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS)build/json
mkdir -p$(BUILDDIR)/json$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS)$(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."

htmlhelp:
mkdir -pbuild/htmlhelpbuild/doctrees
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS)build/htmlhelp
mkdir -p$(BUILDDIR)/htmlhelp$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS)$(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file inbuild/htmlhelp."
".hhp project file in$(BUILDDIR)/htmlhelp."

latex:
mkdir -pbuild/latexbuild/doctrees
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS)build/latex
mkdir -p$(BUILDDIR)/latex$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS)$(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are inbuild/latex."
@echo "Build finished; the LaTeX files are in$(BUILDDIR)/latex."
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
"run these through (pdf)latex."

changes:
mkdir -pbuild/changesbuild/doctrees
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS)build/changes
mkdir -p$(BUILDDIR)/changes$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS)$(BUILDDIR)/changes
@echo
@echo "The overview file is inbuild/changes."
@echo "The overview file is in$(BUILDDIR)/changes."

linkcheck:
mkdir -pbuild/linkcheckbuild/doctrees
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS)build/linkcheck
mkdir -p$(BUILDDIR)/linkcheck$(BUILDDIR)/doctrees
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS)$(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or inbuild/linkcheck/output.txt."
"or in$(BUILDDIR)/linkcheck/output.txt."
Loading

[8]ページ先頭

©2009-2025 Movatter.jp