Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork938
Test in Docker with Alpine Linux on CI#1826
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Merged
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
With only one version of Python, currently 3.11.
This handles the "dubious ownership" error for the Alpine Linuxcontainer using safe.directory as is done in the Cygwin job.Another approach may be to actually use a limited user account inthe container, though, and that may be better, since I expect someof the rmtree tests in test_util.py to fail due to the root userbeing able to perform a delete operation the tests assume cannot bedone.
To overcome "This environment is externally managed" blocker.
- Add a note to the fixture in test_util.py that its ability to create files where rmtree will fail is contingent on not running as root (since root doesn't need to own a dir to delete from it).- Create a non-root user in the container. Give it the same UID as owns the repository files that are shared with the container. Also create a group with the GID of the repository files that are shared with the container and add the user to the group, though that is less important. Actually creating the user ensures it has a home directory and may help some commands work. Passing `options: --user 1001` under `container:` will not work because, even if we didn't make the user, the `apk add` commands still need to run as root.- Run all commands as the new non-root user, except for the system administration commands that install needed apk packages and set up the new non-root user account. To continue clearly expressing each step separately and have them automatically run in the container, this uses the hacky approach of having the default shell be a "sudo" command that runs the script step with "sh" (and passes the desired shell arguments).- Preserve environment variables that may have been set by or for the GHA runner, in commands that run as the non-root user. That is, pass those through, while still removing/resetting others. If this is not done, then the variables such as `CI`, which the init-tests-after-clone.sh uses to proceed without interactive confirmation, will not be set, and that step will fail. However, I think it is also a good idea to do this, which is why I've included all the relevant variables and not just `CI`.- Now that a non-root user is using "pip", stop using a venv, at least for now. The other test jobs don't use one, since the runners are isolated, and a container on a runner is even more isolated. But it may be best to bring the venv back, maybe even on the other test jobs, or alternatively to use "python -m pip" instead of "pip", to ensure expected version of pip is used.- Don't add safe.directory inside the container, in the hope that this may not be necessary because the cloned repository files should have the same UID (and even GID) as the user using them. But I expect this may need to be put back; it seems to be needed separately from that, as actions/checkout automatically attempts it for the git command it finds and attempts to use.This is not the only approach that could work. Another approach isto make use of the container explicit in each step, rather thanusing the `container` key. I think that would make the relationshipbetween the commands here and in other test workflows less apparentand make the workflow a bit less clear, but it could also simplifythings. A third approach is to create an image with the needed apkpackages and user account, which switches to that user, by writinga Dockerfile and building in image, producing it in a previousjob and sharing the image with the job that runs the tests so that`container` can still be used. That might be ideal if it could bedone with upload-artifact and download-artifact, but I think`container` only supports getting images from registries.
As anticipated, it is still needed.
The "error: externally-managed-environment" stoppage occurs evenwhen the Alpine Linux python command is run by a non-root user.
We chown the workspace, so this shouldn't be needed.This commit also removes the "Debug ownership" step.
Byron approved these changesFeb 16, 2024
Member
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thank you so much for this wonderful addition!
GitPython really is tested rigorously by now with the suite improving over time, given that features aren't usually added, and with the platform coverage increasing. So despite being in maintenance mode, it does manage to maintain and increase its quality. Thank you, again and again, for your tremendous work with this!
I appreciate the early PR and believe this setup can change as it has to, and according to your preferences, in the future without problems.
Reading about having a test-matrix for this, I thought maybe multiple containers can run at the same time so they are pre-started? The CI agent should have enough memory to this.
MemberAuthor
EliahKagan commentedFeb 21, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
A GitHub Actions job can run multiple containers at a time without problems. This happens automatically when a workflow defines a job using the But this is unrelated to the matrix strategy feature, since separate jobs always run on separate runners. (GitHub-hosted runners are virtual machines that are created and destroyed on demand per-job. With self-hosted runners, two jobs may run on the same runner, but not, I don't think, concurrently.) A job with a docker run --rm --privileged multiarch/qemu-user-static --reset -p yes However, that or a similar command has to actually run on the runner, or its effect otherwise achieved on the runner. Giving it as the value of a |
Thanks for the clarification - it's definitely quite a bit more complicated 😅. |
renovatebot referenced this pull request in allenporter/flux-localMar 31, 2024
[](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.42` -> `==3.1.43` |[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.43`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.43)[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.42...3.1.43)#### Particularly Important ChangesThese are likely to affect you, please do take a careful look.- Issue and test deprecation warnings by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1886](https://togithub.com/gitpython-developers/GitPython/pull/1886)- Fix version_info cache invalidation, typing, parsing, andserialization by [@​EliahKagan](https://togithub.com/EliahKagan)in[https://github.com/gitpython-developers/GitPython/pull/1838](https://togithub.com/gitpython-developers/GitPython/pull/1838)- Document manual refresh path treatment by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1839](https://togithub.com/gitpython-developers/GitPython/pull/1839)- Improve static typing and docstrings related to git object types by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1859](https://togithub.com/gitpython-developers/GitPython/pull/1859)#### Other Changes- Test in Docker with Alpine Linux on CI by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1826](https://togithub.com/gitpython-developers/GitPython/pull/1826)- Build online docs (RTD) with -W and dependencies by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1843](https://togithub.com/gitpython-developers/GitPython/pull/1843)- Suggest full-path refresh() in failure message by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1844](https://togithub.com/gitpython-developers/GitPython/pull/1844)- `repo.blame` and `repo.blame_incremental` now accept `None` as the`rev` parameter. by [@​Gaubbe](https://togithub.com/Gaubbe) in[https://github.com/gitpython-developers/GitPython/pull/1846](https://togithub.com/gitpython-developers/GitPython/pull/1846)- Make sure diff always uses the default diff driver when`create_patch=True` by[@​can-taslicukur](https://togithub.com/can-taslicukur) in[https://github.com/gitpython-developers/GitPython/pull/1832](https://togithub.com/gitpython-developers/GitPython/pull/1832)- Revise docstrings, comments, and a few messages by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1850](https://togithub.com/gitpython-developers/GitPython/pull/1850)- Expand what is included in the API Reference by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1855](https://togithub.com/gitpython-developers/GitPython/pull/1855)- Restore building of documentation downloads by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1856](https://togithub.com/gitpython-developers/GitPython/pull/1856)- Revise type annotations slightly by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1860](https://togithub.com/gitpython-developers/GitPython/pull/1860)- Updating regex pattern to handle unicode whitespaces. by[@​jcole-crowdstrike](https://togithub.com/jcole-crowdstrike) in[https://github.com/gitpython-developers/GitPython/pull/1853](https://togithub.com/gitpython-developers/GitPython/pull/1853)- Use upgraded pip in test fixture virtual environment by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1864](https://togithub.com/gitpython-developers/GitPython/pull/1864)- lint: replace `flake8` with `ruff` check by[@​Borda](https://togithub.com/Borda) in[https://github.com/gitpython-developers/GitPython/pull/1862](https://togithub.com/gitpython-developers/GitPython/pull/1862)- lint: switch Black with `ruff-format` by[@​Borda](https://togithub.com/Borda) in[https://github.com/gitpython-developers/GitPython/pull/1865](https://togithub.com/gitpython-developers/GitPython/pull/1865)- Update readme and tox.ini for recent tooling changes by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1868](https://togithub.com/gitpython-developers/GitPython/pull/1868)- Split tox lint env into three envs, all safe by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1870](https://togithub.com/gitpython-developers/GitPython/pull/1870)- Slightly broaden Ruff, and update and clarify tool configuration by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1871](https://togithub.com/gitpython-developers/GitPython/pull/1871)- Add a "doc" extra for documentation build dependencies by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1872](https://togithub.com/gitpython-developers/GitPython/pull/1872)- Describe `Submodule.__init__` parent_commit parameter by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1877](https://togithub.com/gitpython-developers/GitPython/pull/1877)- Include TagObject in git.types.Tree_ish by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1878](https://togithub.com/gitpython-developers/GitPython/pull/1878)- Improve Sphinx role usage, including linking Git manpages by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1879](https://togithub.com/gitpython-developers/GitPython/pull/1879)- Replace all wildcard imports with explicit imports by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1880](https://togithub.com/gitpython-developers/GitPython/pull/1880)- Clarify how tag objects are usually tree-ish and commit-ish by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1881](https://togithub.com/gitpython-developers/GitPython/pull/1881)#### New Contributors- [@​Gaubbe](https://togithub.com/Gaubbe) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1846](https://togithub.com/gitpython-developers/GitPython/pull/1846)- [@​can-taslicukur](https://togithub.com/can-taslicukur) madetheir first contribution in[https://github.com/gitpython-developers/GitPython/pull/1832](https://togithub.com/gitpython-developers/GitPython/pull/1832)- [@​jcole-crowdstrike](https://togithub.com/jcole-crowdstrike)made their first contribution in[https://github.com/gitpython-developers/GitPython/pull/1853](https://togithub.com/gitpython-developers/GitPython/pull/1853)- [@​Borda](https://togithub.com/Borda) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1862](https://togithub.com/gitpython-developers/GitPython/pull/1862)**Full Changelog**:gitpython-developers/GitPython@3.1.42...3.1.43</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Enabled.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about this updateagain.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/allenporter/flux-local).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
%5d(https%3a%2f%2frenovatebot.com){kind=link}
lettuce-botbot referenced this pull request in lettuce-financial/github-bot-signed-commitApr 1, 2024
[](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [GitPython](https://togithub.com/gitpython-developers/GitPython) |`==3.1.42` -> `==3.1.43` |[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|[](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>gitpython-developers/GitPython (GitPython)</summary>###[`v3.1.43`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.43)[CompareSource](https://togithub.com/gitpython-developers/GitPython/compare/3.1.42...3.1.43)#### Particularly Important ChangesThese are likely to affect you, please do take a careful look.- Issue and test deprecation warnings by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1886](https://togithub.com/gitpython-developers/GitPython/pull/1886)- Fix version_info cache invalidation, typing, parsing, andserialization by [@​EliahKagan](https://togithub.com/EliahKagan)in[https://github.com/gitpython-developers/GitPython/pull/1838](https://togithub.com/gitpython-developers/GitPython/pull/1838)- Document manual refresh path treatment by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1839](https://togithub.com/gitpython-developers/GitPython/pull/1839)- Improve static typing and docstrings related to git object types by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1859](https://togithub.com/gitpython-developers/GitPython/pull/1859)#### Other Changes- Test in Docker with Alpine Linux on CI by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1826](https://togithub.com/gitpython-developers/GitPython/pull/1826)- Build online docs (RTD) with -W and dependencies by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1843](https://togithub.com/gitpython-developers/GitPython/pull/1843)- Suggest full-path refresh() in failure message by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1844](https://togithub.com/gitpython-developers/GitPython/pull/1844)- `repo.blame` and `repo.blame_incremental` now accept `None` as the`rev` parameter. by [@​Gaubbe](https://togithub.com/Gaubbe) in[https://github.com/gitpython-developers/GitPython/pull/1846](https://togithub.com/gitpython-developers/GitPython/pull/1846)- Make sure diff always uses the default diff driver when`create_patch=True` by[@​can-taslicukur](https://togithub.com/can-taslicukur) in[https://github.com/gitpython-developers/GitPython/pull/1832](https://togithub.com/gitpython-developers/GitPython/pull/1832)- Revise docstrings, comments, and a few messages by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1850](https://togithub.com/gitpython-developers/GitPython/pull/1850)- Expand what is included in the API Reference by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1855](https://togithub.com/gitpython-developers/GitPython/pull/1855)- Restore building of documentation downloads by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1856](https://togithub.com/gitpython-developers/GitPython/pull/1856)- Revise type annotations slightly by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1860](https://togithub.com/gitpython-developers/GitPython/pull/1860)- Updating regex pattern to handle unicode whitespaces. by[@​jcole-crowdstrike](https://togithub.com/jcole-crowdstrike) in[https://github.com/gitpython-developers/GitPython/pull/1853](https://togithub.com/gitpython-developers/GitPython/pull/1853)- Use upgraded pip in test fixture virtual environment by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1864](https://togithub.com/gitpython-developers/GitPython/pull/1864)- lint: replace `flake8` with `ruff` check by[@​Borda](https://togithub.com/Borda) in[https://github.com/gitpython-developers/GitPython/pull/1862](https://togithub.com/gitpython-developers/GitPython/pull/1862)- lint: switch Black with `ruff-format` by[@​Borda](https://togithub.com/Borda) in[https://github.com/gitpython-developers/GitPython/pull/1865](https://togithub.com/gitpython-developers/GitPython/pull/1865)- Update readme and tox.ini for recent tooling changes by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1868](https://togithub.com/gitpython-developers/GitPython/pull/1868)- Split tox lint env into three envs, all safe by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1870](https://togithub.com/gitpython-developers/GitPython/pull/1870)- Slightly broaden Ruff, and update and clarify tool configuration by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1871](https://togithub.com/gitpython-developers/GitPython/pull/1871)- Add a "doc" extra for documentation build dependencies by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1872](https://togithub.com/gitpython-developers/GitPython/pull/1872)- Describe `Submodule.__init__` parent_commit parameter by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1877](https://togithub.com/gitpython-developers/GitPython/pull/1877)- Include TagObject in git.types.Tree_ish by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1878](https://togithub.com/gitpython-developers/GitPython/pull/1878)- Improve Sphinx role usage, including linking Git manpages by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1879](https://togithub.com/gitpython-developers/GitPython/pull/1879)- Replace all wildcard imports with explicit imports by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1880](https://togithub.com/gitpython-developers/GitPython/pull/1880)- Clarify how tag objects are usually tree-ish and commit-ish by[@​EliahKagan](https://togithub.com/EliahKagan) in[https://github.com/gitpython-developers/GitPython/pull/1881](https://togithub.com/gitpython-developers/GitPython/pull/1881)#### New Contributors- [@​Gaubbe](https://togithub.com/Gaubbe) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1846](https://togithub.com/gitpython-developers/GitPython/pull/1846)- [@​can-taslicukur](https://togithub.com/can-taslicukur) madetheir first contribution in[https://github.com/gitpython-developers/GitPython/pull/1832](https://togithub.com/gitpython-developers/GitPython/pull/1832)- [@​jcole-crowdstrike](https://togithub.com/jcole-crowdstrike)made their first contribution in[https://github.com/gitpython-developers/GitPython/pull/1853](https://togithub.com/gitpython-developers/GitPython/pull/1853)- [@​Borda](https://togithub.com/Borda) made their firstcontribution in[https://github.com/gitpython-developers/GitPython/pull/1862](https://togithub.com/gitpython-developers/GitPython/pull/1862)**Full Changelog**:gitpython-developers/GitPython@3.1.42...3.1.43</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Disabled by config. Please merge this manually once youare satisfied.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about theseupdates again.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://developer.mend.io/github/lettuce-financial/github-bot-signed-commit).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->
EliahKagan added a commit to EliahKagan/GitPython that referenced this pull requestJul 24, 2024
Some of the CI tests use WSL. This switches the WSL distributionfrom Debian to Alpine, which might be slightly faster. For the wayit is being used here, the main expected speed improvement would beto how long the image would take to download, as Alpine is smaller.(The reason for this is thus unrelated to the reason for the Alpinedocker CI test job added ingitpython-developers#1826. There, the goal was to test on awider variety of systems and environments, and that runs the wholetest suite in Alpine. This just changes the WSL distro, used by afew tests on Windows, from Debian to Alpine.)Two things have changed that, taken together, have unblocked this:-Vampire/setup-wsl#50 was fixed, so the action we are using is able to install Alpine Linux. See:gitpython-developers#1917 (review)-gitpython-developers#1893 was fixed ingitpython-developers#1888. So if switching the WSL distro from Debian to Alpine breaks any tests, including by making them fail in an unexpected way that raises the wrong exception, we are likely to find out.
This was referencedJul 24, 2024
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading.Please reload this page.
As suggested in#1745 (comment), this adds a test job that runs the tests inside a Docker container.
This can be done with Ubuntu in the container, but instead I used Alpine Linux in the container. This provides a greater variety of platforms GitPython tests on, potentially uncovering some bugs that would otherwise go undetected. In particular, Alpine Linux does not satisfy some of the assumptions that are sometimes made about platforms, as it does not, by default, have
bash; itspsand related tools are not procps; it does not have GNU coreutils or GNU libc; and so forth.No tests had to be marked
xfailfor this. In particular, as noted in#1745 (comment), we have no tests for#1756, so thatpsis not procps in Alpine Linux does not currently result in any failures (and if there were such a test then the existing macOS jobs would still be sufficient for that).This adds only one test job, testing only one version of Python. Currently that version is Python 3.11, since the Alpine Linux
python3apk package provides 3.11 at this time. The specified Docker image isalpine:latest, and the version will presumably advance to Python 3.12 eventually. I added only one job for two reasons:I was able to usethe
containerkey to do this, so that the workflow is still expressed in separate steps in the same style as the other test workflows, such that the operations that are analogous are represented by steps written in the same (or very similar) way across all workflows. I think the benefit of readability and comparability justifies this at this time, but depending what is done in the future, it might end up being changed to pack most of the work into a smaller number of steps (maybe one) that useuses: docker://..., orrun: docker run ...ordocker-run-action, or something else.Specific reasons for existing design and implementation decisions, including for the strange and somewhat hacky
sudousage, are detailed in commit messages.I've proposed this job as its own pull request, rather than waiting and attempting to introduce it with others, because I think this makes sense to review by itself (and I can incorporate any feedback into future changes), and because if I understand#1745 (comment) correctly it seems to be valued as a feature by itself (which I do agree with). I think that even if there are to be more Docker test jobs, they are better added in a subsequent PR.
However, I'm even more motivated by a goal that this PR does not itself achieve: testing on a bigendian architecture. Because GitPython uses
gitfor most of its work, itshouldn't have a problem with that, but maybe some parts ofgitdb/smmapthat GitPython uses (for which GitPython's use is under test), or more importantly some seemingly innocuous aspects of encoding and decoding, might inadvertently assume little-endianness.The good news is that no such bugs appear to be present. I have not yet gotten this to work on CI, but on my local amd64 system I am ableto enable emulation and run s390x images. This is with no hardware acceleration, and some of the performance tests are too slow for me even to have waited for them to finish (which doesnot mean they would have a problem on an actual s390x machine). But when I tell
pytestto deselect the performance tests, the remaining tests run in a reasonable time and all pass.I'd like to do that on CI, which isnot hard to do. If possible I'd like to use a workflow of this form, with the
containerkey, and have it be a matrix job with both architectures, with special logic for enabling emulation and for skipping the performance tests for non-amd64 architectures (which would, initially and also probably forever, be just the one other architecture, s390x). I haven't managed to get it working that way, though. Usingcontainer:, the container is supposed to be able to start before any steps run. Even if I can manage to get emulation enabled without being able to easily run commands directly on the runner, I suspect I can't get that to happen early enough that usingcontainer:is both effective and reasonable.If Docker test jobs for both amd64 and s390x architectures are to exist, I would want both to be written the same way, even if it makes the amd64 one less nice than in this PR. That way, it would be easier to compare their output to troubleshoot any future failures (as well as to understand how they are set up). If you have thoughts on whether or not this could be valuable, and also how much of a problem (if any) it would be to end up later converting the workflow to look morelike that with fewer steps and not using
container:, then that can guide me. However, whether or not you have guidance about that, I think this PR itself can probably stand on its own.(I considered posting this section of the PR description as an issue instead, but I figured I'd put it here in case requested changes to this PR would substantially affect it or make it moot. and also because it provides a possible reason, beyond that discussed in#1745 (comment), to have Docker jobs like this one in the first place.)