Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork938
Attempt 2 - Fix Missing Git Executable Causing ClusterFuzz Crash#1909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Byron merged 1 commit intogitpython-developers:mainfromDaveLak:attempt-two-fuzzing-fix-missing-git-in-clusterfuzzApr 27, 2024
Merged
Attempt 2 - Fix Missing Git Executable Causing ClusterFuzz Crash#1909
Byron merged 1 commit intogitpython-developers:mainfromDaveLak:attempt-two-fuzzing-fix-missing-git-in-clusterfuzzApr 27, 2024
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
This is a second attempt atgitpython-developers#1906 and should resolve:-gitpython-developers#1905-google/oss-fuzz#10600PRgitpython-developers#1906 had the right idea but wrong implementation, and the differences betweenthe ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image wherethe fix was tested led to the issue not being fully resolved.The root cause of the issue is the same: A Git executable is not globallyavailable in the ClusterFuzz container environment where OSS-Fuzz executesfuzz tests.gitpython-developers#1906 attempted to fix the issue by bundling the Git binary and usingGitPython's `git.refresh(<full-path-to-git-executable>)` method to set itinside the `TestOneInput` function of the test harness.However, GitPython attempts to set the binary at import time via its `__init__`hook, and crashes the test if no executable is found during the import.This issue is fixed here by setting the environment variable that GitPythonlooks in before importing it, so it's available for the import. This was testedby setting the `$PATH` to an empty string inside the test files, whichreproduced the crash, then adding the changes introduced here with `$PATH` stillempty, which avoided the crash indicating that the bundled Git executable isworking as expected.
1 task
Byron approved these changesApr 27, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks so much for taking charge of this! No worries about taking more steps either :).
797009d intogitpython-developers:main 26 checks passed
Uh oh!
There was an error while loading.Please reload this page.
It worked! I'm seeing logs from successful fuzz target runs over the last two hours or so! |
DaveLak added a commit to DaveLak/GitPython that referenced this pull requestJun 4, 2024
ClusterFuzz runs of the `fuzz_submodule` target have been failingbecause the `git` import was placed before the condition that sets theGit executable path.The order in which `git` is imported matters because it attempts to finda Git executable as the import is loaded (via `refresh()` in`git/__init__.py`.) As pergitpython-developers#1909, we configure the ClusterFuzzenvironment to use a bundled Git executable via the env variablecondition in all fuzz targets.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading.Please reload this page.
This is a second attempt at#1906 and should resolve:
PR#1906 had the right idea but wrong implementation. The differences between the ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image where it was tested led to the issue not being fully resolved.
The root cause of the issue is the same: A Git executable is not globally available in the ClusterFuzz container environment where OSS-Fuzz executes fuzz tests.
#1906 attempted to fix the issue by bundling the Git binary and using
GitPython's
git.refresh(<full-path-to-git-executable>)method to set it inside theTestOneInputfunction of the test harness.However, GitPython attempts to set the binary at import time via its
__init__hook, and crashes the test if no executable is found during the import.This issue is fixed here by setting the environment variable that GitPython looks in before importing it, so it's available for the import. This was tested by setting the
$PATHto an empty string inside the test files, which reproduced the crash, then adding the changes introduced here with$PATHstill empty, which avoided the crash indicating that the bundled Git executable is working as expected.Note: I didn't test this in a full ClusterFuzz environment, so it's worth noting that while it may be possible for this to continue failing because of the difference between environments that the Git executable was produced in vs. where it's run: I don't believe that will be an issue in practice because the guidance for projects (including Git itself) in OSS-Fuzz is to compile in build.sh with the libraries necessary to create the final binary that is executed in ClusterFuzz. The changes here assume that the apt repo version of Git has everything statically built. If, for some reason, it turns out that a static binary isn't what is installed, the alternative will be to build git from source or mock it. But based on my local testing thus far, I think (and hope) that won't be necessary, and this changeset should be sufficient.