Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork938
Dockerize "Direct Execution of Fuzz Targets"#1904
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Byron merged 2 commits intogitpython-developers:mainfromDaveLak:docker-helper-for-light-weight-fuzzer-executionApr 22, 2024
Merged
Dockerize "Direct Execution of Fuzz Targets"#1904
Byron merged 2 commits intogitpython-developers:mainfromDaveLak:docker-helper-for-light-weight-fuzzer-executionApr 22, 2024
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
Adds a Dockerfile to enable easily executing the fuzz targets directlyinside a container environment instead of directly on a host machine.This addresses concerns raised in PRgitpython-developers#1901 related to how `fuzz_tree.py`writes to the real `/tmp` directory of the file system it is executed onas part of setting up its own test fixtures, but also makes for aneasier to use development workflow.See this related comment on PRgitpython-developers#1901 for additional context:gitpython-developers#1901 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks a lot, much appreciated!
I did take it out for a spin by following the documentation, but encountered an error when building the image (please see below).
Normally I'd ask if this is me, but since it's docker, I'd hope it cannot be. The docker-port is provided by orbstack in my case, and it's possible that that something about that VM is different enough to cause failures. Also, I am doing this on an ARM machine, and probably the VM isn't x86 either.
12.22 /tmp/pip-install-wpyvqb77/atheris_1178434add2a4d5f977b1997318ecda9/setup_utils/find_libfuzzer.sh: line 44: clang: command not found12.22 Failed to find libFuzzer; set either $CLANG_BIN to point to your Clang binary, or $LIBFUZZER_LIB to point directly to your libFuzzer .a file. If needed, download and build the latest version of Clang:12.22 git clone --depth=1 https://github.com/llvm/llvm-project.git12.22 cd llvm-project12.22 cmake -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' -G "Unix Makefiles" -S llvm -B build12.22 NPROC=$(sysctl -n hw.logicalcpu 2>/dev/null || nproc)12.22 cmake --build build --parallel $NPROC # This step is very slow.12.22 Then, set $CLANG_BIN="$(pwd)/bin/clang" and run pip again.12.22 You should use this same Clang for building any Python extensions you plan to fuzz.12.2212.22 Traceback (most recent call last):12.22 File "<string>", line 105, in get_libfuzzer_lib12.22 File "/usr/local/lib/python3.8/subprocess.py", line 415, in check_output12.22 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,12.22 File "/usr/local/lib/python3.8/subprocess.py", line 516, in run12.22 raise CalledProcessError(retcode, process.args,12.22 subprocess.CalledProcessError: Command '['/tmp/pip-install-wpyvqb77/atheris_1178434add2a4d5f977b1997318ecda9/setup_utils/find_libfuzzer.sh']' returned non-zero exit status 1.12.2212.22 During handling of the above exception, another exception occurred:12.2212.22 Traceback (most recent call last):12.22 File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>12.22 main()12.22 File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main12.22 json_out['return_val'] = hook(**hook_input['kwargs'])12.22 File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel12.22 return _build_backend().build_wheel(wheel_directory, config_settings,12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 410, in build_wheel12.22 return self._build_with_temp_dir(12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir12.22 self.run_setup()12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup12.22 super().run_setup(setup_script=setup_script)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup12.22 exec(code, locals())12.22 File "<string>", line 444, in <module>12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 104, in setup12.22 return distutils.core.setup(**attrs)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 184, in setup12.22 return run_commands(dist)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 200, in run_commands12.22 dist.run_commands()12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands12.22 self.run_command(cmd)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command12.22 super().run_command(command)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command12.22 cmd_obj.run()12.22 File "/tmp/pip-build-env-97wquo94/normal/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 368, in run12.22 self.run_command("build")12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command12.22 self.distribution.run_command(command)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command12.22 super().run_command(command)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command12.22 cmd_obj.run()12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run12.22 self.run_command(cmd_name)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command12.22 self.distribution.run_command(command)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 967, in run_command12.22 super().run_command(command)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command12.22 cmd_obj.run()12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 91, in run12.22 _build_ext.run(self)12.22 File "/tmp/pip-build-env-97wquo94/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run12.22 self.build_extensions()12.22 File "<string>", line 333, in build_extensions12.22 File "<string>", line 109, in get_libfuzzer_lib12.22 RuntimeError: Failed to find libFuzzer; set either $CLANG_BIN to point to your Clang binary, or $LIBFUZZER_LIB to point directly to your libFuzzer .a file. If needed, download and build the latest version of Clang:12.22 git clone --depth=1 https://github.com/llvm/llvm-project.git12.22 cd llvm-project12.22 cmake -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' -G "Unix Makefiles" -S llvm -B build12.22 NPROC=$(sysctl -n hw.logicalcpu 2>/dev/null || nproc)12.22 cmake --build build --parallel $NPROC # This step is very slow.12.22 Then, set $CLANG_BIN="$(pwd)/bin/clang" and run pip again.12.22 You should use this same Clang for building any Python extensions you plan to fuzz.12.2212.22 [end of output]12.2212.22 note: This error originates from a subprocess, and is likely not a problem with pip.12.23 ERROR: Failed building wheel for atheris12.23 Failed to build atheris12.23 ERROR: Could not build wheels for atheris, which is required to install pyproject.toml-based projects------Dockerfile:13-------------------- 12 | # Update package managers, install necessary packages, and cleanup unnecessary files in a single RUN to keep the image smaller. 13 | >>> RUN apt-get update && \ 14 | >>> apt-get install --no-install-recommends -y git && \ 15 | >>> python -m pip install --upgrade pip && \ 16 | >>> python -m pip install atheris && \ 17 | >>> python -m pip install -e . && \ 18 | >>> apt-get clean && \ 19 | >>> apt-get autoremove -y && \ 20 | >>> rm -rf /var/lib/apt/lists/* /root/.cache 21 |--------------------ERROR: failed to solve: process "/bin/sh -c apt-get update && apt-get install --no-install-recommends -y git && python -m pip install --upgrade pip && python -m pip install atheris && python -m pip install -e . && apt-get clean && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* /root/.cache" did not complete successfully: exit code: 1Interesting, your image is missing Clang. I'm not familiar with Orbstack (looks neat though) but ARM shouldn't be an issue. I have an ARM machine I can test on and I can probably install Orbstack too if necessary, but before I do would you mind adding E.g, diff --git a/fuzzing/local-dev-helpers/Dockerfile b/fuzzing/local-dev-helpers/Dockerfileindex 77808ed1..e5b171c4 100644--- a/fuzzing/local-dev-helpers/Dockerfile+++ b/fuzzing/local-dev-helpers/Dockerfile@@ -11,7 +11,7 @@ COPY . . # Update package managers, install necessary packages, and cleanup unnecessary files in a single RUN to keep the image smaller. RUN apt-get update && \- apt-get install --no-install-recommends -y git && \+ apt-get install --no-install-recommends -y git clang-15 && \ python -m pip install --upgrade pip && \ python -m pip install atheris && \ python -m pip install -e . && \ |
Thanks for your help! Now I am seeing this, and wonder how it's possible that this ever worked elsewhere - it's docker, right? Maybe it's easiest to install Orbstack as it might have something to do with it. |
Yep, it's Docker. Very strange. I think I might have an idea of what is going on but I'm not totally sure until I can get onto my other machine and test it. I'll let you know if I learn anything.
Yeah, I'll try on my Mac M2 machine with vanilla Docker Desktop as well as with Orbstack to see if I can resolve it. |
The Atheris package bundles a binary that supplies libFuzzer on somehost machines, but in some cases (such as ARM based mac hosts) Atherisseems to require building libFuzzer at install time while pip builds thewheel. In the latter case, clang and related dependencies must bepresent and available for the build, which itself requires using a non"slim" version of the Python base image and not passing the`--no-install-recommends` flag to `apt-get install` as both prevent therequired related libraries from being automatically installed.It is also worth noting that at the time of this commit, the defaultversion of LLVM & Clang installed when `clang` is installed from `apt`is version 14, while the latest stable version is 17 and OSS-Fuzz uses15. The decision to install the default version (14) available via thedebian repos was intentional because a) it appears to work fine for ourneeds and Atheris version b) specifying a different version requiresmore complexity depending on install method, but the goal of thisDockerfile is simplicity and low maintenance.If it becomes neccissary to upgrade Clang/LLVM in the future, one optionto consider besides installing from source is the apt repositorymaintained by the LLVM project:https://apt.llvm.org/See the discussion in this issue for additional context to this change:gitpython-developers#1904
ContributorAuthor
DaveLak commentedApr 21, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
@Byron you should be able to build the image without issue now. The problem and solution are described in the commit message1 forf145121. The image takes a little longer to build now, and is slightly larger overall, but it should work for most folks. Footnotes
|
Byron approved these changesApr 22, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
That's great work right there, thanks so much!
I could validate that the image generation now works, and was able to complete a fuzzer run with 10.000 iterations as well as per the example provided in the README.
The commit message was an interesting read, and shows a lot about the complexities of today's software environment. There is a lot of logic included in installers, that's for sure, and having an image that always works seems like a good basis. The larger size and time to build seems like a price worth paying.
bc7bd22 intogitpython-developers:main 26 checks passed
Uh oh!
There was an error while loading.Please reload this page.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
📣highlight-in-changelog📣Specifically highlight this PR in the changelog after it was created
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading.Please reload this page.
Adds a Dockerfile to enable easily executing the fuzz targets directly inside a container environment instead of directly on a host machine. This addresses concerns raised in PR#1901 related to how
fuzz_tree.pywrites to the real/tmpdirectory of the file system it is executed on as part of setting up its own test fixtures, but also makes for an easier to use development workflow.I also added a warning to the
fuzzing/READEMEstrongly suggesting users only execute the targets inside of Docker.See this related comment on PR#1901 for additional context:#1901 (comment)