Repository files navigation

• Introduction
• Prerequisites
• Workflow
  • Parallelized AFL Execution
  • Other Examples
• Directory and File Structure
• Usage Information
• License
• Contact

afl-cov uses test case files produced by theAFL fuzzerafl-fuzz to generate gcov codecoverage results for a targeted binary. Code coverage is interpreted from onecase to the next byafl-cov in order to determine which new functions andlines are hit by AFL with each new test case. Further,afl-cov allows forspecific lines or functions to be searched for within coverage results, andwhen a match is found the corresponding test case file is displayed. Thisallows the user to discover which AFL test case is the first to exercise aparticular function. In addition,afl-cov produces a "zero coverage" reportof functions and lines that were never executed during any AFL fuzzing run.

Although of no use to AFL itself, the main application ofafl-cov is to wrapsome automation around gcov together with AFL test cases and thereby providedata on how to maximize code coverage with AFL fuzzing runs. Manualinterpretation of cumulative gcov results from AFL test cases is usually stillrequired, but the "fiddly" steps of iterating over all test cases andgenerating code coverage reports (along with the "zero coverage" report) isautomated byafl-cov.

Producing code coverage data for AFL test cases is an important step to tryand maximize code coverage, and thereby help to maximize the effectiveness ofAFL. For example, some binaries have code that is reachable only after acomplicated (or even cryptographic) test is passed, and AFL may not be able toexercise this code without taking special measures. These measures commonlyinclude patching the project code to bypass such tests. (For example, there isa patch to solve this problem for a CRC test in libpng included in the AFLsources atexperimental/libpng_no_checksum/libpng-nocrc.patch.)When a project implements a patch to assist AFL in reaching code that wouldotherwise be inaccessible, a natural question to ask is whether the patch iseffective. Code coverage results can help to verify this.

afl-cov requires the following software:

• afl-fuzz
• python
• gcov, lcov, genhtml

Note thatafl-cov can parse files created byafl-fuzz from a differentsystem, so technicallyafl-fuzz does not need to be installed on the samesystem asafl-cov. This supports scenarios where fuzzing output is collected,say, within a git repository on one system, and coverage results are producedon a different system. However, most workflows typically focus on producingafl-cov results simultaneously for current fuzzing runs on the same system.

At a high level, the general workflow forafl-cov against a targeted projectis:

1. Have a target project compiled and known to work with AFL.
2. Create a spare copy of the project sources, and compile this copy with gcovprofiling support.
3. Runafl-cov against the copy either whileafl-fuzz is building testcases against the original sources, or afterafl-fuzz has been stopped.
4. Review the cumulative code coverage results in the final web report.
5. Iterate to achieve higher coverage results. This might involve buildingbetter initial test cases for AFL, or sometimes changing project sourcesthemselves.

Now, in more detail:

• Copy the project sources to a new directory,/path/to/project-gcov/.This directory should contain the project binaries compiled for gcov profilingsupport (gcc-fprofile-arcs -ftest-coverage).

• Start upafl-cov in--live mode before also starting theafl-fuzzfuzzing cycle. The command line arguments toafl-cov must specify the path tothe output directory used byafl-fuzz, and the command to execute along withassociated arguments. This command and arguments should closely resemble themanner in whichafl-fuzz executes the targeted binary during the fuzzingcycle. If there is already an existing directory of AFL fuzzing results, thenjust omit the--live argument to process the existing results. Here is anexample:

$cd /path/to/project-gcov/$ afl-cov -d /path/to/afl-fuzz-output/ --live --coverage-cmd \"cat AFL_FILE | LD_LIBRARY_PATH=./lib/.libs ./bin/.libs/somebin -a -b -c" \--code-dir.

/path/to/afl-fuzz-output/ is the output directory of afl-fuzz.

TheAFL_FILE string above refers to the test case file that AFL willbuild in thequeue/ directory under/path/to/afl-fuzz-output. Just leave thisstring as-is sinceafl-cov will automatically substitute it with each AFLqueue/id:NNNNNN* in succession as it builds the code coverage reports.

Also, in the above command, this handles the case where the AFL fuzzing cycleis fuzzing the targeted binary via stdin. This explains thecat AFL_FILE | ... ./bin/.lib/somebin ... invocation. For the other style offuzzing with AFL where a file is read from the filesystem, here is an example:

$cd /path/to/project-gcov/$ afl-cov -d /path/to/afl-fuzz-output/ --live --coverage-cmd \"LD_LIBRARY_PATH=./lib/.libs ./bin/.libs/somebin -f AFL_FILE -a -b -c" \--code-dir.

• Withafl-cov running, open a separate terminal/shell, and launchafl-fuzz:

$ LD_LIBRARY_PATH=./lib/.libs afl-fuzz -T somebin -t 1000 \-i /path/to/test-cases/ -o /path/to/afl-fuzz-output/ ./bin/.libs/somebin -a -b -c

The familiar AFL status screen will be displayed, andafl-cov will startgenerating code coverage data.

Note that by defaultafl-cov does not directlcov to include branchcoverage results. This is because there are commonly many hundreds of AFLtest cases in thequeue/ directory, and generating branch coverage across allof these cases may slowafl-cov down significantly. If branch coverage isdesired, just add the--enable-branch-coverage argument toafl-cov.

Here is a sample of what theafl-cov output looks like (note this includesthe--enable-branch-coverage argument as described above):

$ afl-cov -d /path/to/afl-fuzz-output/ --live --coverage-cmd \"LD_LIBRARY_PATH=./lib/.libs ./bin/.libs/somebin -f AFL_FILE -a -b -c" \--code-dir. --enable-branch-coverage[+] Imported 184 files from: /path/to/afl-fuzz-output/queue[+] AFL file: id:000000,orig:somestr.start (1 / 184), cycle: 0    lines......: 18.6% (1122 of 6032 lines)    functions..: 30.7% (100 of 326 functions)    branches...: 14.0% (570 of 4065 branches)[+] AFL file: id:000001,orig:somestr256.start (2 / 184), cycle: 2    lines......: 18.7% (1127 of 6032 lines)    functions..: 30.7% (100 of 326 functions)    branches...: 14.1% (572 of 4065 branches)[+] Coverage diff id:000000,orig:somestr.start id:000001,orig:somestr256.start    Src file: /path/to/project-gcov/lib/proj_decode.c      New'line' coverage: 140      New'line' coverage: 141      New'line' coverage: 142    Src file: /path/to/project-gcov/lib/proj_util.c      New'line' coverage: 217      New'line' coverage: 218[+] AFL file: id:000002,orig:somestr384.start (3 / 184), cycle: 10    lines......: 18.8% (1132 of 6032 lines)    functions..: 30.7% (100 of 326 functions)    branches...: 14.1% (574 of 4065 branches)[+] Coverage diff id:000001,orig:somestr256.start id:000002,orig:somestr384.start    Src file: /path/to/project-gcov/lib/proj_decode.c      New'line' coverage: 145      New'line' coverage: 146      New'line' coverage: 147    Src file: /path/to/project-gcov/lib/proj_util.c      New'line' coverage: 220      New'line' coverage: 221[+] AFL file: id:000003,orig:somestr.start (4 / 184), cycle: 5    lines......: 18.9% (1141 of 6032 lines)    functions..: 31.0% (101 of 326 functions)    branches...: 14.3% (581 of 4065 branches)[+] Coverage diff id:000002,orig:somestr384.start id:000003,orig:somestr.start    Src file: /path/to/project-gcov/lib/proj_message.c      New'function' coverage:validate_cmd_msg()      New'line' coverage: 244      New'line' coverage: 247      New'line' coverage: 248      New'line' coverage: 250      New'line' coverage: 255      New'line' coverage: 262      New'line' coverage: 263      New'line' coverage: 266...[+] Coverage diff id:000182,src:000000,op:havoc,rep:64 id:000184,src:000000,op:havoc,rep:4[+] Processed 184 / 184 files[+] Final zero coverage report: /path/to/afl-fuzz-output/cov/zero-cov[+] Final positive coverage report: /path/to/afl-fuzz-output/cov/pos-cov[+] Final lcov web report: /path/to/afl-fuzz-output/cov/web/lcov-web-final.html

In the last few lines above, the locations of the final web coverage and zerocoverage reports are shown. The zero coverage reports contains function namesthat were never executed across the entireafl-fuzz run.

The code coverage results in/path/to/afl-fuzz-output/cov/web/lcov-web-finalrepresent cumulative code coverage across all AFL test cases. This data can thenbe reviewed to ensure that all expected functions are indeed exercised by AFL -just point a web browser at/path/to/afl-fuzz-output/cov/web/lcov-web-final.html.Below is a sample of what this report looks like for a cumulative AFL fuzzingrun - this is against thefwknop project, andthe full report isavailable here.Note that even though fwknop has a dedicated set ofAFL wrappers, it is stilldifficult to achieve high percentages of code coverage. This provides evidencethat measuring code coverage under AFL fuzzing runs is an important aspect oftrying to achieve maximal fuzzing results. Every branch/line/function that isnot exercised by AFL represents a location for which AFL has not been given theopportunity to find bugs.

With the 0.4 release,afl-cov supports parallelized execution runs ofafl-fuzz. All that is required is to pointafl-cov -d sync_dir at the toplevel sync directory that is used by allafl-fuzz instances(afl-fuzz -o sync_dir). The coverage results are calculated globallyacross all fuzzing instances, and in--live mode new instances will be addedto the coverage results as they are created.

The workflow above is probably the main strategy for usingafl-cov. However,additional use cases are supported such as:

1. Suppose there are a set of wrapper scripts aroundafl-fuzz to run fuzzingcycles against various aspects of a project. By building a set of correspondingafl-cov wrappers, and then using the--disable-coverage-init option on allbut the first of these wrappers, it is possible to generate code coverageresults across the entire set ofafl-fuzz fuzzing runs. (By default,afl-cov resets gcov counters to zero at start time, but the--disable-coverage-init argument stops this behavior.) The end result is aglobal picture of code coverage across all invocations ofafl-fuzz.

2. Specific functions can be searched for in the code coverage results, andafl-cov will return the firstafl-fuzz test case where a given function isexecuted. This allowsafl-cov to be used as a validation tool by other scriptsand testing infrastructure. For example, a test case could be written aroundwhether an important function is executed byafl-fuzz to validate a patchingstrategy mentioned in the introduction.

Here is an example where the first test case that executes the functionvalidate_cmd_msg() is returned (this is after allafl-cov results have beenproduced in the main workflow above):

$ ./afl-cov -d /path/to/afl-fuzz-output --func-search"validate_cmd_msg"[+] Function'validate_cmd_mag()' executed by: id:000002,orig:somestr384.start

An equivalent way of searching the coverage results is to justgrep thefunction from thecov/id-delta-cov file described below. The number"3" inthe output below is the AFL cycle number where the function is first executed:

$ grep validate_cmd_msg /path/to/afl-fuzz-output/cov/id-delta-covid:000002,orig:somestr384.start, 3, /path/to/project-gcov/file.c, function,validate_cmd_msg()

afl-cov creates a few files and directories for coverage results within thespecifiedafl-fuzz directory (-d). These files and directories aredisplayed below, and all are contained within the main/path/to/afl-fuzz-output/cov/ directory and<dirname> refers to thetop level directory name for the fuzzing instance. When AFL is parallelized,there will be one<dirname> directory path for eachafl-fuzz instance.

• cov/diff/<dirname> - contains new code coverage results when aqueue/id:NNNNNN* file causesafl-fuzz to execute new code.
• cov/lcov/<dirname> - contains raw code coverage data produced by the lcovfront-end to gcov.
• cov/web/<dirname> - contains code coverage results in web format producedbygenhtml.
• cov/zero-cov - file that globally lists all functions (and optionallylines) that are never executed by anyafl-fuzz test case.
• cov/pos-cov - file that globally lists all functions (and optionallylines) that are executed at least once by anafl-fuzz testcase.
• cov/id-delta-cov - lists the functions (and optionally lines) that areexecuted by the firstid:000000* test case, and then listsall new functions/lines executed in subsequent test cases.
• cov/afl-cov.log - log file forafl-cov logging output.
• cov/afl-cov-status - status file forafl-cov PID, version number , andcommand line arguments.

Basic--help output appears below:

usage: afl-cov [-h] [-e COVERAGE_CMD] [-d AFL_FUZZING_DIR] [-c CODE_DIR] [-O]           [--disable-cmd-redirection] [--disable-lcov-web]           [--disable-coverage-init] [--coverage-include-lines]           [--enable-branch-coverage] [--live] [--cover-corpus]           [--coverage-at-exit] [--sleep SLEEP] [--gcov-check]           [--gcov-check-bin GCOV_CHECK_BIN] [--background]           [--lcov-web-all] [--disable-lcov-exclude-pattern]           [--lcov-exclude-pattern LCOV_EXCLUDE_PATTERN]           [--func-search FUNC_SEARCH] [--line-search LINE_SEARCH]           [--src-file SRC_FILE] [--afl-queue-id-limit AFL_QUEUE_ID_LIMIT]           [--ignore-core-pattern] [--lcov-path LCOV_PATH]           [--genhtml-path GENHTML_PATH] [--readelf-path READELF_PATH]           [--stop-afl] [--validate-args] [-v] [-V] [-q]optional arguments:  -h, --help            show this help message and exit  -e COVERAGE_CMD, --coverage-cmd COVERAGE_CMD                        Set command to exec (including args, and assumes code                        coverage support)  -d AFL_FUZZING_DIR, --afl-fuzzing-dir AFL_FUZZING_DIR                        top level AFL fuzzing directory  -c CODE_DIR, --code-dir CODE_DIR                        Directory where the code lives (compiled with code                        coverage support)  -O, --overwrite       Overwrite existing coverage results  --disable-cmd-redirection                        Disable redirection of command results to /dev/null  --disable-lcov-web    Disable generation of all lcov web code coverage                        reports  --disable-coverage-init                        Disable initialization of code coverage counters at                        afl-cov startup  --coverage-include-lines                        Include lines in zero-coverage status files  --enable-branch-coverage                        Include branch coverage in code coverage reports (may                        be slow)  --live                Process a live AFL directory, and afl-cov will exit                        when it appears afl-fuzz has been stopped  --cover-corpus        Measure coverage after running all available tests                        instead of individually per queue file  --coverage-at-exit    Only calculate coverage just before afl-cov exit.  --sleep SLEEP         In --live mode, # of seconds to sleep between checking                        for new queue files  --gcov-check          Check to see if there is a binary in --coverage-cmd                        (or in --gcov-check-bin) has coverage support  --gcov-check-bin GCOV_CHECK_BIN                        Test a specific binary for code coverage support  --background          Background mode - if also in --live mode, will exit                        when the alf-fuzz process is finished  --lcov-web-all        Generate lcov web reports for all id:NNNNNN* files                        instead of just the last one  --disable-lcov-exclude-pattern                        Allow default /usr/include/* pattern to be included in                        lcov results  --lcov-exclude-pattern LCOV_EXCLUDE_PATTERN                        Set exclude pattern for lcov results  --func-search FUNC_SEARCH                        Search for coverage of a specific function  --line-search LINE_SEARCH                        Search for coverage of a specific line number                        (requires --src-file)  --src-file SRC_FILE   Restrict function or line search to a specific source                        file  --afl-queue-id-limit AFL_QUEUE_ID_LIMIT                        Limit the number of id:NNNNNN* files processed in the                        AFL queue/ directory  --ignore-core-pattern                        Ignore the /proc/sys/kernel/core_pattern setting in                        --live mode  --lcov-path LCOV_PATH                        Path to lcov command  --genhtml-path GENHTML_PATH                        Path to genhtml command  --readelf-path READELF_PATH                        Path to readelf command  --stop-afl            Stop all running afl-fuzz instances associated with                        --afl-fuzzing-dir <dir>  --validate-args       Validate args and exit  -v, --verbose         Verbose mode  -V, --version         Print version and exit  -q, --quiet           Quiet mode

afl-cov is released as open source software under the terms oftheGNU General Public License (GPL v2+). The latest release can be foundathttps://github.com/mrash/afl-cov/releases

All feature requests and bug fixes are managed through github issues tracking.However, you can also email me (michael.rash_AT_gmail.com), or reach me throughTwitter (@michaelrash).