practicalswift 693247b82b [test] Speed up fuzzing by ~200x when using afl-fuzz

Enable the `afl-clang-fast++` features deferred forkserver (`__AFL_INIT`) and persistent mode (`__AFL_LOOP(1000)`).

Before this patch:

```
$ afl-fuzz -i input -o output -m512 -- src/test/test_bitcoin_fuzzy
[*] Validating target binary...
[!] WARNING: The target binary is pretty slow! See /usr/local/share/doc/afl/perf_tips.txt.
[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 1072 to 1072 bits (average: 1072.00 bits)
        Exec timing : 20.4k to 20.4k us (average: 20.4k us)
…
exec speed : 57.58/sec (slow!)
exec speed : 48.35/sec (slow!)
exec speed : 53.78/sec (slow!)
```

After this patch:

```
$ afl-fuzz -i input -o output -m512 -- src/test/test_bitcoin_fuzzy
[*] Validating target binary...
[+] Persistent mode binary detected.
[+] Deferred forkserver binary detected.
[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 24 to 24 bits (average: 24.00 bits)
        Exec timing : 114 to 114 us (average: 114 us)
…
exec speed : 15.9k/sec
exec speed : 13.1k/sec
exec speed : 15.1k/sec
```

2017-05-19 07:28:46 +02:00

2.1 KiB

Raw Blame History

Fuzz-testing Bitcoin Core

A special test harness test_bitcoin_fuzzy is provided to provide an easy entry point for fuzzers and the like. In this document we'll describe how to use it with AFL.

Building AFL

It is recommended to always use the latest version of afl:

wget http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz
tar -zxvf afl-latest.tgz
cd afl-<version>
make
export AFLPATH=$PWD

Instrumentation

To build Bitcoin Core using AFL instrumentation (this assumes that the AFLPATH was set as above):

./configure --disable-ccache --disable-shared --enable-tests CC=${AFLPATH}/afl-gcc CXX=${AFLPATH}/afl-g++
export AFL_HARDEN=1
cd src/
make test/test_bitcoin_fuzzy

We disable ccache because we don't want to pollute the ccache with instrumented objects, and similarly don't want to use non-instrumented cached objects linked in.

The fuzzing can be sped up significantly (~200x) by using afl-clang-fast and afl-clang-fast++ in place of afl-gcc and afl-g++ when compiling. When compiling using afl-clang-fast/afl-clang-fast++ the resulting test_bitcoin_fuzzy binary will be instrumented in such a way that the AFL features "persistent mode" and "deferred forkserver" can be used. See https://github.com/mcarpenter/afl/tree/master/llvm_mode for details.

Preparing fuzzing

AFL needs an input directory with examples, and an output directory where it will place examples that it found. These can be anywhere in the file system, we'll define environment variables to make it easy to reference them.

mkdir inputs
AFLIN=$PWD/inputs
mkdir outputs
AFLOUT=$PWD/outputs

Example inputs are available from:

Extract these (or other starting inputs) into the inputs directory before starting fuzzing.

Fuzzing

To start the actual fuzzing use:

$AFLPATH/afl-fuzz -i ${AFLIN} -o ${AFLOUT} -m52 -- test/test_bitcoin_fuzzy

You may have to change a few kernel parameters to test optimally - afl-fuzz will print an error and suggestion if so.

2.1 KiB Raw Blame History