Repository files navigation

中文

A command-line benchmarking tool.

Demo: Benchmarkingfd andfind:

A specialthank you goes to our biggestsponsors:

Warp, the intelligent terminal
_{Available on MacOS, Linux, Windows}

Tuple, the premier screen sharing app for developers
_{Available for MacOS & Windows}

• Statistical analysis across multiple runs.
• Support for arbitrary shell commands.
• Constant feedback about the benchmark progress and current estimates.
• Warmup runs can be executed before the actual benchmark.
• Cache-clearing commands can be set up before each timing run.
• Statistical outlier detection to detect interference from other programs and caching effects.
• Export results to various formats: CSV, JSON, Markdown, AsciiDoc.
• Parameterized benchmarks (e.g. vary the number of threads).
• Cross-platform

To run a benchmark, you can simply callhyperfine <command>.... The argument(s) can be anyshell command. For example:

hyperfine'sleep 0.3'

Hyperfine will automatically determine the number of runs to perform for each command. By default,it will performat least 10 benchmarking runs and measure for at least 3 seconds. To change this,you can use the-r/--runs option:

hyperfine --runs 5'sleep 0.3'

If you want to compare the runtimes of different programs, you can pass multiple commands:

hyperfine'hexdump file''xxd file'

For programs that perform a lot of disk I/O, the benchmarking results can be heavily influencedby disk caches and whether they are cold or warm.

If you want to run the benchmark on a warm cache, you can use the-w/--warmup option toperform a certain number of program executions before the actual benchmark:

hyperfine --warmup 3'grep -R TODO *'

Conversely, if you want to run the benchmark for a cold cache, you can use the-p/--prepareoption to run a special command beforeeach timing run. For example, to clear harddisk cacheson Linux, you can run

sync;echo 3| sudo tee /proc/sys/vm/drop_caches

To use this specific command with hyperfine, callsudo -v to temporarily gain sudo permissionsand then call:

hyperfine --prepare'sync; echo 3 | sudo tee /proc/sys/vm/drop_caches''grep -R TODO *'

If you want to run a series of benchmarks where a single parameter is varied (say, the number ofthreads), you can use the-P/--parameter-scan option and call:

hyperfine --prepare'make clean' --parameter-scan num_threads 1 12'make -j {num_threads}'

This also works with decimal numbers. The-D/--parameter-step-size option can be usedto control the step size:

hyperfine --parameter-scan delay 0.3 0.7 -D 0.2'sleep {delay}'

This runssleep 0.3,sleep 0.5 andsleep 0.7.

For non-numeric parameters, you can also supply a list of values with the-L/--parameter-listoption:

hyperfine -L compiler gcc,clang '{compiler} -O2 main.cpp'

By default, commands are executed using a predefined shell (/bin/sh on Unix,cmd.exe on Windows).If you want to use a different shell, you can use the-S, --shell <SHELL> option:

hyperfine --shell zsh'for i in {1..10000}; do echo test; done'

Note that hyperfine alwayscorrects for the shell spawning time. To do this, it performs a calibrationprocedure where it runs the shell with an empty command (multiple times), to measure the startup timeof the shell. It will then subtract this time from the total to show the actual time used by the commandin question.

If you want to run a benchmarkwithout an intermediate shell, you can use the-N or--shell=noneoption. This is helpful for very fast commands (< 5 ms) where the shell startup overhead correction wouldproduce a significant amount of noise. Note that you cannot use shell syntax like* or~ in this case.

hyperfine -N 'grep TODO /home/user'

If you are using bash, you can export shell functions to directly benchmark them with hyperfine:

my_function() { sleep 1; }export -f my_functionhyperfine --shell=bash my_function

Otherwise, inline them into or source them from the benchmarked program:

hyperfine'my_function() { sleep 1; }; my_function'echo'alias my_alias="sleep 1"'> /tmp/my_alias.shhyperfine'. /tmp/my_alias.sh; my_alias'

Hyperfine has multiple options for exporting benchmark results to CSV, JSON, Markdown and otherformats (see--help text for details).

You can use the--export-markdown <file> option to create tables like the following:

The JSON output is useful if you want to analyze the benchmark results in more detail. Thescripts/ folder includes a lotof helpful Python programs to further analyze benchmark results and create helpfulvisualizations, like a histogram of runtimes or a whisker plot to comparemultiple benchmarks:

The following chart explains the execution order of various timing runs when using optionslike--warmup,--prepare <cmd>,--setup <cmd> or--cleanup <cmd>:

On Ubuntu, hyperfine can be installedfrom the official repositories:

apt install hyperfine

Alternatively, for the latest version, you can download the appropriate.deb package from theRelease page and install it viadpkg:

wget https://github.com/sharkdp/hyperfine/releases/download/v1.20.0/hyperfine_1.20.0_amd64.debsudo dpkg -i hyperfine_1.20.0_amd64.deb

On Fedora, hyperfine can be installed from the official repositories:

dnf install hyperfine

On Alpine Linux, hyperfine can be installedfrom the official repositories:

apk add hyperfine

On Arch Linux, hyperfine can be installedfrom the official repositories:

pacman -S hyperfine

On Debian Linux, hyperfine can be installedfrom the official repositories:

apt install hyperfine

On Exherbo Linux, hyperfine can be installedfrom the rust repositories:

cave resolve -x repository/rustcave resolve -x hyperfine

On Funtoo Linux, hyperfine can be installedfrom core-kit:

emerge app-benchmarks/hyperfine

On NixOS, hyperfine can be installedfrom the official repositories:

nix-env -i hyperfine

On Flox, hyperfine can be installed as follows.

flox install hyperfine

Hyperfine's version in Flox follows that of Nix.

On openSUSE, hyperfine can be installedfrom the official repositories:

zypper install hyperfine

Hyperfine can be installed via xbps

xbps-install -S hyperfine

Hyperfine can be installed viaHomebrew:

brew install hyperfine

Or you can install usingMacPorts:

sudo port selfupdatesudo port install hyperfine

Hyperfine can be installed via pkg:

pkg install hyperfine

doas pkg_add hyperfine

Hyperfine can be installed viaChocolatey,Scoop, orWinget:

choco install hyperfine

scoop install hyperfine

winget install hyperfine

Hyperfine can be installed viaconda from theconda-forge channel:

conda install -c conda-forge hyperfine

Hyperfine can be installed from source viacargo:

cargo install --locked hyperfine

Make sure that you use Rust 1.76 or newer.

Download the corresponding archive from theRelease page.

Hyperfine is inspired bybench.

Chronologer is a tool that useshyperfine tovisualize changes in benchmark timings across your Git history.

Bencher is a continuous benchmarking tool that supportshyperfine totrack benchmarks and catch performance regressions in CI.

Drop hyperfine JSON outputs onto theVenz chart to visualize the results,and manage hyperfine configurations.

Make sure to check out thescripts folderin this repository for a set of tools to work withhyperfine benchmark results.

The namehyperfine was chosen in reference to the hyperfine levels of caesium 133 which play a crucial role in thedefinition of our base unit of time— the second.

Thank you for considering to cite hyperfine in your research work. Please see the informationin the sidebar on how to properly cite hyperfine.

hyperfine is dual-licensed under the terms of the MIT License and the Apache License 2.0.

See theLICENSE-APACHE andLICENSE-MIT files for details.