Repository files navigation

This repository benchmarks performance of various Ethereum developmentframeworks by simulating a call to Convex'ssystemShutdown method. This methoduses about 16M gas and performs a number of token transfers

Benchmarks were run at mainnet block 14,445,961. Blocknative only simulates against the latest block (support for simulation at historical blocks is planned), so the Blocknative simulation was run first and other benchmarks were run against the same block.

Notes on benchmarks:

• "Remote RPC" benchmarks used Alchemy as the RPC provider, connected over HTTP.
• "Local RPC" benchmarks used a local Erigon node.
• "Cached" benchmarks use RPC responses that the framework caches locally instead of making HTTP requests. We assume that since Blocknative and Tenderly are infrastructure providers, they aggressively cache and load hot data in-memory to improve performance, which is why their results are also in the "Cached" column.
• Benchmarks were performed on macOS 11.6.2 with a 2.3 GHz 8-Core Intel Core i9 and 32 GB 2667 MHz DDR4.

Notes on gas usage:

• Ganache, Hardhat, and Tenderly all agree on gas usage after refunds and are therefore likely the truth value.
• Foundry usage matches those values: The debugger shows the actual call costs 22,558,945 gas. Adding 21,000 intrinsic gas and 64 gas for calldata (both are excluded from the reported gas number by default) gives the same amount reported by Hardhat and others. The remaining 2995 gas is overhead from the contract based setup and Solidity-generated call checks.
• Blocknative's gas usage does not account for refunds as this is not yet supported by their platform.
• Dapptools' gas usage also does not account for refunds.

1. Runcp .env.example .env, and in the resulting.env file enter a URL to an Ethereum archive node in theETH_RPC_URL environment variable. (Alchemy provides free archive node data). Also fill out theTENDERLY_* variables to benchmark the Tenderly API

2. Runyarn to install dependencies for Ganache and Hardhat

3. Install Foundry's forge and Dapptools using the installation instructionshere andhere respectively

4. Rundapp update to install dependencies for Dapptools and Foundry

5. Run any command in theMakefile to benchmark that tool. For example, usemake benchmark-hardhat to run the simulation against Hardhat. Alternatively, runmake benchmark-all to run all tools

• Setexport CLEAR_CACHE=1 in your.env file to clear the Ganache and Hardhat caches
• Consider running the benchmarks via Docker. See the comment header inthe Dockerfile for details.