Features

Supported Models

Evaluation Indicators

Scoring Rules

Quick Start

Installation

git clone https://github.com/NoFxAiOS/FinBench.gitcd FinBenchgo build -o finbench ./cmd/finbench

Configuration

cp config.template.json config.json

{"models": [    {"name":"DeepSeek-Chat","provider":"deepseek","model":"deepseek-chat","api_key":"your-api-key-here"    }  ]}

Running Benchmarks

# Standard benchmark (10 runs)./finbench run -config=config.json -symbols=BTCUSDT -runs=10 -output=report.json# Quick benchmark (3 runs)./finbench run -config=config.json -symbols=BTCUSDT -runs=3# Multi-symbol benchmark./finbench run -config=config.json -symbols=BTCUSDT,ETHUSDT -runs=10# Using static dataset (reproducible)./finbench run -mode=static -dataset=datasets/snapshots -config=config.json

Generating Visual Reports

python3 scripts/generate_report.py report.json finbench_report.htmlopen finbench_report.html

Command Reference

# View supported models./finbench models# Capture market data snapshot./finbench snapshot -symbols=BTCUSDT,ETHUSDT -output=datasets/snapshots# View help./finbenchhelp

Run Command Parameters

Report Examples

Evaluation Methodology

┌─────────────────────────────────────────┐│         Unified Data Source (Klines)    ││        CoinAnk API / Static Snapshots   │└─────────────────┬───────────────────────┘                  │        ┌─────────┴─────────┐        ▼                   ▼┌───────────────┐   ┌───────────────┐│ Local Calc    │   │  LLM Calc     ││ (Ground Truth)│   │ (Under Test)  │└───────┬───────┘   └───────┬───────┘        │                   │        └─────────┬─────────┘                  ▼        ┌───────────────────┐        │   Compare & Score │        │   Error → Score   │        └───────────────────┘

Project Structure

FinBench/├── cmd/finbench/       # CLI entry point├── benchmark/          # Benchmark engine│   ├── engine.go       # Main engine│   ├── llm.go          # LLM client│   ├── models.go       # Model configuration│   ├── scorer.go       # Scoring logic│   └── calculator.go   # Indicator calculation├── market/             # Market data│   ├── kline.go        # Kline fetching│   └── indicators.go   # Indicator implementation├── provider/           # Data providers│   └── coinank/        # CoinAnk API├── scripts/            # Utility scripts│   └── generate_report.py  # Report generator├── datasets/           # Datasets│   └── snapshots/      # Snapshot data└── docs/               # Documentation

Security Notice

License

Related Projects