Repository files navigation

TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods

🚩News (2025.06)We have also open-sourced the Time Series Anomaly Detection Benchmark (TAB) and Time Series Foundation Model Benchmark (TSFM-Bench).

🚩News (2025.04)TFB has open-sourced thecode for computing time series characteristics, such as trend, seasonality, stationarity, shifting, transition, correlation, and more. BothChinese andEnglish documentation are provided.

🚩News (2025.04)DUET has released the results of a long-term forecasting task with unified hyperparameters, where the input length is fixed at 96 for all experiments. Clickhere to view the results, and clickhere to view the script for reproducing the results.

🚩News (2025.04)TFB has added two new datasets: PEMS03 and PEMS07, bringing the total to 27 multivariate datasets.

🚩News (2025.03)TFB has added a new practical feature: support for predicting only a subset of input variables. BothChinese andEnglish documentation are provided.

🚩News (2025.03)We maintain aWeChat group to facilitate discussions about TFB andOpenTS.

🚩News (2024.09)You can find detailed API documentationhere.

🚩News (2024.08)Introduction video (in Chinese):bilibili.

🚩News (2024.08)TFB achieves 🌟Best Paper Nomination🌟 in PVLDB 2024.

🚩News (2024.08)We have created a leaderboard for time series forecasting，calledOpenTS-Bench.

🚩News (2024.05)Some introduction (in Chinese):intro1,intro2,intro3,intro4,intro5,intro6, andintro7.

Newly added baselines. ☑ means that their codes have already been included into this repo, and their performance results have been included in theOpenTS-Bench leaderboard. ☒ means that just their codes have already been included into this repo.

• TimeBridge - TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecasting[ICML 2025],[Code].

• SparseTSF - SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters[ICML 2024],[Code].

• TimeKAN - TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecasting[ICLR 2025],[Code].

• xPatch - xPatch: Dual-Stream Time Series Forecasting with Exponential Seasonal-Trend Decomposition[AAAI 2025],[Code].

• HDMixer - HDMixer: Hierarchical Dependency with Extendable Patch for Multivariate Time Series Forecasting[AAAI 2024],[Code].

• PatchMLP - PatchMLP: Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting[AAAI 2025],[Code].

• Amplifier - Amplifier: Bringing Attention to Neglected Low-Energy Components in Time Series Forecasting[AAAI 2025],[Code].

• DUET - DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting[KDD 2025],[Code].

• PDF - Periodicity Decoupling Framework for Long-term Series Forecasting[ICLR 2024],[Code].

• Pathformer - Pathformer: Multi-scale transformers with adaptive pathways for time series forecasting[ICLR 2024],[Code].

• FITS - FITS: Modeling Time Series with 10k Parameters[ICLR 2024],[Code].

1. Introduction
2. Quickstart
3. Steps to develop your own method
4. Steps to evaluate on your own time series
5. Time series code bug the drop-last illustration
6. FAQ
7. Citation
8. Acknowledgement
9. Contact

TFB is an open-source library designed for time series forecasting researchers.

We provide a clean codebase for end-to-end evaluation of time series forecasting models, comparing their performance with baseline algorithms under various evaluation strategies and metrics.

The below figure provides a visual overview of TFB's pipeline.

The table below provides a visual overview of how TFB's key features compare to other libraries for time series forecasting.

1. Installation:

• From PyPI

Given a python environment (note: this project is fully tested underpython 3.8), install the dependencies with the following command:

pip install -r requirements.txt

pip install -r requirements-docker.txt

• From Docker

We also provide aDockerfile for you. For this setup to work you need to have a Docker service installed. You can get it atDocker website.

docker build. -t tfb:latest

docker run -it -v$(pwd)/:/app/ tfb:latest bash

2. Data preparation:

You can obtained the well pre-processed datasets fromGoogle Drive orBaidu Drive. Then place the downloaded data under the folder./dataset.

3. Train and evaluate model:

We provide the experiment scripts for all benchmarks under the folder./scripts/multivariate_forecast, and./scripts/univariate_forecast. For example you can reproduce a experiment result as the following:

sh ./scripts/multivariate_forecast/ILI_script/DLinear.sh

We provide tutorial about how to develop your own method. BothChinese andEnglish documentation are provided.

We provide tutorial about how to evaluate on your own time series, you canclick here.

Implementations of existing methods often employ aDrop Last trick in the testing phase. To accelerate the testing, it is common to split the data into batches. However, if we discard the last incomplete batch with fewer instances than the batch size, this causes unfair comparisons. For example, in Figure 4, the ETTh2 has a testing series of length 2,880, and we need to predict 336 future time steps using a look-back window of size 512. If we select the batch sizes to be 32, 64, and 128, the number of samples in the last batch are 17, 49, and 113, respectively.Unless all methods use the same batch size, discarding the last batch of test samples is unfair, as the actual usage length of the test set is inconsistent. Table 2 shows the testing results of PatchTST, DLinear, and FEDformer on the ETTh2 with different batch sizes and the “Drop Last” trick turned on.We observe that the performance of the methods changes when varying the batch size.

Therefore, TFB calls for the testing process to avoid using the drop-last operation to ensure fairness, and TFB did not use the drop-last operation during testing either.

1. How to use Pycharm to run code？

When running under pycharm，please escape the double quotes, remove the spaces, and remove the single quotes at the beginning and end.

Such as:'{"d_ff": 512, "d_model": 256, "horizon": 24}' ---> {\"d_ff\":512,\"d_model\":256,\"horizon\":24}

--config-path"rolling_forecast_config.json" --data-name-list"ILI.csv" --strategy-args {\"horizon\":24} --model-name"time_series_library.DLinear" --model-hyper-params {\"batch_size\":16,\"d_ff\":512,\"d_model\":256,\"lr\":0.01,\"horizon\":24,\"seq_len\":104} --adapter"transformer_adapter"  --gpus 0  --num-workers 1  --timeout 60000  --save-path"ILI/DLinear"

2. How to get models' predicted values and the target values？

We provide tutorial about how to get the models' predicted values and the target values. BothChinese andEnglish documentation are provided.

3. Examples of script writing.

If you want to run datasets in parallel, test multiple datasets, or test multiple algorithms, and so on, you canclick here.

4. How long the look-back window is used for the multivariate forecasting evaluation results?

You can find answerhere.

5. How to train models with multiple gpus using DataParallel?

You can find answerhere.

6. How to predict only a subset of input variables?

You can find answerhere.

7. Is there code in TFB's code base to compute the characteristics of the dataset?

TFB has open-sourced thecode for computing time series characteristics, such as trend, seasonality, stationarity, shifting, transition, correlation, and more. BothChinese andEnglish documentation are provided.

If you find this repo useful, please cite our paper.

@article{qiu2024tfb,  title   = {TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods},  author  = {Xiangfei Qiu and Jilin Hu and Lekui Zhou and Xingjian Wu and Junyang Du and Buang Zhang and Chenjuan Guo and Aoying Zhou and Christian S. Jensen and Zhenli Sheng and Bin Yang},  journal = {Proc. {VLDB} Endow.},  volume  = {17},  number  = {9},  pages   = {2363--2377},  year    = {2024}}@inproceedings{qiu2025duet,  title     = {DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting},  author    = {Xiangfei Qiu and Xingjian Wu and Yan Lin and Chenjuan Guo and Jilin Hu and Bin Yang},  booktitle = {SIGKDD},  pages     = {1185-1196},  year      = {2025}}

The development of this library has been supported byHuawei Cloud, and we would like to acknowledge their contribution and assistance.

If you have any questions or suggestions, feel free to contact:

• Xiangfei Qiu (xfqiu@stu.ecnu.edu.cn)
• Xingjian Wu (xjwu@stu.ecnu.edu.cn)
• Zhengyu Li (lizhengyu@stu.ecnu.edu.cn)
• Junkai Lu (jklu@stu.ecnu.edu.cn)
• Wanghui Qiu (onehui@stu.ecnu.edu.cn)

Or describe it in Issues.

We invite you to join the OpenTS community on WeChat. We run a group chat on WeChat, and you can get the access by scanning theQR code. By joining the community, you can get the latest updates on OpenTS, share your ideas, and discuss with other members.

Those who wish to join can first scan theQR code to contact me via WeChat. Pleaseinclude your name and research direction in the remarks when adding me. After your application is approved, we will invite you to join the group. Once you are in the group, please update your group nickname to"Name + School/Institution + Research Direction." Members who fail to update their remarks within a week will be regularly removed by the administrator.