Repository files navigation

Comparison of learning and inference speed of different GPU with various CNN models inpytorchList of tested AMD and NVIDIA GPUs:

Following benchmark results has been generated with the command: ./show_benchmarks_resuls.shGraph shows the 7700S results both with the pytorch 2.3.1 and with pytorch 2.4.0.ROCM SDK builders pytorch 2.4.0 contains the optimized flashattention support forAMD RX 7700S. (and other gfx1100/gfx1101/gfx1102 and gfx1103 cards)

This command will use pytorch to search all GPUs and will thenrun the benchmark for each of them separately and then in the endthe benchmark that uses all of the GPUs

./run_benchmarks.sh

This command shows how to execute the benchmark for single gpu by using the -i parameter.

python3 benchmark_models.py -i 1 -g 1

First GPU has index 0, second 1, etc...

• New Results are stored under "new_results" folder
• Existing old results are under results folder
• After running the benchmarks, you can create a pull request to githubto request to get them merged
• You can view the results of new benchmarks by addings the name of it's resultfile to plot_benchmarks.py and then running the show_benchmarks.sh script.

• AMD_Radeon_RX_6800
• AMD_Radeon_RX_7900_XTX
• AMD_Radeon_RX_7700S (Framework 16 laptop discrete GPU)
• AMD_Radeon_780M (Framework 16 laptop iGPU)
• 1080TI
• TITAN XP
• TITAN V
• 2080TI
• Titan RTX
• RTX 2060
• RTX 3090
• A100-PCIE
• A100-SXM4

reference site

1. Single & multi GPU with batch size 12: compare training and inference speed of **SequeezeNet, VGG-16, VGG-19, ResNet18, ResNet34, ResNet50, ResNet101,ResNet152, DenseNet121, DenseNet169, DenseNet201, DenseNet161 mobilenet mnasnet ... **

2. Experiments are performed on three types of the datatype. single-precision, double-precision, half-precision

3. making plot(plotly)

././run_benchmarks.sh

• python>=3.6(for f-formatting)
• torchvision
• torch>=1.0.0
• pandas
• psutil
• plotly(for plot)
• cufflinks(for plot)

• Pytorch version2.3
• Number of GPUs on current device4
• CUDA version =10.0
• CUDNN version=7601
• nvcr.io/nvidia/pytorch:20.10-py3 (docker container in A100 and 3090)

• 2024/07/22
  • benchmarks can now be run also on AMD gpus
  • ./run_benchmarks.sh script uses now pytorch to query the gpu countand will first run the tests for each device separately and thenby using all GPU's simultaneously
  • new benchmark results are saved to new_results/<gpu_index>/<gpu_name> folder
  • added new "-i" option which can be used to specify which GPU to use
  • If gpu index is not specified with -i option but the total gpu count specifiedby -g option > 1, then the tests will be run in a way that it uses all gpus simultaneously
• 2021/02/27
  • Addition result in RTX3090
  • Addition result in RTX2060(thanks for gutama)
• 2021/01/07
  • Addition result in TITANXP
• 2021/01/05
  • Addition result in A100 A100-PCIE(PR#14)
• 2021/01/04
  • Addition result in A100 SXM4
  • Addition result in TitanRTX
  • Edit coding style benchmark_model
    • f-formatting
    • save option for json
  • Edit test.sh for bash shell
  • Edit README.md
• 2020/09/01
  • Addition result in windows10
  • Edit README.md
• 2020/01/17
  • Edit coding style and some bug
  • Change plot method
  • Add results of various model experiments(only 2080ti)
• 2019/01/09
  • PR Update typo (thanks for johmathe)
  • Add requirements.txt
  • Add result figures
  • Add ('TkAgg') for cli
  • Addition Muilt GPUS (DGX-station)

• 2021/02/27

• 2021/02/27 thanks for gutamaissue#16

• 2021/01/05 thanks for kirk86pr#14

• 2021/01/05 Thanks for kirk86pr#14

• 2021/01/04

• 2021/01/04

• thanks for olixu

• based on 2020/01/17 update

Each network is fed with 12 images with 224x224x3 dimensions.For training, time durations of 20 passes of forwarding and backward are averaged. For inference, time durations of20 passes of forwarding are averaged. 5 warm-up steps are performed that do not calculate towards the final result.

I conducted the experiment using two RTX 2080ti.

• Results using codes prior to 2020/01/17

If you want to contribute to the experiment in an additional environment, please contribute to the result by subfolder in fig.