- Notifications
You must be signed in to change notification settings - Fork4.5k
Open
Description
Encoder
Collection ofbench results for various platforms and devices.
If you want to submit info about your device, simply run thebench tool or theextra/bench-all.sh and report the results in the comments below.
Suggestions for better summary of the results are welcome
| CPU | OS | Config | Model | Th | Load | Enc. | Commit |
|---|---|---|---|---|---|---|---|
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | tiny | 8 | 71 | 102 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | base | 8 | 96 | 220 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | small | 8 | 233 | 685 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | medium | 8 | 603 | 1928 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | large | 8 | 1158 | 3350 | 206fc93 |
| --- | |||||||
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | small | 1 | 251 | 2605 | 206fc93 |
| MacBook M1 Pro | MacOS 13.0.1 | NEON BLAS | small | 4 | 255 | 884 | 206fc93 |
| --- | |||||||
| Mac Mini M1 | MacOS | NEON BLAS | tiny | 4 | 62 | 194 | fcf515d |
| Mac Mini M1 | MacOS | NEON BLAS | base | 4 | 81 | 380 | fcf515d |
| Mac Mini M1 | MacOS | NEON BLAS | small | 4 | 204 | 1249 | fcf515d |
| Mac Mini M1 | MacOS | NEON BLAS | medium | 4 | 876 | 3980 | fcf515d |
| Mac Mini M1 | MacOS | NEON BLAS | large | 4 | 1876 | 7979 | fcf515d |
| --- | |||||||
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | tiny | 8 | 107 | 422 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | base | 8 | 137 | 880 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | small | 8 | 280 | 2874 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | medium | 8 | 692 | 9610 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 | large | 8 | 1317 | 16917 | fcf515d |
| --- | |||||||
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | tiny | 4 | 120 | 780 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | base | 4 | 151 | 1173 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | small | 4 | 289 | 3062 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | medium | 4 | 711 | 9175 | fcf515d |
| Ryzen 9 3900X | Ubuntu 20.04 | AVX2 BLAS | large | 4 | 1282 | 16050 | fcf515d |
| --- | |||||||
| Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | tiny | 8 | 135 | 197 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | base | 8 | 176 | 421 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | small | 8 | 357 | 1393 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | medium | 8 | 855 | 4404 | fcf515d |
| Ryzen 9 5950X | Ubuntu 22.04 | AVX2 | large | 8 | 1576 | 8118 | fcf515d |
| --- | |||||||
| Raspberry Pi 4 | NEON | tiny | 4 | 1436 | 13839 | fcf515d | |
| Raspberry Pi 4 | NEON | base | 4 | 1894 | 30552 | fcf515d | |
| --- | |||||||
| iPhone 13 Mini | iOS 16.0 | NEON BLAS | base | 4 | 97 | 1091 | fcf515d |
| --- | |||||||
| MacBook M1 Pro | Vivaldi | WASM | tiny | 8 | 133 | 3785 | fcf515d |
| MacBook M1 Pro | Vivaldi | WASM | base | 8 | 172 | 8253 | fcf515d |
| --- | |||||||
| MacBook M1 Pro | Chrome | WASM | tiny | 8 | 134 | 3776 | fcf515d |
| MacBook M1 Pro | Chrome | WASM | base | 8 | 168 | 8200 | fcf515d |
| --- | |||||||
| MacBook M1 Pro | Firefox | WASM | tiny | 8 | 137 | 2626 | fcf515d |
| MacBook M1 Pro | Firefox | WASM | base | 8 | 183 | 6226 | fcf515d |
memcpy
MacBook M1 Pro
./bench -w 1 -t 1memcpy: 37.59 GB/sRyzen 9 5950X
./bench -w 1 -t 1memcpy: 16.74 GB/sggml_mul_mat
MacBook M1 Pro
./bench -w 2 -t 1ggml_mul_mat: 64 x 64: F16 330.6 GFLOPS (128 runs) / F32 466.0 GFLOPS (128 runs)ggml_mul_mat: 128 x 128: F16 737.5 GFLOPS (128 runs) / F32 838.9 GFLOPS (128 runs)ggml_mul_mat: 256 x 256: F16 938.6 GFLOPS (128 runs) / F32 1062.3 GFLOPS (128 runs)ggml_mul_mat: 512 x 512: F16 1312.5 GFLOPS (128 runs) / F32 1835.5 GFLOPS (128 runs)ggml_mul_mat: 1024 x 1024: F16 1765.1 GFLOPS (128 runs) / F32 2041.4 GFLOPS (128 runs)ggml_mul_mat: 2048 x 2048: F16 1784.3 GFLOPS (104 runs) / F32 1859.2 GFLOPS (109 runs)ggml_mul_mat: 4096 x 4096: F16 1855.1 GFLOPS ( 14 runs) / F32 1873.3 GFLOPS ( 14 runs)Ryzen 9 5950X
WHISPER_OPENBLAS=1 make -j bench && ./bench -w 2 -t 1ggml_mul_mat: 64 x 64: F16 56.3 GFLOPS (128 runs) / F32 70.2 GFLOPS (128 runs)ggml_mul_mat: 128 x 128: F16 47.8 GFLOPS (128 runs) / F32 67.0 GFLOPS (128 runs)ggml_mul_mat: 256 x 256: F16 185.1 GFLOPS (128 runs) / F32 332.7 GFLOPS (128 runs)ggml_mul_mat: 512 x 512: F16 386.4 GFLOPS (128 runs) / F32 658.6 GFLOPS (128 runs)ggml_mul_mat: 1024 x 1024: F16 636.2 GFLOPS (128 runs) / F32 1012.0 GFLOPS (128 runs)ggml_mul_mat: 2048 x 2048: F16 950.9 GFLOPS ( 56 runs) / F32 1296.8 GFLOPS ( 76 runs)ggml_mul_mat: 4096 x 4096: F16 1168.6 GFLOPS ( 9 runs) / F32 1403.1 GFLOPS ( 11 runs)