CPU	Config	Model	Th	FA	Enc.	Dec.	Bch5	PP	Commit
M2 ULTRA	METAL	tiny	1	1	7.72	1.05	0.32	0.01	`dc8dda6`
M2 ULTRA	METAL	tiny-q5_0	1	1	8.20	0.98	0.31	0.01	`dc8dda6`
M2 ULTRA	METAL	tiny-q5_1	1	1	8.13	0.99	0.31	0.01	`dc8dda6`
M2 ULTRA	METAL	tiny-q8_0	1	1	7.96	0.93	0.30	0.01	`dc8dda6`
M2 ULTRA	METAL	base	1	1	13.52	1.39	0.35	0.02	`dc8dda6`
M2 ULTRA	METAL	base-q5_0	1	1	14.88	1.31	0.34	0.02	`dc8dda6`
M2 ULTRA	METAL	base-q5_1	1	1	14.76	1.33	0.34	0.02	`dc8dda6`
M2 ULTRA	METAL	base-q8_0	1	1	14.04	1.28	0.34	0.02	`dc8dda6`
M2 ULTRA	METAL	small	1	1	38.78	2.72	0.67	0.04	`dc8dda6`
M2 ULTRA	METAL	small-q5_0	1	1	44.01	2.64	0.69	0.05	`dc8dda6`
M2 ULTRA	METAL	small-q5_1	1	1	44.02	2.66	0.69	0.05	`dc8dda6`
M2 ULTRA	METAL	small-q8_0	1	1	40.79	2.49	0.67	0.05	`dc8dda6`
M2 ULTRA	METAL	medium	1	1	104.48	5.57	1.61	0.10	`dc8dda6`
M2 ULTRA	METAL	medium-q5_0	1	1	122.24	5.00	1.58	0.12	`dc8dda6`
M2 ULTRA	METAL	medium-q5_1	1	1	121.99	5.02	1.59	0.12	`dc8dda6`
M2 ULTRA	METAL	medium-q8_0	1	1	111.68	4.99	1.52	0.11	`dc8dda6`
M2 ULTRA	METAL	medium-dis	1	1	93.23	0.87	0.21	0.01	`dc8dda6`
M2 ULTRA	METAL	large-v2	1	1	189.82	8.36	2.35	0.19	`dc8dda6`
M2 ULTRA	METAL	large-v2-q5_0	1	1	225.73	7.34	2.40	0.22	`dc8dda6`
M2 ULTRA	METAL	large-v2-q5_1	1	1	225.88	7.60	2.40	0.22	`dc8dda6`
M2 ULTRA	METAL	large-v2-q8_0	1	1	203.55	7.32	2.26	0.20	`dc8dda6`
M2 ULTRA	METAL	large-v2-dis	1	1	168.20	0.98	0.24	0.02	`dc8dda6`
M2 ULTRA	METAL	large-v3-turbo	1	1	170.22	1.46	0.37	0.03	`dc8dda6`
M2 ULTRA	METAL	large-v3-turbo-q5_0	1	1	201.88	1.27	0.38	0.04	`dc8dda6`
M2 ULTRA	METAL	large-v3-turbo-q8_0	1	1	182.37	1.24	0.36	0.03	`dc8dda6`

Flash Attention OFF:

CPU	Config	Model	Th	Enc.	Dec.	Bch5	PP	Commit
M2 ULTRA	METAL	tiny	1	10.15	1.20	0.36	0.01	`dc8dda6`
M2 ULTRA	METAL	tiny-q5_0	1	10.21	1.15	0.39	0.01	`dc8dda6`
M2 ULTRA	METAL	tiny-q5_1	1	9.26	1.15	0.38	0.01	`dc8dda6`
M2 ULTRA	METAL	tiny-q8_0	1	9.00	1.12	0.37	0.01	`dc8dda6`
M2 ULTRA	METAL	base	1	15.77	1.73	0.45	0.02	`dc8dda6`
M2 ULTRA	METAL	base-q5_0	1	16.90	1.63	0.44	0.02	`dc8dda6`
M2 ULTRA	METAL	base-q5_1	1	16.93	1.64	0.44	0.02	`dc8dda6`
M2 ULTRA	METAL	base-q8_0	1	16.13	1.63	0.43	0.02	`dc8dda6`
M2 ULTRA	METAL	small	1	45.15	3.45	0.92	0.05	`dc8dda6`
M2 ULTRA	METAL	small-q5_0	1	50.63	3.36	0.94	0.06	`dc8dda6`
M2 ULTRA	METAL	small-q5_1	1	50.56	3.36	0.94	0.06	`dc8dda6`
M2 ULTRA	METAL	small-q8_0	1	47.52	3.20	0.92	0.05	`dc8dda6`
M2 ULTRA	METAL	medium	1	122.55	7.38	1.95	0.12	`dc8dda6`
M2 ULTRA	METAL	medium-q5_0	1	140.61	6.73	2.02	0.14	`dc8dda6`
M2 ULTRA	METAL	medium-q5_1	1	140.48	6.76	2.04	0.14	`dc8dda6`
M2 ULTRA	METAL	medium-q8_0	1	131.00	6.57	1.96	0.13	`dc8dda6`
M2 ULTRA	METAL	medium-dis	1	110.85	1.00	0.24	0.02	`dc8dda6`
M2 ULTRA	METAL	large-v2	1	222.28	10.96	3.03	0.21	`dc8dda6`
M2 ULTRA	METAL	large-v2-q5_0	1	258.64	9.79	3.04	0.25	`dc8dda6`
M2 ULTRA	METAL	large-v2-q5_1	1	258.32	9.87	3.05	0.24	`dc8dda6`
M2 ULTRA	METAL	large-v2-q8_0	1	236.55	9.61	2.87	0.23	`dc8dda6`
M2 ULTRA	METAL	large-v2-dis	1	199.84	1.14	0.27	0.02	`dc8dda6`
M2 ULTRA	METAL	large-v3-turbo	1	201.52	1.77	0.45	0.03	`dc8dda6`
M2 ULTRA	METAL	large-v3-turbo-q5_0	1	233.14	1.56	0.47	0.04	`dc8dda6`
M2 ULTRA	METAL	large-v3-turbo-q8_0	1	214.23	1.53	0.44	0.04	`dc8dda6`

What's Changed

docs : add xcframework section to README.md [no ci] by@danbev in#2997
sync : ggml by@ggerganov in#2992
whisper.wasm : fix unknown language issue by@danbev in#3000
examples : update server.py to match github pages app [no ci] by@danbev in#3004
rename : ggerganov -> ggml-org by@ggerganov in#3005
whisper : fix "bench-all outputs an invalid result on larger models" by@fujimotos in#3002
tests : add script to benchmark whisper.cpp on LibriSpeech corpus by@fujimotos in#2999
ruby : Change homepage URI in Ruby gemspec by@KitaitiMakoto in#3007
fix dead link to models in readme by@gregsadetsky in#3006
Update uri.rb by@Olli in#3016
Update ruby_whisper_params.c by@Olli in#3022
xcf : use check for visionos build version by@danbev in#3021
Fix README.md by@ekaitz-zarraga in#3024
docs : document how to use 'WHISPER_FFMPEG' build option by@fujimotos in#3029
whisper : reduce delta_min from 1000ms to 100ms by@ggerganov in#3028
support max_context api for addon.node by@buxuku in#3025
Update README.md to note newer NVIDIA GPUs by@jeffklassen in#3031
ruby: use CMake in build process by@KitaitiMakoto in#3043
examples : add FFmpeg v7.0 support to ffmpeg-transcode.cpp by@fujimotos in#3038
feat: Add no-context option to server by@sachaarbonel in#3045
ruby : make Ruby bindings installed with build options by@KitaitiMakoto in#3056
examples : add HEAPU8 to exported runtime methods by@danbev in#3062
ci : disable freeBSD job in build.yml by@danbev in#3064
coreml : set convert_to="mlprogram" in convert by@danbev in#3060
sync : ggml by@ggerganov in#3071
ci : enable bindings java job by@danbev in#3070
ruby : add encoder begin callback related methods by@KitaitiMakoto in#3076
Fix deprecated FFmpeg functions by@Podre-Henrique in#3073
Add Moore Threads GPU support and update GitHub workflow for MUSA build by@yeahdongcn in#3069
ci : disable publishing of java binding [no ci] by@danbev in#3086
talk-llama : sync llama.cpp by@ggerganov in#3084
whisper : remove empty .gitmodules file [no ci] by@danbev in#3085
feat: expose language detection probabilities to server example by@sachaarbonel in#3044
whisper : fix grammar advance stack warning by@danbev in#3087
ggml : suppress Windows compiler warnings by@danbev in#3075
make : fix samples glob pattern by@ggerganov in#3100
ruby : ignore "Downloading" output in test_log_suppress by@danbev in#3106
server : add --no-gpu option to print usage output by@danbev in#3098
whisper : add check that target name exists by@danbev in#3103
sync : ggml by@ggerganov in#3108
ruby : refine HTTP cache feature by@KitaitiMakoto in#3109
feat(server): bump httplib.h to version 0.20.0 by@sachaarbonel in#3101
removed stdc++fs by@JaredTweed in#3097
docs : Update cli documentation by@antimatter96 in#3102
cli : Support "-" for stdout like stdin by@danielzgtg in#3050
feat(server): Improve HTTP Connection Abort Handling by@sachaarbonel in#3112
whisper: remove MSVC warnings pragmas by@danbev in#3090
ci : add zip extension to xcframework artifact name by@danbev in#3120
ci : zip windows artifacts for release uploading by@danbev in#3124
sync : ggml by@ggerganov in#3125
ci : add bindings-java jar artifact to release by@danbev in#3126
sync : ggml by@ggerganov in#3127
stream.wasm : add HEAPU8 to exported runtime methods by@danbev in#3130
whisper : deprecate WHISPER_CCACHE CMake option by@danbev in#3131
wasm : add note about worker.js file generation [no ci] by@danbev in#3133
examples : add HEAPU8 to all of the exported runtime methods by@enesgrahovac in#3134
ruby : omit test_build_options locally by@danbev in#3132
ruby : test extra build options only when env var specified by@KitaitiMakoto in#3136
examples : update link to Paul Tol's color scheme [no ci] by@danbev in#3140
Update Readme to recommend same Openvino as Python tools by@peardox in#3138
cli : print color scheme info for --print-colors by@danbev in#3141
docs : fix -owts flag typo karaoke section [no ci] by@danbev in#3142
whisper : remove dummy commit comment [no ci] by@danbev in#3143
vad : add initial Voice Activity Detection (VAD) support by@danbev in#3065
docs: Update README.md for whisper.objc app by@tomer-schles in#2569
vad : remove shortform for --vad option in cli.cpp by@danbev in#3145
examples : add VAD speech segments example by@danbev in#3147
sync : ggml by@ggerganov in#3148
server : add --flash-attn usage output by@danbev in#3152
vad : add download-vad-model scripts by@danbev in#3149
examples : add --print-confidence option to cli by@danbev in#3150
whisper : add build_*/ to .gitignore [no ci] by@danbev in#3157
vad : store VAD context in whisper_state by@danbev in#3156
vad : return early if no vad segments are detected by@danbev in#3158
examples : add vad-speech-segments to win warns [no ci] by@danbev in#3170
sync : ggml by@ggerganov in#3171
ruby : add GGML_SYCL_DNN option to ruby bindings by@danbev in#3172
Fix CMakeLists.txt to handle deprecated gpu Warnings by@jugal-sheth in#3163
whisper : remove redundant assignments by@danbev in#3178
docs: fix very minor typo in README by@alpaim in#3179
docs : add VAD model download instructions [no ci] by@danbev in#3180
feat(server): Add k6 Load Testing Script by@sachaarbonel in#3175
ci : use dynamic libopenblas.dll for window-blas by@danbev in#3177
docs : Fix VAD section heading levels in README by@KitaitiMakoto in#3186
Quickfix for swedish umlauts + expose model inference settings in talk-llama.cpp by@matteng1 in#3187
node : enable no_prints to suppress all output by@danbev in#3189
docs : convert README_sycl.md to utf8 format [no ci] by@danbev in#3191
sync : ggml by@ggerganov in#3193
ci : update windows artifact uploads steps by@danbev in#3192
tests : add a new benchmark test for long-form audio by@fujimotos in#3185
Install shared libs when using GGML_BACKEND_DL by@peardox in#3195
ruby : add VAD support, migration to Ruby's newer API by@KitaitiMakoto in#3197
whisper : remove whisper_load_backends function by@danbev in#3196
sync : ggml by@ggerganov in#3202
ggml : Fix backtrace breaking Windows build by@danielzgtg in#3203
ruby : handle build options on installation by@KitaitiMakoto in#3206
vad : revisit timestamp alignment/mapping by@danbev in#3173
ruby : add Core ML support by@KitaitiMakoto in#3214
sync : ggml by@ggerganov in#3215
node : add language detection support by@danbev in#3190
ci : use mirrors.kernel.org for Ubuntu packages by@danbev in#3220
musa: correct MUSA SDK rc4.0.1 download URL by@yeahdongcn in#3217
fix: apply whisperParams in fullTranscribeWithTime by@Jo4sDev in#3201
ci : add mirror for ports.ubuntu.com (ARM packages) by@danbev in#3221
ruby : Add parallel transcription support by@KitaitiMakoto in#3222
devops: add main-intel dockerfile by@kwaa in#3229
ci(docker): build and publish main-intel image by@kwaa in#3231
ruby : output format by@KitaitiMakoto in#3237
sync : whisper.cpp by@ggerganov in#3239
ruby : add cleaning of library names in dependencies by@danbev in#3241
ci : update windows runner to windows-2022 by@danbev in#3242
ruby : add .gitignore entries for ext directory by@danbev in#3245
cli : fix short name conflict for vad options [no ci] by@danbev in#3247
server : add Voice Activity Detection (VAD) support by@danbev in#3246
whisper : fix VAD processing for skipped audio segments by@danbev in#3230
feat(server): graceful shutdown, atomic server state, and health endpoint Improvements by@sachaarbonel in#3243
let people select the model the wanna use with yt-wsp.sh using a second argument by@w1redch4d in#3251
examples : set the C++ standard to C++17 for server by@danbev in#3261
whisper : clear result_all if vad_samples is empty by@danbev in#3262
sync : ggml by@ggerganov in#3264
examples : add stereo to mono conversion in read_audio_data by@danbev in#3266
cmake : fix android build by@ggerganov in#3265
android : update CMakeLists.txt to use FetchContent for ggml by@danbev in#3268
sync : ggml by@ggerganov in#3272
ruby : specify Apple frameworks explicitly on build by@KitaitiMakoto in#3270
ci : add apt-get clean to musa Dockerfile by@danbev in#3275
whisper-command: write commands to text file by@aaron-ang in#3273
chore: add .gitignore entries for OpenVINO support by@mochiya98 in#3276
ci : reduce musa image size by@danbev in#3277
backport CoreML features to macos < 14 by@glaszig in#3255
feat: update java example by@Jo4sDev in#3281
ci : enable main-cuda build by@danbev in#3282
stream : add nullptr check of whisper_context by@danbev in#3283