Despite significant advances in automatic speech recognition (ASR) accuracy, challenges remain. Naturally occurring conversation often involves multiple overlapping speakers, of different ages, accents and genders, as well as noisy environments and suboptimal audio recording equipment, all of which reduce ASR accuracy. In this study, we evaluate the accuracy of state of the art open source ASR systems across diverse conversational speech datasets, examining the impact of audio and speaker characteristics on WER. We then explore the potential of ASR ensembling and post-ASR correction methods to improve transcription accuracy. Our findings emphasize the need for robust error correction techniques and of continuing to address demographic biases to enhance ASR performance and inclusivity.
@inproceedings{imai-etal-2025-evaluating, title = "Evaluating Open-Source {ASR} Systems: Performance Across Diverse Audio Conditions and Error Correction Methods", author = "Imai, Saki and Chowdhury, Tahiya and Stent, Amanda J.", editor = "Rambow, Owen and Wanner, Leo and Apidianaki, Marianna and Al-Khalifa, Hend and Eugenio, Barbara Di and Schockaert, Steven", booktitle = "Proceedings of the 31st International Conference on Computational Linguistics", month = jan, year = "2025", address = "Abu Dhabi, UAE", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.coling-main.336/", pages = "5027--5039", abstract = "Despite significant advances in automatic speech recognition (ASR) accuracy, challenges remain. Naturally occurring conversation often involves multiple overlapping speakers, of different ages, accents and genders, as well as noisy environments and suboptimal audio recording equipment, all of which reduce ASR accuracy. In this study, we evaluate the accuracy of state of the art open source ASR systems across diverse conversational speech datasets, examining the impact of audio and speaker characteristics on WER. We then explore the potential of ASR ensembling and post-ASR correction methods to improve transcription accuracy. Our findings emphasize the need for robust error correction techniques and of continuing to address demographic biases to enhance ASR performance and inclusivity."}
%0 Conference Proceedings%T Evaluating Open-Source ASR Systems: Performance Across Diverse Audio Conditions and Error Correction Methods%A Imai, Saki%A Chowdhury, Tahiya%A Stent, Amanda J.%Y Rambow, Owen%Y Wanner, Leo%Y Apidianaki, Marianna%Y Al-Khalifa, Hend%Y Eugenio, Barbara Di%Y Schockaert, Steven%S Proceedings of the 31st International Conference on Computational Linguistics%D 2025%8 January%I Association for Computational Linguistics%C Abu Dhabi, UAE%F imai-etal-2025-evaluating%X Despite significant advances in automatic speech recognition (ASR) accuracy, challenges remain. Naturally occurring conversation often involves multiple overlapping speakers, of different ages, accents and genders, as well as noisy environments and suboptimal audio recording equipment, all of which reduce ASR accuracy. In this study, we evaluate the accuracy of state of the art open source ASR systems across diverse conversational speech datasets, examining the impact of audio and speaker characteristics on WER. We then explore the potential of ASR ensembling and post-ASR correction methods to improve transcription accuracy. Our findings emphasize the need for robust error correction techniques and of continuing to address demographic biases to enhance ASR performance and inclusivity.%U https://aclanthology.org/2025.coling-main.336/%P 5027-5039
[Evaluating Open-Source ASR Systems: Performance Across Diverse Audio Conditions and Error Correction Methods](https://aclanthology.org/2025.coling-main.336/) (Imai et al., COLING 2025)