Apple proposes compact seq2seq model for ASR error correction with correction-first decoding

Apple’s Machine Learning Research team proposes revisiting automatic speech recognition (ASR) error correction with compact sequence-to-sequence (seq2seq) models trained on both real and synthetic ASR errors. The work argues that most existing methods rely on text-only models that are unaware of ASR-specific error patterns, and that recent large language model (LLM)-based approaches introduce latency and hallucination concerns.

To scale training, the researchers construct synthetic corpora using cascaded text-to-speech (TTS) and ASR pipelines, emphasizing the importance of matching the diversity of realistic error distributions. They introduce correction-first decoding, a method where the correction model generates candidate corrections that are then rescored using ASR acoustic scores.

The proposed compact seq2seq model achieves a word error rate (WER) of 1.5% on the LibriSpeech test-clean set and 3.3% on test-other, outperforming LLM-based approaches while using 15 times fewer parameters. The model generalizes across multiple ASR architectures, including CTC, seq2seq, and transducer-based systems, and performs particularly well in low-error regimes where LLMs typically struggle.

The research builds on prior work from Apple’s team, including a January 2025 paper on integrating LLMs into first-pass decoding for end-to-end ASR, and a September 2024 study on retrieval-augmented correction of named entity speech recognition errors.