Apple researchers propose iterative pseudo-labeling to improve Mandarin-English code-switching speech recognition

Apple’s Machine Learning Research group describes a three-phase iterative pseudo-labeling approach designed to improve Mandarin-English code-switching automatic speech recognition (ASR). The method addresses data scarcity by generating pseudo-labels from a large unlabeled corpus to create a semi-supervised dataset.

The approach proceeds in three stages: pseudo-label generation from unlabeled speech, two-stage bilingual model training (pre-training followed by fine-tuning on supervised code-switching data), and iterative refinements to improve handling of complex code-switching scenarios.

In evaluations on the SEAME benchmark, the method achieved a Mix Error Rate (MER) reduction of 6.35% on the devman subset and 8.29% on the devsge subset, indicating measurable gains in recognition accuracy for mixed-language utterances.

The authors—Qu Yang, Cakra Wardhana, and Tim Ng—note that code-switching poses significant challenges for ASR due to limited labeled training data, and argue that leveraging unlabeled data via iterative pseudo-labeling can substantially improve system performance.