Apple proposes LINK method to improve multilingual knowledge transfer in low-resource languages

Apple’s Machine Learning Research team introduced LINK (Lexical Interventions for Knowledge transfer), a method designed to improve cross-lingual knowledge transfer during the pretraining of multilingual language models. The approach targets scenarios where target language data is scarce, making it difficult to acquire task-specific knowledge locally. Instead, the method relies on transferring knowledge primarily from high-resource languages like English.

Existing methods for enhancing cross-lingual knowledge transfer typically require large amounts of parallel data, translation systems, auxiliary models, or additional training stages—resources that are largely unavailable for many languages. LINK addresses this gap by performing lexical substitutions in the high-resource portion of the pretraining data. Specifically, randomly selected words in a portion of the English training corpus are replaced with their word-level translations using bilingual vocabularies.

The method requires no additional model training and only a bilingual vocabulary, which the authors note can be obtained at near-zero cost for virtually any language. In evaluations across eight languages and five model sizes, LINK delivered notable improvements on downstream tasks in the target languages. The team also reported up to a 2x speedup in training time to reach equivalent performance compared to baseline approaches.

The paper is authored by Anastasiia Sedova, Natalie Schluter, Skyler Seto, and Maartje ter Hoeve, with equal contribution indicated among the latter three. It is published under Apple’s Machine Learning Research umbrella with a stated publication date of August 2026.