Researchers train language models on a K–5 curriculum to study knowledge acquisition boundaries
A controlled study finds scaling, post-training, and in-context learning amplify taught skills but do not meaningfully improve performance beyond the training distribution.
2 sources · cross-referenced
- A team trained three language models (0.6B, 1.3B, 5B) from scratch on an 88B-token corpus aligned with U.S. grades K–5, explicitly excluding material above Grade 5.
- Scaling, supervised fine-tuning with GRPO, and in-context learning improved performance within the curriculum but did not meaningfully improve out-of-scope performance.
- The authors release the models, dataset pipeline, and live chat demo to enable further study of knowledge boundaries and educational comparisons.
Researchers trained three language models at 0.6B, 1.3B, and 5B scales from scratch on an 88B-token corpus filtered to the U.S. elementary-school curriculum (K–5), excluding concepts, facts, and vocabulary above Grade 5. The dataset, LittleCurriculum, was distilled from FineWeb-Edu through a five-stage filtering pipeline aligned with Common Core standards.
Each model was paired with a matched unfiltered control sharing the same architecture, token count, and training recipe to enable clean comparisons. The authors released base, GRPO (math-specialized), and chatty variants, along with a live chat demo in the browser.
Experiments showed that scaling, supervised fine-tuning with GRPO, and in-context learning improved performance within the curriculum and modestly extended performance along the same learning trajectory, but none meaningfully improved performance on problems requiring capabilities outside the training exposure.
The authors highlight three research directions enabled by the controlled setting: using reinforcement learning to study capability emergence under a restricted prior, studying continual learning by introducing new concepts near the boundary, and comparing machine and child learners on the same exposure.
The paper and models are available on the project site and arXiv, with a live chat interface to interact with the 5B model.
- Nov 11, 2026 · The Verge — AI
Google DeepMind unveils AlphaGenome 2 to predict genetic variant effects on cell types
Trust79 - Nov 11, 2026 · Google DeepMind — Blog
Google DeepMind unveils AlphaGenome 2 to predict DNA variant effects in specific cell types
Trust84 - Nov 11, 2026 · arXiv cs.AI
Researchers demonstrate auditable human-agent teams for autonomous synchrotron X-ray experiments
Trust79