Granite 4.1 LLMs use five-stage pre-training and multi-stage reinforcement learning to achieve dense model efficiency

IBM's Granite 4.1 family comprises three decoder-only models—sized at 3B, 8B, and 30B parameters—all trained on approximately 15 trillion tokens using a five-phase pre-training strategy. The approach prioritizes data quality and composition changes over simple scale expansion, with each phase employing a distinct data mixture and learning-rate schedule.

The first two phases establish foundational language understanding: Phase 1 draws on general web data (59% CommonCrawl), code (20%), and math (7%), while Phase 2 sharpens reasoning capability by increasing math to 35% and code to 30%. Phases 3 and 4 shift toward mid-training with progressively higher-quality data, incorporating chain-of-thought reasoning trajectories and instruction data. Phase 5 extends the context window from 4K to 512K tokens through staged long-context extension on books and code repositories.

Supervised fine-tuning relies on an LLM-as-Judge framework that automatically filters ~4.1M samples against structural, semantic, and behavioral criteria. The system rejects hallucinations, false premises, and incorrect computations while maintaining contextual information such as retrieved documents and tool outputs as non-evaluated context.

The 8B instruct variant achieves performance parity with or exceeds Granite 4.0-H-Small, a 32B mixture-of-experts model, despite operating with fewer parameters and simpler dense architecture. This result was achieved through multi-stage reinforcement learning using on-policy GRPO with DAPO loss (as documented in Yu et al., 2025). All three model sizes employ identical training procedures, differing only in dimensional parameters such as embedding size and layer counts.

The models are released under the Apache 2.0 license with code available on GitHub and model weights accessible via Hugging Face, enabling external validation and reproducibility of the training methodology.