Gemma 4 released with multimodal, encoder-free architecture and reasoning traces

Google’s Gemma team released Gemma 4, a new generation of open-weight, natively multimodal language models that expands the Gemma family with dense and Mixture-of-Experts (MoE) architectures spanning 2.3 billion to 31 billion parameters. The suite introduces a unified, encoder-free 12B model capable of ingesting raw audio and image patches directly, eliminating the need for separate vision and audio encoders in that configuration.

The Gemma 4 models integrate a "thinking mode" that prompts the model to generate intermediate reasoning traces before emitting a final response, aiming to improve transparency and problem-solving reliability. The technical report highlights improvements in inference speed, memory efficiency, compute efficiency, and long-context capabilities through targeted design choices.

According to the report, Gemma 4 delivers measurable gains across STEM, multimodal, and long-context benchmarks, and in human-rated evaluations it rivals larger frontier open models. The authors emphasize compute efficiency and reasoning as core design goals for the new generation.

The release is documented in a technical report on arXiv, listing over 300 authors and detailing architectural decisions, benchmark outcomes, and efficiency improvements across the model family.