Transformers 5.8.0 adds DeepSeek-V4, Gemma 4 Assistant, and five new models

Hugging Face released Transformers version 5.8.0 on May 5, 2025, adding support for six new model families. The release spans language models, specialized assistants, and multimodal systems, indicating both market adoption velocity and the library's role as a central distribution point for emerging architectures.

DeepSeek-V4 represents a step forward in mixture-of-experts design. The model departs from DeepSeek-V3's multi-head latent attention in favor of hybrid local and long-range attention. It replaces residual connections with manifold-constrained hyper-connections and uses a static token-id to expert-id hash table for its first MoE layers. The release covers Flash, Pro, and Base variants, acknowledging different scale and performance trade-offs.

Gemma 4 Assistant is purpose-built for speculative decoding on top of Gemma 4 base models. It uses multi-token prediction and KV cache sharing with the target model to draft tokens faster, reducing latency for inference pipelines. The architecture includes cross-attention to improve prediction accuracy.

Four additional models round out the release: Granite Speech Plus adds multimodal speech-to-text transcription with speaker annotation; Granite Vision 4.1 targets document extraction for enterprise use with chart-to-code and table recognition; EXAONE-4.5 is LG AI's first open-weight vision language model with 33 billion parameters and 256K context support; PP-FormulaNet handles mathematical formula and table structure recognition from images.

The release includes a breaking change: removal of Apex integration across the library and affected models like T5. Users relying on Apex for mixed precision or fused operations must switch to PyTorch native equivalents. Tokenization improvements addressed mapping inconsistencies for DeepSeek R1 Distilled and resolved a significant performance regression in special token handling that yielded approximately 300x speedup in convert_ids_to_tokens.