Skip to content
Tools · Oct 29, 2026

Hugging Face Transformers v5.22.0 adds support for Mistral Large 4 and FP8 adapter training

The update introduces native integration for Mistral’s new Mixture-of-Experts model and enables trainable FP8 quantization for adapters, alongside bug fixes and performance improvements.

Trust79
HypeLow hype

1 source · cross-referenced

ShareXLinkedInEmail
TL;DR
  • Hugging Face Transformers v5.22.0 adds native support for Mistral Large 4, a 1T-parameter multimodal MoE model with 52B active parameters.
  • The release enables FP8 quantized adapter training with backward passes, reducing peak memory in MoE layers from ~28.9 GiB to ~2.8 GiB at 8k tokens.
  • Bug fixes include processor warnings, boolean padding in feature extractors, and improved context handling for RoPE.

Hugging Face Transformers v5.22.0 integrates Mistral Large 4, Mistral’s largest and most capable model to date, as a natively supported model in the library. The model is a 1 trillion-parameter Mixture-of-Experts architecture with 49 billion active parameters per token (52 billion including embeddings and output layers), trained on 3,800 NVIDIA Grace Blackwell GPUs in Mistral’s European datacenters.

The update also introduces support for FP8 quantized adapter training, enabling backward passes on dense and grouped FP8 matrix multiplications. This reduces peak memory usage in MoE layers during backward passes from approximately 28.9 GiB to 2.8 GiB at 8k tokens, improving efficiency for fine-tuning workflows.

Additional improvements include fixes for processor-related warnings, boolean padding in the ClapFeatureExtractor, and enhanced handling of context for RoPE (Rotary Position Embedding) via the PermuteForRope utility.

The release resolves several bugs, such as unnecessary processor warnings, incorrect boolean padding in feature extraction, and issues with uninitialized layer cropping. It also updates documentation and removes deprecated benchmarking infrastructure to streamline future development.

Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights and supports a 1M context window. It accepts both text and image inputs with text output, and is natively fluent in over 160 languages, including all official EU languages.

Sources
  1. 01GitHub · huggingface/transformers releases — v5.22.0: Mistral Large 4 (Le Chonk)
Also on Tools

Stories may contain errors. Dispatch is assembled with AI assistance and curated by human editors; despite the trust-score filter, mistakes happen. We correct publicly — every article links to its revision history. Nothing here is financial, legal, or medical advice. Verify before relying on any claim.

© 2026 Dispatch. No ads. No sponsorships. No paid placement. Reader-supported via Ko-fi.

Built by a person who cares about honest AI news.