Hugging Face Transformers v5.21.0 adds support for MiniCPM-OCR, Flint, UltraVBert, and Pocket TTS models
The release introduces new model support, breaking changes, and performance improvements in the transformers library.
1 source · cross-referenced
- Adds native support for MiniCPM-OCR, Flint, UltraVBert, and Pocket TTS models
- Introduces breaking changes to Gemma4 processor imports and tensor parallelism behavior
- Includes new FP8 backend for QuantizedCache and fixes for Gemma4 cross-imports
Hugging Face released version 5.21.0 of the transformers library, adding native support for four new models: MiniCPM-OCR, Flint, UltraVBert, and Pocket TTS. MiniCPM-OCR is a document-understanding model from OpenBMB's MiniCPM-V family, specialized for reading text-dense images such as scanned pages, screenshots, receipts, and tables. It retains the MiniCPM-V 4.6 architecture, which combines a SigLIP vision encoder with a window-attention merger and a Qwen3.5 language model.
Flint, developed by Recursive, is a mixture-of-experts language model that introduces hybrid attention—sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers. It also incorporates learned attention sinks and per-head RMS-normalized queries and keys. Each sparse layer routes tokens to 6 of 256 experts using sigmoid scoring, with an ungated shared expert, and the first two layers are dense.
UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view, and the connector merges each 4 x 4 block of vision patches into one token, producing 64 image tokens per tile. The max_image_tiles setting caps the number of tiles per page at 27 by default.
Pocket TTS, developed by Kyutai, is a 100M-parameter text-to-speech model built as a Continuous Audio Language Model that runs faster than real time on CPU. It emits one continuous audio latent per frame at 12.5 Hz through a one-step flow head, with an end-of-speech head determining when to stop. Voices can be cloned from a recording, passed as precomputed latents, or loaded from cached KV states. The release also includes the Pocket TTS audio tokenizer, which converts 24 kHz audio into latents at 12.5 Hz and back, using a modified Mimi architecture with a continuous 32-dimensional bottleneck.
The release introduces breaking changes, including fixes for Gemma4 processor imports that previously failed in environments with PIL but without PyTorch. It also adds an FP8 backend to QuantizedCache, supporting torch.float8_e4m3fn with a single scale per tensor, enabled via cache_implementation="quantized" and cache_config={"backend": "fp8"}. Tensor parallelism now has a dedicated embedding style, shards every model's input embeddings by default, and supports widths not divisible by the tensor parallelism size. Custom tensor parallelism plans that previously applied to embeddings must switch to the new embedding style.
The changes were implemented across multiple pull requests, including updates for MiniCPM-OCR (#49920), Flint (#49824), UltraVBert (#49685), Pocket TTS (#49589), and the Pocket TTS audio tokenizer (#49588). Breaking changes were addressed in PRs such as fixes for Gemma4 cross-imports (#49748), the addition of the FP8 backend (#48677), and updates to tensor parallelism behavior (#49570).
- Nov 11, 2026 · Hugging Face
Hugging Face proposes token pooling-aware training to improve compressed multi-vector retrieval
Trust79 - Nov 11, 2026 · TechCrunch — AI
Fractile raises $720M Series C to develop AI chips and compiler stack
Trust79 - Nov 11, 2026 · The Verge — AI
Google’s Tabletop AI experiment lets users build custom collaborative apps via waitlist
Trust79