Hugging Face Transformers 5.17.0 adds support for five new models and frameworks

Hugging Face released Transformers 5.17.0, which adds native support for seven new models and frameworks, expanding the library’s coverage of speech, audio, and multimodal tasks.

The release introduces HYV4, a 780-billion-parameter mixture-of-experts language model with a 1-million-token context window. Each MoE layer contains 256 routed experts plus one always-active shared expert, routing every token to eight experts per layer. The architecture combines Multi-head Latent Attention, DeepSeek Sparse Attention, gated MLA with learnable attention sinks, and Independent Hyper-Connections, though the implementation does not execute multi-token prediction layers at load time.

VibeVoice is a new framework for synthesizing high-fidelity, long-form speech with multiple speakers using a next-token diffusion approach within an LLM structure, targeting podcasts and multi-participant audiobooks.

NeoMME is a family of multimodal-native multilingual foundation encoders (260M and 800M parameters) that processes multilingual text tokens and raw image patches in a single bidirectional Transformer encoder, without a separately pretrained vision tower or causal language model. The release also adds NeoMME-Retriever, a model fine-tuned for visual document retrieval with joint late-interaction and dense objectives.

Fun-ASR-Nano is an 800-million-parameter end-to-end speech recognition model supporting Chinese, English, Japanese, seven Chinese dialects, and 26 regional accents. It offers hotword customization and native punctuation output without requiring a separate punctuation model.

KimiLinear introduces a hybrid linear attention architecture with Kimi Delta Attention, a refinement of Gated DeltaNet that applies per-channel forget gates. Most layers use KDA, while every fourth layer retains a full-attention block using DeepSeek-V3’s Multi-head Latent Attention, and feed-forward blocks follow a DeepSeek-V3-style MoE with a shared expert.

Canary-1B-v2 is a multilingual ASR and speech-to-text translation model that reuses the Fast Conformer encoder from Parakeet and pairs it with a Transformer decoder using fixed sinusoidal positional embeddings and tied input/output embeddings. Task selection is controlled via a decoder prompt prefix.

NeuCodec is a neural audio codec that uses Finite Scalar Quantization and operates at 50 tokens per second and 16 bits per token, yielding an overall bitrate of 0.8 kbps. It accepts 16 kHz input and outputs 24 kHz audio via upsampling.