Hugging Face Transformers v5.14.0 released with Inkling model support, performance improvements, and breaking changes

Hugging Face published Transformers v5.14.0, which adds native support for Inkling, a 975-billion-parameter multimodal model accepting text, image, and audio inputs to produce text outputs. The model is released with open weights for research, fine-tuning, and integration into downstream products, and is positioned for use in agentic systems, coding assistants, chatbots, and retrieval-augmented generation workflows.

Performance work in the release targets attention kernels and decoding. SDPA prefill leverages FlashAttention with StaticCache to deliver up to 260% speedups at large input sizes, and a FlashAttention performance regression affecting models such as Qwen3-VL was fixed. A separate decode optimization corrects a bug where grouped-to-batched matrix multiplication was not applied to experts residing in submodels, improving MoE decoding scenarios.

Breaking changes include renaming GPTNeoX's embed_out to lm_head and enabling _supports_attention_backend = True for GPTBigCode to align with vLLM compatibility requirements; users relying on prior weight names or attention backend behavior should update their code.

New decoding features include Multi-Token Prediction support and static ensemble verification for speculative decoding to raise draft token acceptance rates. A misleading double-negative warning about synced_gpus in continuous batching mode was corrected, and a crash in greedy assisted generation with mismatched tokenizers was resolved.

Cache routing was simplified via explicit layer-type mappings for sliding and static layers, reducing complexity in cache dispatch. Additional fixes address read-only cache failures in CPU CI environments and MPS graph cache growth during variable-length batch training on Apple Silicon.

Kernel-related updates include pinning the kernels dependency to a compatible version in benchmark workflows, removing a deprecated package_name argument from LocalLayerRepository, and making the DeepGEMM Triton fallback more robust when CUDA_HOME is misconfigured.

The release also contains routine bugfixes and documentation improvements, including expanded CI coverage, TokenizersBackend fallback handling, and runtime resolution of continuous batching XPU availability checks.