Hugging Face Transformers v5.15.0 released with new model support, breaking changes, and performance improvements

Hugging Face released Transformers v5.15.0, which adds native support for several new models including Meta’s Muse Glimmer, GraniteSWA, GraniteMoeSWA, A.X-K1, A.X-K2, and Cosmos3 Edge. Muse Glimmer is a 30-billion-parameter multimodal model distilled from Muse and released under the Apache 2.0 license, designed for agentic use cases such as coding, document analysis, and personal assistants.

The release includes breaking changes to the cache cropping API, which now only accepts negative values (relative offsets) instead of absolute sizes, requiring users to update calling code. It also changes the default attention backend for T5 and its family (MT5, LongT5, etc.) to support SDPA and other backends via ALL_ATTENTION_FUNCTIONS, which may alter the default attention implementation. Users relying on the previous eager-only path are advised to explicitly set attn_implementation="eager".

Performance improvements target vision-language models, with faster image preprocessing for models such as GLM4V, MiniMaxM3-VL, and others by eliminating redundant tensor copies. The release also refactors linear attention models for better maintainability and updates attention mechanisms, including fixes for Multi-Head Latent Attention (MLA) cache compression and SDPA prefill with position bias.

Additional changes include optimizations for Flash Attention’s max sequence length computation in vision models, improved MPS support via metal-flash-sdpa integration, and bug fixes for dtype alignment in Kosmos2/Kosmos2_5 embedding merges and position-embedding initialization in Phi4Multimodal.