Hugging Face Transformers 5.18.0 adds Nemotron 3 Diarization, NemotronH Omni, HyperCLOVAX Vision V2, and GTE

Version 5.18.0 of the Hugging Face Transformers library introduces four new models: Nemotron 3 Diarization, NemotronH Omni, HyperCLOVAX Vision V2, and GTE.

Nemotron 3 Diarization is an open-weight streaming speaker diarization model designed to determine "who spoke when" in real-world audio. It supports both streaming and offline inference, handles up to eight speakers, and orders speaker outputs by each speaker's first arrival in the input audio. The model uses the Arrival-Order Speaker Cache (AOSC) and FIFO queue introduced for Streaming Sortformer. A single checkpoint supports configurable latency profiles, from an 80 ms input buffer to a 30.4 s offline-style buffer, and configurable output frame resolution in multiples of 10 ms. With chunked inference, the maximum audio duration is not limited.

NemotronH Omni is a multimodal reasoning model from NVIDIA that pairs the NemotronH hybrid Mamba-Transformer language model with a RADIO vision encoder and an optional Parakeet-based sound encoder. Image and video patches are projected through a RADIO tower and a pixel-shuffle MLP into the language model's embedding space at the <image> / <video> context-token positions; audio clips are projected similarly at <audio> positions. The result is a single autoregressive model that reasons jointly over text, images, video, and sound.

HyperCLOVAX Vision V2 is a multimodal vision-language model developed by NAVER. It combines the HyperClovaX language model backbone with a Qwen2.5-VL vision encoder. The model supports text, image, and video inputs and is capable of chain-of-thought reasoning via built-in thinking tokens (<think>...</think>).

GTE is a BERT-style bidirectional encoder proposed in the paper mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval. It replaces absolute position embeddings with RoPE, uses a gated MLP, and applies layer normalization after each residual connection. The same architecture backs multiple existing checkpoints, including Alibaba’s gte-*-v1.5, gte-multilingual-*, gte-en-mlm-*, and Snowflake’s snowflake-arctic-embed-m-v2.0.

The release also includes breaking changes and bug fixes. Breaking changes affect ROCm and vLLM compatibility, image processing speed in DETR, layer type remapping, and kernel version bumps. Bug fixes address issues in tokenizer classes, integration tests, activation kernel registration on XPU, and audio label tensor handling.