Hugging Face adds native PyTorch backend for AWS Trainium via TorchNeuron

Hugging Face announced TorchNeuron, a new native backend for PyTorch on AWS Trainium that integrates directly into the PyTorch runtime as a device. This change enables eager execution, torch.compile, and native distributed APIs such as FSDP and TP without requiring users to adopt compiler-centric workflows.

The update allows Hugging Face libraries—Transformers, Diffusers, Accelerate, PEFT, and TRL—to run on Trainium using the same classes and scripts used for other accelerators. Hugging Face recommends users adopt TorchNeuron over the previous Optimum Neuron pathway, which relied on PyTorch XLA and required static shapes and lazy tensors.

For inference, TorchNeuron natively supports text generation across a wide range of model architectures. According to the announcement, 89% of the top 100 most-used text-generation model architectures are supported out of the box with TorchNeuron, compared to 46% with Optimum Neuron v0.4.5. Users can move models to Neuron devices by passing device_map=neuron in their code.

The public beta of TorchNeuron will be available starting Oct. 20, 2026, and can be installed via pip install torch-neuronx. The announcement includes example code for text generation using tensor parallelism and static cache to avoid frequent recompilations.

For training, TorchNeuron enables standard Hugging Face workflows on Trainium. A supervised fine-tuning script using TRL's SFTTrainer requires only minor Neuron-specific settings, such as disabling automatic dynamic shapes and configuring distributed parallelism via DistributedConfig with tensor-parallel and FSDP sizes. The same script can run on other hardware without modification.

Parallelism is handled natively by Transformers, with distributed configuration passed through model_init_kwargs and SFTTrainer. The model is sharded as weights are loaded, and the Trainer does not wrap the model again, eliminating the need for hardware-specific flags or configurations.