PyTorch 2.15 brings FP8, faster LLM decoding, and memory reductions to Apple MPS

PyTorch 2.15 introduces FP8 tensor support on Apple MPS, allowing float8_e4m3fn values to be stored in half the space of BF16 while enabling scaled matrix multiplication directly on the GPU. This reduces memory usage and accelerates operations for models running on macOS.

Mixture-of-experts (MoE) models see dramatic decoding speed improvements in PyTorch 2.15, with throughput increasing 5–6.5× over PyTorch 2.14. For example, Qwen3.6-35B-A3B on an M3 Ultra rises from 9.4 to 61.3 tokens/s, and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s, both measured with torch.compile.

Dense language models also benefit from faster linear layers and an 8× speedup for softmax operations during sampling, improving overall inference efficiency. These gains come from specialized matrix-vector kernels and optimized paths for single-row decoding inputs.

Linear algebra operations receive broader GPU acceleration, including complex LU and triangular solves, faster small-matrix inverses, and QR decomposition. Convolution, padding, dropout, interpolation backward, and reductions are also optimized for MPS.

Video decoding memory usage is drastically reduced. Wan2.2’s tiled VAE decode, which previously peaked at 270 GiB, now uses about 8 GiB, making high-resolution video workflows feasible on Apple silicon Macs.

Image editing workflows are corrected with improved padding handling. The Qwen-Image-2.1 VAE encoder no longer corrupts images on MPS, and an encode-decode round trip now matches CPU quality at 29.95 dB PSNR, up from 10.06 dB in PyTorch 2.14.

PyTorch 2.15 requires macOS 15 or later and Python 3.11 or later. The update also introduces public MPS streams for better control from Python and more reliable event timing, alongside expanded operation coverage on MPS.

The PyTorch team at Hugging Face notes that earlier versions of PyTorch on MPS often required workarounds due to unsupported operations and performance cliffs. Many of the improvements in 2.15 stem from replacing MPSGraph with dedicated Metal kernels, developed in collaboration with PyTorch maintainers and community contributors.