Apple proposes CalibAtt, a training-free method to accelerate video generation via calibrated sparse attention

Apple’s Machine Learning Research team describes CalibAtt, a training-free method designed to accelerate diffusion-based text-to-video generation by exploiting stable sparsity in attention computations. The core observation is that a significant fraction of token-to-token connections consistently yield negligible attention scores across inputs, and these patterns repeat across queries. CalibAtt performs an offline calibration pass to identify block-level sparsity and repetition patterns that are stable across inputs, then compiles these patterns into optimized attention operations for each layer, head, and diffusion timestep.

At inference time, CalibAtt computes only the selected input-dependent connections densely and skips the unselected ones in a hardware-efficient manner. This reduces the computational overhead from spatiotemporal attention, which is a known bottleneck in large transformer-based video generation backbones.

In experiments across multiple models—including Wan 2.1 14B, Mochi 1, and few-step distilled variants at various resolutions—CalibAtt achieves up to 1.58× end-to-end speedup over prior training-free acceleration methods. Crucially, the authors report that video generation quality and text-video alignment are maintained despite the sparsification.

The paper positions CalibAtt as a practical, deployment-friendly technique: it does not require retraining, adapts to existing model weights, and targets inference-time efficiency. The authors suggest this approach could broaden access to high-quality video generation by lowering compute costs and latency without sacrificing output fidelity.