Apple researchers propose Dynamically Scaled Activation Steering to adaptively guide generative models

Apple’s Machine Learning Research team describes Dynamically Scaled Activation Steering (DSAS), a framework designed to improve how generative models are guided toward desired behaviors such as toxicity mitigation. Unlike prior methods that apply steering uniformly across all inputs, DSAS decouples the decision of when to steer from how to steer, intervening only when undesired behavior is detected.

The method adaptively modulates the strength of steering transformations across layers and inputs, computing context-dependent scaling factors at generation time. This allows DSAS to selectively adjust the intensity of any steering method, improving the trade-off between steering effectiveness and utility preservation.

In experiments, DSAS consistently improves the Pareto front when combined with existing steering techniques, achieving better toxicity mitigation without sacrificing model performance. The authors also demonstrate DSAS’s generality by applying it to a text-to-image diffusion model, showing how adaptive steering can modulate specific concepts.

DSAS introduces minimal computational overhead and enhances interpretability by pinpointing which tokens require steering and by how much. The code is planned for release on GitHub.

The paper was accepted at the Workshop on Unifying Representations in Neural Models (UniReps) at NeurIPS 2025 and is scheduled for publication in the Transactions on Machine Learning Research (TMLR).