Apple proposes unlearning method that cuts computational cost by up to 50% by focusing on low-influence data points

Apple’s Machine Learning Research team published a paper proposing a method to reduce the computational cost of machine unlearning by focusing on data points with negligible influence on model outputs. The work challenges the prevailing approach in state-of-the-art unlearning, which typically treats all points in the forget set as equally important to remove.

The researchers argue that not all training data points contribute meaningfully to a model’s learned behavior. Through a comparative analysis of influence functions across language and vision tasks, they identify subsets of training data whose removal has minimal impact on model outputs. By excluding these low-influence points from the unlearning process, the team reports significant computational savings—up to approximately 50%—on real-world empirical examples.

The proposed framework introduces an efficient unlearning pipeline that reduces the size of datasets before applying unlearning algorithms. This pre-processing step leverages influence estimation to filter out points that do not meaningfully affect the model, thereby lowering the computational burden of subsequent unlearning steps without degrading model performance.

Authors Anat Kleiman, Robert Fisher, Ben Deaner, Udi Wieder, and Vitaly Feldman situate the work within broader efforts to improve privacy-preserving machine learning. The paper includes a comparative analysis across multiple tasks, demonstrating the method’s applicability to both language and vision domains.