Apple proposes MoMo, a two-stage imitation-learning framework for robot motion control

Apple’s Machine Learning Research team describes MoMo, a two-stage imitation-learning framework designed to learn reusable behavioral factors—referred to as motion modes—that govern how robot actions unfold across tasks, objects, and interaction settings. The approach treats motion mode as a continuous control variable that can be varied to produce steady, dynamic, or intermediate behaviors, which human raters can distinguish based on measurable differences in joint speed, acceleration, and end-effector approach pitch.

The framework consists of a spatiotemporal action tokenizer and a behavior-cloning transformer. The tokenizer discretizes continuous action trajectories into tokens, while the transformer conditions on both the task and a continuous motion-mode input to generate appropriate action sequences. This design enables the model to reuse learned motion modes across different tasks without requiring new demonstrations for each mode.

Across six real-robot manipulation tasks, MoMo’s authors report that varying the motion-mode condition produced behaviors that differed in joint speed, acceleration, and end-effector approach pitch. In experiments where tasks were demonstrated in only one motion mode, MoMo successfully transferred to unseen requested modes while largely preserving task success rates. These results suggest the system can generalize to novel task–mode combinations, a form of compositional generalization.

The paper frames motion mode as a reusable behavioral factor that can be shared across tasks, potentially reducing the data and demonstration burden for training versatile manipulation policies. By decoupling the control of execution style from task-specific objectives, MoMo points toward more scalable and adaptable robot learning systems.