Apple proposes Internalized Visual Thinking to cut video reasoning latency by more than fivefold

Apple’s Machine Learning Research team proposes Internalized Visual Thinking (IVT), a post-training framework designed to address the inference overhead of visual chain-of-thought (Visual CoT) in multimodal large language models. Visual CoT improves interpretability by generating intermediate reasoning images, but it introduces significant computational costs, particularly for proactive video reasoning where timely responses are critical.

IVT trains models to perform visual reasoning without relying on explicit pixel-space generation at inference. Instead, it jointly optimizes textual prediction and next-embedding prediction over unlabeled videos during training. Given a partially observed video, the model predicts latent representations of future frames alongside the target textual answer, capturing motion, object transitions, interactions, and latent intent.

At inference, IVT generates answers directly without synthesizing or re-encoding future frames, streamlining the reasoning process. In controlled studies across six evaluation settings—including variations in target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives—IVT improved performance over text-only post-training while maintaining the same efficient inference pathway.

Compared with Visual CoT, IVT achieved comparable or better performance while reducing end-to-end latency by more than fivefold. The findings suggest that explicit pixel-space generation at inference may not be necessary for proactive video reasoning, and that predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.