Apple proposes STARFlow2, a unified multimodal model architecture combining normalizing flows with frozen pretrained vision-language models

Apple’s Machine Learning Research team describes STARFlow2, a model architecture that bridges language models and normalizing flows for unified multimodal generation. The work targets a key structural fragmentation in existing multimodal systems: the trade-offs between visual fidelity, architectural symmetry, and preservation of pretrained understanding.

The authors argue that autoregressive normalizing flows share the same causal mask, KV-cache mechanism, and left-to-right structure as LLMs, making them a natural fit for unified multimodal generation that is continuous, single-pass, and purely causal. This contrasts with prior approaches that rely on discrete tokenization or diffusion-based denoising, which introduce structural asymmetry or degrade pretrained capabilities.

STARFlow2 is built on the Pretzel architecture, which vertically interleaves a frozen pretrained vision-language model (VLM) stream with a TARFlow stream via residual skip connections, both operating under the same causal mask. This design aims to preserve pretrained multimodal understanding while enabling high-fidelity continuous image generation.

The paper introduces a deep-shallow flow design and a unified FAE latent space to support cache-friendly interleaved generation. In this setup, both text and visual outputs directly enter the KV-cache without re-encoding, streamlining the generation process.

Experiments in the paper demonstrate strong performance across image generation and multimodal understanding benchmarks, suggesting that autoregressive flows can serve as a viable foundation for unified multimodal modeling. The authors position STARFlow2 as a step toward structurally unified multimodal systems that avoid the pitfalls of prior hybrid approaches.