Researchers demonstrate first in-orbit use of a vision-language model for autonomous Earth observation

Researchers report the first in-orbit demonstration of a vision-language model performing autonomous multi-modal inference entirely onboard a Low Earth Orbit spacecraft. The NAVI-Orbital system was deployed on a LEO spacecraft and executed zero-shot classification, scene description, and natural-language dialogue using a local vision-language model (Gemma 3).

The system enables operators to re-task the spacecraft using plain-English prompts instead of conventional command sequences. It is orchestrated by a graph-based state machine (LangGraph) that coordinates dedicated agents for detection and dialogue.

Ground benchmarking results show 88.16% accuracy on the 7,960-image curated AID benchmark. Flatsat validation and live in-orbit captures of previously unseen Earth imagery—including uncorrected YAM-9 imagery—were processed onboard using hardware-accelerated GPU inference without fine-tuning for the flight instrument.

The authors argue this approach addresses the growing gap between onboard data collection and actionable ground intelligence by semantically compressing Earth observations in orbit, thereby reducing the need to downlink raw data.