Google DeepMind launches Gemini Robotics ER 2 for real-time robot orchestration and multi-robot collaboration

Google DeepMind introduced Gemini Robotics ER 2, a model intended to serve as a high-level controller for robots. It supports real-time spatial reasoning, multi-step task planning, and collaboration among multiple robots.

The model accepts multimodal inputs—including streaming video, audio, and text—via the Gemini API, Google AI Studio, and the Gemini Enterprise Agent Platform, enabling developers to build physical AI agents.

Gemini Robotics ER 2 can orchestrate low-level control interfaces such as Vision-Language-Action (VLA) models or navigation APIs, and it can call external tools like Google Search or user-defined functions to assist with task execution.

Compared to its predecessor, ER 1.6, ER 2 improves real-time progress tracking by analyzing continuous video feeds, allowing robots to detect errors, self-correct, and determine when to advance to the next step in a task.

The model introduces multi-robot collaboration, enabling coordinated workflows across multiple robots that would be infeasible for a single robot to perform alone.

In evaluations, ER 2 outperformed ER 1.6 in tool orchestration across three control modes: real-world VLA, simulated VLA, and human teleoperation.

ER 2 integrates with the Gemini Live API using a bidirectional streaming endpoint optimized for low-latency tasks, reducing the need for stop-and-think pauses during execution.

A demonstration with Boston Dynamics’ Spot robot shows ER 2 orchestrating navigation and manipulation APIs to fetch objects on natural language command, with code examples available on GitHub.