Microsoft Research finds offloading robot AI inference to edge or cloud improves performance and battery life

Microsoft Research describes a systematic study of robotics workloads that challenges the prevailing approach of running physical AI inference exclusively on onboard GPUs. The team argues that as physical AI models grow in size and complexity, onboard compute constraints—power draw, battery life, cost, weight, and model capability—become limiting factors for robot performance and scalability.

The researchers focused on mobile robotic manipulation tasks, such as retrieving and disposing of rubbish in a kitchen, which require coordinated capabilities in semantic mapping and planning, navigation, and manipulation. They evaluated representative models across onboard, edge, and cloud compute configurations, using hardware ranging from lightweight boards like Raspberry Pi-5 to high-end GPUs such as the A100.

Results showed offloading inference to edge or cloud GPUs improved robot performance across mapping, planning, navigation, and manipulation workloads. For example, mapping and planning slowed by up to 383% on smaller onboard GPUs compared to an A100, and navigation showed a 30% drop in timely obstacle detection with lighter GPUs. Vision-language-action models also suffered accuracy drops of up to 50% when run on smaller onboard GPUs.

Beyond performance, offloading inference significantly extended battery life. In experiments with a Stretch-3 robot, replacing an onboard GPU with a Raspberry Pi-5 and offloading inference to a remote GPU reduced battery drain by up to 160%, translating to several additional hours of operation between charges.

To enable distributed inference across robots, edge infrastructure, and cloud, the team introduced a new capability in the Physical AI Toolchain: containerized deployment and orchestration using Kubernetes-based tooling. This allows developers to manage robotics AI workloads across heterogeneous hardware and locations, supporting scalable and flexible inference pipelines for physical AI applications.