Microsoft Research introduces MindTopo benchmark to test AI spatial reasoning and planning

Microsoft Research introduced MindTopo, a benchmark designed to evaluate how multimodal large language models (VLMs) understand topological relationships such as connectivity, enclosure, order, separation, and knots. Unlike traditional spatial benchmarks that focus on Euclidean properties like distance or shape, MindTopo emphasizes structural relationships that persist despite deformation or movement.

The benchmark evaluates models across two cognitive levels: reasoning and planning. In reasoning tasks, models analyze static scenes to answer questions about topological structure, such as whether two points in a maze are connected or whether a rope is truly knotted. In planning tasks, models interact with simulated environments to perform actions that must preserve or alter topological relationships, such as rotating pipe segments or untangling ropes without allowing strands to pass through one another.

Across a range of proprietary and open-weight models, performance was consistently stronger on static reasoning than on interactive planning, with both falling well below human performance. Models often succeeded in recognizing topological relationships in single scenes but failed to maintain those relationships during sequences of actions, revealing a breakdown in tracking structural constraints over time.

Error analysis showed that static mistakes typically stemmed from perception failures, such as missing a wall or crossing, while planning mistakes occurred after the scene was understood. Models frequently proposed locally plausible actions that ignored long-term consequences, lost track of the task over multiple steps, or violated the environment’s physical constraints.

The researchers also tested whether image or video generation could help models maintain topological understanding. While image generation sometimes aided static reasoning, it proved unreliable for preserving relationships across sequences of actions. Video rollouts often altered topology or violated task dynamics, limiting their utility for improving spatial reasoning in dynamic settings.

MindTopo’s scenes are generated from controlled simulators, enabling precise ground truth and adjustable difficulty. This control helps distinguish between failures caused by visual complexity and those caused by an inability to maintain underlying topological relationships as objects move or scenes change.