NIST launches global online competition to benchmark robot manipulation skills

NIST has opened ManipulationNet, a global online competition where researchers can benchmark robot manipulation skills against standardized tasks. Competitors log in to the portal and have their robots perform jobs on camera, such as inserting a peg into progressively smaller holes. The tasks escalate in difficulty, and an AI system scores performance in real time, with NIST researchers verifying the AI’s assessments.

Previously, these competitions were conducted locally or only in simulation, limiting comparability and participation. ManipulationNet expands access by enabling competitors worldwide to test any robot hardware at any time and compare results on a public leaderboard. NIST researchers Kenneth Kimble and Robert Seney are among the experts overseeing the AI scoring and verification process.

The portal will add more tasks over time to reflect real-world challenges, broadening the benchmark’s scope beyond simple peg insertion. The initiative aims to provide a transparent, repeatable way to measure robot dexterity and inform improvements in robotic systems for manufacturing and other industrial applications.