Allen Institute for AI releases ToyBlocks, an open simulation framework for robot learning and evaluation
ToyBlocks provides controlled simulated environments, automatic task and demonstration generation, and metrics to study robotics models and their failures in a constrained domain.
1 source · cross-referenced
- Allen Institute for AI (AI2) released ToyBlocks, an open simulation framework for robot learning and evaluation.
- The framework includes a controlled tabletop environment with a robot arm and blocks, enabling studies of reasoning, dexterity, and generalization.
- ToyBlocks automatically generates tasks, demonstrations, and metrics at scale using reinforcement learning and motion planning in Nvidia’s Isaac Lab.
- Researchers can vary training mixtures, evaluate models beyond success rates, and connect frontier models via a web interface.
The Allen Institute for AI (AI2) released ToyBlocks, an open simulation framework designed to provide controlled environments for studying robot learning and evaluation. The framework pairs a simulated tabletop world with tools for automatically generating tasks, training demonstrations, and metrics to measure a robot’s behavior over time.
ToyBlocks is the first in a planned series called FoundationWorlds, which aims to extend the approach to increasingly complex environments. The initial environment focuses on a robot arm manipulating blocks, deliberately constrained to enable realistic physics, automatic data generation, and challenges that combine reasoning with dexterity.
The framework uses reinforcement learning and motion planning in Nvidia’s Isaac Lab to generate skills for short-horizon tasks such as pushing, flipping, or stacking blocks. It can produce up to 24 times real-time performance by running hundreds of plans in parallel on a single GPU.
ToyBlocks automatically generates demonstrations and labels by combining task-and-motion planning with simulation, producing datasets that describe which task or skill the robot performs. Researchers can vary training mixtures to emphasize specific skills or fill gaps in existing datasets, and evaluate models on tasks kept out of training to test generalization.
Evaluation metrics go beyond success rates, including execution speeds, forces exerted on objects, and task progress measured via the same specifications used by the planning module. A web interface lets users connect models like Claude to instruct the robot or try tasks using natural language, enabling exploration of planning and action without requiring robotics expertise.
AI2 states that ToyBlocks is designed to fulfill four requirements: realistic physics and appearance, automatic generation of tasks and demonstrations at scale, challenges that combine reasoning with dexterity, and broad coverage of tasks within its domain. The narrow scope of ToyBlocks makes it feasible to meet all these requirements, unlike open-ended domains.
- Nov 11, 2026 · Hugging Face
Hugging Face proposes token pooling-aware training to improve compressed multi-vector retrieval
Trust79 - Nov 11, 2026 · TechCrunch — AI
Fractile raises $720M Series C to develop AI chips and compiler stack
Trust79 - Nov 11, 2026 · The Verge — AI
Google’s Tabletop AI experiment lets users build custom collaborative apps via waitlist
Trust79