Hugging Face releases Grabette, an open handheld system for recording robot-manipulation data

Hugging Face published details of Grabette, an open system designed to record robot-manipulation data using a handheld gripper and cameras. The system aims to make data collection for robot learning more accessible by reducing reliance on expensive teleoperation setups or robotic hardware.

The hardware consists of a handheld device (Grabette) with two cameras, an IMU, and a gripper, and a motorized gripper (Gripette) for robotic arms. The handheld device records demonstrations with a wide fisheye camera for context and an RGBD camera for 6-DoF tracking, while the motorized gripper serves as the robotic counterpart for executing learned movements.

Grabette’s capture service runs on a Raspberry Pi and records synchronized data from the observation camera, tracking camera (color, depth, and IMU), and gripper encoder values. Users can stop recording to save episodes locally, then process the data via a browser-based dashboard that uploads episodes to the Hugging Face Hub.

The processing pipeline converts recorded episodes into the LeRobot dataset format using SLAM (RTAB-MAP) for trajectory verification. The resulting datasets are compatible with the LeRobot training stack and can be used to train policies, such as a Diffusion Policy with a ResNet18 encoder, which the team demonstrated on a 7-DoF arm with the Gripette gripper.

The project emphasizes open-source components, including CAD files for Grabette and Gripette, the capture service software, and the processing pipeline. The team states that the handheld device can be built from off-the-shelf sensors and components, with a bill of materials costing approximately €490 for Grabette and €120 for Gripette.

Hugging Face positions Grabette as a community-driven effort to seed a large, open, collaborative manipulation dataset on the Hugging Face Hub, enabling contributions from users without access to robotic hardware.