Encord and Zander Labs test brain-wave sensors to generate robotics training data

Encord, a data tooling startup focused on machine-vision applications, is running a trial with Zander Labs to explore whether brain-wave sensors can generate richer training data for physical AI models. In a San Leandro, California warehouse, pilots like Andrew Ceja wear headsets equipped with cameras and Zander Labs’ brain-wave sensors while performing tasks such as disassembling a Jenga tower. The sensors capture mental states like intent, error, and surprise, which Encord and Zander hope can provide signals that improve robotics model training.

The collaboration is currently a trial run. Encord’s goal is to build an initial dataset tagged with brain-wave data and evaluate whether it improves the performance of customer robotics models before deciding whether to scale the approach. Lucas Gehrke, a neuroscientist at Zander Labs supervising the work, suggests that brain activity patterns could help model builders determine when to deploy higher-effort models during specific tasks.

The effort reflects a broader push to address the scarcity of real-world physical training data, a bottleneck for advancing humanoid and warehouse robotics. Encord’s head of robot learning, Vineeth Velmurugan—formerly of OpenAI’s robot lab and Berkshire Grey—notes that the data required to train physical AI models does not exist at the scale needed, and that generating it is a distinct challenge from collecting text or video data.

Encord collects "egocentric" video data from factories worldwide and uses its San Leandro facility to experiment with new modalities, including brain waves and muscle-signal sensors. Pilots also use leader-follower rigs to create datasets for tasks like pouring coffee or stacking poker chips. Velmurugan estimates that dense annotation of physical tasks—such as describing hand movements like "right hand tightens bolt"—is worth 100 times the value of lower-quality ego data for training specific tasks, though it costs 20 times more to produce.

The comparison to large language models (LLMs) highlights the economic challenge: scraping text from the internet is inexpensive, but generating physical training data requires active manufacturing, making it a costly and labor-intensive process. Velmurugan, who has visibility into programs across the industry, says progress is being made as startups and labs identify what techniques work for improving physical AI models.