Robot training is facing a more practical limitation: a lack of high-quality real-world data. TechCrunch reports that data tools company Encord is testing new data acquisition methods, such as brainwaves and electromyography, in California, hoping to provide more usable training samples for physical AI and humanoid robots.

Encord test brainwave annotation data

In the experiment, staff members wore head-mounted displays equipped with cameras and brainwave sensors while performing tasks such as disassembling building blocks. The cameras recorded first-person perspective footage, while the sensors measured brain activity during the process.

This device comes from the German neuroscience startup Zander Labs. The idea is to infer the operator's intentions, errors, and unexpected reactions during a task through brain activity, and then convert these signals into labels that can be used for model training.

First, perform small-scale validation.

Encord stated that the collaboration is currently in the pilot phase. The company plans to first create a dataset with brainwave annotations, then provide it to the client's robot model for testing to observe whether it can improve performance, before deciding whether to scale up.

The report notes that while language models can be trained on massive amounts of text from the internet, it's difficult to find equally large amounts of original material when robots are learning physical operations. Autonomous driving companies typically collect their own data, but this approach is costly to scale; training solely with videos often lacks the details of real-world operations.

First-person perspective videos remain the mainstream.

Currently, robotics companies primarily rely on two types of data sources: one is first-person perspective video captured by staff wearing cameras, and the other is data collected by remotely controlling robots to complete tasks and recording their movements. Encord utilizes both methods.

The company is testing more detailed training tasks at its California facility, including pouring coffee, stacking chips, plugging and unplugging network cables, and handling tasks in a home setting such as vases, books, plastic vegetables, and cable management. These tasks may seem simple, but they require a high degree of precision from the robot.

Electromyographic signals were also included in the acquisition.

In addition to brainwaves, Encord is also developing another data model: attaching sensors to the forearm to read muscle electrical signals. Since ordinary videos often cannot fully record every angle and movement of the hand, the company hopes to use these signals to reconstruct hand position changes that are closer to 3D.

Encord also adds denser action description tags to the videos, such as "right hand tightening a bolt." The company believes that while this type of high-density annotation is more expensive to produce, it is more valuable for training specific tasks. For physical AI, whoever can produce high-quality data more efficiently is more likely to gain an advantage.