Ego–Exo Data: Connecting First-Person and External Views

As embodied intelligence moves from the laboratory to industrialization, the data bottleneck has become an unavoidable core challenge for every robotics team. Within this bottleneck, a more fundamental question is emerging: what kind of 'eyes' do robots truly need to learn, like humans, to act in complex environments?

The Limits of Single-View Capture: Why Ego Data Alone Is Not Enough

For a long time, first-person-view (Ego) data has been regarded as the important reference for embodied AI training. The logic is intuitive: if robots are expected to mimic human operations in the future, having a person wear a camera to record actions from a perspective close to the robot's future task viewpoint seems like the natural choice. Ego data captures the operator's gaze direction, hand trajectories, close-up details of target objects, and accompanying body movements during operation. These are indeed foundational pieces of information required for robot imitation learning.

However, as research advances, the limitations of single-view Ego data have become increasingly apparent. First, the Ego perspective is inherently egocentric. The operator's torso, arms, and tools frequently occlude critical areas, preventing the model from fully understanding the global spatial relationships of the operation. Second, the Ego view lacks awareness of the overall environmental layout. A task such as organizing a shelf requires not only seeing the item being picked up, but also understanding shelf structure, surrounding item distribution, aisle space, and the movements of other people. Third, Ego data often lacks a panoramic verification of execution outcomes. The model may learn the motion of 'extending the arm,' but it struggles to determine whether that motion actually achieved the goal.

For these reasons, academia and industry have turned their attention to another perspective: the external view (Exo). Exo cameras record the entire operation scene from a third-person perspective, providing global spatial information, action execution results, and the relative relationship between humans and the environment. When Ego and Exo are fused, data is no longer a thin 'what I see,' but a three-dimensional 'how I act within the environment.'

Ego + Exo: Building Binocular Coordination for Embodied Intelligence

The relationship between Ego and Exo mirrors the binocular coordination of humans themselves. Humans do not act solely based on the images projected onto their retinas; instead, they build a three-dimensional understanding of the world by integrating information from multiple visual sources. Embodied intelligence models require the same multi-view fusion capability. Ego provides intention and fine-grained motion, while Exo provides spatial context and outcome feedback; only their combination can form a complete 'perception-decision-execution' loop.

From a data-science perspective, the value of Ego-Exo dual-view data is evident at three levels. First is information complementarity. Ego excels at capturing hand posture, object texture, and operational intention; Exo excels at recording whole-body motion, environmental constraints, and task outcomes. Second is temporal consistency. When two or more video streams are aligned via precise timestamps, the model can learn multiple representations of the same action from different viewpoints, and this multi-view supervisory signal significantly improves the robustness of representation learning. Third is enhanced transferability. Models trained on Ego-Exo aligned data are more easily transferred across different robot platforms because they simultaneously grasp both 'the world through the operator's eyes' and 'the world through the observer's eyes.'

TOBI E2 in an Ego–Exo capture workflow

E2 can serve as the first-person component of an Ego–Exo collection setup. Its current published specification lists two RGB global cameras at 1600×1200 and up to 30 FPS. Preserve the original observations and calibration metadata when designing a research capture workflow. External-camera integration must be validated for the intended configuration.

Ego–Exo datasets require a documented time-alignment method. E2’s binocular hardware synchronization specification concerns its own two cameras; it must not be treated as a guarantee for external-camera, wrist-camera or glove synchronization. Measure cross-device time offsets and drift on the actual setup and record the accepted tolerance in the data card.

In TOBI's data collection methodology, real industrial tasks, natural whole-body motion, Ego+Exo multi-view capture, 1600×1200 at up to 30 FPS raw image quality, wide-angle or even ultra-wide-angle field of view, no stabilization, and QR-code time synchronization together form a complete data-quality standard. This is not merely a stack of technical parameters, but a systematic answer to the question of 'what robots need to learn from real humans.'

Dual-View Data Is Catalyzing a New Generation of Model Paradigms

The rise of Ego-Exo dual-view data is driving an upgrade in embodied intelligence model paradigms. In imitation learning, dual-view data enables policy networks to simultaneously leverage first-person fine-grained information and third-person spatial information, significantly improving generalization on long-horizon tasks and in complex environments. In world-model training, Ego-Exo data provides richer dynamic scene observations, helping models learn more accurate physical predictions and state transitions. In cross-embodiment transfer research, dual-view data serves as a critical bridge between human videos and robot skills: human demonstration Ego videos provide direct supervision for policy learning, while Exo videos help robots establish their own spatial localization in the world.

Research presented at RSS 2026 and EIRCS 2026 shows that Ego-Exo aligned data consistently outperforms single-view methods across multiple benchmarks. Projects such as ACE-Ego-0, which unify human video and robot data, make dual-view capture a core requirement of their data specifications. Xiaomi's validation of a 100,000-hour real-data Scaling Law in its robotics R&D likewise depends on the support of a multi-view capture system for data density and diversity. In short, dual-view capture has evolved from an 'optional enhancement' into 'infrastructure.'

From Data to Deployment: Engineering Challenges in Dual-View Capture

Although the theoretical value of Ego-Exo dual-view data is clear, scaling its collection in real-world scenarios still faces multiple engineering challenges. The first is flexibility in camera deployment. The position, angle, and number of Exo cameras must be dynamically adjusted to the task scenario to ensure comprehensive coverage of critical areas. The second is robustness to lighting and environmental variation. Factory, warehouse, and home environments differ enormously in lighting conditions, requiring the capture system to have strong adaptive capability. The third is management of massive data volumes. Multi-view, multi-modal data creates enormous storage and bandwidth pressure; without a powerful data operations platform, raw material can easily fall into the trap of 'collect and forget.'

Through the EGO Data Operations Platform and the EGO Embodied Intelligence Data Annotation Platform, TOBI provides customers with end-to-end capabilities spanning capture task management, device scheduling, data upload, quality monitoring, semantic annotation, action segmentation, and success/failure labeling. The value of dual-view data can only be truly converted into model-training assets when supported by a complete toolchain.

First-person and external views answer different questions. Combining them can make a demonstration easier to interpret, provided synchronization, calibration and task labels are reliable.

Conclusion: Giving Robots a More Complete Pair of Eyes

The competition in embodied intelligence ultimately comes down to a competition over data quality. And the competition over data quality is shifting from 'whether you have data' to 'whether the data is complete, authentic, and capable of multi-view coordination.' Ego-Exo dual-view capture is not a simple replacement for single-view capture, but a paradigm upgrade in dimensionality. It allows robots not only to see what the operator sees, but also to understand the operator's position in the world, the global impact of actions, and the true outcome of task execution.

E2 provides a first-person capture component; the quality of the resulting Ego–Exo dataset depends on the complete setup. Validate camera calibration, cross-device timing, coverage and labels before scaling collection.