From observing the world to acting verifiably: embodied data collection white paper

This white paper is based on Tuobi Technology's observations at the 2026 World Robot Conference and concurrent industry exchanges, user-provided conference materials, and public research available as of August 20, 2026. Starting from data engineering rather than a single model approach, the article discusses how to organize first-person manipulation, multi-view geometry, actions, contact, quality evidence, and data governance into verifiable embodied data assets.

Abstract

In August 2026, the World Robot Conference was held in the Beijing Economic-Technological Development Area, with the theme "Symbiosis of Humans and Machines, Integration of Production and Demand." According to the official conference release, this edition placed frontier technology exchange, application-driven adoption, ecosystem co-building, and industry collaboration within a single framework, and highlighted embodied intelligence, real-world scenarios, and large-scale deployment as key topics.[1] For TOBI, the value of attending the conference and the concurrent industry exchanges goes beyond observing new products, tracking specifications, or judging hot topics — it lies in re-answering a fundamental question:When a robot needs to understand, predict, and change the environment in an open world, what kind of data qualifies as evidence for learning the world?

This white paper argues that the core competition in embodied intelligence is shifting from "acquiring more video" to "forming more usable evidence." High-quality embodied data is not an isolated video, action, or annotation file, but a set of verifiable observations carrying time, space, action, contact, task, quality, ownership, and version information. It must not only let the model see people and objects, but also let the model know where the camera is, when actions occur, why objects change, whether the record is trustworthy, and whether the data can be reproduced, compared, and audited across different teams, different robotic platforms, and different versions.

Based on this assessment, this article proposes "Ego data capture—spatial alignment—task modeling—quality evidence—model closed loop" five-layer data infrastructure roadmap. It treats first-person perspective as the natural entry point for human operational intent and operational process, multi-view geometry, inertial, and optional contact signals as physical constraints, task semantics and failure samples as learnable decision boundaries, data cards and lineage information as procurable, reproducible, and governable delivery units, and places world models, VLA, and policy evaluation back into the same closed loop for collaborative optimization.

Core judgment: World models, VLA, visual foundation models, and control policies should not be reduced to a "debate over routes." They address problems at different levels: VLA is responsible for converting current observations and goals into actions, world models are responsible for inferring the possible futures caused by actions, and data infrastructure is responsible for making the associations learned by both verifiable in terms of time, space, task, and quality.[2] [3]

The "forward-looking" perspective discussed in this article is not a promise that any technology will automatically solve generalization, causality, or safety problems in the short term. On the contrary, as model capabilities enter real-world scenarios, data collection, calibration, authorization, quality control, and evaluation protocols will move from back-office work to front-line capabilities that determine system boundaries. TOBI hopes to use auditable, reproducible, and procurable embodied data assets as a lever to jointly build a physical AI data foundation that can withstand rigorous verification, together with robot OEMs, model developers, site owners, research institutions, and service networks.

I. From the Conference Floor to the Data Proposition: Why Embodied Intelligence Enters the "Evidence Engineering" Phase in 2026

Official information from the World Robot Conference shows that the 2026 event will be held from August 19 to 23, focusing on frontier technologies, application scenarios, industrial ecosystems, and international cooperation; related releases also emphasized topics such as dataset construction, application trials, pilot-scale validation, and full-lifecycle traceability.[1] [4] These signals collectively point to a change: the robotics industry no longer only discusses "whether a single action can be performed," but increasingly needs to answer "whether actions can be stably reproduced across different scenarios, whether failures can be explained, and whether experience can be transferred to more embodiments and tasks."

A simple yet critical expression repeatedly appeared in on-site exchanges: if the current state is $s_t$ and the action is $a_t$, then what the system cares about is not only "what action should be output at this moment," but also the state transition represented by $s_{t+1}=f(s_t,a_t)$. This expression is consistent with the core definition of robot world model research: a world model should describe how the environment evolves under the conditions of actions, and serve planning, evaluation, simulation, data generation, or policy optimization.[2] However, from an engineering perspective, the premise for the formula to hold is not that the model is large enough, but that the training and evaluation data can reliably answer four questions:What is the state, what is the action, under what temporal relationship do the two occur, and whether the state change can be verified by spatial and physical constraints.

Therefore, the difficulty of embodied data is not connecting more cameras to a storage system. The movement of the same hand in a video, if lacking temporal synchronization, camera pose, task phase, object identity, action boundaries, or success criteria, may only be a highly correlated but non-transferable visual clip during model training. Conversely, a trajectory that has been synchronized, calibrated, and quality-audited, even if limited in duration, allows researchers to inspect every link from observation to action: whether the target appears, whether hand-object contact occurs, whether the viewpoint is occluded, whether the end-effector trajectory is continuous, whether the task succeeds, whether a failure stems from perception or control, and whether the data is permitted for the intended training purpose.

This is what this article refers to asEvidence engineering. It does not replace algorithm innovation, but establishes verifiable inputs for it. A VLA data survey indicates that future progress increasingly relies on the co-design of datasets, benchmarks, and data engines; real data, synthetic data, video-to-data pipelines, and hardware-assisted capture each have their own advantages and boundaries, and a single metric can hardly cover the tension between realism, cost, scale, and generalization.[3] For the industry, the next stage of competition should not merely ask "how many hours of data are there," but rather "what level of evidence can each hour of data provide for which models, which tasks, and which evaluations."

From a "capture volume" mindset to an "evidence engineering" mindset	Quantity-centric approach	Verifiability-centric approach
Core unit	Hours, frames, video files	Task-oriented, versioned, traceable trajectories and data cards
Visual information	Visible imagery from a single viewpoint	Ego, external, wrist-mounted viewpoints and camera geometric relationships
Action information	Optional or inferred post hoc	Pose, joint, end-effector, or action labels synchronized with the observations
Quality judgment	Clarity, whether recorded	Synchronization, calibration, coverage, continuity, success/failure, anomalies and authorization
Delivery method	Material package	Datasets, samples, quality reports, data cards, licenses and version records
Value outcome	Available for browsing or coarse training	Usable for training, evaluation, audit, procurement and the iteration closed loop

II. Reformulation of Scientific Questions: What VLA, World Models, and Spatial Intelligence Each Solve

Setting VLA and world models against each other is the most common source of misunderstanding in current discussions. A VLA can be expressed as a policy function $\pi$: given a visual observation $o_t$ and a language goal $l$, it outputs an action or action segment $a_{t:t+k}$. Its advantage is that it places visual understanding, language instructions, and control output into a unified framework, enabling robots to learn behaviors for more open-ended sets of instructions and objects.[2] [3] However, a direct observation-to-action mapping faces error accumulation and causal attribution challenges in long-horizon tasks, under occlusion, with branching outcomes, under contact uncertainty, and in out-of-distribution scenarios.

World models attempt to learn $p(x_{t+1:t+H}\mid x_t,a_{t:t+H-1},l)$: predicting future states or future observations conditioned on the current state, action sequence, and high-level goal.[2] Their significance lies not in generating a video that "looks real," but in generatingfutures that are sensitive to candidate actions, self-consistent in spatial structure, and useful for subsequent decisions. If a model predicts the same object to contradictory positions across different camera viewpoints, or makes a gripper appear to complete an action in the image that does not conform to actual geometry and contact constraints, then even if the prediction is visually fluent, it cannot directly serve as a reliable basis for control.

Spatial intelligence occupies a critical position between the two. It concerns the geometric relationships among objects, hands, tools, robots, and the environment within the same coordinate system, as well as how these relationships evolve over time. For embodied data capture, spatial intelligence does not mean "adding an extra depth map file." It means: the capture system needs to provide a unified description of camera intrinsics, extrinsics, IMU, SLAM poses, hand or end-effector trajectories, object states, and the coordinate transformation chain, while continuously recording their calibration versions and validity intervals. Only in this way can a model's visual representations have the opportunity to establish stable links with executable motion, contact, and workspace.

Research in 2026 further reinforced this judgment. The PAIWorld paper points out that multi-camera robot manipulation requires maintaining consistency of object position, depth, and texture across viewpoints such as ego, external, and wrist; simply concatenating tokens from multiple viewpoints makes it difficult to explicitly establish cross-view geometric communication and 3D constraints.[5] Ctrl-World shows that controllable multi-view world models must address long-horizon consistency, frame-level action conditioning, and prediction under novel camera positions; in its specific research setting, it also explored using imagined trajectories to rank and improve policies.[6] These results do not mean that industry can skip real data collection; rather, they show that high-quality real collection requires letting the model learn camera relationships, action conditioning, and quality boundaries.

TOBI's R&D position: First-person data is not "cheap video" that replaces robot data, and multi-view data is not a "collection of materials" formed by stacking more cameras. Both should form a convertible, inspectable, and versionable evidence chain through a unified temporal baseline, spatial calibration, and task semantics, thereby serving the prediction of world models, the actions of VLA, and system-level evaluation.

III. Why Ego Data Becomes a Key Entry Point to Transferable Skills

When a human unscrews a bottle cap, tidies a countertop, assembles parts, places or removes a container, or uses a tool, the first-person viewpoint naturally captures task-relevant regions of attention, hand–object relative relationships, occlusion patterns, operational rhythm, and intent progression. It is not automatically equivalent to a robot's viewpoint, yet it shares important structure with head, wrist, or near-field views in many robot operating systems: the target object typically lies within the reachable workspace, the hand or end effector engages in continuous interaction around the target, and the outcome of each action is immediately visible in the local field of view. The value of ego data lies in recording "how the task is seen and advanced by a human" as a temporally continuous process, rather than merely storing static images of the task's start and end.

Research released by EgoDex at ICLR 2026 provides a clear case: this dataset records human dexterous manipulation using first-person video with joint 3D hand and finger tracking, covering 829 hours across 194 categories of tabletop tasks.[7] Its important insight is not the scale figures themselves, butVision, hand motion, and spatial tracking are paired at recording time。For algorithms that aim to learn manipulation structure from human demonstrations, video alone can easily conflate "seeing a hand move" with "knowing how the hand moves in 3D space"; paired spatiotemporal records provide a clearer foundation for imitation, inverse dynamics, action representation learning, and cross-platform mapping.

However, the first-person perspective also has inherent limitations. It may occlude key contact surfaces and provides insufficient visibility of tool end-effectors, global layout, and external safety boundaries; there are also differences in morphology, degrees of freedom, and dynamics between the human hand and the robot end-effector. Therefore, TOBI advocates treating ego as a high-value primary viewpoint, not the only viewpoint. For high-value tasks, ego, external observation, wrist or end-effector close-ups, IMU/SLAM poses, and optional tactile, force, and proprioceptive states should be unified into a single trajectory. This design preserves the local advantage of "where the manipulation intent originates" while also obtaining external evidence of "whether the action truly occurred in the 3D environment."

Data level	Recommended recorded content	Primary problem addressed	Applicable tasks
Ego First-Person View	First-person RGB/video, head or device pose, IMU	Intent, attention, hand-object local interaction, natural manipulation rhythm	Human demonstrations, process learning, manipulation segmentation
External spatial viewpoint	Fixed or mobile third-person view, camera calibration, depth/point cloud (optional)	Global layout, occlusion compensation, spatial geometry, evaluation visibility	Assembly, placement, bimanual coordination, long-horizon tasks
Close-up of the end effector	Wrist/tool camera, end-effector pose	Local geometry before and after contact, micro-manipulation and tool state	Plugging, grasping, tightening, fine-grained placement
Platform and Control	Joints, end-effector, gripper, velocity, commands, control frequency	Video-to-action, action alignment, cross-embodiment transfer	Robot teleoperation, policy training and evaluation
Contact and Physics	Force, torque, tactile, pressure, object state (as needed)	Contact events, friction, deformation, stability	Soft objects, assembly, articulated objects, rapid placement
Task and quality	instructions, steps, success/failure, anomalies, coverage labels, authorization	Supervision, retrieval, evaluation, compliance, and procurement	All deliverable data assets

The DROID real-world manipulation dataset likewise illustrates that scaling data collection is not merely about deploying equipment in more locations. It uses synchronized external and wrist cameras, camera calibration, multiple action representations, collector and task metadata, success/failure labels, and improves reproducibility in distributed collection settings through unified hardware and procedures.[8] The implication for TOBI is that egocentric data acquisition devices must serve two roles simultaneously: an "observation window" and a "data evidence collection node." On the device side, stable generation of synchronized, continuous, calibratable raw streams is required; on the platform side, every stream must be traceable back to the device, time, task, version, and quality result.

IV. From Camera to Space: Multi-View and Spatiotemporal Synchronization as the First Gate to Data Usability

The most insidious and costly errors in embodied data are often not "blurry footage," but silently distorted temporal and spatial relationships. For example, visual frames lagging the IMU by tens of milliseconds, hand trajectories offset from object contact moments, or camera extrinsics still using stale values after device collisions or loose wearing. To a human viewer, these deviations may be unnoticeable; to a model that must learn action-conditioned state transitions, they dilute genuine causal relationships into spurious correlations.

Therefore, the first principle of high-quality data capture should beDefine time first, then accumulate frames. TOBI recommends organizing all sensor streams around a unified clock, explicit timestamp semantics, and a verifiable synchronization error bound. For each collection session, the clock source, sensor frequency, frame drop/retransmission strategy, synchronization method, offset detection results, and applicable validity interval should be recorded. Quality reports should not merely state "synchronized"; the acceptable temporal error should be part of the task specification: high-speed grasping, rapid placement, and contact moments require tighter bounds, while low-dynamics process observation can use more economical configurations.

The second principle isdefine the coordinate chain first, then discuss 3D understanding. Every data package should at minimum clearly answer: what is the camera's positional relationship to the wearer, tool, robot base, and scene coordinate frames; which version of intrinsic and extrinsic parameters was used; under what quality thresholds are SLAM, calibration boards, visual reconstruction, or other methods valid; and through which traceable steps are coordinate transforms generated. 3D is not a depth map, but an inspectable transform system. If this system cannot be reproduced, a model may appear to have "multi-view" capability during training but be unable to convert perception into stable action at deployment.

The third principle isExplicitly flag spatial anomalies as part of the data. Reflections, transparent objects, rapid motion, severe occlusion, relocalization, calibration failure, device displacement during wearing, map drift, and strong lighting changes should not be silently removed during post-processing. They are both quality risks and hard cases needed for model robustness. The appropriate approach is not to mix all anomalies into "normal data," but to provide filterable quality grades, anomaly types, and disposition records, enabling research teams to select strict subsets, robust subsets, or dedicated challenge sets according to task requirements.

V. New Requirements the World Model Places on Data: Not "Future Frames," but "Controllable Futures"

Traditional video prediction focuses on visual continuity and natural appearance of people and objects; robot world models must go further: predicted futures must be consistent with candidate actions, view geometry, object constraints, and contact events. The world model survey summarizes this direction as policy-oriented predictive structure, whose value includes lookahead, imagination-driven planning, and data augmentation; the key is not visual plausibility but operational meaning for the consequences of actions.[2]

This means the acquisition system should serve counterfactual and control questions from the outset. If only one successful action is recorded for the same initial state, the model cannot distinguish "which actions would produce different outcomes." Therefore, a high-quality data engine should record reasonable action variants, different initial conditions, near-success failures, retry processes, and recovery behaviors within the bounds of safety and task feasibility. Failure data is not waste to be discarded; with clear failure causes and safety and privacy boundaries ensured, it is critical evidence for defining policy boundaries, training risk recognition, and improving world models.

Ctrl-World's research places multi-view, frame-level action conditioning, and long-horizon consistency into a single model, and under specific evaluation settings explores using an imagined space to assist policy ranking and supervised fine-tuning.[6] This direction reminds the industry: future datasets must not only provide "what happened," but also, through high-quality alignment of states, actions, and outcomes, give models the opportunity to learn "what might happen if this is done." However, imagined trajectories cannot replace real-world validation. They are better suited as tools for reducing screening cost, generating hypotheses, and assisting training; whether something is ultimately deployable still requires closed-loop testing in real scenarios, on real hardware, under real safety constraints.

For TOBI, the data strategy in the world model era should comprise three categories of collaborative assets. The first isReal base data, used to preserve physical facts that are not easily simulated, such as camera noise, contact friction, environmental interference, and human operational variation; the second isstructured task data, used to provide state, action, goal, phase, success conditions, and exception boundaries; the third category isverifiable derived dataFor example, reconstruction, annotation, segmentation, alignment, synthetic augmentation, and quality labels. These three must be connected by lineage relationships to prevent derived data from becoming difficult to attribute or reproduce once separated from the original evidence.

VI. Beyond "Video + Text": How Contact, Force Sensing, and Object State Enter High-Value Datasets

The truly difficult tasks for robots are often not about seeing objects, but about handling invisible physical quantities: how much damping when twisting a bottle cap, whether a plug has made contact, whether a soft object is being squeezed, whether a drawer is obstructed, whether a cup is about to slip. Policies relying solely on vision can accomplish some open-world tasks, but in contact-rich, visibly deformable, or safety-sensitive scenarios, vision is often limited by occlusion, material properties, and viewing angle. In such cases, force, torque, tactile, pressure, proprioceptive state, and object state are not icing on the cake, but supplementary evidence of "what is happening."

The Hoi! dataset at CVPR 2026 jointly records cross-view video with force, tactile, and articulated object manipulation, covering human hands, wrist cameras, and multiple tool embodiments, demonstrating that "seeing, doing, and feeling" can be organized within the same data structure.[9] Another data study on humanoid contact manipulation shows that dense tactile signals can express time-varying contact patterns under soft objects and varying pressure conditions, but also bring higher-dimensional, more complex optimization and quality control challenges.[10] This illustrates that multimodality is not "more signals are better," but must be configured around the observable physical quantities required by the task.

TOBI recommends structuring the multimodal strategy into three tiers. The base tier includes vision, time, space, and action as mandatory modalities, supporting the vast majority of perception-action learning. The enhanced tier adds wrist-view, depth, end-effector state, or object state for tool use, fine-grained grasping, assembly, and mobile manipulation. The high-value tier adds force/torque/tactile/pressure for tasks such as plugging, tightening, soft-object handling, rapid placement, and hazardous contact, with dedicated sensor calibration and quality rules. This tiering avoids indiscriminately deploying expensive sensors across all tasks while also avoiding delivering only surface-level video for tasks that most require physical evidence.

In terms of data structure, "contact events" should be elevated from post-hoc labels to queryable entities. A single grasp can include phases such as approach, pre-contact, initial contact, stable grasp, movement, release, and outcome confirmation; each phase should be associated, where possible, with the corresponding visual segments, spatiotemporal states, action segments, contact signals, object states, and quality conclusions. This way, researchers can both train end-to-end models and build diagnostic models targeting failure points, such as distinguishing perception failures, path planning failures, force control failures, object state estimation failures, or task semantic understanding failures.

VII. Defining "High Quality": From Single Sharpness Checks to Six-Dimensional Quality Evidence

High quality is not an adjective but a quality definition that can be executed, spot-checked, and reviewed. A common misconception in the industry is equating high resolution, long duration, or large scale with high quality; these attributes are certainly important, but they cannot prove that data is suitable for a specific task. A 4K video without trustworthy timestamps, with unverifiable camera poses, ambiguous task success criteria, or unclear authorization scope still struggles to become a reliable asset for robot policy training. Conversely, a trajectory with moderate resolution but clearly defined actions, space, tasks, and quality boundaries may be more suitable as a foundation for model evaluation, failure diagnosis, and reproducible training.

TOBI proposes using "Six-Dimensional Quality Evidence Model" to define the delivery quality of embodied data. The first dimension isPerception completeness, checking for missing images, depth, IMU, audio, or other required streams, exposure, motion blur, occlusion, and visibility. The second dimension isSpatiotemporal consistency, checking the unified clock, cross-stream synchronization, camera intrinsics and extrinsics, pose continuity, coordinate transforms, and relocalization anomalies. The third dimension isAction usability, check the action representation, control frequency, end-effector/joint states, action continuity, command-versus-execution discrepancy, and action boundaries. The fourth dimension isTask semantic correctness, checking whether language instructions, objects, tools, steps, success/failure, anomalies, and goal states match the raw evidence. The fifth dimension isPhysical validity, to check whether contact, force perception, object state, and safety boundaries reach the level of observability required by the task. The sixth dimension isGovernance traceability, checking source, licensing, privacy handling, version, processing pipeline, QC records, and usage restrictions.

six-dimensional quality evidence	Key Issues	Typical measurement or spot-check methods	Deliverables
Perception completeness	Did the model see what was needed to complete the task?	Frame drop rate, exposure/blur detection, key object visibility rate, occlusion spot checks	Stream integrity report, anomalous segment index
Spatiotemporal consistency	Did different sensors record the same event?	Temporal offset, drift, reprojection error, pose continuity, calibration validity period	Synchronization report, calibration files, coordinate system documentation
Action usability	Can observations be associated with executable actions?	Frequency, continuity, joint / end-effector conversion, command–execution error	Action dictionary, control interface documentation
Task semantic correctness	Can the task, steps, and results be verified?	Dual-person spot checks, rule consistency, object/instruction alignment, failure-reason labels	Task ontology, annotation specification, spot-check records
Physical validity	Are the records of contact, deformation, and constraints sufficient?	Force/tactile calibration, event alignment, object state, contact coverage	Modality descriptions, physical event tables, sensor reports
Governance traceability	Can the data be used safely, legally, and reproducibly?	Source chain, license verification, de-identification records, hashes, version comparison	Data cards, licenses, lineage graphs, version logs

This model emphasizes one principle:Quality is relative to the intended use. For large-scale visual pretraining, prioritize diversity in scenes, objects, and actions. For data used in 3D world models, requirements for calibration, cross-view correspondence, and spatial consistency must be raised. For data used in high-risk contact tasks, stricter action, force-sensing, and failure records are required. Treating all data against a single "full-score standard" wastes resources, while delivering data for different purposes against a single "minimum standard" creates uncontrollable risk. The correct approach is to specify the purpose, task, modalities, and quality thresholds in the SOW, collection plan, and data card, so that acceptance shifts from subjective impressions to verifiable evidence.

8. Data Cards and Data Lineage: Making Embodied Data a Procurable, Reproducible Engineering Asset

High-quality data should not merely be a file directory; it should also have an "identity card" understandable to both humans and machines. Robot data lineage research proposes that the relationships between source, configuration, execution, and post-processing should be recorded during data generation; supplementing explanations only at release time often cannot recover the true inputs, versions, and processing paths.[11] This aligns closely with the characteristics of embodied data: many sensors, fast data streams, frequent hardware and software version iterations, widely distributed scenes and personnel — any missing record can deprive subsequent training or evaluation of interpretability.

Therefore, SKYWORTH XR recommends usingTrajectory level — batch level — dataset levelThree-tier data card organization for delivery. The trajectory-level data card records the task, time, device, sensors, coordinate frame, quality conclusion, success/failure, anomalies, and authorization status of the current operation; the batch-level data card describes a set of trajectories collected under the same specification, including anonymized information about the scene, objects, operator/collection role, version, sampling ratio, and quality distribution; the dataset-level data card explains the overall purpose, coverage, modalities, data format, license, known limitations, recommended uses, non-recommended uses, version changes, and how to obtain the data.

Data card field groups	Recommended Fields	Engineering Problems Solved
Identity and Version	Dataset ID, trajectory ID, hash, creation time, version, parent version	Prevents file confusion; supports incremental updates and rollback
Capture Environment	Scene type, object category, lighting, workspace, risk level	Supports scene stratification, domain-shift analysis, and task retrieval
Devices and Calibration	Device model, sensor configuration, firmware, intrinsic/extrinsic parameters, coordinate frames, calibration time	Supports spatial reproducibility and multi-device comparison
Time and Synchronization	Clock source, frequency, timestamps, offsets, dropped frames, valid intervals	Supporting cross-modal alignment and event analysis
Tasks and Actions	Natural-language goals, steps, objects, tools, action representations, success criteria	Supports VLA supervision, process retrieval, and task evaluation
Quality and Anomalies	Automated checks, manual spot checks, quality grades, failure types, anomalous segments	Supports filtering by use case, training filtering, and error diagnosis
Ownership and Governance	Capture authorization, licensing, de-identification, access level, usage restrictions, retention policy	Supporting compliant procurement, cross-organization collaboration, and accountability definition

This kind of data card is not an administrative appendix; it is an interface for model engineering. It helps the algorithm team determine, before training, which trajectories should enter foundational pretraining, which should enter spatial-consistency learning, which should enter contact-skill fine-tuning, and which can only be used for internal evaluation. It also helps the business team clarify whether the data meets the deliverable scope, whether it is reusable, and whether additional capture is needed. For buyers, the data card provides a starting point for technical due diligence: rather than asking "how much data do you have," ask "what are the spatiotemporal reference frame, task coverage, quality thresholds, licensing boundaries, and known limitations of this batch of data?"

Reproducibility research further indicates that sensors, actuators, environmental conditions, materials, software, and measurement accuracy in embodied systems can all alter experimental conclusions, so transparent reporting, variable control, and end-to-end experimental records are required.[12] This means the data card should also include a description of "what cannot be done." For example, a batch of human egocentric video may be suitable for learning task segmentation and semantic priors, but not necessarily for directly generating robot joint actions; a batch of multi-view data may contain rich visual information, yet be unsuitable for 3D supervision due to calibration drift; a batch of contact data may cover only rigid objects and cannot be used to infer pressure control for soft objects. Stating boundaries clearly is the most valuable honesty for customers, researchers, and the industry.

IX. TOBI's Recommended R&D Roadmap: Using Ego Data as the Entry Point to Build a Closed-Loop Data Engine for Physical AI

Based on observations from the conference, conference materials, and published research, TOBI proposes a recommended R&D roadmap for embodied data:Rather than first settling on a single "ultimate model" and passively collecting data, establish a closed loop of data—model—evaluation—recapture around verifiable tasks. This direction applies both to internal R&D and to data projects built jointly with customers, service providers, whole-machine OEMs, and research institutions.

The first phase isTask Definition and Scene Modeling. Before capture, "helping the robot learn to do things" should be decomposed into verifiable tasks: what is the goal, what are the objects and tools, what are the environmental constraints, what observable state change indicates completion, what failures are acceptable, and what action precision and physical perception are required. For the same "placement" task, gentle placement, rapid placement, narrow-aperture insertion, soft-object placement, and liquid-container placement require completely different data. Task definition should simultaneously produce capture scripts, quality thresholds, annotation ontologies, and final evaluation protocols, rather than deferring this work until after capture is complete.

The second phase isEgocentric-First Multimodal Capture. First-person devices record natural manipulation and local interaction; external and wrist-mounted viewpoints fill in global spatial context and pre-/post-contact detail; IMU/SLAM/calibration systems establish a traceable spatial reference. For complex tasks, data capture should be organized into scalable modality packages: a base package for large-scale scene and action understanding, a spatial package for geometry and multi-view world models, and a contact package for force control and fine manipulation. This controls cost while giving each data type a clear purpose.

The third stage isOnline Quality Gating and Edge Preprocessing. Many serious quality problems should be detected on-site during capture rather than surfacing weeks later during training. Devices and platforms can flag timestamp anomalies, dropped frames, unusable images, calibration failure, device displacement, visual occlusion, out-of-range motion, and deviations from the task script. Online gating is not about mechanically pursuing "zero anomalies" by capture operators; it is about turning anomalies into visible, actionable, and labelable events. High-value trajectories can trigger immediate recapture or manual review; difficult cases with research value can enter a robustness data pool with explicit labels.

The fourth stage isData Production and Versioned Delivery. Raw data, aligned data, annotated data, derived features, training subsets, and evaluation subsets should not be mixed in the same directory. Versioning, hashing, and lineage relationships should maintain traceability, and data processing should be defined as re-runnable pipelines. Truly efficient data engineering is not merely about one-time project delivery; it is about enabling newly added scenes, revised calibrations, improved labels, and model feedback to continuously enter the asset repository without breaking historical versions.

The fifth stage isModel-Driven Active Recapture. Model training and evaluation should in turn tell the data engine where data is missing: do certain object categories, lighting conditions, poses, contact conditions, language expressions, viewpoint combinations, or failure recoveries cause systematic weaknesses? If a world model fails on cross-view consistency, which well-calibrated multi-view trajectories should be prioritized for supplementation? If a VLA fails on long-horizon tasks, which task segments, transition states, and recovery actions should be prioritized? The essence of data scaling is not unlimited accumulation, but using evaluation evidence to guide incremental collection, so that each round of new data narrows the known capability boundary.

the closing loop	Key Deliverables Recommended by TOBI	Core questions to be answered
Task definition	SOW, task tree, risk list, acceptance thresholds	What must the robot ultimately learn, and how is success determined?
Collection Design	Modality packages, device configuration, coordinate frames, time base, scripts	Which observables should be recorded, and what are the precision boundaries?
On-site Execution	Raw streams, online alerts, anomaly flags, re-collection records	Is the current trajectory usable, and were anomalies handled correctly?
Data Production	Alignment, annotation, quality control, data cards, versions and licensing	Can the data be accurately retrieved, trained on and audited?
Model Evaluation	Coverage matrix, failure clusters, reference set, regression report	Where does the system succeed, where does it fail, and why?
Active Re-collection	Gap lists, hard-case tasks, incremental versions	How can the next batch of data maximally reduce uncertainty?

X. From Correlation to Causal Usability: What Constitutes "Action-Verifiable Data"

The conference materials raised the point that purely image-to-action learning is constrained by data insufficiency and the limits of correlation. This view should be understood with care: VLA is not ineffective, and world models do not automatically confer causal understanding. A more accurate statement is,The causal usability of an embodied system must be jointly established by models, data and experiments. Models can propose predictions about the future; data can provide aligned evidence of actions and their outcomes; experiments and evaluation are responsible for testing whether predictions still hold when conditions are changed.

Verifiable action data should provide at least three layers of comparison. The first layer is temporal comparison: whether there are sufficient continuous observations before, during, and after the action to determine when the change occurred. The second layer is spatial comparison: whether there are verifiable relative positions and geometric relationships among the object, hand, tool, robot, and environment. The third layer is task comparison: whether the same goal has comparable samples under different initial conditions, object properties, ambient lighting, viewpoints, or manipulation methods. Only with these comparisons are researchers more likely to distinguish "the model memorized a particular frame" from "the model learned a transferable task structure."

For industrial data projects, causal usability does not require every trajectory to be designed as a rigorous scientific experiment — that would be unaffordable. What it requires is deliberate coverage at the level of data composition. For example, for grasping tasks, controlled variants can be designed across object material, size, placement, lighting, occlusion and approach method; for tool-use tasks, normal procedures, common misuses, contact failures and recoveries can be recorded; for assembly tasks, clear difficulty gradients can be formed across assembly tolerances, insertion angles and force-control boundaries. This way, the model does not have to guess what matters most from an unstructured mass of material — the data itself already provides organizational structure for generalization and diagnosis.

The combination of world models and VLA can be more meaningful on this foundation. VLA provides the capability of "how to act given the current observation and goal"; the world model provides the prediction of "what may be seen and encountered if one acts this way"; the evaluator checks the deviation between prediction and actual outcome; and the data engine then collects new evidence targeting that deviation. NVIDIA's industry observations on World-Action Models also suggest that the future may not see a single winner of pure VLA or pure world model, but rather joint optimization of visual-language understanding, future prediction and action generation within hybrid systems.[13] This remains a technical judgment in rapid evolution rather than a fully resolved industry fact, but it is sufficient to show that data collection should simultaneously serve observation, prediction and action, rather than catering only to whichever model interface happens to be popular at the moment.

11. Standardize Before Scaling: A Shared Language for Coverage, Difficulty, Failure, and Evaluation

The most common risk in scaling data is the rapid accumulation of heterogeneous noise in the absence of a shared language. Different collectors name the same task differently, different teams use different coordinate frames, different devices record actions at different frequencies, and different projects annotate "success" by different standards — ultimately resulting in data that can be aggregated at the file level but is difficult to compare at the model level. DROID's distributed collection experience shows that a unified hardware/control stack, explicit camera calibration, task metadata and collection protocols are the key engineering conditions for collecting real-world manipulation data across multiple sites.[8]

TOBI recommends dividing standardization into four levels.Task StandardizationDefine objectives, objects, steps, success criteria and failure types;Action StandardizationDefine the representation and coordinate frames for end-effector, joint, gripper, hand or tool actions;Observation StandardizationDefine the time, spatial, format, and quality rules for each sensor;Governance standardizationDefine the data card, licensing, de-identification, access, versioning, and boundaries of responsibility. Standardization should not compress the uniqueness of every scenario, but rather give that uniqueness a place that can be recorded and compared.

Coverage should also be upgraded from simple category counts to a joint matrix of task — environment — object — viewpoint — action — physical conditions. If a batch of data has 100 "task names" but all are performed on the same table, with the same class of objects, at the same angle and under the same lighting, the model's extrapolation boundary remains limited. Conversely, data with fewer task names but systematic coverage across multiple scenes, multiple objects, different camera relationships and different difficulty gradients may be better suited for studying robustness. Data teams should regularly produce coverage heatmaps and gap lists, not just total-volume dashboards.

Failures also need to be standardized. At a minimum, the following should be distinguished: perception failures, localization/calibration failures, path failures, grasping failures, contact/force-control failures, object-state failures, language/task-understanding failures, safety interruptions, equipment malfunctions, and unknown causes. Failure labels are not meant to place unnecessary pressure on data collectors, but to convert "not successful" into structured information that models, data, and engineering can all learn from. For high-quality datasets, failure trajectories should have a clear purpose: as robustness training samples, as policy refusal/recovery learning samples, as risk assessment samples, or as quality anomalies that are excluded while retaining the reason for handling.

XII. Data Governance and Responsible Release: Physical-world data needs boundaries even more

Embodied AI data comes directly from people, places, tools, production processes and real physical environments; its governance cannot follow the mindset that "internet material is usable by default." A NeurIPS 2025 position paper cautions that embodied AI simultaneously involves physical harm, information and privacy, and economic and social risks, and that existing industrial robotics or autonomous-driving governance frameworks may not cover all of these issues.[14] For data collection, this means that authorization, de-identification, usage boundaries, access control, retention policies, incident response and accountability division must be designed in from the very start of a project.

First, collection scenarios should be classified into public, restricted and sensitive tiers. Public scenarios should still specify the scope of filming and intended use; restricted scenarios require agreed handling of personnel, equipment, process information and third-party identifiers; sensitive scenarios should, where necessary, employ minimal collection, on-device de-identification, tiered access, encrypted storage and strict audit logs. For human operation data, participant informed consent, withdrawal mechanisms, handling of identity information, and whether the data may be used for model training, demonstration, external sharing or derivative products must be clearly specified.

Second, licenses should be written as rules understandable to both machines and business, rather than vague text attached to the contract. A license should at minimum specify data ownership / usage rights, training purposes, whether redistribution is allowed, whether synthetic derivatives are allowed, geographic scope and duration, whether commercialization is allowed, whether deletion or updates are required, and how to handle violations of usage boundaries. For customer-customized data, the data card should correspond to the contract, SOW, and delivery checklist, avoiding gaps between "technically replicable" and "commercially usable."

Finally, governance does not obstruct data scaling — it is the precondition for scaling. Traceable provenance, clear licensing, reproducible processing pipelines and auditable versions reduce the cost of trust in cross-institutional collaboration. As the robotics industry enters real-world settings, data quality and data rights will increasingly become foundational capabilities, much like hardware reliability: only by clearly knowing where data comes from, what processing it has undergone, what it may be used for and what it may not be used for, can data be turned into a long-term accumulable asset rather than short-term consumable material.

Section 13: Action Framework for 2026–2028 — From Single-Point Capture Projects to Data Infrastructure

TOBI believes that over the next two to three years, embodied data infrastructure will undergo three parallel evolutions. First, fromRecord behaviorDirectionRecord status transitions. The collection system will place greater emphasis on action conditions, object states, spatial geometry, and outcome verification, making the data more suitable for world models and closed-loop control. Second, fromsingle-modality visionDirectionhierarchical multi-modality. Ego, external, wrist, depth, pose, proprioceptive, and optional tactile/force sensors are combined based on task value, rather than stacked indiscriminately. Third, fromproject deliveryDirectionOperationalizable data assets. Data cards, versions, licensing, quality reports, samples and feedback loops will become standard components of customer procurement and long-term partnerships.

To turn trends into actionable projects, this article offers four recommendations. First, build a high-value task library before pursuing an industry-wide task list; select tasks that represent the customer's core business, have observable success criteria, and offer room for continuous incremental gains. Second, configure appropriate modalities and quality thresholds for each task; do not rely on a single equipment configuration for all tasks, and do not let low-value data drag down the quality of high-value tasks. Third, run small-scale end-to-end validation before the first batch of data delivery, including data reading, alignment, training, evaluation, failure traceback, and permission checks; only when the entire chain runs through does scaling become meaningful. Fourth, establish a joint data-model review mechanism so that capture, algorithm, product, scenario, and compliance personnel jointly decide the next round of data gaps, rather than each optimizing local metrics.

Time Window	Recommended Focus	Key Outputs	Success Criteria
0–3 Months	Select high-value tasks, freeze the base specification	Task tree, SOW, modality plan, quality metrics, data card template	Small-batch samples jointly accepted by both algorithm and business teams
3–6 Months	Establishing a Stable Data Acquisition and Production Pipeline	Synchronization and calibration workflows, quality-control rules, versioning system, and sample sets	Key quality metrics remain stable; anomalies are traceable
6–12 months	Establish an evaluation and active re-collection loop	Coverage matrix, failure clusters, regression set, gap list	Newly acquired data can explain and improve known weak points
More than 12 months	Expand scenarios and partnership networks to accumulate procurable assets	Data product catalog, licensing policy, service provider specifications, cross-project asset layer	Data can be safely reused, procured to specification, and continuously iterated upon

Conclusion: Transforming the complexity of the physical world into a verifiable foundation for learning

In 2026 embodied AI discussions, world models, VLA, spatial intelligence, video generation, teleoperation, tactile sensing, and humanoid robots are all advancing rapidly. What truly determines whether these technologies can form stable capabilities in the real world is not the popularity of any single concept, but whether there is a data foundation sufficient to support observation, prediction, action, and verification. A high-quality embodied dataset is not about "shooting more," but about "better defining, recording, aligning, proving, and iterating."

What TOBI hopes to promote is not a closed model narrative, but an open data engineering methodology: using Ego data to record natural manipulation, using multi-view and spatial calibration to establish geometric evidence, using action and optional contact signals to connect physical processes, using task and failure labels to present decision boundaries, using data cards, licenses, and versions to complete auditable delivery, and using model evaluation feedback to guide the next round of data collection. Only when data can be verified, reproduced, and used correctly does embodied intelligence have the opportunity to move from demonstrations to reliable productivity.

TOBI's forward-looking commitments: For embodied-AI data projects, every high-value trajectory should answer five questions as fully as possible: What task does it record? Under what spatiotemporal conditions did it occur? What actions and outcomes is it linked to? What is its quality evidence? Who may use it, and within what authorization scope? These five questions are the starting point for embodied data assets that are auditable, reproducible, and procurable.

---

On-Site Material Notes

The images in this article come from on-site materials of an exchange on embodied intelligence and world models provided by the user on August 20, 2026. The images are used to present the industry exchange environment and demonstration topics and do not constitute cooperation, affiliation, commercial endorsement, or performance claims regarding any institution, personnel, model, competition, product, or viewpoint depicted. Factual descriptions of research progress in the text are based on public sources; content about TOBI is primarily advisory R&D methodology oriented toward embodied data collection and data engineering.

References and Sources

1. World Robot Conference: "Human-Robot Symbiosis, Integration of Production and Demand" — 2026 World Robot Conference Press Conference Held in Beijing
2. Hou et al. (2026), World Model for Robot Learning: A Comprehensive Survey
3. Wang et al. (2026), Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines
4. Beijing Municipal Science and Technology Commission: 2026 World Robot Conference Opens in Beijing in August
5. PAIWorld Team (2026), A 3D-Consistent World Foundation Model for Robotic Manipulation
6. Guo et al. (2026), Ctrl-World: A Controllable Generative World Model for Robot Manipulation
7. Hoque et al. (2026), EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
8. Khazatsky et al. (2025 version), DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
9. Engelbracht et al. (2026), Hoi! - A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation
10. Kwon et al. (2025), A Humanoid Visual-Tactile-Action Dataset for Contact-Rich Manipulation
11. Ortega et al. (2026), Replicable Simulation-Based Robot Validation through Provenance
12. Howard (2026), Operationalising reproducibility in soft robotics
13. NVIDIA Technical Blog (2026), Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models
14. Perlo et al. (2025), Emerging Risks from Embodied AI Require Urgent Policy Action