Mid-morning on our capture floor, the operator at the humanoid teleop station finishes her fourth tote-packing attempt of the hour; two of the four will survive QA, and the delivered-hours counter for that program advances by about twenty minutes. One bay over, six workers wearing stereo capture rigs will bank more usable manipulation hours before lunch than that robot produces in a week. That gap, between what a robot can experience per day and what a policy needs to learn, is the defining constraint in humanoid robotics right now. Figure has said Helix was trained on roughly 500 hours of teleoperation, NVIDIA built GR00T N1 around a data pyramid that leans on human video precisely because robot-collected hours are scarce and expensive (arxiv.org/abs/2503.14734), and Optimus data collection has reportedly shifted toward human-worn capture. The bottleneck is not compute or models. It is data.
Our thesis, and the argument this guide carries through every section: a humanoid data program that works is shaped like a pyramid, a wide base of cheap human hours, a narrow apex of on-robot teleoperation, with retargeting QA deciding whether the base is an asset or a liability. This guide explains what humanoid robot training data actually is, the five ways teams capture it, what each method costs per usable hour, and how to blend them into a budget that survives contact with reality. The numbers come from DexSet’s own capture operations: our egocentric rigs, our teleop stations, and the QA pipeline we run on every delivered hour.
TL;DR: Key Takeaways – Humanoid robot training data is multimodal recorded experience (video, joint states, actions, contact events) used to train control policies and vision-language-action (VLA) models for human-shaped robots. – It splits into two regimes: whole-body control data (locomotion, balance) and manipulation data (grasping, tool use). Most teams under-budget the second. – Human motion is the largest available data source for humanoids. Egocentric human video retargets to humanoid embodiments at roughly one tenth the cost of robot-collected teleoperation. – Typical 2026 costs we see: $15 to $40 per hour for human egocentric capture, $40 to $90 for mocap suit capture, $150 to $400 all-in for on-robot teleoperation, under $5 per simulated-hour equivalent in Isaac Lab or MuJoCo. – Retargeting error is the tax on cheap human data. Budget QA for it or the savings evaporate downstream.
What Is Humanoid Robot Training Data?
Humanoid robot training data is the recorded sensory input and motor output used to train the control policies of human-shaped robots, spanning camera streams, joint positions and torques, IMU readings, end-effector poses, contact and tactile signals, and the action sequences that produced them. A single usable episode pairs what the robot (or a human stand-in) perceived with what it did, timestamp-aligned tightly enough that a model can learn the mapping from observation to action.
The definition matters because “training data” means something different for a humanoid than for a warehouse arm. A stationary manipulator needs wrist and scene cameras plus 6 or 7 joint states. A Unitree G1 has 23 or more actuated degrees of freedom, a Figure or Apptronik Apollo platform carries dexterous hands on top of that, and all of them must balance while acting. That pushes humanoid data toward higher dimensionality, stricter synchronization, and a modality mix that includes whole-body proprioception, not just vision and gripper state.
In practice the field trains two model families on this data. Reinforcement learning policies for locomotion and balance are trained mostly in simulation (NVIDIA Isaac Lab, MuJoCo) with domain randomization. Vision-language-action models such as Figure’s Helix, OpenVLA, Physical Intelligence’s pi-0, and NVIDIA’s GR00T N1 are trained on real demonstrations: teleoperation episodes, retargeted human video, or both. Understanding which regime your data feeds is the first budgeting decision, and we cover the split next.
Whole-Body Control Data vs Manipulation Data
Whole-body control data captures how a humanoid balances and moves its entire kinematic chain, while manipulation data captures how its hands interact with objects; the two have different sources, different costs, and different failure modes. Treating them as one line item is the most common budgeting mistake we see in RFPs.
Locomotion and balance are largely a solved-by-simulation problem. Massively parallel RL in Isaac Lab with domain randomization over friction, mass, and terrain produces walking policies that transfer to real hardware, which is why Unitree G1s walk off the pallet and why Agility’s Digit has, per the company’s public announcements, been working logistics pilots. Simulated hours cost cents. The catch is that simulators remain weak exactly where manipulation lives: contact dynamics, deformable objects, friction at the fingertip, cloth, liquids.
Manipulation therefore still runs on real demonstrations. The ALOHA line of work (Zhao et al., arxiv.org/abs/2304.13705) showed that a few hundred well-collected episodes can train fine manipulation, and Mobile ALOHA (arxiv.org/abs/2401.02117) extended that to whole-body tasks. But “a few hundred episodes per task” multiplied across the hundreds of tasks a general humanoid needs is where budgets die. This is the gap that human-sourced data exists to close, so let us define the capture methods precisely.
Core Modalities: What a Humanoid Episode Contains
A modality is a distinct sensor or signal stream inside an episode, and a humanoid episode typically carries five to eight of them, all sharing one clock. Getting the modality stack right at capture time is cheaper than regretting it at training time, so here is the stack we specify by default:
- Mono vs stereo vision. Mono egocentric video is the minimum viable stream. Stereo adds the metric depth cues manipulation policies use for reach and pre-grasp alignment; on our rigs the marginal hardware cost of the second camera is small next to the labor, so we default to stereo at 30 fps, 1080p or better per eye.
- Depth. Active depth sensors help in low-texture scenes but fight sunlight and reflective surfaces. We treat depth as a supplement to stereo, not a substitute.
- Proprioception. Joint positions and velocities for every actuated degree of freedom, 23 or more channels on G1-class platforms, sampled at or above control rate. For human capture, the analog is mocap joint trajectories plus reconstructed hand pose.
- IMU. Base and head orientation and angular velocity. Cheap to log, painful to retrofit, and required for anything involving balance.
- Contact and tactile. The scarcest modality. Grasp-event labels at minimum, fingertip force where hardware allows. Vision-only corpora cap performance on insertion and in-hand tasks, which is why tactile is climbing the priority list across the field.
- Language. Task annotations per episode, since VLA training assumes instruction-conditioned data.
Whatever the stack, synchronization is the contract that holds it together. Our tolerance is one frame at 30 fps across all streams, verified per episode rather than assumed from hardware specs.
The Five Capture Methods for Humanoid Training Data
Humanoid training data comes from five capture methods: human egocentric video, mocap suit capture, on-robot teleoperation, exoskeleton or leader-arm capture, and simulation. Each trades cost against fidelity to the target embodiment.
1. Human Egocentric Video
Human egocentric video is first-person footage captured from head-mounted cameras (Project Aria glasses, Quest 3 passthrough, or custom stereo rigs) while a person performs tasks with their own body. It is the cheapest data per hour because the “robot” is a person who needs no maintenance, no safety cage, and no operator. Datasets such as Ego4D (arxiv.org/abs/2110.07058) and EgoExo4D (arxiv.org/abs/2311.18259) established the format at scale, and the human-data hypothesis behind GR00T N1 (arxiv.org/abs/2503.14734) treats this human layer as the wide base of the data pyramid. The limitation: there are no robot action labels, so the motion must be retargeted to the humanoid’s kinematics, and hands must be reconstructed from video. On our rigs we capture synchronized stereo egocentric video, hand pose, and gaze at $15 to $40 per hour depending on task complexity and annotation depth.
2. Mocap Suit Capture
Mocap suit capture uses IMU-based body suits (Xsens-style) to record full-body human kinematics at 60 to 240 Hz, giving clean joint trajectories without cameras or occlusion problems. It is the strongest source for whole-body motion style: walking gaits, turning, crouching, carrying. Works like H2O (arxiv.org/abs/2403.04436) and HumanPlus (arxiv.org/abs/2406.10454) retarget exactly this kind of human motion to humanoids in real time. Weaknesses are IMU drift over long takes and poor fingertip detail, so we pair suits with hand-tracking gloves or egocentric video when the task involves manipulation.
3. On-Robot Teleoperation
On-robot teleoperation records demonstrations directly on the target humanoid while a human pilots it through VR controllers, motion capture, or retargeted body tracking. It produces the highest-fidelity data because observations and actions come from the true embodiment; this is how Figure, by its own account, collected the roughly 500 teleoperated hours behind Helix. It is also the most expensive method: you need the robot (a Unitree G1 lists near $16,000, per unitree.com; full-size platforms run into six figures), a trained operator, a safety setup, and you accept 2 to 3 usable hours per robot-day after resets and retries. All-in, we benchmark humanoid teleop at $150 to $400 per delivered hour, roughly 10x the cost of human egocentric capture.
4. Exoskeleton and Leader-Arm Capture
Exoskeleton capture puts the human inside a passive kinematic structure that matches the robot’s arms, so joint-space actions are recorded directly without inverse-kinematics guesswork. ALOHA’s leader-follower arms are the low-cost version of this idea; humanoid programs in China (Fourier, AgiBot-style rigs) have scaled full upper-body exoskeleton stations. Fidelity sits between mocap and true teleop, at $80 to $200 per hour in our benchmarks, with the main constraint being workspace and task variety.
5. Simulation
Simulation generates synthetic episodes in physics engines (Isaac Lab, MuJoCo) where domain randomization varies dynamics, textures, and lighting to force policies to generalize. Marginal cost per hour-equivalent is under $5 and often under $1 at scale. It dominates locomotion training and is increasingly used to multiply real demonstrations (as in GR00T’s synthetic middle layer). Its ceiling remains contact-rich manipulation, where the sim-to-real gap still costs real-world success rates.
Comparison Table: Capture Methods Side by Side
The table below summarizes the trade space using DexSet benchmark ranges from our own capture operations (Level 1 evidence: first-hand).
| Method | Hardware cost | Cost per usable hour | Usable hours per station-day | Action labels | Best for | Key failure mode |
|---|---|---|---|---|---|---|
| Human egocentric video | $500 to $6,000 per rig | $15 to $40 | 5 to 6 | None (retargeted) | Pretraining, visual priors, task diversity | Retargeting error, no force signal |
| Mocap suit | $10,000 to $45,000 per suit | $40 to $90 | 4 to 5 | Human joint space | Whole-body motion, locomotion style | IMU drift, weak hand detail |
| On-robot teleop | $16,000 to $200,000+ per robot | $150 to $400 | 2 to 3 | Native robot actions | Post-training, task finishing | Throughput, operator fatigue |
| Exoskeleton capture | $20,000 to $60,000 per station | $80 to $200 | 3 to 4 | Matched joint space | Dexterous upper-body tasks | Workspace limits |
| Simulation | Compute only | $0.50 to $5 per hr-equiv | Unlimited (parallel) | Perfect | Locomotion RL, augmentation | Sim-to-real gap on contact |
Human Motion Retargeting: The Bridge and the Tax
Human motion retargeting is the process of mapping recorded human kinematics onto a humanoid’s different skeleton, joint limits, and mass distribution so the robot can learn from human demonstrations. It is what makes the cheap 80 percent of the pyramid usable, and it is where quality is won or lost.
The core problem is embodiment mismatch. A human shoulder has more usable range than most humanoid shoulders. Human proportions differ from a G1’s. Naive retargeting produces joint-limit violations, feet that slide through the floor, and end-effector paths that miss the object by centimeters. Research systems like H2O, OmniH2O (arxiv.org/abs/2406.08858), and HumanPlus each spend most of their machinery on exactly this mapping.
Our position, stated as first-hand operational experience: retargeting quality is a QA problem, not just an algorithm problem. Every retargeted batch that leaves DexSet passes a fixed gate set, and we publish the checks to clients:
- Joint-limit violation rate below 0.5 percent of frames per episode
- Post-retargeting end-effector position error under 2 cm against reconstructed human hand pose
- Foot-skate detection (stance-foot translation above 1 cm per frame flags the episode)
- Contact consistency: detected grasp events must survive retargeting with object-relative pose intact
- Sensor synchronization within one frame at 30 fps across all streams
Batches failing any gate get re-solved or rejected before delivery. This is the difference between a cheap corpus and a cheap corpus your model can actually use.
Cost and Economics: Budgeting a Humanoid Data Program
A humanoid data budget is best modeled as a pyramid: a wide, cheap base of human data, a middle layer of simulation and augmentation, and a narrow, expensive apex of on-robot teleoperation. This mirrors the structure NVIDIA describes for GR00T N1 and matches what we see working across client programs.
Consider a 10,000-hour pretraining corpus, using midpoint DexSet benchmark rates:
| Strategy | Composition | Approximate cost | Notes |
|---|---|---|---|
| Teleop-only | 10,000 hrs teleop at $250/hr | $2,500,000 | Plus robot fleet capex and 12+ months of station-days |
| Human-first (recommended) | 8,000 hrs egocentric at $25 + 1,500 hrs mocap/exo at $100 + 500 hrs teleop at $250 | $475,000 | Teleop reserved for post-training on target tasks |
| Sim-heavy | 9,500 hrs sim at $2 + 500 hrs teleop at $250 | $144,000 | Strong for locomotion; expect manipulation gaps |
The human-first strategy costs roughly one fifth of teleop-only and, in our experience, reaches comparable downstream task success once retargeting QA is enforced, because the teleop budget is spent exactly where embodiment fidelity matters: final-mile policy tuning. The sim-heavy strategy is the right shape for a locomotion-centric program (Digit-style logistics work) but under-serves dexterous manipulation.
One more number worth planning around: robot-collected hours do not just cost 10x per hour, they scale 10x worse. Doubling egocentric throughput means hiring and equipping more capture workers in days. Doubling teleop throughput means procuring robots with lead times measured in months.
Case Study Proof: Scaling Data for a Humanoid VLA
A humanoid foundation model team came to us with a VLA architecture, a fleet of four robots, and about 60 hours of teleoperation covering 12 manipulation tasks. Success rates plateaued in the 40s (percent) on held-out object arrangements. Their diagnosis was correct: not enough visual and behavioral diversity, and no budget to teleop their way out.
Over 16 weeks we delivered 4,000 hours of task-matched human egocentric capture (kitchen, warehouse tote, and assembly scenarios mirroring their task list) plus 300 additional teleop hours concentrated on the six worst-performing tasks. Retargeted human data went into pretraining; teleop went into post-training. Their reported outcome: held-out task success moved from the low 40s into the 70s, with the largest gains on tasks where human data introduced object and layout diversity their lab could not stage. Total data spend was under a quarter of what the equivalent teleop-only corpus would have cost. Full write-up: Case Study: How We Scaled Humanoid Robot Training Data for a VLA Model.
Where the Field Is Heading
Three shifts define humanoid data strategy through 2027; we hold this as an informed opinion, not a fact. First, human data becomes the default pretraining substrate: Optimus data collection has reportedly moved toward human-worn capture, and the academic pipeline (EgoExo4D onward) keeps improving hand and pose reconstruction. Second, tactile and force sensing enters the standard modality stack, because vision-only data caps performance on insertion and in-hand tasks. Third, cross-embodiment corpora in the Open X-Embodiment tradition (arxiv.org/abs/2310.08864) extend to humanoids, making standardized formats (LeRobot datasets on Hugging Face) a procurement requirement rather than a nice-to-have.
For deeper treatments, see this week’s supporting posts: why data is the bottleneck in physical AI, a cost comparison of capture approaches, and five hidden challenges in humanoid data.
Download: The Humanoid Data RFP Scorecard
Most data RFPs we receive ask about price and volume and skip the questions that predict corpus quality. We built a one-page scorecard covering the ten dimensions that matter: synchronization guarantees, retargeting QA gates, hand-data fidelity, format compatibility (LeRobot/HDF5), re-collection policy for rejected batches, and unit economics per modality. It is the same rubric we score ourselves against. Download it, use it on us, use it on our competitors.
Next Step
If you are scoping a humanoid data program, two options. Download the Humanoid Data RFP Scorecard and pressure-test every vendor with it, including us. Or book a demo and we will walk through sample egocentric and teleop episodes from our rigs, with the QA reports attached, so you can judge the data before you budget for it.
Frequently Asked Questions
What is humanoid robot training data?
Humanoid robot training data is timestamp-aligned multimodal recordings (video, joint states, IMU, contact signals, and actions) used to train locomotion and manipulation policies for human-shaped robots. It comes from human egocentric video, mocap suits, on-robot teleoperation, exoskeleton capture, and simulation.
How much does humanoid robot training data cost in 2026?
In our benchmarks: $15 to $40 per hour for human egocentric capture, $40 to $90 for mocap suit capture, $80 to $200 for exoskeleton capture, $150 to $400 all-in for on-robot teleoperation on a humanoid platform (fixed arm rigs like ALOHA-class stations run far cheaper, $28 to $60 per hour; see our teleoperation data collection guide), and under $5 per simulated hour-equivalent.
Why not train humanoids entirely in simulation?
Simulation excels at locomotion and balance thanks to domain randomization in Isaac Lab and MuJoCo, but it still models contact-rich manipulation poorly. Deformable objects, fingertip friction, and cloth remain unreliable in sim, so manipulation policies still need real demonstrations.
What is human motion retargeting and why does it matter?
Retargeting maps recorded human motion onto a humanoid’s different joint limits, link lengths, and mass distribution. It is what makes cheap human data usable for robot training, and poor retargeting (joint violations, foot skate, end-effector drift) silently corrupts a corpus unless it is caught by QA gates.
How many hours of data does a humanoid VLA model need?
There is no fixed number, but public reference points help: Figure’s Helix used roughly 500 hours of teleoperation for its manipulation capabilities, while pretraining corpora built on human video run into the thousands of hours. Post-training on the target robot typically needs tens to hundreds of hours per task family.
Is human egocentric video really usable without robot action labels?
Yes, with caveats. Retargeting and hand-pose reconstruction convert human video into supervision for pretraining, as demonstrated by HumanPlus, H2O, and the GR00T N1 data pyramid. Final-mile performance still benefits from a smaller layer of native teleoperation on the target embodiment.
