Picture yourself as the head of data at a humanoid startup: your researchers have just validated the experiment that could anchor the next funding round, co-training human egocentric video with your teleop corpus improves policy generalization, and then you run the arithmetic on your own collection capacity and find the experiment wants ten times more human data than your team can produce this year. That is exactly where one humanoid foundation model team (unnamed here by agreement) was standing when they contacted us. Their result was in line with what EgoMimic-style research had reported (arXiv:2410.24221); their teleop dataset was around 400 hours and growing slowly.
Why was a well-funded team stuck? Because collection was competing with research for the same people. Every hour their engineers spent managing capture logistics was an hour not spent on training runs, and their pilot attempts at crowdsourced collection had returned footage with unusable sync and a rejection rate far beyond anything a program can absorb. Familiar story; we hear a version of it most months.
This post is the anonymized production log of what happened next: how we scoped, staffed, and delivered 4,000 QA-passed hours of egocentric data in 14 weeks, what it cost per hour, where we lost yield, and the three decisions that mattered most. Numbers are real from our program tracking, rounded to protect the client. The thesis the whole log supports: scaled collection is a manufacturing problem, and the returns came from three protocol decisions made before hour one, not from capture volume.
Key Takeaways – 4,000 QA-passed egocentric hours delivered in 14 weeks across three task families (kitchen, warehouse shelving, assembly bench) for a humanoid VLA co-training pipeline. – Blended cost landed in the $22 to $34 per hour band: mono-heavy capture for breadth, stereo for depth-critical tasks, dense hand-pose annotation on only the 30 percent of hours flagged retarget-critical. – First-pass QA yield started at 71 percent in week one and reached 86 percent by week six after protocol fixes; the delta was worth roughly $18K. – Camera height matched to the robot’s head frame and constrained “robot-plausible” motion instructions were the two protocol choices the client’s researchers valued most. – The client reported improved success on unseen-object manipulation evaluations versus their robot-only baseline after co-training on the mixed corpus.
The Brief: What “Scaled” Actually Meant
Scaling egocentric data collection means hitting a volume target at a defined quality bar and annotation depth on a fixed calendar, and all four variables were specified before we captured an hour. The brief we agreed with the client’s head of data:
- Volume: 4,000 QA-passed hours (not raw hours; the distinction is the whole game).
- Task families: kitchen manipulation, warehouse shelf pick-and-place, small-parts assembly bench, weighted 45/35/20.
- Viewpoint constraint: camera height within 10 cm of the robot’s head frame; field of view overlapping the deployment camera’s.
- Annotation: temporal task segmentation on everything; 3D hand pose and object tracks only on episodes their researchers flagged as retarget-critical, which settled at about 30 percent of hours.
- Format: episode-structured delivery compatible with their LeRobot-based ingestion (github.com/huggingface/lerobot).
That last 30 percent decision deserves emphasis. Dense annotation on all 4,000 hours would have added roughly $10 per hour across the corpus, $40K of spend their training plan did not need. Annotation depth should follow the training mechanism; this program is why we repeat that line so often.
The Setup: Rigs, People, Protocol
Our production setup for this program combined two rig families, 26 trained collectors, and a task protocol written jointly with the client’s researchers in week zero. Rig split:
- GoPro-class mono head rigs at 4K/60 for the breadth corpus, where the consumer was encoder pretraining and temporal understanding.
- RealSense D455 helmet rigs for depth-critical assembly and shelving tasks, where metric 3D fed their grasp analysis.
The protocol work is what separated this from “wearing cameras at work.” Every task became a card: start state photo, goal condition, allowed object set, retry rule, and, critically, motion constraints. Collectors were instructed to keep grasps within the robot’s gripper envelope (no five-finger tricks a parallel-jaw gripper cannot imitate), avoid motions outside the robot’s reach profile, and re-stage rather than improvise when a scene drifted off-spec. We borrowed episode discipline from how public robot datasets are structured; if it cannot be expressed as a clean episode, the way entries in Open X-Embodiment are (arXiv:2310.08864), it does not ship.
The Numbers: Timeline, Yield, and Cost
Program metrics are the honest core of any case study, so here is the phase table straight from our tracking, rounded:
| Phase | Weeks | Hours Delivered (QA-Passed) | First-Pass Yield | Notes |
|---|---|---|---|---|
| Pilot + protocol iteration | 1 to 2 | 120 | 71% | Blur and sync failures dominated rejections |
| Ramp | 3 to 6 | 900 | 78% rising to 86% | Fixed exposure presets; re-trained 4 collectors |
| Steady state | 7 to 12 | 2,400 | 85 to 88% | ~420 hours/week across 26 collectors |
| Completion + re-delivery | 13 to 14 | 580 | 87% | Backfilled task-family gaps flagged by client |
Cost landed where the pilot predicted: mono breadth hours in the low $20s per QA-passed hour, stereo depth-critical hours with dense annotation in the low-to-mid $30s, blending to $22 to $34 across the corpus. For context on how those bands compare across rigs and vendors, the economics section of our pillar guide has the full tables: The Complete Guide to Egocentric Data Collection for Robotics.
Where did the 12 to 29 percent of rejected hours go? Motion blur in fast kitchen tasks (fixed with shutter-priority presets), IMU packet loss on two aging rigs (retired), off-protocol grasps (re-training), and a stubborn tail of episodes with ambiguous outcomes (protocol clarification). Yield improvement from 71 to 86 percent was worth roughly $18K over the program, which is why we staff a protocol engineer for the first six weeks rather than treating capture as a solved problem.
The Three Decisions That Mattered
Looking back through the retro notes, three decisions carried most of the outcome:
1. Matching viewpoint geometry before hour one. Camera height within 10 cm of the robot’s head frame and overlapping FOV meant the client’s encoder saw deployment-like images from the first batch. This costs nothing at capture time and is nearly impossible to fix afterward.
2. Robot-plausible motion constraints. Instructing collectors to stay inside the gripper envelope narrowed the embodiment gap at the source instead of hoping retargeting would absorb it. Their researchers called this the highest-value protocol clause.
3. Two-tier annotation. Dense hand pose on 30 percent of hours, selected by the researchers who consumed it, kept the budget aligned with the training plan and cut roughly $40K of unneeded spend.
An honorable mention: weekly delivery instead of end-of-program delivery. The client’s researchers trained on batch one while we captured batch four, surfaced a task-family gap in week five, and we corrected course while the collectors were still staffed. On a single-delivery program, that gap would have been discovered after demobilization, when fixing it costs triple.
The Outcome, Stated Carefully
The client reported improved success rates on unseen-object manipulation evaluations after co-training on the mixed corpus versus their robot-only baseline; direction consistent with published ego-plus-robot co-training results, and we will not dress it in precise percentages that are theirs to publish, not ours. What we can state as first-hand fact: the corpus passed their ingestion validation on first delivery, the program hit its calendar, and they extended into a second phase covering exocentric paired capture.
Opinion, clearly flagged: I think the 10x-human-to-robot-hours ratio this team converged on will look conservative within two years. Human egocentric capture scales with people; teleop scales with robot fleets, and people are the cheaper fleet.
Running This Play for Your Team
If your co-training experiments are data-starved, the fastest way to de-risk a program like this is a scoped pilot: 50 to 100 hours against your protocol, with QA reports attached. Book a scoping call and bring your task list; we will bring the yield curves from programs like this one.
Frequently Asked Questions
How long does it take to collect 4,000 hours of egocentric robotics data?
In this program, 14 weeks end to end with 26 trained collectors, including a two-week pilot and protocol iteration phase. Steady-state throughput ran about 420 QA-passed hours per week.
What did the egocentric data cost per hour?
The blended rate landed between $22 and $34 per QA-passed hour: mono breadth capture in the low $20s, stereo depth-critical hours with dense hand-pose annotation in the low-to-mid $30s.
How do you keep human demonstration data useful for a robot embodiment?
Match camera height and field of view to the robot’s head frame, constrain grasps to the robot’s gripper envelope, keep motions inside its reach profile, and structure everything as clean episodes with defined start states and outcomes.
What QA yield should a mature program expect?
This program started at 71 percent first-pass yield and stabilized at 85 to 88 percent by week six. Budget for 15 to 25 percent rejection if you are starting fresh.
Did the egocentric data actually improve the client’s model?
The client reported improved success on unseen-object manipulation evaluations after co-training the mixed corpus versus their robot-only baseline, consistent with published results such as EgoMimic. Exact figures are theirs to publish.