Skip to main content

Dexset

dexset
Menu
Request a Demo

Tag: Case Study

Case Study: How We Scaled VLA Training Data for a Humanoid Foundation Model Team

Posted on by Enoch Pakanati

On station four of the collection floor, an operator sets down her leader arms after a clean bowl-to-sink transfer, and the batch dashboard flags the episode anyway. The instruction field reads “put the bowl in the sink,” the same sentence shape as two hundred episodes before it, and the template-share gate has just tripped. That flag, firing in the middle of a live program, is the single most important control in the engagement this post describes.

The client behind that program, a humanoid foundation model team, did not come to us asking for data. They came asking why their model was failing. Their OpenVLA-style fine-tune scored well on standard tabletop evaluations, then collapsed on the bimanual kitchen tasks their product actually needed. Worse, the policy only responded reliably to instructions phrased exactly like the training labels. “Put the bowl in the sink” worked. “Move that bowl into the sink” mostly did not.

The failure had a clean explanation in their data, and it is the most common one we see. Their existing corpus was large by episode count, roughly three times what we ended up collecting, but it covered only 9 task families and used 14 instruction templates. The model had learned those 9 tasks and those 14 sentence shapes, thoroughly, and almost nothing else. Volume was never the problem. Distribution was.

This post walks through what we scoped, what the QA gates caught, what it cost in structure (with our standard benchmark rates), and what changed in their evaluations. If you are planning a fine-tuning collection, the transferable part is the method: diversity targets set as hard gates, not aspirations.

Key Takeaways – 1,400 hours of bimanual teleoperation across 42 task families replaced a corpus 3x larger in episode count but 5x narrower in tasks. – A hard instruction diversity gate (no template above 8% of any batch) fixed paraphrase brittleness; language annotation ran at our standard $12-15 per data-hour, segment-level. – Blending new data 60/40 with the legacy corpus roughly doubled held-out kitchen task success in the client’s own evaluation harness. – QA rejected 11% of collected episodes; the top three causes were teleoperator mid-chunk pauses, frame-action sync drift, and failed-but-unlabeled episodes. – Total program ran 19 weeks from scoping to final delivery in LeRobot-compatible format.

The Starting Point: A Large but Narrow Corpus

A narrow corpus is a dataset whose episode count outruns its task, scene, and language diversity, and it trains models that look strong on in-distribution tests and brittle everywhere else. The client’s legacy data was a textbook case: high volume, 9 task families, 14 templates, single kitchen scene, one camera geometry.

The published evidence already pointed to the fix. Open X-Embodiment showed pooled diversity beating single-domain volume (arxiv.org/abs/2310.08864), and Mobile ALOHA reached strong per-task results co-training with roughly 50 demonstrations per task over a broad base (arxiv.org/abs/2401.02117). Our job was to apply that at commercial fine-tuning scale on their embodiment, where cross-embodiment transfer is weakest.

What We Scoped

The collection specification is the deliverable that matters most, so we fixed it before a single episode was recorded:

  • Volume and spread: 1,400 robot-hours of bimanual teleoperation across 42 task families, capped at 60 hours per family to force breadth.
  • Scenes and objects: 6 kitchen layouts, 300+ object instances, mandated lighting variation across sessions.
  • Capture spec: 4 camera streams per station (two exocentric, head, wrist), 30 Hz, hardware-timestamped, frame-to-action skew held under 10 ms.
  • Language spec: natural free-form instructions written at collection time, segment-level labels with timestamps on all long-horizon episodes, and the diversity gate below.
  • Format: LeRobot-compatible episodes so their training stack ingested data on day one (github.com/huggingface/lerobot).

The Instruction Diversity Gate

An instruction diversity gate is a batch-level QA rule that rejects any 1,000-episode batch where a single instruction template exceeds a set share, in this case 8%. Annotators drift toward comfortable phrasings; without a gate, “pick up the X and place it in the Y” quietly becomes half your labels.

We enforced three thresholds per batch: template share under 8%, minimum unique verb count, and minimum spatial-relation phrase coverage (“behind the cutting board”, “second shelf from the top”). Batches that failed went back for paraphrase augmentation before delivery. This is also where the money went: segment-level annotation with paraphrase passes ran at our standard $12-15 per data-hour, versus $8-10 for a plain episode-level pass. The client initially pushed back on that line item. It ended up being the line item their evaluation results vindicated most directly.

How the Program Ran

A collection program’s timeline is set by station-hours and reset speed, not by calendar ambition, so we planned backwards from throughput. The program ran 19 weeks across six bimanual stations: two weeks of scoping and task-family design with the client’s ML team, one week of operator training and pilot batches, fourteen weeks of production collection, and two weeks of final QA, re-collection of rejected hours, and packaging.

The pilot week matters more than its size suggests. We shipped the first 40 hours to the client immediately and they ran a smoke-test fine-tune before full production started. That surfaced two spec problems while they were still cheap to fix: their training stack expected a different gripper action convention, and one camera’s mounting height put the left workspace edge out of frame on tall-object tasks. Catching either in week twelve instead of week three would have cost hundreds of re-collected hours. If a vendor does not offer a pilot-batch gate before full-rate production, build one into the contract yourself.

Weekly deliveries went out in LeRobot format with per-batch QA reports, so the client’s dataset grew continuously instead of arriving as one end-of-program surprise.

What QA Caught

Episode rejection is the health metric of a collection program, and ours ran at 11% across the engagement. Where those episodes died:

Rejection CauseShare of RejectsWhy It Matters for VLA Training
Teleoperator mid-chunk pauses34%Corrupts action-chunk targets for chunked policies (ACT-style and continuous heads)
Frame-action sync drift > 10 ms27%Measurably degrades continuous-action fine-tunes
Failed episode, unlabeled22%Success mislabeling flattens gains on precision tasks
Dropped frames / camera faults11%Breaks observation continuity
Instruction-episode mismatch6%Directly poisons language conditioning

Every rejected hour was recollected within the fixed budget because our per-hour rates ($40-60 for bimanual leader-follower work, our standard benchmark range) price in an expected rejection band. If a vendor quotes you a rate with no stated rejection policy, that risk is sitting on your side of the table.

Results

The client fine-tuned on the new corpus blended 60/40 with their legacy data and ran their internal evaluation harness before and after. Directional results, their harness, not an audited benchmark:

  • Held-out kitchen task success roughly doubled against the legacy-data baseline.
  • Paraphrased-instruction success moved from near-random to parity with template phrasing.
  • Long-horizon task completion improved most on tasks with segment-level labels, consistent with what the pi-zero pipeline reports about post-training data quality (arxiv.org/abs/2410.24164).

The result we quote most often is the second one, because it isolates the language layer. Nothing about the model changed. The labels changed.

What Transfers to Your Program

Four rules from this engagement apply to almost any VLA fine-tuning collection:

  • Audit distribution before buying volume; count task families and unique templates in your existing corpus first.
  • Set diversity thresholds as batch-level rejection gates, not guidelines.
  • Price annotation and QA as separate line items so you can see what you are cutting when you cut.
  • Cap hours per task family; the marginal episode of a covered task is your most overpriced purchase.

The full scoping framework, cost tables, and the RFP scorecard we build these programs from are in the complete guide to VLA training data.

Next Step

if your evaluations look like this client’s before-picture, book a scoping call. We will audit your corpus distribution before recommending a single collection hour, or start with the complete VLA training data guide.

Request a Quote

Frequently Asked Questions

How many hours of data does it take to fine-tune a VLA for a humanoid?

This program used 1,400 hours across 42 task families, blended with existing data. Humanoids sit at the expensive end because cross-embodiment transfer from open datasets is weakest for them; standard single arms often need only 200-500 hours.

A batch-level QA rule that rejects annotation batches where any single instruction template exceeds a set share (we used 8%), with minimum thresholds for unique verbs and spatial-relation phrases. It prevents the label monoculture that makes VLA policies brittle to paraphrasing.

The legacy data still carried valid task knowledge; the 60/40 blend kept that signal while the new data corrected the distribution. Full replacement wastes paid-for coverage and can regress previously working tasks.

We ran 11% on this program, and 10-15% is the band we consider healthy for bimanual long-horizon work. A vendor reporting near-zero rejection is usually not checking sync, pauses, or success labels.

Case Study: How We Scaled Data Capture Hardware for Embodied AI for a VLA Model

Posted on by Enoch Pakanati

Two numbers sat in the same planning deck: 40 captured hours a week, and 8,000 hours due in five months. Both were accurate, and they could not both survive. Hitting 8,000 hours in roughly 22 weeks means averaging about 370 delivered hours a week, nine times what the floor was producing; at the actual run rate, the dataset would have arrived roughly four years late, well after the funding milestone it was supposed to unblock. This is the point where a VLA foundation model team brought the problem to us.

The gap was not effort, and that is the thesis of this case study: scaling capture is an architecture problem (sync, formats, parallel cheap seats), not an effort or premium-hardware problem. Their two-person capture team worked hard on good hardware, a single ALOHA-class station and a Vision Pro teleop pilot. The gap was architecture: no sync strategy across devices, no shared episode format, and a fleet plan that scaled capex faster than throughput. This is the most common shape of the scaling problem we see, so we are publishing the anonymized playbook.

This post walks through the three phases of the engagement: the audit, the hardware re-spec, and the scale-out, with the real throughput and cost numbers at each stage. If you are sizing a capture floor for VLA fine-tuning, the tables below are the ones we wish every team had before their first purchase order.

DexSet operates egocentric, teleoperation, and exocentric capture lines in production; this engagement ran on our floors with the client’s task list and QA criteria. Per our client agreements, the team is anonymized (“a VLA foundation model team”) and all numbers are rounded operational figures.

Key Takeaways – A VLA foundation model team needed 8,000 hours of manipulation data in five months; their in-house rate was 40 hours a week. We reached a sustained 400 hours a week by week nine. – The fix was mostly architectural: hardware sync, one episode format (LeRobot-compatible HDF5), and parallel cheap seats instead of premium single rigs. – Fleet mix at full scale: 6 GELLO-equipped arm cells, 3 ALOHA-class bimanual stations, 12 Quest 3 teleop seats, 20 UMI-style grippers, and one 6-camera D455 evaluation studio. – Delivered cost settled at $31-42 per teleop hour across task families, inside our standard $28-60 benchmark range.

Phase 1: The Audit (Weeks 1-2)

A capture audit is a measured baseline of usable hours per seat per shift, and it always starts by separating captured hours from delivered hours. The client’s 40 weekly hours were captured hours; after QA, delivered hours were closer to 26. Two findings explained most of the loss.

First, synchronization. The Vision Pro pilot streamed hand pose to the robot over Wi-Fi with no shared clock, and QA was rejecting roughly a quarter of episodes for observation-action misalignment beyond their 25 ms tolerance. Second, format debt: the ALOHA station wrote its own HDF5 layout, the Vision Pro stack wrote JSON pose logs plus video, and an engineer spent about a day a week converting between them.

Neither problem is exotic. Both are invisible until you measure delivered hours instead of captured hours.

Phase 2: The Hardware Re-Spec (Weeks 3-5)

A hardware re-spec is a redesign of the fleet around delivered cost per hour rather than device quality, and ours cut the premium hardware first. The Vision Pro seats moved to evaluation-only duty. For fleet teleop we standardized on Quest 3 headsets at roughly $500 per seat: seven times cheaper per seat, no measurable difference in demonstrations per operator-hour on the client’s pick, place, and insertion task families, and cheap enough to hold 20 percent spares.

For arm-based capture we kept the existing ALOHA-class station (fine bimanual tasks justified it) and added GELLO leader arms (arxiv.org/abs/2309.13037) at about $300 per arm on follower robots the client already owned. For task families that did not need force or proprioception fidelity, we added UMI-style handheld grippers (arxiv.org/abs/2402.10329) at roughly $400 per unit, with a budgeted SLAM QA pass.

The last addition was a six-camera RealSense D455 evaluation studio, hardware-synced through the D455’s trigger pin, wrapped around one teleop cell. Every evaluation episode was therefore captured ego and exo simultaneously, which the client’s research team used to test viewpoint generalization in the style of Ego-Exo4D (arxiv.org/abs/2311.18259). The studio cost about $2,700 in cameras and roughly the same again in truss, sync wiring, lighting, and NVMe ingest, a ratio worth budgeting for in advance.

Every line wrote one format: LeRobot-compatible episodes (github.com/huggingface/lerobot) with per-frame timestamps from a shared NTP-disciplined clock and a hardware sync signal on the camera side. The conversion engineer got their day back.

Phase 3: Scale-Out (Weeks 6-9)

Scale-out is the replication of a proven seat design in parallel, and it only works after yield per seat is stable. We added seats in weekly increments, holding the QA gate constant, until the floor hit the target run rate.

Metric Baseline (client in-house) Week 5 (re-spec live) Week 9 (full scale)
Captured hours / week 40 130 ~460
Delivered hours / week (post-QA) 26 110 ~400
QA rejection rate ~35% 15% 10-12%
Active seats 3 14 42
Fleet capex (cumulative) ~$32k ~$68k ~$118k
Delivered cost per teleop hour not tracked $46 $31-42

Two numbers deserve emphasis. The rejection rate fell from 35 percent to under 12 percent before most of the new seats arrived; sync and format fixes, not hardware volume, did that. And full-scale fleet capex, about $118k for 42 seats, is less than many teams spend on four premium bimanual cells. Cheap synchronized seats, bought after the yield problem was solved, are what made the 8,000-hour target arrive on schedule.

The non-hardware work mattered too, and it is easy to undercount. Operator onboarding became a two-day curriculum with a certification task instead of an afternoon of shadowing, which pulled new-seat ramp time down from three weeks to one. The QA gate ran four automated checks (sync residual, calibration freshness, trajectory smoothness, format validity) plus human review on a 10 percent sample, and its thresholds never moved during scale-out. Holding the gate constant is what made the weekly delivered-hours number trustworthy enough to plan a training schedule against.

What We Would Do Differently

Honest retrospectives are rarer than case studies, so here is ours. We would have moved the UMI-style gripper line earlier in the schedule; its SLAM post-processing pipeline took three weeks to tune, and starting it in week 3 instead of week 6 would have added several hundred delivered hours. We also underestimated storage: the exo studio’s terabyte-per-shift output forced a mid-project NVMe retrofit that a day of arithmetic would have predicted. Neither mistake threatened the deadline. Both are now checklist items in every new engagement.

What Generalizes From This Engagement

The transferable lessons are hardware-selection rules, and they match the decision matrix in our full guide to data capture hardware for embodied AI:

  • Measure delivered hours, not captured hours. The 14-hour weekly gap at baseline was invisible in the client’s dashboards.
  • Fix sync before buying anything. A 25 ms alignment tolerance is unreachable over ad-hoc Wi-Fi streaming without a shared clock.
  • Premium headsets are evaluation tools, not fleet tools. Quest 3 matched Vision Pro on demonstrations per operator-hour for standard manipulation families.
  • Match rig fidelity to task family. ALOHA-class cells for fine bimanual work, GELLO for arm tasks on owned robots, UMI for volume, one format across all of them.
  • Scale last. Every seat added before yield stabilizes multiplies your rejection rate.

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

How long does it take to scale a robot data capture operation?

In this engagement, nine weeks from audit to a sustained 400 delivered hours per week. The first five weeks fixed synchronization, formats, and seat design; scaling seats was the fast part.

About $118,000 in cumulative fleet capex for 42 seats: 12 Quest 3 teleop seats, 6 GELLO arm cells, 3 ALOHA-class stations, 20 UMI-style grippers, and a 6-camera RealSense D455 evaluation studio.

On a well-synchronized floor we hold 10 to 12 percent. Rates above 25 percent almost always trace to clock misalignment or ergonomic problems, not operator error.

Not for fleet capture. On pick, place, and insertion tasks, ~$500 Quest 3 seats matched the $3,499 Vision Pro on demonstrations per operator-hour. The Vision Pros stayed in service for gaze-labeled evaluation episodes, where their eye tracking earns the price.

Yes; the phases (audit, re-spec, scale-out) and rules above are the playbook. The vendor case is throughput on demand without capex and hiring, which mattered here because of the five-month deadline.

[Book a Demo] and we will run the throughput math on your task list, or start with the full guide to data capture hardware for embodied AI.

Case Study: Scaling Teleoperation Data for a VLA Model, 1,900 to 12,600 Episodes in Ten Weeks

Posted on by Enoch Pakanati

A head of data once asked us which of their 1,900 collected episodes had actually made their policy better, and we could not tell them; neither could they, because nothing in their pipeline had ever connected episodes to outcomes. That question came from the team this case study is about: eight weeks into their own collection effort, 1,900 episodes against a 12,000-episode target, a 34 percent QA rejection rate, and a training deadline that had already slipped once. Their ML team was strong. Their collection operation was two engineers moonlighting as floor managers, and it showed.

This is the common failure shape, and it is the thesis of this case study. Collection stalls not because anyone lacks skill but because demonstration data is an operations problem wearing an ML costume: staffing, ergonomics, and QA gating, none of which appear in the papers. The papers do tell you what is possible; ACT hit 80 to 90 percent success from about 50 clean demos per task (arxiv.org/abs/2304.13705). Getting 12,000 clean demos across 40 tasks is a different discipline.

Here is exactly what we changed, week by week, including the two things that failed. Use it as a template for your own program, or as an argument for not running one in-house.

Key Takeaways – Baseline: 1,900 episodes in 8 weeks, 34 percent rejection. Outcome: 12,600 QA-passed episodes in 10 weeks, 12 percent final rejection. – Six leader-follower stations plus a two-week operator certification program did most of the work. – Automated ingest QA (sync skew, smoothness, completion gates) cut silent data loss to near zero. – Client’s ACT baseline improved from 41 to 68 percent average success across their internal 40-task eval. – Peak steady-state: about 1,400 usable episodes per week from 8 operators.

The Starting Point: Why 1,900 Episodes Took Eight Weeks

The client’s bottleneck was a stack of small losses, not one big one. Their audit told the story: two VR stations shared among rotating volunteers, no defined operator roles, no reset protocol, and QA performed manually by an ML engineer, at training time, weeks after collection. Rejection ran 34 percent, dominated by timestamp skew between their 50 Hz action logs and 30 fps cameras, plus inconsistent initial states that made half the “passing” episodes weak training signal anyway.

Their effective yield was under 60 usable episodes per day. At that rate the 12,000-episode target sat more than eight months out.

What We Changed

The intervention had four parts: stations, people, protocol, and gates. In order of impact:

1. Stations. We replaced the two shared VR rigs with six bimanual leader-follower stations (GELLO-class leaders on the client’s arms, roughly $300 per leader plus fabrication), keeping one VR station for the mobile-base tasks where it fit better. Joint-level control cut trajectory hesitancy immediately and shortened operator ramp.

2. People. Eight dedicated operators on a two-week certification program: task cards, reset drills, and a throughput gate (20 usable episodes per hour on a reference task) before touching production tasks. Our standard ramp curve applied; novices started at 8 to 12 usable episodes per hour and certified operators stabilized at 25 to 40.

3. Protocol. Written task cards for all 40 tasks, marked randomization zones for object placement, success criteria defined at collection time, and per-episode metadata (task, operator, rig, success) attached at the source.

4. Gates. Automated ingest QA running nightly: inter-stream sync skew over 33 ms flagged, dropped-frame counts, jerk thresholds on joint trajectories, and completion labels verified against a sampled human review. Failures routed back to the floor as next-morning feedback per operator, which mattered as much as the gate itself.

The Week-by-Week Numbers

Weekly usable output tells the ramp story better than any narrative.

Week Usable episodes Rejection rate Notes
1 310 29% Station bring-up; operators in certification
2 540 26% First cohort certified; jig redesign after reset failures
3 980 19% All 8 operators on production tasks
4 1,390 15% Steady state reached
5 to 9 1,300 to 1,450/wk 11 to 14% Two task cards rewritten after eval feedback
10 1,280 12% Target passed: 12,600 total QA-passed

Two failures worth admitting. In week two, our first reset jig for a deformable-object task increased reset time instead of cutting it; operators beat the jig by hand and we scrapped it. And one of the 40 tasks, bimanual jar opening, ran a stubborn 25 percent rejection until week six, when slowing the commanded demonstration pace fixed what better hardware had not.

Did the Data Actually Help? Policy Results

The point of episode counts is policy performance, so the client re-trained their ACT baseline on the new corpus at week eleven. Average success across their internal 40-task evaluation went from 41 percent (trained on their original 1,900 mixed-quality episodes) to 68 percent, with the largest gains on precision tasks where the old VR data had been noisiest. Same architecture, same hyperparameter budget, different data. Their team’s summary, which we will keep quoting: “We spent six months on model changes for less improvement than ten weeks of better data.”

That is one program, not a controlled study, and we flag it as such. But it matches the pattern across our floors and the published record: demonstration quality moves policy success more reliably than architecture iteration at this stage of the field.

What We Would Do Differently

Every program teaches something, and this one taught us three corrections we have since made standard. First, we would start the operator certification cohort a full week before station bring-up finishes, not after. Waiting for hardware serialized two ramps that could have run in parallel, and it cost the program most of week one’s output. Second, we would pilot every task card ourselves before handing it to operators. Two of the 40 cards contained ambiguous success criteria (“lid secured” without a torque or rotation definition), and the resulting label inconsistency did not surface until the week-five training run flagged suspicious loss on exactly those tasks. A ten-minute pilot per card would have caught both. Third, we would set the deformable-object tasks to a slower commanded pace from day one. The week-six discovery that pace, not hardware, drove the jar-opening rejection rate had been sitting in the per-task rejection data since week two; we were looking at floor-level averages and missed it. Per-task dashboards are now default on every program we run.

None of these are exotic lessons. That is rather the point: scaling demonstration data is a discipline of small, boring corrections applied early, and the gap between 1,900 episodes and 12,600 episodes was made of exactly such corrections.

The Reusable Checklist

Scaling teleoperation collection comes down to seven decisions made early:

  • Match rig to task family (leader-follower for fine bimanual work; VR for mobile/humanoid tasks)
  • Dedicated, certified operators, never rotating volunteers
  • A throughput gate before production collection
  • Written task cards with randomization zones and success criteria
  • Hardware timestamps on every stream; sync checked at ingest, not at training
  • Nightly automated QA with per-operator feedback loops
  • A rejection buffer of at least 15 percent in the episode target

Rig options, cost-per-episode math, and QA thresholds are covered in depth in the complete teleoperation data collection guide.

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

How many episodes does a VLA model program need?

Program-dependent, but the humanoid teams we support typically target 10,000 to 50,000 QA-passed episodes per quarter across dozens of tasks. Single-task ACT policies can work from about 50 clean demos; multi-task generalization is what drives volume.

With rigs, certified operators, and ingest QA in place, expect three to four weeks to steady state. This program reached about 1,400 usable episodes per week from 8 operators by week four.

At least 15 percent at maturity and 25 to 30 percent during the first two weeks of any new task or rig. Budgeting zero rejection is the most common planning error we see.

In-house costs more than teams expect once engineer time is counted; the client here spent eight engineer-weeks producing data that mostly failed QA. Outsourcing wins on time-to-data in most cases; in-house wins when collection is itself a core competency you intend to build.

Rig fidelity and QA gating together. Joint-level leader-follower control removed trajectory hesitancy, and ingest gates stopped skewed episodes from reaching training. Success rose from 41 to 68 percent with no model changes.

Case Study: How We Scaled Egocentric Data Collection for Robotics for a VLA Model

Posted on by Sainath Gupta

Picture yourself as the head of data at a humanoid startup: your researchers have just validated the experiment that could anchor the next funding round, co-training human egocentric video with your teleop corpus improves policy generalization, and then you run the arithmetic on your own collection capacity and find the experiment wants ten times more human data than your team can produce this year. That is exactly where one humanoid foundation model team (unnamed here by agreement) was standing when they contacted us. Their result was in line with what EgoMimic-style research had reported (arXiv:2410.24221); their teleop dataset was around 400 hours and growing slowly.

Why was a well-funded team stuck? Because collection was competing with research for the same people. Every hour their engineers spent managing capture logistics was an hour not spent on training runs, and their pilot attempts at crowdsourced collection had returned footage with unusable sync and a rejection rate far beyond anything a program can absorb. Familiar story; we hear a version of it most months.

This post is the anonymized production log of what happened next: how we scoped, staffed, and delivered 4,000 QA-passed hours of egocentric data in 14 weeks, what it cost per hour, where we lost yield, and the three decisions that mattered most. Numbers are real from our program tracking, rounded to protect the client. The thesis the whole log supports: scaled collection is a manufacturing problem, and the returns came from three protocol decisions made before hour one, not from capture volume.

Key Takeaways – 4,000 QA-passed egocentric hours delivered in 14 weeks across three task families (kitchen, warehouse shelving, assembly bench) for a humanoid VLA co-training pipeline. – Blended cost landed in the $22 to $34 per hour band: mono-heavy capture for breadth, stereo for depth-critical tasks, dense hand-pose annotation on only the 30 percent of hours flagged retarget-critical. – First-pass QA yield started at 71 percent in week one and reached 86 percent by week six after protocol fixes; the delta was worth roughly $18K. – Camera height matched to the robot’s head frame and constrained “robot-plausible” motion instructions were the two protocol choices the client’s researchers valued most. – The client reported improved success on unseen-object manipulation evaluations versus their robot-only baseline after co-training on the mixed corpus.

The Brief: What “Scaled” Actually Meant

Scaling egocentric data collection means hitting a volume target at a defined quality bar and annotation depth on a fixed calendar, and all four variables were specified before we captured an hour. The brief we agreed with the client’s head of data:

  • Volume: 4,000 QA-passed hours (not raw hours; the distinction is the whole game).
  • Task families: kitchen manipulation, warehouse shelf pick-and-place, small-parts assembly bench, weighted 45/35/20.
  • Viewpoint constraint: camera height within 10 cm of the robot’s head frame; field of view overlapping the deployment camera’s.
  • Annotation: temporal task segmentation on everything; 3D hand pose and object tracks only on episodes their researchers flagged as retarget-critical, which settled at about 30 percent of hours.
  • Format: episode-structured delivery compatible with their LeRobot-based ingestion (github.com/huggingface/lerobot).

That last 30 percent decision deserves emphasis. Dense annotation on all 4,000 hours would have added roughly $10 per hour across the corpus, $40K of spend their training plan did not need. Annotation depth should follow the training mechanism; this program is why we repeat that line so often.

The Setup: Rigs, People, Protocol

Our production setup for this program combined two rig families, 26 trained collectors, and a task protocol written jointly with the client’s researchers in week zero. Rig split:

  • GoPro-class mono head rigs at 4K/60 for the breadth corpus, where the consumer was encoder pretraining and temporal understanding.
  • RealSense D455 helmet rigs for depth-critical assembly and shelving tasks, where metric 3D fed their grasp analysis.

The protocol work is what separated this from “wearing cameras at work.” Every task became a card: start state photo, goal condition, allowed object set, retry rule, and, critically, motion constraints. Collectors were instructed to keep grasps within the robot’s gripper envelope (no five-finger tricks a parallel-jaw gripper cannot imitate), avoid motions outside the robot’s reach profile, and re-stage rather than improvise when a scene drifted off-spec. We borrowed episode discipline from how public robot datasets are structured; if it cannot be expressed as a clean episode, the way entries in Open X-Embodiment are (arXiv:2310.08864), it does not ship.

The Numbers: Timeline, Yield, and Cost

Program metrics are the honest core of any case study, so here is the phase table straight from our tracking, rounded:

Phase Weeks Hours Delivered (QA-Passed) First-Pass Yield Notes
Pilot + protocol iteration 1 to 2 120 71% Blur and sync failures dominated rejections
Ramp 3 to 6 900 78% rising to 86% Fixed exposure presets; re-trained 4 collectors
Steady state 7 to 12 2,400 85 to 88% ~420 hours/week across 26 collectors
Completion + re-delivery 13 to 14 580 87% Backfilled task-family gaps flagged by client

Cost landed where the pilot predicted: mono breadth hours in the low $20s per QA-passed hour, stereo depth-critical hours with dense annotation in the low-to-mid $30s, blending to $22 to $34 across the corpus. For context on how those bands compare across rigs and vendors, the economics section of our pillar guide has the full tables: The Complete Guide to Egocentric Data Collection for Robotics.

Where did the 12 to 29 percent of rejected hours go? Motion blur in fast kitchen tasks (fixed with shutter-priority presets), IMU packet loss on two aging rigs (retired), off-protocol grasps (re-training), and a stubborn tail of episodes with ambiguous outcomes (protocol clarification). Yield improvement from 71 to 86 percent was worth roughly $18K over the program, which is why we staff a protocol engineer for the first six weeks rather than treating capture as a solved problem.

The Three Decisions That Mattered

Looking back through the retro notes, three decisions carried most of the outcome:

1. Matching viewpoint geometry before hour one. Camera height within 10 cm of the robot’s head frame and overlapping FOV meant the client’s encoder saw deployment-like images from the first batch. This costs nothing at capture time and is nearly impossible to fix afterward.

2. Robot-plausible motion constraints. Instructing collectors to stay inside the gripper envelope narrowed the embodiment gap at the source instead of hoping retargeting would absorb it. Their researchers called this the highest-value protocol clause.

3. Two-tier annotation. Dense hand pose on 30 percent of hours, selected by the researchers who consumed it, kept the budget aligned with the training plan and cut roughly $40K of unneeded spend.

An honorable mention: weekly delivery instead of end-of-program delivery. The client’s researchers trained on batch one while we captured batch four, surfaced a task-family gap in week five, and we corrected course while the collectors were still staffed. On a single-delivery program, that gap would have been discovered after demobilization, when fixing it costs triple.

The Outcome, Stated Carefully

The client reported improved success rates on unseen-object manipulation evaluations after co-training on the mixed corpus versus their robot-only baseline; direction consistent with published ego-plus-robot co-training results, and we will not dress it in precise percentages that are theirs to publish, not ours. What we can state as first-hand fact: the corpus passed their ingestion validation on first delivery, the program hit its calendar, and they extended into a second phase covering exocentric paired capture.

Opinion, clearly flagged: I think the 10x-human-to-robot-hours ratio this team converged on will look conservative within two years. Human egocentric capture scales with people; teleop scales with robot fleets, and people are the cheaper fleet.

Running This Play for Your Team

If your co-training experiments are data-starved, the fastest way to de-risk a program like this is a scoped pilot: 50 to 100 hours against your protocol, with QA reports attached. Book a scoping call and bring your task list; we will bring the yield curves from programs like this one.

Request a Quote

Frequently Asked Questions

How long does it take to collect 4,000 hours of egocentric robotics data?

In this program, 14 weeks end to end with 26 trained collectors, including a two-week pilot and protocol iteration phase. Steady-state throughput ran about 420 QA-passed hours per week.

The blended rate landed between $22 and $34 per QA-passed hour: mono breadth capture in the low $20s, stereo depth-critical hours with dense hand-pose annotation in the low-to-mid $30s.

Match camera height and field of view to the robot’s head frame, constrain grasps to the robot’s gripper envelope, keep motions inside its reach profile, and structure everything as clean episodes with defined start states and outcomes.

This program started at 71 percent first-pass yield and stabilized at 85 to 88 percent by week six. Budget for 15 to 25 percent rejection if you are starting fresh.

The client reported improved success on unseen-object manipulation evaluations after co-training the mixed corpus versus their robot-only baseline, consistent with published results such as EgoMimic. Exact figures are theirs to publish.

Case Study: How a Series B Humanoid Startup Rebuilt Data Procurement for Its VLA Model

Posted on by Enoch Pakanati

Grasp success on the team’s fixed eval suite dropped six points in a single week, and for two days nobody could explain it. The model had not changed. The training code had not changed. What had changed, it eventually emerged, was a data batch: one of their three vendors had silently rescaled gripper actions between deliveries. The team had money, talent, and a working humanoid platform. What stalled their VLA program was procurement: three data vendors, three delivery formats, no shared acceptance spec, and an ML team that had quietly stopped trusting incoming data. By their own estimate, roughly a quarter of purchased hours never reached a training run.

Nothing about that situation was unusual. It is what happens when data purchasing grows by accretion: a vendor added for a demo push here, another for a task category there, each with its own definitions of “hour,” “annotated,” and “calibrated.” The result is not one data pipeline but several, with the training team as the manual integration layer.

This post walks through how the team rebuilt procurement in one quarter using the four artifacts from the Robotics Data Buyer’s Playbook: a spec-first RFP, a weighted scorecard, a red flags screen, and a 50-hour paid pilot. The client is anonymized (a Series B humanoid startup; we do not name customers), but the process and the numbers are the real pattern we see when buyers switch from improvisation to protocol. The thesis this case argues: pilots run under production conditions predict production performance, and paper evaluation alone does not.

DexSet was one of the vendors evaluated in this process, so we saw the RFP, the pilot, and the contract from the receiving side. That is exactly the vantage point a buyer should want documented.

Key Takeaways

  • Starting point: three vendors, three formats, ~74 percent average usable-hour yield, about one engineer-day of triage per delivery.
  • The rebuild: one written acceptance spec, an RFP to six vendors, a 10-criterion weighted scorecard, paid 50-hour pilots for the top two.
  • Pilot results diverged sharply: 90 percent yield with a positive policy delta versus 78 percent yield and a failed loader test.
  • Twelve weeks later: yield held within two points of the pilot across the first 1,000 contracted hours, and per-delivery engineering time dropped from a day to under an hour.

The Starting Point: Accretion, Not Architecture

The team’s data supply had accreted rather than been designed: each vendor relationship was created for a short-term need, and none was ever re-evaluated. Vendor A delivered undocumented HDF5 with a schema that shifted between batches. Vendor B delivered rosbag2 that had to be converted and re-timestamped. Vendor C delivered mp4s plus CSV joint logs with no sync guarantees at all.

The measurable symptoms, from their own tracking:

  • Average usable-hour yield around 74 percent across vendors, measured against the acceptance checks they eventually wrote down.
  • Roughly one engineer-day per delivery spent on conversion and triage.
  • Two training regressions in one quarter traced back to data batches, one to a sync drift issue, one to inconsistent gripper action scaling between vendors.

The trigger for change was mundane: a planning meeting where nobody could answer what an incremental 1,000 hours of manipulation data would actually cost, because nobody could say what fraction would be usable.

Step 1: Write the Spec Before Talking to Anyone

The acceptance spec is a short document that defines what a usable hour means for a specific program, and writing it was the most valuable week of the whole rebuild. Theirs fit on three pages: stereo egocentric plus wrist cameras at defined resolutions and frame rates, proprioception at 50 Hz minimum, time sync under 10 ms across all streams, a 60-task manipulation list with reset criteria, an annotation rubric, and delivery in LeRobot format with a schema doc, following the conventions the ecosystem inherited from Open X-Embodiment and the ALOHA line of work (arxiv.org/abs/2310.08864; arxiv.org/abs/2304.13705).

The immediate effect was that vendor quotes became comparable for the first time. The secondary effect was internal: the ML and data teams had to agree on what they were actually buying, which surfaced two silent disagreements about camera placement that had been corrupting cross-vendor consistency for months.

Step 2: RFP and Scorecard

The RFP went to six vendors with fifteen questions requiring numeric answers: committed usable-hour yield, sync tolerance and recheck cadence, audit accuracy, rig counts behind throughput claims, per-hour pricing by task complexity, pilot terms, and licensing. Four vendors answered with numbers. Two answered with adjectives and were dropped without further calls, which is the red flags screen doing its job cheaply.

The four remaining responses were scored independently by two people on the 10-criterion weighted scorecard (quality SLAs 15, modality coverage 12, calibration/sync 12, down to IP/licensing at 7). Independent scoring earned its keep in the reconciliation meeting: the two scorers disagreed by more than a point on only three criteria, and every disagreement traced to an ambiguous vendor answer rather than a difference in judgment. Those ambiguities went back to the vendors as written clarification requests, which is a politer and more useful outcome than one person’s optimism deciding the ranking.

The scoring separated the field more on contract terms than on hardware: two vendors wanted broad rights to reuse delivered episodes for other customers, which the team treated as disqualifying on the licensing criterion. Two finalists advanced.

Step 3: The 50-Hour Paid Pilots

Both finalists ran paid 50-hour pilots under production conditions, judged on four numbers fixed in the pilot agreements before capture began. This is the step that paper evaluation cannot replace, and the results diverged more than the scorecards had predicted.

Pilot metric (agreed in advance) Threshold Finalist 1 Finalist 2
Usable-hour yield vs acceptance spec ≥85% 90% 78%
Annotation accuracy (independent 5% re-label) ≥97% 98.2% 96.1%
Policy success delta on fixed 12-task eval Positive +6 points vs baseline +1 point
Loader time into LeRobot pipeline ≤1 engineer-day ~2 hours Failed (schema mismatches, 3+ days)

The policy-delta test deserves a note. The team fine-tuned the same fixed baseline policy twice, once on their existing mix, once with the 50 pilot hours added, and evaluated both on the same 12 tasks. Fifty hours is a small delta at VLA scale, and they treated it that way: not as proof of final performance, but as a canary. Data that helps at 50 hours might help at 1,000; data that does nothing at 50 hours is a warning you can act on before signing an annual contract.

Step 4: Contract and the Twelve Weeks After

The contract encoded the pilot numbers rather than replacing them with prose: yield commitment at 88 percent with redelivery of failures, sync tolerance and recalibration cadence as delivery requirements, LeRobot schema versioned in an appendix, quarterly re-audit rights, and no vendor reuse of delivered episodes.

The results over the first 1,000 contracted hours were unremarkable in the best sense. Yield held between 88 and 90 percent, within two points of the pilot. Engineering time per delivery fell from roughly a day to under an hour, because the format and schema were locked before signature instead of negotiated after each batch. The two silent disagreements about camera placement did not recur, because the spec, not tribal memory, was the reference.

The honest caveat: this process cost about five weeks end to end and roughly $7,000 in pilot fees across two finalists. For a team spending six figures annually on data, that insurance premium was around 5 percent of program cost. For a team buying 20 exploratory hours, it would be overkill, and we say so in the full playbook.

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

How much did the procurement rebuild cost compared to what it saved?

About five weeks of process time and roughly $7,000 in paid pilot fees. Against it: yield moved from ~74 to ~90 percent, and per-delivery engineering time dropped from a day to under an hour, which on a six-figure annual program repaid the process cost within the first deliveries.

Because scorecards rank paper, and paper can lie by omission. In this case the two finalists scored within a few points of each other, then diverged by 12 yield points and a failed loader test under production conditions.

It is enough to judge the vendor’s pipeline, not the model’s ceiling. Treat the policy success delta as a canary: positive movement at 50 hours justifies scaling; zero movement is a cheap warning.

The pilot numbers themselves: yield commitment with redelivery terms, sync tolerance as a delivery requirement, a versioned schema appendix, re-audit rights, and an explicit ban on vendor reuse of delivered episodes.

Case Study: Scaling a VLA Training Data Program from 500 to 12,000 Usable Hours Without Doubling the Budget

Posted on by Enoch Pakanati

Cell F14 of the spreadsheet a VLA startup sent us in January read cost_per_usable_hr = $71. Two rows down sat the scaling plan: 12,000 usable hours by July, up from the 500 they had collected on an internal ALOHA-class cell before their policy plateaued. At $71 per usable hour, the plan cost $852,000. The approved budget in the adjacent column was $560,000. The engagement began, literally, as a request to make F14 say something else.

The gap behind that cell was not caused by anything exotic. Their single rig sat idle 60 percent of the day. Their rejection rate was 31 percent because task specs lived in a Slack thread. And every hour they collected was premium bimanual teleop, even the hours that existed only to teach the visual encoder what countertops look like.

This is the most common shape of scaling problem we see, and it carries the thesis this case study argues: data budgets are rescued by operational variables, rejection rate, modality mix, and rig utilization, not by rate negotiation. Names and identifying details are removed at the client’s request; every number is from the actual program ledger.

What follows: the starting cost baseline, the three changes that moved it, the month-by-month rejection curve, and the final economics. If you are scaling any demonstration data program past a few thousand hours, at least one of these three changes applies to you.

Key Takeaways

  • Starting point: $71 per usable hour, 31% QA rejection, one under-utilized rig. Ending point: $46 per usable hour, 11% rejection, 12,100 usable hours delivered.
  • Fix 1: a written, versioned task spec cut rejection from 31% to 14% in five weeks. Cost: about a week of engineering time.
  • Fix 2: a 65/35 modality split (egocentric pretraining at $22/hr blended, teleop post-training at $44/hr blended) cut the blended rate by 29%.
  • Fix 3: two-shift rig utilization dropped amortization per hour from $11 to $5.
  • Total spend: $557,000 against a $560,000 budget. The policy’s task success rate improved 19 points on their internal eval.

The Baseline: Why $71 per Usable Hour Happens

A cost baseline is the fully loaded spend per QA-passed hour at the start of an engagement, and it is almost always worse than the team believes because rejection losses hide inside it. The startup quoted their internal cost as “about $49 an hour.” That was spend divided by raw hours. Divided by usable hours, it was $71.

Their baseline decomposed like this:

Line item Their baseline Notes
Rig amortization $11/raw hr One ~$26k cell, single shift, idle 60% of the day
Operator labor $29/raw hr Two part-time operators, still early on the learning curve
QA review $9/raw hr Manual review, no automated pre-checks
Raw total $49/raw hr The number they quoted internally
Rejection rate 31% Ambiguous specs, no calibration checklist
Usable total $71/usable hr $49 / (1 - 0.31)

Nothing in that table is negligence. It is what a research-grade pipeline looks like when a training run suddenly demands production volume. The ALOHA hardware itself was fine; a roughly $20k bimanual rig is a proven design (https://arxiv.org/abs/2304.13705). The economics around the hardware were the problem.

Fix 1: A Versioned Task Spec Cut Rejection by More Than Half

A task specification is a written, versioned document defining episode start states, success criteria, camera framing requirements, and reset procedures, and it is the highest-ROI artifact in any capture program. Theirs was a Slack thread and tribal memory. Operators guessed, and QA rejected the guesses.

We replaced it with a two-page spec per task family: exact object sets, start-state photos, an explicit success predicate, and a pre-episode calibration checklist. Rejections got tagged with a reason code so the spec could be patched weekly.

The rejection curve over the program:

Month Rejection rate Main rejection driver
Jan (baseline) 31% Ambiguous success criteria
Feb 22% Camera framing drift
Mar 14% Occluded end-effector
Apr 12% Long-horizon task failures
May to Jul 11% Steady state

Total cost of the fix: roughly one engineer-week plus 30 minutes of weekly maintenance. Effect: every subsequent dollar bought 29 percent more usable data. No other intervention in this program came close on ROI.

Fix 2: Stop Buying Premium Hours for Non-Premium Jobs

Modality staging is the practice of assigning cheap capture modalities to pretraining objectives and reserving teleoperation for action-supervised post-training. The startup was training their visual encoder on $49-per-hour bimanual teleop footage. The encoder did not care that the footage had action labels; it needed diverse pixels of hands, objects, and scenes.

We restructured the volume: 65 percent egocentric human video for representation pretraining, collected across 40+ real kitchens and workspaces at a blended $22 per hour, and 35 percent bimanual teleop for post-training at a blended $44 per hour. This mirrors what the public record already suggested: cross-source mixtures train stronger generalists, which is the entire premise of Open X-Embodiment’s 1M+ trajectories across 22 embodiments (https://arxiv.org/abs/2310.08864) and DROID’s 76k episodes across 52 buildings (https://arxiv.org/abs/2403.12945).

Blended raw rate after the split: $29.70 per hour, a 29 percent reduction against their all-teleop plan. Their ML lead’s initial worry, that ego data would not move policy metrics, resolved in evaluation: the encoder pretrained on the ego corpus reached the same downstream success with 22 percent fewer teleop post-training hours. The cheap data made the expensive data go further.

Fix 3: Utilization Is a Pricing Decision

Rig utilization is the fraction of available hours a capture station spends collecting, and it directly divides your amortization cost per hour. One cell on one shift with 60 percent idle time was charging them $11 per hour just for hardware.

Changes: we added a second shift on their existing cell, stood up two DexSet production cells in parallel, and scheduled resets and calibration into shift handovers instead of collection time. Amortization dropped to $5 per hour. Operator throughput also compounded, as it always does; our operators improve 30 to 50 percent over their first 200 hours, and by April the program’s episodes-per-shift was up 41 percent against baseline.

The Final Ledger

Program economics are only credible as a closed ledger, so here is the whole thing:

Item Plan (their original) Actual (restructured)
Usable hours target 12,000 12,100 delivered
Blended raw rate $49/hr (all teleop) $29.70/hr (65/35 mix)
Rejection rate 31% 11% steady state
Cost per usable hour $71 $46
Language annotation not budgeted $10/usable hr on teleop subset ($42,400)
Total spend $852,000 projected $557,000 actual

Against their internal eval suite, task success improved 19 points over the plateaued January checkpoint. We do not claim the data mix deserves all of that credit; their team shipped model improvements in the same window. We do claim the counterfactual: at $71 per usable hour, the volume that enabled those experiments would not have fit the budget.

Every formula behind this ledger, including the cost-per-usable-hour tables and the rig amortization model, is published in our robot training data costs and pricing guide.

What Transfers to Your Program

A transferable lesson is one that holds regardless of embodiment or task list, and this program had three. Write the task spec before you scale, because rejection rate is the multiplier on everything. Split modalities by training objective, because encoders do not need action labels. And treat utilization as pricing, because an idle rig is a fee you pay for nothing. None of these require our involvement to implement. They do require someone to own cost per usable hour as a weekly metric.

Apply This Playbook to Your Program

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

How many usable hours does a VLA program actually need?

It varies by task breadth and model scale, but the programs we run for post-training typically land between 5,000 and 30,000 usable teleop hours, on top of larger and cheaper pretraining corpora. Budget from your eval targets backward, not from a round number.

With a versioned task spec and calibration checklists, 10 to 15 percent is a realistic steady state in our pipelines. Above 25 percent, fix your spec before you buy another hour.

In this program, encoder pretraining on egocentric video reached equal downstream success with 22 percent fewer teleop post-training hours. Results vary with task overlap, but the direction is consistent across our engagements.

$46, down from a $71 baseline: a 35 percent reduction from spec discipline, modality staging, and rig utilization combined, with no reduction in QA standards.

The task spec and utilization fixes, yes, immediately. The modality split needs access to egocentric collection at scale, which is harder to stand up internally; that is typically where teams engage a provider.

The Robot Training Data Costs and Pricing Guide contains every benchmark and formula used in this program. Or book a scoping call and we will baseline your current cost per usable hour in one session.

Case Study: How We Scaled Exocentric & Multi-View Data for a VLA Model

Posted on by Enoch Pakanati

We once shipped a pilot batch with an 18 millisecond wrist-camera clock offset, and this case study exists partly because of it. The mistake was ours: the wrist stream was timestamped off the arm controller instead of the rig’s PTP clock domain, invisible in playback, and it was the customer’s smoke-test training run at hour 20 that surfaced it. We fixed it, changed our rig standard, and kept the lesson. Vendor case studies that report zero mistakes are describing a project that never touched hardware.

This one covers eight weeks of work with a humanoid foundation model team (anonymized by agreement) whose manipulation policy had stalled on cluttered-scene tasks. The claim we will argue with the numbers below: their ceiling was informational, not architectural, and a disciplined multi-view capture operation, pilot batches, per-session calibration gates, one shared clock domain, is what removed it.

The team arrived with a specific complaint. Their policy trained fine, evaluated fine on open scenes, then dropped hard on cluttered tabletops. Their dataset: tens of thousands of teleoperated episodes, every one recorded from a single fixed camera. Failure review showed what you would expect. When clutter occluded the target from that one viewpoint during approach, the policy guessed.

They asked us for multi-view capture at a rate their internal rig could not hit: hundreds of hours within a quarter, calibrated, synchronized, and QA’d to a standard their ML leads could defend. What follows is how we built it, what it cost, what broke, and what the retrain showed. For the general framework behind these choices, the pillar guide is here: The Complete Guide to Exocentric & Multi-View Data for Robot Learning.

Key Takeaways – 600 hours of DROID-style multi-view episodes (two external stereo + wrist) delivered across 8 weeks on 4 parallel stations. – Per-session calibration gates rejected 4.1 percent of sessions; every rejection would have shipped corrupted extrinsics without the gate. – All-in capture cost landed at $31 per hour, inside our standard $26 to $38 benchmark range. – Retrained on matched episode counts, the customer’s occlusion-heavy split improved by double digits; open-scene performance held.

The Starting Point: Diagnosing a Data Ceiling

A data ceiling is a performance plateau caused by information missing from the training set rather than by model capacity. Confirming one is cheap and worth doing before any capture contract: tag evaluation failures by whether the target object was visible to the training viewpoint at decision time. On this team’s cluttered-scene split, occlusion-correlated failures dominated by a wide margin. No architecture sweep fixes pixels that were never captured.

The finding matches the public record. The RoboMimic study showed observation space design materially changes imitation outcomes on identical demonstrations (arXiv:2108.03298), and DROID’s authors considered multi-view important enough to mandate two calibrated external stereo views plus wrist across all 76,000 episodes (arXiv:2403.12945). We proposed the same geometry rather than inventing one.

The Rig Spec: Boring by Design

The capture spec is the contract between operations and the ML team, and ours fit on one page. Four identical stations, each with:

  • Two ZED 2i stereo cameras on rigid tripod mounts at roughly 45 degrees off the workspace centerline, 1 m from task center, opposite sides.
  • One wrist camera on the arm.
  • Extrinsics calibrated with a ChArUco target via OpenCV, cross-checked in Kalibr; camera-to-robot-base transform verified against known end-effector poses.
  • Sync via PTP-disciplined clocks, with a hard gate: cross-camera skew under 10 ms or the session does not ship.
  • A 20-second ChArUco verification sweep at every session start; reprojection error above 0.5 px blocks capture until recalibration.

Identical stations mattered more than any individual choice. One geometry means one calibration procedure, one QA script, one training data schema, and operators who can rotate between stations without retraining.

What Eight Weeks Actually Looked Like

Scaling capture is a throughput problem with a quality constraint, and the numbers tell the story better than prose:

Metric Value
Capture stations 4 (identical DROID-style geometry)
Calendar time 8 weeks
Delivered episodes ~58,000 across 41 task variants
Delivered hours (multi-view) 600
Sessions rejected at calibration/sync gate 4.1%
Episodes rejected at QA review 2.7%
Storage delivered ~38 TB (H.265, with per-frame extrinsics and sync metadata)
All-in operated cost $31 / hr

Weeks one and two ran at half throughput on purpose. We shipped a 20-hour pilot batch first so the customer’s ML team could confirm schema, load episodes into their LeRobot-based training stack, and run a smoke-test train before we committed the fleet. They caught the wrist-camera clock defect described at the top of this post: a consistent 18 ms offset from timestamping off the arm controller instead of the PTP domain. Catching that at hour 20 instead of hour 600 is the entire argument for pilot batches.

Throughput past the pilot came down to two decisions that had nothing to do with cameras. The first was task design: the customer’s 41 task variants were sequenced so that each station ran one object set per half-day block, which cut resets and scene changes to minutes instead of the constant churn you get when operators bounce between tasks. The second was operator rotation. Because all four stations shared one geometry and one procedure, any operator could run any station, and we scheduled captures to keep stations busy through breaks and calibration stops. Utilization across the fleet held near 85 percent of scheduled hours; on prior projects with heterogeneous rigs, we had struggled to hold 65.

The other failure worth naming: in week five, a boom mount on station three sagged after a fixture swap, and the morning verification sweep caught reprojection error at 1.3 px. The gate rejected the session, recalibration took 25 minutes, and no corrupted data shipped. Before we ran per-session gates, that class of drift used to surface weeks later as unexplainable training noise.

The Result: What the Retrain Showed

The customer retrained the same architecture on matched episode counts, single-view versus our multi-view data, which is the only comparison that isolates the data effect. On their occlusion-heavy cluttered split, success improved by double digits. On open scenes, performance held flat, confirming the ceiling had been informational, not architectural. Their engineers also reported a second-order win: with three calibrated views per episode, failure triage became visual inspection instead of guesswork, because someone could always see what happened.

We report ranges rather than their exact internal metrics by agreement, and we would flag any vendor who publishes a client’s precise evaluation numbers as a reason to negotiate confidentiality carefully. What we can say precisely is what the comparison controlled for: same architecture, same hyperparameters, same episode count, same evaluation protocol. The only variable was the data.

What We Would Repeat, and What We Changed

Three practices carried the project and are now standard on every DexSet engagement: pilot batches before fleet commitment, per-session calibration and sync gates with hard thresholds, and extrinsics embedded in every episode’s metadata rather than in a side document that drifts out of date.

One thing we changed afterward: we now put the wrist camera on the PTP domain from day zero, on every rig, because the 18 ms lesson generalizes. And we stopped quoting capture programs without a failure-tagging pass on the customer’s existing evaluation data first; twice since, that pass showed the bottleneck was not viewpoint at all, and we said so.

Test the Pattern on Your Own Failures

If your evaluation failures cluster around occlusion and your dataset is single-view, the pattern in this case study probably applies to you. Book a demo and we will walk through the pipeline with real sample episodes, calibration metadata included, and run the failure-tagging pass on your evaluation data before anyone talks about a contract.

Request a Quote

Frequently Asked Questions

How long does it take to scale a multi-view capture program?

In this engagement, 600 hours of calibrated multi-view data took 8 weeks on 4 parallel stations, including a deliberately slow 2-week pilot phase. Throughput scales roughly linearly with identical stations once the geometry and QA gates are standardized.

All-in operated capture landed at $31 per hour, inside DexSet’s standard $26 to $38 benchmark range for DROID-style rigs, plus one-time rig builds in the $4,500 to $7,000 range per station.

A 20-second ChArUco verification sweep at every session start, gated on reprojection error under 0.5 px, with extrinsics cross-checked in Kalibr and camera-to-base transforms verified against known end-effector poses. Sessions failing the gate are recalibrated before any capture ships.

Yes. Retrained on matched episode counts, the customer’s occlusion-heavy evaluation split improved by double digits while open-scene performance held, isolating viewpoint coverage as the binding constraint.

Because schema and sync defects are cheap at hour 20 and expensive at hour 600. The pilot here caught a wrist-camera clock offset of 18 ms that would otherwise have contaminated the full delivery.