Skip to main content

Dexset

dexset
Menu
Request a Demo

Tag: Decision Matrix

Teleop, Egocentric, Exocentric, or Handheld: Comparing Robot Training Data Approaches by Cost

Posted on by Sainath Gupta

In February, we priced the same 5,000-hour manipulation corpus three different ways for one buyer: all bimanual teleoperation at a blended $44 per hour, a 65/35 egocentric-to-teleop mix at just under $30, and a handheld-gripper-heavy plan in between. Same task list, same acceptance spec. The spread between the first two plans came to roughly $70,000.

I build teleop cells for a living, so this is not an argument against teleoperation. It is the thesis that quoting exercise made unavoidable: modality selection is a budgeting decision, and the right modality for each training objective is the cheapest one that actually satisfies it. For plenty of what your model needs to learn, the premium teleop hour is simply the wrong purchase.

Teams get this wrong in both directions. Some buy 20,000 hours of premium teleop and burn budget teaching their encoder what a kitchen looks like, a job $18-per-hour egocentric video does fine. Others go all-in on cheap human video and then discover their policy has beautiful representations and no idea how to move an actual gripper.

The reason the mistake is so common is that modality costs and modality capabilities are usually discussed separately. Cost tables live in procurement decks; capability arguments live in arXiv papers. This post puts them in one place, with DexSet’s operating benchmarks attached, so you can match each dollar to the learning objective it actually serves.

By the end you will have per-hour costs for four collection approaches, an honest pros-and-cons list for each, and a decision matrix that maps training objectives to the cheapest modality that satisfies them.

Key Takeaways

  • Teleoperation ($28 to $60/hr) is the only approach that outputs executable robot actions natively. Pay for it where action supervision matters.
  • Egocentric human video ($15 to $40/hr) is the cheapest volume play, best for representation pretraining, worst for the embodiment gap.
  • Multi-view exocentric capture ($20 to $50/hr) buys scene context and cross-view consistency; calibration labor is its hidden cost.
  • UMI-style handheld grippers (under $1k per device by our estimates) collect gripper-centric data at near-egocentric labor cost, with heavier post-processing.
  • QA rejection (10 to 30 percent in our pipelines) and annotation ($8 to $25/hr per pass) apply to all four. Compare on cost per usable hour.

The Four Approaches, Defined

A data collection approach is the pairing of a capture device with a control source: robot teleoperation, head-mounted egocentric capture, calibrated exocentric camera arrays, or handheld instrumented grippers. Everything else, mono versus stereo, camera count, annotation depth, is a variation within these four.

Teleoperation: $28 to $60 per Hour

Teleoperation data is produced by a human driving a real robot through leader-follower arms or a VR interface, so every recorded frame pairs observations with executable actions in the robot’s own action space. The ALOHA project showed this could be done on a roughly $20k bimanual rig (https://arxiv.org/abs/2304.13705), and Mobile ALOHA extended it to whole-body mobile tasks on roughly $32k of hardware (https://arxiv.org/abs/2401.02117).

Pros

  • Native action labels; feeds imitation learning and VLA post-training directly
  • Matches your exact embodiment, gripper, and camera placement
  • Long-horizon, contact-rich tasks are demonstrable at production quality

Cons

  • Highest labor cost: trained operators at $18 to $38 per hour, plus rig amortization and QA
  • Throughput capped by operator skill and episode reset time
  • Data is embodiment-specific; switching robots strands some of its value

Egocentric Human Video: $15 to $40 per Hour

Egocentric data is first-person video from a head-mounted camera while a human performs tasks with their own hands, capturing human-level dexterity with no robot in the loop. Ego4D and EgoExo4D (https://arxiv.org/abs/2311.18259) made the research case; the commercial case is pure economics, since the collector works at natural speed on tasks they already know.

Pros

  • Cheapest per hour; scales to thousands of hours quickly
  • Enormous task and scene diversity, including real homes
  • Strong pretraining signal for visual encoders and hand-object interaction priors

Cons

  • No robot actions; the embodiment gap means it rarely supervises control directly
  • QA rejection skews high (motion blur, gaze drift, occlusion), 15 to 30 percent in our pipelines
  • Needs retargeting or paired data to transfer to a gripper

Multi-View Exocentric Capture: $20 to $50 per Hour

Exocentric data is third-person video from multiple calibrated, synchronized cameras observing the same task, giving models scene-level context and cross-view consistency that neither ego nor teleop streams provide alone. Stereo pairs add 15 to 25 percent over mono at the same view count and buy metric depth in return.

Pros

  • Full-scene coverage; occlusions in one view are recovered in another
  • Calibrated multi-view supports 3D reconstruction and world-model training
  • Pairs well with egocentric streams (the EgoExo4D recipe)

Cons

  • Calibration and synchronization labor at every scene change is the silent budget eater
  • Fixed arrays limit scene diversity; mobile arrays raise cost
  • Still no action labels without a paired control source

Handheld Instrumented Grippers (UMI-style): Near-Egocentric Cost, Gripper-Centric Output

Handheld gripper capture uses a portable, wrist-camera-equipped gripper operated by a human, producing gripper-centric trajectories without any robot present at collection time. The UMI paper (https://arxiv.org/abs/2402.10329) defined the category; our build estimate is under $1,000 per device including the camera.

Pros

  • Capex is trivial next to a $20k to $32k teleop cell
  • Collection happens anywhere a person can walk, at near-egocentric labor rates
  • Output is closer to robot action space than raw human video

Cons

  • Heavier post-processing to recover clean actions (SLAM drift, kinematic mismatch)
  • Gripper form factor constrains which tasks are demonstrable
  • QA tooling for this modality is younger; expect iteration

Master Comparison Table

Approach DexSet cost/raw hr Capex per station Action labels QA rejection Best use
Teleoperation $28 to $60 $20k to $32k Native 10 to 25% VLA post-training, imitation learning
Egocentric video $15 to $40 $300 to $3.5k None 15 to 30% Encoder pretraining, dexterity priors
Exocentric multi-view $20 to $50 $5k to $15k (3 to 8 cams) None 10 to 20% Scene context, world models, 3D
Handheld gripper (UMI-style) $18 to $42 (our estimate) Under $1k/device Recoverable 15 to 25% Diverse-scene manipulation at low capex

Annotation is additive to every row: $8 to $12 per hour for language instructions, up to $18 to $25 for dense masks and contact labels. And every row’s real price is its cost per usable hour: divide by (1 minus rejection rate). The full math, with a budget spreadsheet, lives in our robot training data costs and pricing guide.

Decision Matrix: Match the Dollar to the Objective

A modality decision matrix assigns each training objective the cheapest approach that actually satisfies it, instead of defaulting everything to the premium modality. Here is the one we use in scoping calls:

Your objective Buy this Not this Why
Post-train a VLA on your robot Teleoperation Egocentric You need native actions on your embodiment
Pretrain visual encoders at volume Egocentric Teleoperation Paying $42/hr for pixels is waste
Scene diversity across 100+ homes Handheld gripper or egocentric Fixed exo array Portability beats calibration
Depth-dependent manipulation Stereo exo + teleop Mono anything Metric depth earns its 15 to 25% premium
World-model or video-prediction training Exo multi-view + ego pairs Teleop only Cross-view consistency is the signal
Bimanual, contact-rich skills Teleoperation (ALOHA-class) Handheld gripper Two grippers, force-aware demos

The pattern behind the matrix: mix modalities and stage them. Open X-Embodiment’s 1M+ trajectories across 22 embodiments (https://arxiv.org/abs/2310.08864) and DROID’s 76k episodes (https://arxiv.org/abs/2403.12945) already prove cross-source data mixes train better generalists. Your budget should look like a portfolio, not a single line item. A 70/30 split of cheap pretraining hours to teleop post-training hours routinely cuts blended cost by a third in programs we run, with no loss on the action-supervised objectives.

Match the Modality to the Objective

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

What is the cheapest way to collect robot training data?

Egocentric human video, at $15 to $40 per hour in our benchmarks, since hardware is a wearable camera and collectors work at natural speed. It is cheapest per hour but supplies no robot actions, so it cannot carry a program alone.

When you need executable actions on your specific embodiment: VLA post-training, imitation learning for contact-rich or bimanual skills. For those objectives nothing cheaper substitutes, which is exactly why you should not spend teleop dollars on anything else.

They are production-useful with caveats. Capex is under $1,000 per device by our estimates and scene diversity is the widest of the four approaches, but plan for heavier post-processing and a 15 to 25 percent rejection rate while the QA tooling matures.

Stereo adds 15 to 25 percent to exocentric capture cost and pays for itself on depth-dependent manipulation tasks. For pure representation pretraining, mono volume usually beats stereo precision per dollar.

Convert everything to cost per usable hour: quoted rate divided by (1 minus the measured QA rejection rate), plus annotation per pass. Our pricing guide includes the worked tables.

Start with the Robot Training Data Costs and Pricing Guide for the full benchmark tables, then book a scoping call and we will run your task list through the decision matrix above.

Comparing Exocentric & Multi-View Data Approaches for Robot Learning: Pros, Cons & Costs

Posted on by Sainath Gupta

A buyer on a scoping call last month put the field’s confusion into one sentence: everyone tells him to collect multi-view data, and nobody will tell him which multi-view. He was right to push. “Multi-view” describes at least five distinct capture strategies with different rig costs, different failure modes, and different value per training hour, and the honest answer, the thesis of this post, is that the right configuration is determined by your task list and training strategy, not by your budget or by whichever public dataset you read about first. Choosing the wrong one is not a small mistake. A team that builds a six-camera arc when their tasks needed a wrist camera and one tripod has burned rig budget, tripled their storage bill, and slowed capture throughput for nothing.

The confusion is understandable. Public datasets each embody one choice without explaining the alternatives: DROID picked two external stereo cameras plus wrist, Ego-Exo4D picked glasses plus stationary exo arrays, and Open X-Embodiment inherited whatever its 22 source labs happened to mount. The papers report what was captured, not the decision tree.

This post is that decision tree. We compare the five approaches we quote and build most often at DexSet, with honest pros, cons, and cost ranges from our own rigs, and end with a matrix mapping task types to configurations. For the underlying camera specs, calibration toolchain, and sync engineering, see the pillar guide: The Complete Guide to Exocentric & Multi-View Data for Robot Learning.

Key Takeaways – Five capture approaches dominate: single exo + wrist, DROID-style (2 exo + wrist), ego+exo paired human capture, dense arrays (4-8 cameras), and sim-rendered multi-view. – DROID-style is the default for tabletop manipulation and VLA training data: $4,500 to $7,000 rig, $26 to $38 per operated hour in our benchmarks. – Ego+exo paired capture is the only approach that supports human-video pretraining with cross-view transfer; it costs more in sync engineering than in cameras. – Sim-rendered views are nearly free per view but inherit the sim-to-real gap; they complement real capture, they do not replace it.

The Five Approaches, Defined

A capture approach is the combination of camera count, camera placement, actor type (robot or human), and sync method used to record training episodes. The five that cover almost every real program:

  • Single exocentric + wrist. One fixed external camera plus a wrist camera on the robot. The minimum viable multi-view setup.
  • DROID-style: two exocentric stereo + wrist. Two external stereo cameras (ZED 2i class) at distinct poses plus a wrist camera, calibrated extrinsics, as used across DROID’s 76,000 episodes (arXiv:2403.12945).
  • Ego + exo paired human capture. Glasses or head-mounted camera on a human demonstrator plus stationary exocentric cameras, the Ego-Exo4D pattern (arXiv:2311.18259).
  • Dense array (4-8 cameras). Hardware-triggered ring or arc around the workspace for reconstruction-grade coverage.
  • Sim-rendered multi-view. Arbitrary virtual cameras rendered from simulation, optionally mixed with real data.

Master Comparison Table

Approach Rig Build Capture Cost / Hr Sync Difficulty Occlusion Coverage Human-Video Pretraining Main Risk
Single exo + wrist $2,500 to $4,000 $20 to $28 Low Partial No Blind spots remain
DROID-style (2 exo + wrist) $4,500 to $7,000 $26 to $38 Moderate Good No Calibration upkeep
Ego + exo paired $6,000 to $9,500 $32 to $44 High (moving ego cam) Good Yes Ego-exo time alignment
Dense array (4-8 cams) $9,000 to $12,000 $40 to $50 High (trigger/genlock) Excellent No Storage, diminishing returns
Sim-rendered multi-view Compute only ~$1 to $5 equivalent None Perfect Limited Sim-to-real gap

Rig and capture figures are DexSet benchmarks, including session calibration checks and QA; sim figures are rough GPU-time equivalents.

Where Each Approach Wins and Loses

Single exo + wrist earns its place as a starting point. Pros: cheapest real multi-view, simple calibration (one extrinsic pair), enough to break the wrist-only occlusion ceiling for many tasks. Cons: one blocked view and you are back to single-view; no view redundancy for QA cross-checks. We recommend it for prototyping and single-task policies, and we recommend planning the mount points for camera two on day one.

DROID-style is the workhorse, and not by accident. Two external views mean occlusion of one is usually covered by the other; three total views give the RoboMimic-style observation flexibility that lets ML teams ablate view combinations later (arXiv:2108.03298). Cons: per-session calibration verification becomes mandatory, because three cameras drift three ways. In our operations the added QA overhead is roughly 5 percent of session time. This is what we quote when a VLA team asks for a default.

Ego + exo paired solves a different problem: it is the only configuration that produces the ego-exo correspondences needed to pretrain on human demonstration video and transfer to robot viewpoints, the exact gap Ego-Exo4D was built to close. Pros: human demonstrators are fast and cheap per episode; the data doubles as a bridge to large human-video corpora. Cons: the ego camera moves, so extrinsics to the world frame change every frame and must be recovered via SLAM or the glasses’ own tracking; time alignment between glasses and fixed cameras is the hardest sync problem on this list. Choose it when your training strategy explicitly includes human video.

Dense arrays buy near-complete coverage and reconstruction-grade geometry for humanoid whole-body work and world-model data. The cons compound quietly: hardware triggering or genlock is effectively mandatory, storage runs 3 to 4 TB per capture day at 1080p30 in our pipelines, and, in every ablation we have run on single-arm manipulation, cameras five through eight never moved the success metric. Buy this coverage for reconstruction, not for policy learning on tabletop tasks.

Sim-rendered multi-view costs almost nothing per additional view, which is genuinely useful for view-invariance augmentation and architecture prototyping. But every rendered view inherits the simulator’s gap in contact dynamics, materials, and lighting. Teams in the Open X-Embodiment consortium (arXiv:2310.08864) mix sim and real rather than substituting one for the other, and that matches our experience: sim views stretch a real multi-view dataset, they do not replace it.

Decision Matrix: Match the Approach to the Program

Your Situation Recommended Approach
Prototyping one task, tight budget Single exo + wrist, mounts pre-planned for a second exo
Training VLA / manipulation foundation data at scale DROID-style (2 exo + wrist)
Pretraining on human demonstrations or video Ego + exo paired
Humanoid whole-body, reconstruction, world models Dense array, hardware-triggered
Need view diversity beyond rig budget DROID-style real capture + sim-rendered augmentation

One category the table cannot capture: switching costs. Moving from single-exo to DROID-style mid-program is cheap if the mount points and calibration workflow were planned for it, and painful if they were not, because your existing episodes and your new episodes will differ in geometry and your training pipeline has to reconcile them. Moving from robot-only capture to ego+exo is a bigger jump; it changes your demonstrator pool, your sync architecture, and your annotation scheme at once. Teams that expect to make either move should write the target configuration into their schema now, even if the extra cameras arrive next quarter.

Two cross-cutting rules. First, whatever you choose, log extrinsics and sync metadata into every episode; the approach you pick today is the aggregation problem someone inherits in two years. Second, ablate before you scale: run 20 hours in the candidate configuration, train, and let the success metric pick the rig.

Turn the Matrix Into a Procurement Rubric

If you are scoping a capture program or comparing vendors, download the Multi-View Rig RFP Scorecard. It turns this decision matrix into weighted evaluation questions on calibration verification, sync tolerances, and deliverable formats, the same rubric we hold our own rigs to.

Request a Quote

Frequently Asked Questions

What is the cheapest way to get multi-view robot data?

A single external camera plus a wrist camera, at roughly $2,500 to $4,000 for the rig and $20 to $28 per operated capture hour in DexSet benchmarks. It breaks the wrist-only occlusion ceiling for many tasks but leaves blind spots a second external view would cover.

For VLA training data and tabletop manipulation at scale, usually yes: the second external view covers occlusions the first misses and enables view ablations later. The premium over single-exo is about $2,000 to $3,000 in rig cost and $6 to $10 per hour.

No. Rendered views are nearly free and useful for view-invariance augmentation, but they inherit the simulator’s gaps in contact dynamics, materials, and lighting. Production programs mix sim views with real calibrated capture rather than substituting.

When your training plan includes learning from human demonstration video. Paired capture, as in Ego-Exo4D, provides the cross-view correspondences needed to transfer first-person human priors to third-person robot viewpoints.

Task-dependent, but in our single-arm manipulation ablations, cameras beyond the third stopped moving policy success while adding roughly 25 percent storage and QA cost per view. Dense arrays of 4 to 8 cameras are justified for reconstruction and whole-body humanoid work, not tabletop policies.