Skip to main content

Dexset

dexset
Menu
Request a Demo

Tag: Robotics Data Buyers

Comparing Training Data Approaches for Vision-Language-Action (VLA) Models: Pros, Cons & Costs

Posted on by Sainath Gupta

Zero robot action labels exist in the world’s largest manipulation video corpora. Thousands of hours of humans pouring, folding, and assembling, on YouTube and in Ego4D-scale datasets, and not one recorded motor command among them. That single number is why the VLA data sourcing debate never resolves cleanly: the cheapest, most scalable source of manipulation data is missing the one field an action model trains on, and every source that has the field costs real money per hour.

So the budget conversation inside most teams collapses into familiar camps. Someone wants to teleoperate everything because actions are the labels. Someone wants to generate everything in simulation because it is cheap. Someone read the GR00T paper and wants to pretrain on YouTube. All three are right about the strengths and quiet about the weaknesses, and the budget only covers one of them properly.

Our thesis, argued below with published results and our own collection benchmarks: these approaches are not substitutes, they are layers, with different label completeness, different transfer properties, and per-hour costs that differ by two orders of magnitude. Treating them as interchangeable is how teams end up with 40,000 synthetic episodes that fine-tune into a policy that cannot close a real gripper on a real deformable object.

This post compares the four practical sourcing approaches: teleoperation, human video, synthetic generation, and open datasets. For each you get what it is, where it wins, where it fails, and what it costs, ending with the decision matrix we use when scoping client collections. Costs marked as DexSet benchmarks are from our own production pipeline; the rest cite primary sources.

Key Takeaways – Teleoperation is the only approach that produces complete observation-action-language triples; it costs $28-60 per robot-hour in our benchmarks. – Human video is the cheapest per hour and the most scalable, but has no action labels; it works as a pretraining layer, never as the whole diet. – Synthetic data has near-zero marginal cost and perfect labels, but pays a sim-to-real tax on contact-rich tasks. – Open datasets (Open X-Embodiment, DROID, BridgeData V2) are effectively free and belong in every mixture; their limit is embodiment and domain mismatch. – The winning pattern in published systems is a pyramid: open + web data at the base, synthetic in the middle, embodiment-matched teleoperation on top.

Approach 1: Teleoperation

Teleoperation data is collected by a human operator directly controlling the robot while every observation and command is recorded, making the robot’s own action stream the training label. This is the approach behind ALOHA/ACT (arxiv.org/abs/2304.13705), DROID (arxiv.org/abs/2403.12945), and the top layer of essentially every serious VLA pipeline including pi-zero’s roughly 10,000-hour corpus (arxiv.org/abs/2410.24164).

Pros: complete and exact action labels; embodiment-matched by construction; supports contact-rich and bimanual tasks; language can be attached at collection time while context is fresh.

Cons: real-time throughput ceiling (20-40 usable tabletop episodes per hour on our rigs, 8-15 for long-horizon work); operator skill variance shows up in the data; rigs cost real money (an ALOHA-class bimanual station is around $20k before cameras).

Cost (DexSet benchmark): $28-38 per robot-hour for VR-rig tabletop collection, $40-60 for leader-follower bimanual work, plus $8-15 per data-hour for language annotation and $5-9 for QA.

Approach 2: Human Egocentric Video

Human video data is footage of people performing tasks, captured from head-mounted or fixed cameras, containing rich visual and behavioral signal but no robot action labels. Ego4D-scale corpora and in-house egocentric capture both fall here.

Pros: an order of magnitude cheaper per hour than teleoperation; humans perform tasks at natural speed with natural strategies; enormous diversity is reachable; head-mounted capture rigs are cheap to scale across many wearers.

Cons: no action labels, so it cannot train the action head directly; embodiment gap between human hands and robot grippers; requires retargeting or representation-learning machinery to become useful.

Cost (DexSet benchmark): $15-25 per hour for structured egocentric capture with task scripts and QA, roughly half the cost of teleoperation for the same wall-clock hour.

Where it fits: as pretraining and co-training signal. NVIDIA’s GR00T N1 explicitly structures this as the wide base of its data pyramid (arxiv.org/abs/2503.14734).

Approach 3: Synthetic and Simulation Data

Synthetic data is generated in physics simulators or by generative models, with actions and annotations produced programmatically at near-zero marginal cost. Perfect labels, infinite resets, no hardware wear.

Pros: marginal cost approaches zero after environment setup; perfect ground truth including states no sensor can capture; safe exploration of failure states; trivially parallelizable.

Cons: the sim-to-real gap is worst exactly where manipulation is hardest: contact dynamics, deformables, friction, specular and transparent objects. Environment engineering is a real up-front cost that budget lines often forget. Policies trained purely in sim routinely need real-data fine-tuning anyway.

Cost: environment and asset engineering is the dominant term (weeks of engineer time per task family); generation itself is compute-priced. Treat it as high fixed cost, near-zero marginal cost.

Approach 4: Open Datasets

Open datasets are published robot demonstration corpora, led by Open X-Embodiment with over one million trajectories across 22 embodiments (arxiv.org/abs/2310.08864), DROID with 76k episodes across 564 scenes, and BridgeData V2 with roughly 60k trajectories (arxiv.org/abs/2308.12952).

Pros: free; instantly available; proven at scale (OpenVLA pretrained on ~970k OXE episodes, arxiv.org/abs/2406.09246); increasingly standardized via LeRobot formats.

Cons: your embodiment, cameras, and tasks are probably not in there; instruction quality varies wildly between constituent datasets; license terms need actual reading; everyone else trains on the same data, so it confers no advantage by itself.

Side-by-Side Comparison

DimensionTeleoperationHuman VideoSyntheticOpen Datasets
Action labelsComplete, exactNonePerfect, programmaticComplete, variable quality
Cost per data-hour$28-60 + annotation (DexSet benchmark)$15-25 (DexSet benchmark)Near-zero marginal, high fixedFree
Embodiment matchExactPoor (human)ConfigurableRarely exact
Contact-rich fidelityHighVisual onlyWeak to moderateHigh where present
Scale ceilingLow (real time)HighVery highFixed (~1M episodes)
Language qualityControllable at sourceMust be addedTemplated by defaultInconsistent
Best roleFine-tuning top layerPretraining baseMid-layer augmentationPretraining base

The Decision Matrix We Actually Use

The right mixture follows from two questions: how far is your embodiment from the open corpora, and how contact-rich are your target tasks?

  • Standard single arm, tabletop tasks: open datasets + a thin teleop fine-tuning layer (200-500 hours). Cheapest viable path.
  • Standard arm, contact-rich or precision tasks: add stereo teleop collection on the target tasks; synthetic helps for coverage but will not close the contact gap alone.
  • Dexterous hands or humanoids: cross-embodiment transfer is weakest here. Budget for substantial embodiment-matched teleoperation (1,000+ hours) over a human-video and open-data base.
  • Long-horizon mobile manipulation: prioritize segment-level language labels; Mobile ALOHA showed strong results co-training with roughly 50 demos per task on a broad base (arxiv.org/abs/2401.02117), so spread hours across many tasks.

Two budgeting notes that surprise first-time buyers. First, annotation and QA scale with data-hours, not with approach, so a cheap collection method with poor labels often costs more per useful training hour than an expensive one delivered clean; a $30 teleop hour that ships with diverse language labels and verified success flags beats a $15 hour that needs a relabeling pass and a 20% cull. Second, the approaches interact: a few hundred hours of well-instrumented teleoperation is also the calibration set that makes your synthetic pipeline honest, because it gives you real contact dynamics to validate against. Teams that sequence it that way (teleop first, sim scaled second) report far less sim-to-real disappointment than teams that generate first and verify never.

The pattern across every row: nobody should pay collection rates for the base of the pyramid, and nobody should expect the free layers to cover the top. For the full scoping framework, mixture ratios, and our vendor RFP scorecard, see the complete guide to VLA training data.

Next Step

if you are weighing these approaches for a real budget, read the complete VLA training data guide or book a demo and we will pressure-test your planned mixture against our collection benchmarks.

Request a Quote

Frequently Asked Questions

What is the cheapest way to get training data for a VLA model?

Start with free open datasets (Open X-Embodiment, DROID, BridgeData V2) for pretraining, then buy only embodiment-matched teleoperation for fine-tuning. In our benchmarks that fine-tuning layer costs $28-60 per robot-hour plus annotation, and 200-500 hours is often enough for standard manipulators.

Not alone. Synthetic data provides cheap coverage and perfect labels but carries a sim-to-real gap that is worst on contact dynamics and deformable objects. Published systems use it as a middle layer between web-scale video and real teleoperation.

In DexSet’s pipeline, structured egocentric human video runs $15-25 per hour versus $28-60 per robot-hour for teleoperation. The difference is labels: teleoperation records executable robot actions, human video does not, so they occupy different layers of the training mixture rather than competing.

Both. OXE-scale open data is the standard pretraining base and costs nothing. Custom collection earns its price where open data is thin: your specific embodiment, your camera geometry, your tasks, and instruction phrasing diverse enough to make language conditioning work.

The Complete Guide to Training Data for Vision-Language-Action (VLA) Models (2026)

Posted on by Sainath Gupta

Five billion image-text pairs sit in LAION-5B (arxiv.org/abs/2210.08402); just over one million robot trajectories sit in Open X-Embodiment, the largest open robot dataset ever assembled (arxiv.org/abs/2310.08864). Those two numbers should not both describe the training diet of a single model family, yet every production VLA pairs a vision-language half fed at web scale with an action half starved at roughly one-thousandth of it. That mismatch, more than any architecture choice, is the defining constraint of physical AI in 2026.

The gap exists because the two halves come from different worlds. The vision-language half was pretrained on image-text pairs that already existed. The action half needs robot trajectories that do not exist until someone physically collects them, frame-synchronized, action-labeled, and paired with language instructions. You cannot scrape your way out of that. Which is why, on most VLA programs, the architecture question gets settled in a week and the compute budget in a month, while the question of how many language-annotated trajectories the model needs, at what task diversity, on which embodiments, goes unanswered the longest.

This guide gives you the full picture: what VLA training data is, how the landmark models (RT-1, RT-2, OpenVLA, Octo, pi-zero, GR00T) actually sourced their data, how action representation changes your labeling requirements, what collection and annotation cost per hour, and how to scope a dataset that transfers to your embodiment instead of overfitting to someone else’s lab. The thesis we argue throughout, and the one our delivery data keeps confirming: once you stand on the open-data base layer, dataset composition (task diversity, instruction diversity, capture fidelity) predicts VLA capability better than episode count, so budget should follow distribution, not volume.

DexSet supplies egocentric, exocentric, teleoperation, mono, and stereo data to robot foundation model teams. The numbers in this guide come from our own rigs, QA pipelines, and cost benchmarks, alongside the primary papers. Where a figure is our internal benchmark rather than a published result, we say so.

TL;DR: Key Takeaways – VLA training data consists of robot trajectories (synchronized camera frames + robot actions) paired with natural-language instructions, usually collected via teleoperation. – Landmark scale points: RT-1 used ~130k episodes over 17 months; OpenVLA pretrained on ~970k episodes from Open X-Embodiment; pi-zero reports on the order of 10,000 hours of dexterous robot data. – Action representation drives labeling spec: discrete action tokens (RT-2, OpenVLA) tolerate lower control rates; continuous-action heads (pi-zero flow matching, Octo diffusion) reward 50 Hz, low-jitter capture. – In our benchmarks, teleoperation collection runs $28-60 per robot-hour depending on rig and task complexity; a proper language annotation pass adds $8-15 per hour of data. – Data mixture beats raw volume: co-training on web-scale vision-language data plus a well-balanced robot mixture is what gives VLAs semantic generalization. – Task and scene diversity predict generalization better than episode count once you pass a per-task floor (roughly 50-150 demonstrations per task in the imitation-learning literature).

What Is Training Data for Vision-Language-Action (VLA) Models?

Training data for VLA models is a corpus of robot demonstration episodes in which each timestep pairs visual observations, robot proprioceptive state, and executed actions, and each episode (or segment) carries a natural-language instruction describing the task. A VLA model consumes images and an instruction as input and produces robot actions as output, so its training data must contain all three aligned in time.

A single well-formed episode contains:

  • Observations. One or more camera streams (wrist-mounted, over-the-shoulder exocentric, or head-mounted egocentric), typically 224×224 to 1080p RGB at 10-60 fps, optionally stereo pairs or depth.
  • Proprioception. Joint positions, gripper state, end-effector pose, usually at the control rate.
  • Actions. The command stream that produced the motion: delta end-effector poses, joint targets, or gripper commands, timestamped against frames.
  • Language. An instruction such as “put the red mug on the top shelf,” attached at episode level, and increasingly at sub-episode segment level for long-horizon tasks.

The entity chain matters for anyone mapping this space: teleoperation rigs (ALOHA-style leader-follower arms, VR controllers, exoskeletons) produce demonstrations; demonstrations feed imitation learning; imitation learning at scale, on top of a pretrained vision-language backbone, is what turns a VLM into a VLA. The Hugging Face LeRobot project has become the de facto open format for storing these episodes, and its dataset cards are a good reference for what a clean schema looks like.

How the Landmark VLA Models Sourced Their Data

Every major VLA result is downstream of a specific, documented data strategy, and the strategies differ more than the architectures do. Reading the data sections of these papers teaches you more about building a VLA than reading the model sections.

Model (Year)OrgParamsRobot DataLanguage SourceAction Representation
RT-1 (2022)Google35M~130k episodes, 13 robots, 17 months, ~700 tasksTemplated instructionsDiscrete tokens, 256 bins (arxiv.org/abs/2212.06817)
RT-2 (2023)Google DeepMind12B / 55BRT-1 data, co-fine-tuned with web-scale VLM dataTemplated + web textActions as text tokens (arxiv.org/abs/2307.15818)
Octo (2024)UC Berkeley et al.27M / 93M800k trajectories from Open X-EmbodimentMixed, dataset-dependentDiffusion head, continuous (arxiv.org/abs/2405.12213)
OpenVLA (2024)Stanford et al.7B970k episodes from Open X-EmbodimentMixed OXE instructionsDiscrete tokens (arxiv.org/abs/2406.09246)
pi-zero (2024)Physical Intelligence~3BOn the order of 10k hours, 7 platforms, 68 task familiesNatural + segment-level relabelingFlow matching, 50 Hz continuous (arxiv.org/abs/2410.24164)
GR00T N1 (2025)NVIDIA2BData pyramid: web video + synthetic + real robot dataMixed human + generatedContinuous, diffusion-based (arxiv.org/abs/2503.14734)

Three lessons sit in that table. First, scale climbed from 130k episodes to roughly a million in two years, but the biggest capability jumps (RT-2 over RT-1, pi-zero over Octo-class models) came from mixture changes, not just volume. Second, OpenVLA at 7B outperformed the 55B RT-2-X on the paper’s 29-task evaluation suite, which tells you curated cross-embodiment data can substitute for an order of magnitude of parameters. Third, nobody serious trains on robot data alone anymore; co-training against web-scale vision-language corpora is standard because it is what keeps semantic knowledge (what “the leftmost ripe banana” means) alive while the model learns motor control.

Core Data Modalities for VLA Training

Modality choice determines what your VLA can learn and what your collection rig must capture. The main axes are viewpoint (egocentric vs exocentric), depth strategy (mono vs stereo), and collection method (teleoperation vs human video).

Egocentric vs Exocentric Views

Egocentric data is captured from the robot’s or demonstrator’s own viewpoint; exocentric data is captured from external cameras watching the workspace. Wrist and head cameras give the policy view-invariant, occlusion-realistic inputs and tend to transfer better across scene layouts. Exocentric views give annotators and QA reviewers the context to verify task success and label segments. In our pipelines we treat exocentric streams as partly a QA instrument: an episode with only a wrist view is roughly twice as expensive to verify because reviewers must scrub proprioception logs to confirm success.

Mono vs Stereo

Mono capture records a single RGB stream per viewpoint; stereo capture records calibrated pairs that support depth estimation. Most published VLAs train on mono RGB because the backbone VLMs are mono. Stereo earns its extra cost on contact-rich and clutter-heavy manipulation, where depth ambiguity is a dominant failure mode. Our rule of thumb: mono for pretraining breadth, stereo for the fine-tuning set on your target tasks.

Teleoperation vs Human Video

Teleoperation produces the gold standard because the action stream is the robot’s own. Leader-follower rigs like ALOHA (arxiv.org/abs/2304.13705) hit high dexterity at a hardware cost around $20k per station; VR-based rigs are cheaper and faster to train operators on but add pose-retargeting noise. Human egocentric video (Ego4D, EgoExo4D scale) has no action labels, so it enters VLA training as a co-training or pretraining signal, the approach GR00T N1 formalizes as a data pyramid. Human video buys you visual and behavioral priors cheaply; it never replaces the teleop layer where actions live.

Action Representation: Tokenization vs Continuous Actions

Action representation is the choice of how robot actions are encoded for the model to predict, and it silently rewrites your data spec. There are two dominant families.

Discrete action tokenization (RT-1, RT-2, OpenVLA) bins each action dimension, typically into 256 buckets, and treats action prediction as next-token prediction. It plugs directly into a language-model backbone and tolerates modest control rates. The FAST tokenizer (arxiv.org/abs/2501.09747) pushed this further with a compression-based scheme that makes autoregressive VLAs viable on high-frequency data.

Continuous action heads (Octo’s diffusion head, pi-zero’s flow matching) regress action chunks directly. They excel at dexterous, high-rate control (pi-zero runs 50 Hz) but they punish sloppy capture. Timestamp jitter between frames and actions that a tokenized model shrugs off will smear a flow-matching target distribution.

What this means for data buyers, from our QA logs:

  • For tokenized targets, 10-15 Hz aligned capture is usually sufficient; invest the savings in task diversity.
  • For continuous heads, capture at 30-60 Hz with hardware-timestamped sync; we hold frame-to-action skew under 10 ms on our teleop rigs because beyond that we can measure it in fine-tune loss.
  • Action-chunk prediction (introduced with ACT on ALOHA) means episodes must be free of teleoperator pauses mid-chunk, which is a QA check most teams add only after their first bad fine-tune.

Language Annotation: The Layer Everyone Underestimates

Language annotation is the process of attaching instructions to robot episodes, and it is where VLA datasets most often quietly fail. A VLA is only as language-conditioned as its labels are diverse.

Templated vs natural instructions. RT-1-era datasets used templates (“pick X”, “move X near Y”). Templates are cheap and consistent, and they teach the model a grammar of about a dozen verbs. Models trained on templated instructions follow paraphrases poorly; the semantic generalization RT-2 demonstrated came from the web co-training, not the robot labels. Current practice, visible in the pi-zero and DROID (arxiv.org/abs/2403.12945) pipelines, is natural free-form instructions plus paraphrase augmentation.

Episode-level vs segment-level labeling. Episode-level labeling attaches one instruction to a whole demonstration. Segment-level labeling breaks a long-horizon episode into sub-tasks (“open the drawer”, “place the spoon inside”, “close the drawer”) with aligned timestamps. Segment labels cost more but unlock long-horizon training and better credit assignment. In our pipeline, segment-level labeling costs roughly 2.5x the episode-level pass because annotators must scrub video and mark boundaries, not just watch and describe.

Our first-hand benchmark: a language annotation pass adds $8-15 per hour of robot data depending on instruction density and whether we run single or dual annotation. We also run an instruction diversity QA gate: across any 1,000-episode batch we require minimum thresholds for unique verbs, unique object references, and spatial-relation phrases, because we have watched fine-tunes on label-poor batches lose the ability to follow anything but the majority phrasing.

Data Mixtures, Co-Training, and Cross-Embodiment Transfer

A data mixture is the weighted combination of datasets a VLA trains on, and mixture design is now a first-class engineering discipline. Three decisions dominate.

Web co-training ratio. RT-2 established that co-fine-tuning on web vision-language data alongside robot data preserves semantic breadth. Teams that fine-tune on robot data alone consistently report the model “forgetting” open-vocabulary grounding.

Cross-embodiment weighting. Open X-Embodiment showed that pooling data across 22 robot types improves average success, and OpenVLA and Octo both hand-tuned per-dataset mixture weights rather than sampling uniformly. Transfer across embodiments is real but not free: it appears strongest between similar morphologies (single-arm manipulators with parallel grippers) and weakest into dexterous hands and humanoids, which is exactly why humanoid programs still commission embodiment-matched collection.

There is no published universal mixture ratio, and be suspicious of anyone who quotes one. What the papers do agree on is the shape of the curve: heavily downweight any single dataset that would dominate by timestep count, keep a meaningful web co-training fraction throughout fine-tuning rather than dropping it to zero, and upweight the small datasets that match your target embodiment and camera geometry. In practice we see teams land between 10% and 40% embodiment-matched data by timestep in their final fine-tuning mixtures, with the exact point set by held-out evaluation rather than by principle. Measure mixtures in timesteps or tokens, not episodes; episode lengths vary enough between sources that episode-level accounting quietly misstates composition by 2-3x.

Diversity vs depth. Once a task passes a demonstration floor (Mobile ALOHA showed strong results co-training with as few as 50 demonstrations per task on top of a broad base, arxiv.org/abs/2401.02117), the marginal episode of task 12 is worth less than the first episode of task 13. When we rebalance client collections, we usually shift budget from episode count toward scene, object, and instruction diversity, and the downstream evaluations reward it.

Cost and Economics: What VLA Training Data Actually Costs

VLA data cost is driven by four stacked line items: rig amortization, operator time, annotation, and QA. Vendors who quote a single opaque per-episode price are hiding this stack. Here is ours, from DexSet production benchmarks (round figures, typical ranges we see):

Cost ComponentTypical RangeNotes
Teleop collection (VR rig, tabletop tasks)$28-38 per robot-hourIncludes operator, station amortization, resets
Teleop collection (leader-follower, bimanual/contact-rich)$40-60 per robot-hourALOHA-class rigs, slower resets, higher operator skill
Language annotation, episode-level$8-10 per data-hourNatural instructions, single pass
Language annotation, segment-level + paraphrases$12-15 per data-hourTimestamped sub-task boundaries, dual pass on 10%
QA and success verification$5-9 per data-hourSuccess labels, sync checks, instruction diversity gate
Useful episodes per collection hour20-40 tabletop; 8-15 long-horizonAfter discarding failed/aborted episodes

Worked example: a 2,000-hour bimanual manipulation corpus with segment-level language and full QA lands between $114k and $168k. Against that, note that pi-zero-class results were built on roughly 10,000 hours. This is why mixture design and cross-embodiment pretraining matter commercially: nobody fine-tuning a foundation model should be paying to recollect the base layer that OXE-scale open data already provides. Pay for the embodiment-matched, task-matched, well-labeled top of the pyramid.

Case Study Proof: Scaling a Fine-Tuning Corpus for a Humanoid Team

A humanoid foundation model team came to us with an OpenVLA-style base model and a fine-tuning problem: strong tabletop scores, weak performance on their bimanual kitchen tasks, and instructions that only worked when phrased like the training templates. We scoped a 1,400-hour teleoperation collection across 42 task families with a hard instruction-diversity requirement (no template allowed to exceed 8% of a batch) and segment-level labels on all long-horizon episodes.

The mixture, not the volume, did the work. Their prior corpus had 3x our episode count but 9 task families and 14 instruction templates. After fine-tuning on the new corpus blended 60/40 with their legacy data, their internal evaluation showed task success on held-out kitchen tasks roughly doubling, and paraphrased-instruction success moved from near-random to parity with template phrasing. We report those as directional results from the client’s own evaluation harness, not audited benchmarks. The full breakdown is in our case study write-up.

Scoping Your Own Collection: The RFP Scorecard

A VLA data RFP scorecard is a structured checklist for comparing data vendors on the dimensions that predict fine-tune quality. If you take one action from this guide, make vendors answer these in writing:

  • Frame-to-action sync tolerance, in milliseconds, with measurement method.
  • Control and capture rates, and whether timestamps are hardware or software.
  • Instruction sourcing: templated, natural, or paraphrase-augmented, with a diversity metric per batch.
  • Episode-level vs segment-level labeling, priced separately.
  • Success-labeling protocol and inter-annotator agreement.
  • Cost per robot-hour, broken into collection, annotation, and QA. Refuse bundled per-episode pricing.
  • Delivery format (LeRobot-compatible schemas save your team weeks).

We publish ours as a downloadable template. [Download the VLA Data RFP Scorecard] and use it against us too.

Related reading this week:

Next Step

If you are scoping a VLA data collection this quarter, two options. Book a demo and we will walk through our teleop rigs, annotation pipeline, and the instruction diversity QA gate live. Or download sample data: a free LeRobot-format sample pack of bimanual teleop episodes with segment-level language labels, so your team can inspect sync quality before you talk to anyone.

Request a Quote

Frequently Asked Questions

What is training data for vision-language-action (VLA) models?

It is a corpus of robot demonstration episodes pairing synchronized camera observations and robot actions with natural-language instructions. VLA models train on it, usually via imitation learning on top of a pretrained vision-language backbone, to map images plus an instruction to robot actions.

Pretraining-scale efforts use hundreds of thousands of episodes (OpenVLA used ~970k from Open X-Embodiment; pi-zero reports on the order of 10,000 hours). Fine-tuning an existing VLA to a new embodiment or task set typically needs far less: the literature shows meaningful results from 50-150 demonstrations per task on top of a broad base, with 500-2,000 hours being a common commercial fine-tuning corpus.

In DexSet’s benchmarks, teleoperation collection runs $28-60 per robot-hour depending on rig and task complexity, language annotation adds $8-15 per data-hour, and QA adds $5-9. A 2,000-hour fully annotated corpus typically lands between $110k and $170k.

Not for most pretraining, since mainstream VLA backbones consume mono RGB. Stereo or depth earns its cost on contact-rich, cluttered, or precision tasks in your fine-tuning set, where mono depth ambiguity is a leading failure mode.

Only partially. Human egocentric video has no robot action labels, so it serves as a pretraining or co-training signal (the GR00T N1 data pyramid approach), improving visual and behavioral priors. The action-labeled teleoperation layer remains mandatory for the policy to output executable control.

Tokenized VLAs (RT-2, OpenVLA) discretize each action dimension into bins and predict actions as tokens, which suits language-model backbones and moderate control rates. Continuous-action VLAs (pi-zero with flow matching, Octo with diffusion) regress action chunks directly, enabling 50 Hz dexterous control but demanding tighter timestamp sync in the training data.

The Complete Guide to Data Capture Hardware for Embodied AI (2026)

Posted on by Sainath Gupta

A Head of Data on a scoping call last quarter asked us which single capture device her team should standardize on, and for the first time in months, we had no quick answer. We had run every device she named in production. The problem was the question: it has no device-shaped answer, because a capture device is not a purchase, it is a commitment. The hardware you choose locks your pipeline into what that device can timestamp, sync, and survive, and teams that fail on data usually failed at that moment, months before anyone touched a model. That is this guide’s thesis: capture hardware is a data-strategy decision, judged on seven axes and on cost per delivered hour, never on spec sheets.

The reason the question stumps buyers is that capture hardware for embodied AI sits in a gap between consumer electronics and lab equipment. A Quest 3 was built for gaming, not for exporting synchronized RGB, IMU, and hand pose at training quality. A RealSense was built for prototyping, not for 40 units running ten hours a day. Vendor spec sheets answer none of the questions that matter for foundation model training, so teams learn the hard lessons after the purchase order clears.

This guide gives you the full picture: every major hardware category (headsets, research glasses, action cameras, stereo depth cameras, teleoperation rigs, handheld grippers, tactile sensors), compared on the seven axes that decide whether your dataset trains a VLA model or fills a hard drive. You will leave with real prices, a cost-per-hour model, and a decision matrix mapped to the three capture strategies that matter: egocentric human data, teleoperation, and multi-view exocentric capture.

At DexSet we run these devices in production. Our teams operate teleop cells, headset fleets, and multi-camera capture studios that deliver egocentric, exocentric, mono, and stereo data to robotics foundation model teams. Every number in this guide comes from our own rigs, published research, or a linked primary source.

TL;DR: Key Takeaways – Data capture hardware for embodied AI spans four categories: wearable egocentric devices ($350 to $3,499), stereo depth cameras ($250 to $500 per unit), teleoperation rigs ($300 per arm to $20,000+ per station), and tactile sensors. – The seven evaluation axes are price, resolution/fps, depth quality, sync capability, wearability, SDK maturity, and data formats. Sync capability is the axis teams most often ignore and most often regret. – A Meta Quest 3 at ~$500 covers most teleoperation UI needs; the Apple Vision Pro at $3,499 earns its price only for eye tracking and high-resolution passthrough research. – Teleop capex is lower than most CTOs assume: a GELLO leader arm costs roughly $300 per arm, against $20,000+ for a full ALOHA-class bimanual station. – In our production benchmarks, delivered teleop data runs $28 to $60 per hour depending on rig and task complexity; egocentric human data runs meaningfully cheaper per hour but trains different capabilities.

What Is Data Capture Hardware for Embodied AI?

Data capture hardware for embodied AI is the set of physical devices, wearables, cameras, teleoperation interfaces, and sensors, used to record the visual, spatial, inertial, tactile, and action data that trains robots and vision-language-action (VLA) models. It is the acquisition layer of the robot learning stack: everything between the physical world and your training dataset.

The category matters because embodied AI models learn from demonstrations, not from text scraped off the internet. A VLA model like OpenVLA (arxiv.org/abs/2406.09246) or Physical Intelligence’s pi-0 (arxiv.org/abs/2410.24164) consumes synchronized streams of camera frames, proprioception, and actions. The hardware that records those streams sets a hard ceiling on model quality. No amount of post-processing recovers depth a sensor never measured or a timestamp a device never wrote.

Core Concepts and Data Modalities

Data modalities are the distinct sensor streams a capture device records, and every hardware decision starts by listing which modalities your model actually needs. The main ones for embodied AI are RGB video, stereo pairs, depth, IMU, eye gaze, hand and body pose, robot proprioception, and tactile signal.

Mono vs. Stereo RGB

Mono RGB is a single camera stream; stereo RGB is a calibrated pair that allows depth to be recovered by triangulation. Mono is cheaper and lighter and dominates large egocentric datasets like Ego4D (arxiv.org/abs/2110.07058). Stereo gives your model, or your post-processing pipeline, metric 3D structure, which matters for manipulation policies that must reason about distance to a grasp point. On our rigs, stereo roughly doubles storage and calibration workload, so we specify it only when the downstream model consumes it.

Depth

Depth is a per-pixel distance measurement, produced either by active sensors (structured light, time-of-flight) or by stereo matching. Active depth from a RealSense-class sensor works well indoors at 0.3 to 6 meters but degrades in sunlight and on reflective surfaces. Neural stereo depth (ZED 2i class) extends range but requires a GPU at capture time or in post.

IMU and Time Synchronization

An IMU (inertial measurement unit) records acceleration and angular velocity, typically at 200 Hz to 1,000 Hz, and it is only useful if its clock aligns with the camera clock. Cross-device sync is the silent killer of embodied AI datasets: a 40 ms offset between video and proprioception is invisible in playback and fatal for action-labeled training data. Devices differ enormously here. Project Aria timestamps all sensors on a shared clock; a GoPro gives you a self-consistent file but no native way to sync with a robot’s ROS 2 clock without an external signal.

Proprioception and Action Data

Proprioception is the robot’s own joint state (positions, velocities, torques), and action data is the command stream sent to the robot. These come from the robot or the teleoperation interface, not from cameras, which is why teleop rigs are capture hardware in their own right. Imitation learning methods like ACT, introduced with the ALOHA rig (arxiv.org/abs/2304.13705), train directly on this stream. The entity chain to keep straight: teleoperation produces demonstrations, demonstrations feed imitation learning, imitation learning trains the VLA policies that run on robots.

Tactile

Tactile sensing records contact geometry and force at the gripper, usually via vision-based sensors in the GelSight style, where a camera watches a deformable gel pad. Tactile is the newest modality in production pipelines and the least standardized; we cover the hardware below.

The Four Hardware Categories

Capture hardware for embodied AI falls into four categories: wearable egocentric devices, stereo and depth cameras for exocentric capture, teleoperation rigs, and handheld or tactile interfaces. Most production pipelines combine at least two.

Category 1: Wearable Egocentric Devices

Wearable egocentric devices are head-mounted cameras and headsets that record the world from a human’s first-person viewpoint, the same viewpoint a humanoid robot’s head camera will have. This category powers egocentric pretraining datasets like Ego-Exo4D (arxiv.org/abs/2311.18259) and doubles as the operator interface for immersive teleoperation systems like Open-TeleVision (arxiv.org/abs/2407.01512).

Device Price Sensors & Resolution Depth Sync Wearability (8h shift) SDK Maturity Native Data Formats
Meta Quest 3 ~$500 Color passthrough cameras, 6DoF inside-out tracking, hand tracking, IMU Passthrough depth estimate, not export-grade Good within headset; WebXR/OpenXR timestamps for teleop Moderate; 515 g, front-heavy after ~2h Strong (OpenXR, Meta SDK, large dev community) App-level capture; pose + hand joints via OpenXR
Apple Vision Pro $3,499 High-res micro-OLED displays, high-quality color passthrough, eye tracking, hand tracking, LiDAR LiDAR scene depth Excellent within device; ARKit timestamps Poor for long shifts; 600-650 g, external battery Strong but restrictive (visionOS, ARKit); raw camera access gated ARKit poses, hand skeletons; limited raw sensor export
Project Aria / Aria Gen 2 Research access (not retail) RGB camera, 2 mono SLAM cameras, eye tracking cameras, spatial mics, dual IMUs (Gen 2 adds PPG, improved compute) No active depth; SLAM-derived Best in class; all sensors on shared clock, VRS format Excellent; ~75 g glasses form factor Good (Project Aria Tools, MPS cloud services) VRS files, MPS trajectories, gaze (projectaria.com)
GoPro head mount (Hero-class) ~$400 Single wide FOV RGB, 5.3K/60 or 4K/120, IMU telemetry None Weak across devices; GPMF telemetry internal only Good; ~150 g with mount Moderate (GPMF parsers, no live SDK) MP4 + GPMF metadata track

The pattern to notice: the $3,499 device is not seven times better than the $500 device for data capture. Vision Pro wins when you need eye gaze and the highest-fidelity passthrough for perception research or premium teleop. Quest 3 wins on fleet economics: at our scale, one damaged Vision Pro costs the same as seven Quest 3 replacements. Aria wins on data quality per gram but is research-access only, which makes it a partnership decision rather than a purchasing decision. The GoPro remains the workhorse for raw egocentric video volume, and it is the camera inside the UMI gripper stack discussed below.

Category 2: Stereo Depth Cameras for Exocentric Capture

Stereo depth cameras are fixed or tripod-mounted sensors that record a scene from outside the actor, providing the third-person views used in multi-view exocentric datasets and in robot workspace monitoring. Datasets like DROID (arxiv.org/abs/2403.12945) standardized on this class of sensor.

Device Price RGB / Depth Resolution & FPS Depth Quality Sync Capability SDK Maturity Data Formats
Intel RealSense D435i ~$300-350 Depth 1280x720 @ up to 90 fps; RGB 1080p @ 30; IMU Good 0.3-3 m; global-shutter depth sensors Hardware sync pin across units Mature (librealsense, ROS 2 wrapper) ROS 2 bags, .bag/.ply, raw streams
Intel RealSense D455 ~$420-450 Depth 1280x720 @ 90 fps; wider 95 mm baseline; global-shutter RGB; IMU Better than D435i at 4-6 m range Hardware sync pin Mature (same stack) Same as D435i
Stereolabs ZED 2i ~$500 Stereo 2K @ 15, 1080p @ 30, 720p @ 60; IMU, barometer, magnetometer Neural stereo depth, usable 0.3-20 m Software sync; multi-cam via ZED Hub Mature but GPU-dependent (ZED SDK, needs NVIDIA) SVO recordings, ROS 2 wrapper
Luxonis OAK-D ~$250-350 4K RGB + 2x mono stereo pair; on-device compute Good short-range stereo depth Hardware FSYNC on some variants Growing (DepthAI, ROS 2 driver) DepthAI streams, ROS 2

For multi-view exo capture, the deciding axis is sync. RealSense’s hardware sync pin lets you drive eight cameras off one trigger, which is why it still dominates academic capture cells despite Intel’s on-again-off-again commitment to the product line. ZED 2i buys you longer range and better outdoor behavior at the cost of an NVIDIA GPU per capture node. OAK-D is the value play when you want on-device neural inference to filter or annotate at capture time.

Category 3: Teleoperation Rigs

Teleoperation rigs are the leader-follower arm pairs, exoskeletons, and VR-based control systems through which a human operator produces robot action data. This is the category where hardware choice most directly sets your cost per demonstration.

System Price What It Is Action Fidelity Operator Fatigue SDK / Ecosystem Data Formats
ALOHA / ALOHA 2 rig ~$20,000+ per bimanual station Two leader + two follower arms, joint-space mapping, 50 Hz High; fine bimanual tasks Moderate; seated, arms supported Strong research ecosystem (ACT, Mobile ALOHA, LeRobot support) HDF5 episodes; LeRobot dataset format
GELLO ~$300 per arm (plus follower robot) 3D-printed, Dynamixel-based kinematic-twin leader arm (arxiv.org/abs/2309.13037) High for the matched follower Low-moderate Open source, LeRobot-compatible Joint streams to ROS 2 / LeRobot
UMI gripper ~$400 build (printed gripper + GoPro) Handheld gripper with wrist-mounted fisheye GoPro; robot-free demonstrations (arxiv.org/abs/2402.10329) Good; SLAM-recovered actions, no proprioception Low; natural human motion Open source, active community MP4 + GPMF, post-processed to episodes
VR teleop (Quest 3 / Vision Pro + arms) $500-$3,499 + robot Headset streams head/hand pose to robot controller (Open-TeleVision style) Good; depends on retargeting Moderate; passthrough latency drives fatigue Growing; OpenXR + custom stacks Pose streams + robot logs
Tactile (DIGIT / GelSight-style) A few hundred dollars per fingertip (DIGIT-class); research-grade GelSight units cost thousands Vision-based tactile sensor in gripper fingertips Adds contact-rich signal N/A (robot-mounted) Moderate (open DIGIT designs; PyTouch-era tooling) Compressed tactile image streams

The $20,000 ALOHA figure and the $300 GELLO figure describe different points on the same curve, not competing products. ALOHA is a complete bimanual station including four arms. GELLO is a leader-arm design you bolt onto follower robots you already own. UMI removes the robot from data collection entirely, which is why it produces the cheapest demonstrations and the weakest proprioception. Every serious pipeline we run mixes at least two of these.

Category 4: Multi-View Exocentric Studios

A multi-view exocentric studio is an array of four to twelve synchronized cameras surrounding a workspace, recording the same manipulation from multiple third-person angles. Ego-Exo4D paired Aria glasses with four to five stationary GoPros per scene precisely because models trained on both viewpoints generalize better than either alone. Budget $2,500 to $6,000 per studio in cameras and mounts, then double it for the sync, lighting, and storage infrastructure that spec sheets never mention.

Decision Matrix: Which Hardware for Which Use Case

The right hardware follows from the capture strategy, and there are three dominant strategies: egocentric human data, teleoperation, and multi-view exocentric capture.

Use Case First Choice Budget Alternative Premium Option Avoid
Egocentric human data at scale (pretraining) GoPro head mounts (~$400/operator) Used Hero-class cameras Project Aria via research partnership Vision Pro (cost, battery, comfort)
Egocentric with gaze + trajectories Project Aria / Aria Gen 2 Quest 3 with logging app Aria Gen 2 GoPro (no gaze, no shared clock)
Immersive teleoperation UI Quest 3 (~$500) Quest 3 Vision Pro (eye tracking, passthrough fidelity) GoPro (not a display)
Bimanual fine manipulation demos ALOHA-class station (~$20k) GELLO leaders on existing arms (~$300/arm) ALOHA 2 with tactile fingertips UMI (no bimanual joint coupling)
Cheap single-arm demonstrations at volume UMI grippers (~$400 each) GELLO + one follower arm ALOHA station Vision Pro teleop (capex per seat)
Multi-view exo studio 4-8x RealSense D455, hardware-synced OAK-D array ZED 2i array with GPU nodes Unsynced consumer webcams
Contact-rich tasks DIGIT-class fingertips on any rig above Single tactile finger + RGB GelSight-style research sensors Skipping tactile, then labeling contact by eye

Cost, Economics, and ROI

Capture hardware economics are dominated by throughput and operator time, not by device price, so the correct unit of analysis is cost per delivered data hour. This is the section competitors leave out, so here are the numbers we actually use.

Capex per seat, typical ranges:

  • Egocentric human capture seat: $400 to $700 (camera, mount, spare batteries, storage cards)
  • VR teleop seat: $1,200 to $5,500 (headset, workstation, networking; excludes the robot)
  • GELLO teleop seat: $600 to $1,000 on top of an existing follower arm
  • ALOHA-class bimanual cell: $22,000 to $30,000 fully commissioned
  • Exo studio (6 cameras): $2,500 to $6,000 in sensors, roughly the same again in sync, mounting, lighting, and storage

Cost per delivered hour, our production benchmarks: teleoperation data lands at $28 to $60 per hour depending on rig type, task complexity, and QA depth. Egocentric human data typically lands at one third to one half of that, because a human wearing a camera needs no robot, no retargeting stack, and far less resets-per-hour overhead. These are the typical ranges we see across our own pipelines; treat them as planning numbers, not quotes.

The ROI logic is straightforward. A $20,000 ALOHA cell running two shifts amortizes to under $2 per data hour within a year; the operators cost ten times that. Buying cheaper hardware that cuts operator throughput by 20 percent is a false economy. Buying $3,499 headsets when $500 ones sustain the same demonstrations per hour is the opposite mistake.

Case Study: A Humanoid Foundation Model Team’s Capture Stack

The proof is in a deployment, so here is an anonymized configuration from our own work. A humanoid foundation model team came to us needing three data types simultaneously: egocentric human video for pretraining, bimanual teleop demonstrations for fine-tuning, and multi-view exo recordings for evaluation.

We stood up three lines. Line one: 30 GoPro head-mount seats for kitchen and warehouse egocentric capture, delivering several hundred hours per week. Line two: four ALOHA-class stations plus six GELLO-equipped single-arm cells, producing task-labeled demonstrations in LeRobot format (github.com/huggingface/lerobot). Line three: a six-camera RealSense D455 studio, hardware-synced, wrapped around one teleop cell for paired ego-exo evaluation episodes.

Two hardware lessons generalized. First, the D455’s wider baseline halved our depth QA rejections at 2 to 4 meters compared with the D435i units we started with. Second, moving VR teleop from Vision Pro pilots to Quest 3 fleet units cut capex per seat by 85 percent with no measurable drop in demonstrations per operator-hour on pick-and-place classes of tasks. The Vision Pros stayed in service for the subset of tasks where gaze labels earned their cost.

Get the RFP Scorecard

A hardware decision this size deserves a structured evaluation, so we built one you can reuse. The DexSet Data Capture Hardware RFP Scorecard is a one-page matrix that scores any device or vendor on the seven axes in this guide (price, resolution/fps, depth quality, sync, wearability, SDK maturity, data formats) plus delivery terms and QA guarantees. Teams use it to compare quotes on equal footing instead of vendor spec sheets. Download it below, and if you want the comparison done for you, that is literally our job.

Related reading from this series:

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

What is data capture hardware for embodied AI?

Data capture hardware for embodied AI is the set of wearable cameras, headsets, depth sensors, teleoperation rigs, and tactile sensors used to record the synchronized visual, inertial, action, and contact data that trains robot policies and vision-language-action models.

Only for specific needs. The Vision Pro’s eye tracking and passthrough fidelity justify its $3,499 price for gaze-labeled data and premium teleoperation research. For fleet teleop and general egocentric capture, the ~$500 Quest 3 delivers comparable demonstrations per operator-hour at one seventh the capex.

Between $300 and $30,000 depending on class. A GELLO leader arm costs roughly $300 per arm on top of a follower robot you already own; a UMI handheld gripper builds for about $400; a complete ALOHA-class bimanual station runs $20,000 or more fully commissioned.

Egocentric capture records from the actor’s first-person viewpoint using head-mounted devices; exocentric capture records the actor from external, third-person cameras. Datasets like Ego-Exo4D pair both because policies trained on combined viewpoints generalize better.

Because embodied AI models train on aligned observation-action pairs. A clock offset of tens of milliseconds between camera, IMU, and robot proprioception silently corrupts action labels, and it cannot be fixed after the fact without shared timestamps or an external sync signal.

For hardware-synchronized multi-view rigs, the Intel RealSense D455 (~$420-450) is the default choice because of its sync pin, 95 mm baseline, and mature ROS 2 support. The ZED 2i suits longer-range or outdoor scenes if each node has an NVIDIA GPU.

DexSet already runs the rigs in this guide at production scale. [Download the RFP Scorecard] to evaluate vendors on equal terms, or [Book a Demo] to see sample egocentric, teleop, and multi-view datasets from these exact devices. [Download Sample Data] is one click away.

Teleoperation Data Collection for Robot Learning: The Complete 2026 Guide

Posted on by Sainath Gupta

Midway through a humanoid data program last year, our cost per usable episode rose almost 40 percent in a single week, and for two days nobody could explain it. Every rig passed its health checks. The QA gates were green. The answer had nothing to do with hardware or pipelines: two trained operators had left, their replacements were producing at novice rates, and the ramp math did the rest. Papers make demonstration collection look like a solved problem. In production, it is a staffing, hardware, and QA operation that most ML teams have never run before.

That week is this guide in miniature, and it carries the thesis the whole page argues: teleoperation data collection is an industrial operation, not an ML task, and it succeeds or fails on operational levers. It is part hardware engineering (leader arms, cameras, time sync), part human operations (hiring, training, and retaining operators), and part data engineering (formats, QA, versioning). Underestimate any one leg and the dataset comes out late, expensive, or unusable.

This guide covers the full stack: what teleoperation data collection actually is, the five main rig approaches and what they cost, the economics of operator labor, how demonstrations turn into trained policies, and the QA thresholds that separate usable datasets from expensive noise. The numbers on operator throughput, cost per hour, and rejection rates come from our own collection floors; the hardware and algorithm claims come from the primary literature (ALOHA, GELLO, UMI, DROID, Open-TeleVision), cited throughout.

TL;DR – Teleoperation data collection is the process of recording synchronized robot actions and sensor observations while a human remotely controls the robot through a task, producing demonstrations for imitation learning. – Five rig families dominate: leader-follower arms (ALOHA, ~$20k bimanual), low-cost kinematic leaders (GELLO, roughly $300 per arm), VR controllers/headsets (Quest 3, Vision Pro), 6-DoF desktop devices (SpaceMouse), and robot-free handheld capture (UMI). – Realistic all-in operator cost runs $28 to $60 per teleop hour. Novice operators produce 8 to 12 usable episodes per hour; trained operators reach 25 to 40 within two to three weeks. – Expect 10 to 30 percent QA rejection. Budget for it up front or your episode targets will slip. – 50 clean demonstrations trained with ACT reached 80 to 90 percent success on fine bimanual tasks in the original ALOHA work. Quality compounds; volume alone does not.

What Is Teleoperation Data Collection?

Teleoperation data collection is the process of recording a robot’s proprioceptive states, actions, and camera observations while a human operator remotely controls the robot through a task, producing time-aligned demonstration episodes used to train imitation learning policies. Each episode is a synchronized log: joint positions and velocities, commanded actions, gripper state, and multiple camera streams, typically stored in HDF5 or the LeRobot dataset format (github.com/huggingface/lerobot).

The entity chain matters for anyone mapping this space. Teleoperation produces demonstrations. Demonstrations feed imitation learning algorithms such as ACT (Action Chunking with Transformers, arxiv.org/abs/2304.13705) and Diffusion Policy (arxiv.org/abs/2303.04137). Those policies, scaled across tasks and embodiments, become the training substrate for vision-language-action (VLA) models. Open X-Embodiment aggregated over one million trajectories across 22 robot embodiments largely from teleoperated sources (arxiv.org/abs/2310.08864). No teleop data, no VLA.

A useful mental model: a teleop episode is worth exactly as much as its worst-synchronized stream. A 50 Hz action log paired with a camera that drifted 80 ms out of alignment teaches the policy that objects move before the hand does. That single failure mode wastes more training budget than any other we see.

Core Concepts and Data Modalities

A teleoperation dataset is defined by four things: the action space, the observation streams, the control and capture rates, and the episode structure. Get these right before collecting a single demo, because retrofitting a format after 10,000 episodes is miserable.

Action Space and Leader-Follower Mapping

Leader-follower joint mapping is the control scheme in which a human moves a small “leader” arm and a “follower” robot arm mirrors those joint positions in real time. ALOHA uses this directly: two WidowX-class leader arms drive two ViperX 300 followers, joint to joint, which gives operators immediate kinesthetic intuition and avoids inverse kinematics singularities (arxiv.org/abs/2304.13705). GELLO takes the same idea and rebuilds the leader from hobby-grade Dynamixel servos and printed parts for roughly $300 per arm (arxiv.org/abs/2309.13037). VR and SpaceMouse rigs instead command end-effector poses and rely on an IK solver, which is cheaper but introduces its own artifacts near joint limits.

Observation Streams: Mono, Stereo, Wrist, and Overhead

Observation streams are the camera and sensor feeds recorded alongside actions, and their placement determines what the policy can actually learn. The standard manipulation setup is one or two exocentric (scene) cameras plus one wrist camera per arm. Stereo pairs or depth add geometric grounding for tasks with occlusion. Our default rig records three RGB streams at 640×480, 30 fps, plus joint states at the control rate. Egocentric head cameras matter for humanoid teleop, where systems like Open-TeleVision stream an actively controlled head view into the operator’s headset (arxiv.org/abs/2407.01512).

Control Rate vs Camera Rate: The 50 Hz / 30 fps Problem

The rate mismatch problem is the timing gap created when actions are logged at 50 Hz but cameras capture at 30 fps, leaving no camera frame for two out of every five action steps. ACT-style pipelines handle this by training on the nearest or interpolated frame and predicting action chunks (sequences of 50 to 100 future actions) rather than single steps, which smooths over per-step jitter. What the algorithm cannot fix is unsynchronized clocks. Hardware-timestamp every stream at the source, log in one process or use a shared monotonic clock, and record the offsets. In our QA pipeline, any episode with inter-stream skew above 33 ms (one camera frame) is auto-flagged, and it is the single most common rejection reason on new rigs.

Episode Structure and Task Definition

An episode is one complete attempt at a defined task, from a randomized initial state to a labeled terminal state. Discipline here is what makes a dataset trainable: written task cards, explicit reset procedures, randomized object poses within a marked region, and a per-episode success label assigned at collection time. Datasets that skip initial-state randomization produce policies that memorize a table layout instead of learning a skill.

The Five Rig Approaches Compared

Rig choice is the decision that fixes your cost floor, your data quality ceiling, and which tasks you can collect at all. There is no universal winner; there is a right answer per task family and budget.

Approach Example system Hardware cost Control fidelity Operator ramp Best for Weakness
Bimanual leader-follower ALOHA (Stanford) ~$20k; Mobile ALOHA ~$32k with base Joint-level, 50 Hz, kinesthetic Fast (days) Fine bimanual manipulation Cost per station; fixed embodiment
Low-cost kinematic leader GELLO ~$300 per leader arm + robot Joint-level, high Fast Scaling stations cheaply on arms you own Build/maintenance effort; per-robot tuning
VR headset + controllers Quest 3 rigs; Open-TeleVision style $500 to $3,500 + robot Pose-level via IK, medium-high Medium (1 to 2 weeks) Humanoids, mobile manipulation, immersive tasks IK artifacts, some operator fatigue/sim sickness
6-DoF desktop device 3Dconnexion SpaceMouse $150 to $450 + robot Pose-level, low-medium Slow for contact-rich work Simple pick-place, lab prototyping Slow, unintuitive for dexterous or bimanual tasks
Robot-free handheld UMI gripper Under $1,500 per device Demonstrations without a robot Fast In-the-wild diversity, pre-robot data Embodiment gap; SLAM-dependent pose quality

Three field notes that the spec sheets miss. First, DROID, one of the largest open manipulation datasets (76,000 episodes, 350 hours, 564 scenes), was collected with a Meta Quest 2 controller driving a Franka arm, proof that consumer VR hardware scales to serious datasets (arxiv.org/abs/2403.12945). Second, GELLO is the highest-throughput-per-dollar option if you already own the follower arms, but plan for a part-time technician; printed leaders drift and servos strip. Third, UMI (arxiv.org/abs/2402.10329) is the only approach that collects before your robot ships, which makes it a strong hedge for teams still finalizing hardware.

For a deeper head-to-head, see our companion piece on comparing teleoperation approaches.

Economics: What Teleoperation Data Actually Costs

The true cost of teleoperation data is operator hours multiplied by an all-in rate, divided by usable episode yield, and most budgets get all three numbers wrong. Hardware is the smaller line item; a $20k ALOHA station running two shifts amortizes to under $3 per collection hour over a year. Labor and QA dominate.

Our first-hand benchmarks, consistent across programs we run:

Cost and throughput factor Typical range (our benchmarks)
All-in operator cost (wages, supervision, facility, hardware amortization) $28 to $60 per teleop hour
Novice operator throughput (weeks 1 to 2) 8 to 12 usable episodes per hour
Trained operator throughput (week 3 onward) 25 to 40 usable episodes per hour
QA rejection rate (mature pipeline) 10 to 15 percent
QA rejection rate (new task or new rig, first 2 weeks) 20 to 30 percent
Effective cost per usable episode roughly $0.90 to $6.00 depending on task complexity

Two implications fall out of that table. The operator ramp curve is a 3x throughput difference, so operator retention is a data-cost lever bigger than any hardware decision; churn resets you to novice yield. And QA rejection compounds against you: a 10,000-episode target with 25 percent rejection means collecting 13,300, not 10,000. We cover the bottleneck math in detail in why teleoperation data is the bottleneck in physical AI.

ROI framing for buyers: at $40 per hour all-in and 30 usable episodes per hour, a 50,000-episode dataset costs about $67k in collection labor before QA overhead. Compare that against three to six months of two ML engineers building and debugging an in-house collection operation, and outsourced collection frequently wins on time even when it ties on cost.

From Demonstrations to Policy: The Imitation Learning Pipeline

The imitation learning pipeline is the sequence that converts teleoperated demonstrations into an evaluated robot policy: collect episodes, convert to a training format, train ACT or a diffusion policy, then measure real-world success rate. The pipeline is short; the feedback loop is what teams underuse.

  • Collect and QA. Episodes stream into storage with per-episode metadata (task, operator, rig, success label). QA gates on sync skew, dropped frames, trajectory smoothness, and task success.
  • Convert. HDF5 or ROS 2 bags convert to LeRobot format for training and sharing on Hugging Face.
  • Train. ACT with action chunking (predicting ~100-step action sequences at 50 Hz) or Diffusion Policy for multimodal action distributions.
  • Evaluate. Fixed evaluation protocol: N rollouts per task with randomized initial states, report success rate.
  • Loop. Failure modes from evaluation define next week’s collection priorities.

The proof point worth anchoring on: the original ACT work reached 80 to 90 percent success on fine bimanual tasks such as cable routing and battery insertion from only 50 demonstrations per task (arxiv.org/abs/2304.13705). Demonstration quality and consistency, not raw volume, set that outcome. We have watched 400 sloppy episodes lose to 120 clean ones on the same task, same architecture. Collect less, reject more, retrain sooner.

Case Study Proof: Scaling a VLA Data Program

A humanoid foundation model team came to us needing 12,000 QA-passed bimanual episodes across 40 household tasks in ten weeks, after their in-house effort had produced 1,900 episodes in eight weeks with a 34 percent rejection rate. We stood up six leader-follower stations, ran a two-week operator certification program, and enforced automated sync and smoothness gates at ingest. By week four the floor was producing 1,400 usable episodes per week; the program closed at 12,600 QA-passed episodes with a 12 percent final rejection rate, and the team’s ACT baseline improved from 41 to 68 percent average task success on their internal eval. The full breakdown, including what failed in weeks one and two, is in the VLA case study.

Standing Up a Collection Program: The First 30 Days

A collection program launch is a 30-day sequence of hardware bring-up, protocol authoring, operator certification, and QA gate activation, in that order. Teams that compress or reorder it pay the cost back with interest in weeks five through eight. Our standard sequence:

Days 1 to 7: rigs and clocks. Assemble stations, calibrate cameras (record intrinsics and extrinsics per rig, per calibration), and prove time sync before anyone collects a demonstration. Log a physical sync event (a clapboard works) and verify every stream lands within one camera frame. A day spent here saves a re-collection later; skew discovered after 5,000 episodes is not fixable in post.

Days 8 to 14: protocol. Write task cards for the first five tasks: success criteria, reset procedure, randomization zones marked physically on the workspace, and expected episode duration. Pilot each card yourself before an operator touches it. Every ambiguity you leave in a task card becomes a labeling inconsistency in the dataset.

Days 15 to 28: certification. Run operators through reference tasks with a throughput bar (we use 20 usable episodes per hour) and a quality bar (QA pass rate above 80 percent) before production collection begins. Expect the standard ramp: 8 to 12 usable episodes per hour early, 25 to 40 after two to three weeks.

Day 30: gates on. Nightly automated QA becomes mandatory, with per-operator, per-rig reports every morning. From this point, the program self-corrects.

Build vs Buy: The Honest Decision Criteria

The build-versus-buy question comes down to whether demonstration collection is a competency your team needs to own or a supply problem it needs solved. Building makes sense in three cases: your embodiment is unusual enough that no vendor has matching rigs, your data requirements are small (under roughly 5,000 episodes), or collection know-how is itself part of your product thesis. Buying makes sense when the deadline is the constraint. The math from the economics section applies either way, but in-house programs carry two costs that rarely appear in the comparison spreadsheet: the two to four engineer-months of pipeline and QA tooling before the first reliable episode, and the management overhead of running an operations floor inside an ML organization. We are a vendor, so discount our view accordingly; then ask any team that has run both models which cost they underestimated.

Free Resource: Teleoperation Data RFP Scorecard

A vendor RFP scorecard is a weighted rubric for comparing teleoperation data providers on the criteria that actually predict dataset quality. Ours scores ten dimensions, including sync guarantees, QA rejection transparency, operator training programs, format deliverables, and per-episode pricing disclosure. Most vendors will not publish cost per hour; make them fill in the sheet. [Download the RFP scorecard (XLSX)] and use it on us too.

Related reading: 5 hidden challenges in teleoperation data collection.

Next Step

If you are scoping a demonstration data program, two options. Download 10 free teleoperation episodes (HDF5 plus synchronized MP4, three camera views, 50 Hz actions) and run them through your training stack. Or book a 30-minute scoping call with our data operations team and leave with a per-episode cost estimate for your task list. [Download Sample Data] | [Book a Demo]

Request a Quote

Frequently Asked Questions

What is teleoperation data collection?

Teleoperation data collection is the recording of synchronized robot actions, joint states, and camera observations while a human operator remotely controls the robot through a task. The resulting episodes train imitation learning policies such as ACT and diffusion policies, which underpin VLA models.

Our all-in benchmark is $28 to $60 per operator hour, covering wages, supervision, facility, and hardware amortization. With trained operators producing 25 to 40 usable episodes per hour, effective cost lands between roughly $0.90 and $6.00 per usable episode depending on task complexity.

Fewer than most teams assume, if quality is high. The original ALOHA/ACT results reached 80 to 90 percent success on fine bimanual tasks with about 50 demonstrations per task. Multi-task and VLA training raises the requirement into the thousands per skill family, which is where collection economics start to dominate.

It depends on the task family and budget, per the comparison table above. Fine bimanual manipulation favors leader-follower rigs (ALOHA at ~$20k, or GELLO leaders at ~$300 per arm on robots you own). Humanoid and mobile tasks favor VR teleop. Robot-free UMI capture wins when hardware has not shipped yet.

The main causes are inter-stream time skew above one camera frame, dropped frames, jerky or hesitant trajectories, failed task completions, and reset protocol violations. Expect 20 to 30 percent rejection in the first two weeks on a new task and 10 to 15 percent once the pipeline matures.

Yes. UMI-style handheld grippers record demonstrations with a wrist-mounted camera and SLAM-derived poses, no robot required. The trade-off is an embodiment gap when transferring to your platform, so most teams use it to supplement, not replace, on-robot teleop.

HDF5 per episode and the LeRobot dataset format are the current de facto standards; ROS 2 bags are common at capture time. Whatever the container, require hardware timestamps on every stream and per-episode metadata (task, operator ID, rig ID, success label).

Why Data Procurement, Not Data Capture, Is the Biggest Bottleneck in Physical AI

Posted on by Enoch Pakanati

Two teams approached us in the same quarter with nearly identical VLA data programs. The first sent a two-page acceptance spec and an RFP that demanded numbers, then ran a paid pilot before signing with anyone. The second compared three price sheets and took the lowest. Half a year later, the first team’s deliveries were entering training runs the day they arrived; the second team was still bisecting a training regression that traced back to a sync tolerance no contract had ever specified. Same budget class, same architecture, divergent quarters. The difference was not capture quality. It was procurement.

The uncomfortable part is that capture itself has scaled. ALOHA-class rigs are reproducible from public documentation (arxiv.org/abs/2304.13705). Open X-Embodiment pooled over a million episodes across 22 embodiments (arxiv.org/abs/2310.08864). DROID collected 76,000 teleop episodes across 13 institutions (arxiv.org/abs/2403.12945). The hardware and process knowledge exist. What has not scaled is the buying side: most teams still purchase demonstration data with less rigor than they apply to a laptop refresh.

This post makes the case that procurement is now the binding constraint, shows what the gap costs in numbers, and gives you the four artifacts that close it. It condenses the full Robotics Data Buyer’s Playbook, which includes the complete scorecard and RFP question bank.

We see this from the vendor side at DexSet. The buyers who send a real spec get better data at better prices than the buyers who send a budget and a hope, because a real spec lets us commit to numbers instead of hedging against unknowns.

Key Takeaways

  • Capture capacity has commoditized; vendor selection has not. Weak procurement is now the most common cause of stalled VLA data programs.
  • The cost of a bad pick is measured in usable hours: a cheap vendor at 70 percent yield can cost more per usable hour than a pricier one at 90 percent, once rework is priced in.
  • Latent defects (sync error, calibration drift) are invisible in previews and only surface in training, which is why sample-based buying fails.
  • The fix is procedural, not heroic: a spec-first RFP, a 10-criterion weighted scorecard, a red flags list, and a 50-hour paid pilot.

What Makes Procurement the Bottleneck

The procurement bottleneck is the delay and waste created when robot data purchasing decisions are made without a specification, a scoring method, or a pilot, forcing quality problems to surface downstream during training. It shows up as three concrete failure patterns.

Pattern one: the invisible defect. Robot data defects are latent. A 40 ms sync offset between camera frames and joint states will not appear in video playback, but it corrupts the state-action mapping your imitation learning policy depends on. Teams that buy on sample previews systematically miss this class of problem, then spend weeks bisecting training regressions that were purchased, not coded.

Pattern two: the spec vacuum. When the buyer has no written acceptance spec, every vendor quote describes a different product. One vendor’s “hour of manipulation data” is stereo egocentric plus wrist cameras with dense annotations; another’s is a single 720p exocentric mono stream. Comparing their prices is meaningless, and procurement stalls in clarification loops that a two-page spec would have prevented.

Pattern three: the format tax. Deliveries arrive in whatever the vendor uses internally: undocumented HDF5, half-converted rosbag2, a proprietary container. Your engineers pay the conversion cost into LeRobot or RLDS on every delivery. In our experience that tax runs 15 to 25 percent of program cost when format compatibility was never contracted.

The Cost of Buying Badly, in Numbers

The cost of weak procurement is best expressed as price per usable hour, which is the delivered price divided by the fraction of hours that pass your acceptance spec. The sticker price is the number vendors compete on; the usable-hour price is the number your training run experiences.

Scenario Sticker price Usable-hour yield True price per usable hour 1,000-hour program cost (usable basis)
Cheapest bid, no pilot $32/hr 70% $45.71 $45,710
Mid bid, sample-only check $40/hr 82% $48.78 $48,780
Higher bid, passed 50-hr pilot $46/hr 90% $51.11 $51,110
Cheapest bid after rework and triage time $32/hr + engineering time 70% $55–$60 effective $55,000–$60,000

Read the last row carefully. The cheapest vendor is the most expensive one once you price the engineering time spent triaging failures and patching training mixes, and that is before counting the schedule slip, which no spreadsheet captures but every roadmap feels. These are typical ranges from our benchmarks; your yields will vary, which is exactly why you measure them in a pilot instead of assuming them.

The Fix: Four Artifacts, Not More Meetings

The fix for the procurement bottleneck is a set of four reusable artifacts that convert vendor selection from judgment calls into measurements. Each one is small. Together they remove the guesswork that creates the bottleneck.

  • A spec-first RFP. Write the acceptance spec (cameras, sync tolerance under 10 ms, task list, annotation rubric, delivery format) before contacting anyone. Then send 15 questions that force numeric answers: committed usable-hour yield, audit accuracy, rig counts, per-hour pricing by task complexity.
  • A weighted scorecard. Ten criteria, weights summing to 100, with data quality SLAs, modality coverage, and calibration/sync spec carrying 39 points between them. Two independent scorers, reconciled. Automatic disqualification for weak answers on quality, calibration, or licensing.
  • A red flags list. Some behaviors end the conversation regardless of score: pricing only after discovery calls, refusal of paid pilots, proprietary-only formats, no consent documentation, broad data reuse rights.
  • A 50-hour paid pilot. Production rigs, production operators, and four predefined numbers: usable-hour yield (85 percent or higher), annotation audit accuracy (97 percent or higher on a 5 percent sample), policy success delta on a fixed eval set, and loader time into your LeRobot or RLDS pipeline (one engineer-day or less).

The full versions, including the complete scorecard weights and all 15 RFP questions, are in the Robotics Data Buyer’s Playbook.

How to Tell If Procurement Is Your Bottleneck

A procurement bottleneck diagnosis is a check of where data-program time actually goes, and it takes one honest hour with your delivery logs. Run through five questions:

  • Can anyone state your acceptance spec from memory, or point to the document? If the spec lives in tribal knowledge, every vendor conversation is renegotiating it implicitly.
  • What fraction of delivered hours reached a training run last quarter? If nobody tracks this number, assume it is worse than you think; teams that start measuring usually find 15 to 30 percent of purchased hours never trained anything.
  • How long does a delivery take to enter the pipeline? More than a few hours of engineer time per batch means you are paying the format tax on every delivery.
  • Could you defend your current vendor choice to your board with numbers? A scorecard produces that defense as a byproduct. A demo-based decision cannot.
  • When did a data defect last cost you a training debugging cycle? If the answer is “this quarter,” the defect entered through procurement, not through capture.

Two or more uncomfortable answers means the bottleneck is upstream of your training code, and the fix below is cheaper than the symptom.

Scaling the Playbook Across Programs

Scaling this process means running the same artifacts on every purchase rather than reinventing evaluation per deal. The spec becomes a living document versioned alongside your model releases. The scorecard weights shift as your risks shift: teams early in data collection weight modality coverage higher; teams scaling a proven recipe weight throughput and SLAs higher. Pilot results accumulate into an internal vendor database, which is the closest thing this industry has to a track record. After three or four cycles, vendor evaluation drops from weeks of meetings to days of scoring, and that is the bottleneck removed.

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

Why is procurement a bigger bottleneck than capture in physical AI?

Because capture methods are now well documented (ALOHA, DROID, Open X-Embodiment) while buying practices are not. Latent defects like sync error survive sample review and surface in training, so unstructured purchasing converts directly into stalled training runs.

Delivered price divided by the fraction of hours passing your acceptance spec. A $32/hr vendor at 70 percent yield costs $45.71 per usable hour, which is why sticker-price comparisons mislead.

Around 50 paid hours. That volume exposes process problems (calibration drift, operator variance, QA gaps) while keeping a failed pilot cheap: typically $1,500 to $4,500 at market rates.

Native delivery in LeRobot, RLDS/TFDS, or documented HDF5, with MCAP or rosbag2 as ROS 2 options. Proprietary-only delivery is a disqualifier because it adds conversion cost to every delivery and blocks independent audits.

The Robotics Data Buyer’s Playbook: RFPs, Scorecards, and Pilots That Actually Work (2026 Guide)

Posted on by Sainath Gupta

QA_FAIL episode_0847.hdf5: sync_offset_max 41ms (cam_high vs qpos). That log line, or one very like it, is how many robot data programs discover they have a procurement problem: the flag fires weeks after signature, the contract never defined a sync tolerance, and the vendor technically delivered exactly what was ordered. The team can tell you its GPU budget to the dollar. It cannot tell you which of three vendors quoting $30, $45, and $80 per teleoperation hour would have caught that flag before shipping.

That gap exists for a structural reason. Robot training data has no established procurement discipline. Software buyers inherited decades of RFP practice from enterprise IT. Robotics data buyers inherited nothing, because the category barely existed before large-scale imitation learning and VLA models made demonstration data a line item worth millions. So teams improvise: they buy on price, on a demo video, or on whoever answered the email fastest. Then they discover, three months in, that 30 percent of delivered hours fail their own QA, the cameras were never time-synced to spec, and the contract says nothing about redelivery.

Our thesis, argued with numbers throughout this guide: buying robot data is buying a vendor’s pipeline, and because the defects that matter are latent, only three instruments filter them before signature: a written acceptance spec, a weighted scorecard, and a paid pilot. This guide gives you that procurement discipline, which the category is missing. You will get a 10-criterion weighted vendor scorecard, a 15-question RFP bank grouped by category, a red flags list built from real deals, and a 50-hour paid pilot protocol with pass/fail metrics. Everything here is designed to be copied into your next vendor evaluation.

We run teleoperation and egocentric capture pipelines at DexSet, which means we sit on the receiving end of RFPs every week. We have seen which questions separate serious buyers from tourists, and we have watched deals go sideways when those questions were never asked. This playbook is the document we wish every buyer sent us.

TL;DR: Key Takeaways

  • A robotics data buyer’s playbook is a structured process (RFP, weighted scorecard, red flags, paid pilot) for selecting and managing robot training data vendors.
  • Score vendors on 10 weighted criteria. Data quality SLAs, modality coverage, and calibration/sync spec carry the most weight (39 of 100 points combined).
  • Never sign an annual contract without a paid pilot. Our recommended protocol: 50 hours, measure usable-hour yield (target 85 percent or higher) and policy success delta on fixed eval tasks.
  • Pricing opacity is a signal, not an inconvenience. Vendors who publish per-hour ranges tend to survive QA audits; vendors who quote only after “discovery calls” often cannot.
  • Require delivery in a standard format (LeRobot, RLDS, or HDF5 with documented schema). A proprietary-format-only vendor is a lock-in risk.

What Is a Robotics Data Buyer’s Playbook?

A robotics data buyer’s playbook is a repeatable procurement process for evaluating, piloting, and contracting robot training data vendors, built around four artifacts: a request for proposal (RFP), a weighted vendor scorecard, a red flags checklist, and a pilot evaluation protocol. The playbook exists because robot demonstration data is a specification-heavy purchase, closer to contract manufacturing than to SaaS: what you receive is physical work (teleoperation, human video capture, annotation) frozen into files, and defects are expensive to detect after the fact.

The four artifacts map to four decisions:

  • RFP: which vendors are worth talking to at all.
  • Scorecard: how to compare the ones who respond, on the same axes, with weights that reflect your actual risk.
  • Red flags: which vendors to eliminate regardless of score.
  • Pilot protocol: whether the winning vendor’s production output matches their sample, before you commit annual budget.

Entity chain, stated plainly: your VLA model (OpenVLA, π0, GR00T-class) is trained by imitation learning on demonstration episodes; those episodes come from teleoperation rigs (ALOHA-style bimanual arms, VR-based systems) or egocentric human capture; the vendor’s calibration, sync, and QA pipeline determines whether those episodes are usable; your procurement process determines which vendor pipeline you inherit. Buying data is buying a pipeline.

Why Robot Training Data Is a Different Kind of Purchase

Robot training data procurement differs from every other data purchase because quality is defined by physics, not labels. In text or image annotation, a bad label is visible on screen. In robot data, the defects that kill model performance are invisible in a preview: a 40 ms sync offset between camera frames and joint states, extrinsics that drifted after someone bumped a wrist camera, gripper actions clipped at the edge of the calibration range. The episode looks fine. The policy trained on it does not.

Three properties follow from this:

Defects are latent. You often cannot see them until you train. This is why the pilot protocol below trains an actual policy instead of just eyeballing playback.

Specs are multidimensional. A single “hour of data” bundles camera count and placement (egocentric, exocentric, wrist), mono versus stereo, resolution and fps, depth, proprioception rate, action space, task diversity, and annotation depth. Two vendors quoting the same price are almost never quoting the same product.

Formats determine integration cost. The ecosystem has converged on a handful of formats: the LeRobot dataset format from Hugging Face (github.com/huggingface/lerobot), RLDS/TFDS episodes used by Open X-Embodiment (github.com/google-research/rlds; arxiv.org/abs/2310.08864), raw HDF5 in ALOHA conventions (arxiv.org/abs/2304.13705), and log formats like MCAP (github.com/foxglove/mcap) and rosbag2 (github.com/ros2/rosbag2) for ROS 2 stacks (docs.ros.org). A vendor who cannot deliver in at least one of these adds weeks of conversion work to every delivery.

Here is how the common delivery formats compare from a buyer’s perspective:

Format Origin / ecosystem Best for Random access Tooling maturity Buyer risk if this is the only option
LeRobot v2.x Hugging Face LeRobot Training loops, HF hub distribution Good (parquet + mp4) High, active community Low
RLDS / TFDS Google, Open X-Embodiment TF/JAX pipelines, OXE mixing Good High in TF world, thinner in PyTorch Low to medium
HDF5 (ALOHA-style) ACT / ALOHA papers Bimanual teleop episodes Good Medium, schema varies by lab Medium (demand a schema doc)
MCAP Foxglove Multimodal logging, ROS 2 Good Growing fast Medium
rosbag2 ROS 2 Robot-native logging Moderate High in ROS, low outside Medium
Proprietary format Single vendor Nothing you control Unknown None outside vendor High: lock-in, conversion cost, audit difficulty

The 10-Criterion Weighted Vendor Scorecard

The DexSet vendor scorecard evaluates a robotics data provider on ten criteria, each scored 1 to 5 and multiplied by a weight, for a maximum of 500 weighted points. The weights below reflect where we see deals actually fail; adjust them to your program, but keep quality, modality, and calibration at the top.

# Criterion Weight What a 5 looks like What a 1 looks like
1 Data quality SLAs 15 Contractual usable-hour yield (e.g., 90 percent pass your acceptance spec), free redelivery of failures “We do QA” with no numbers
2 Modality coverage 12 Ego + exo + wrist cams, stereo option, depth, synced proprioception, tactile roadmap Single exocentric mono camera
3 Calibration & sync spec 12 Published intrinsics/extrinsics per rig, time sync under 10 ms across streams, drift checks per shift “Cameras are calibrated” with no numbers or recheck cadence
4 Annotation accuracy 10 Documented rubric, dual-pass or audit sampling, 97 percent or higher on independent audit Unversioned labels, no audit trail
5 Throughput (hours/week) 10 Stated capacity per rig type, evidence of sustained delivery at your volume “We can scale to anything” without rig counts
6 Pricing transparency 10 Published per-hour ranges by rig and task complexity Pricing only after discovery calls
7 Privacy & consent chain 8 Signed operator/participant consent on file, face/bystander handling policy, GDPR posture No documented consent process
8 Format compatibility 8 Native LeRobot and RLDS export, HDF5/MCAP on request, schema docs Proprietary format only
9 Pilot terms 8 Offers paid pilot with buyer-defined acceptance criteria Refuses pilots or offers only cherry-picked samples
10 IP & licensing 7 You own delivered data and models trained on it; no reuse without consent Vendor retains broad reuse rights, ambiguous model ownership

How to use it: have two people score independently from the RFP responses and sample data, then reconcile. Anything below 350/500 exits the process. Anything scoring 1 or 2 on criteria 1, 3, or 10 exits regardless of total, because quality SLAs, calibration, and licensing are the three areas where a weak answer becomes an unrecoverable problem after signature.

The RFP Question Bank: 15 Questions That Do the Work

A robotics data RFP is a short, specific document (five pages beats fifty) whose questions force vendors to commit to numbers. Below are 15 questions we recommend, grouped by category. Vendors who answer all 15 with specifics belong on your shortlist. Vendors who answer with adjectives do not.

Quality and QA 1. What usable-hour yield do you commit to contractually against an acceptance spec we define, and what happens to hours that fail (redelivery, credit, or refund)? 2. Describe your per-episode QA pipeline: what is checked automatically, what is checked by humans, and what percentage of episodes get a second-pass review? 3. Provide annotation accuracy from your most recent independent audit, and the rubric it was measured against.

Hardware, calibration, and sync 4. For each rig type you would use on our program, list cameras (placement, mono/stereo, resolution, fps), depth sensors, and proprioception rates. 5. What is your maximum time-sync error across camera, joint-state, and action streams, and how is it measured and rechecked during production? 6. How often are intrinsics and extrinsics recalibrated, and do delivered episodes include per-rig calibration files?

Operations and throughput 7. How many rigs and operators would be dedicated to our program, and what sustained hours per week does that support at our spec? 8. What was your actual delivered volume for your largest program in the last six months (hours, not episodes)? 9. What is your process when task success rates drop or instructions change mid-program?

Commercial 10. Provide your per-hour pricing range by rig type and task complexity, and state every cost that is not included in it (setup, annotation, redelivery, format conversion). 11. What are your paid pilot terms: minimum hours, price, timeline, and whether buyer-defined acceptance criteria apply? 12. In which formats do you deliver natively (LeRobot, RLDS, HDF5, MCAP, rosbag2), and can you share a schema document today?

Legal and privacy 13. Who owns the delivered data and any models trained on it, and do you retain any right to reuse our episodes for other customers or your own models? 14. Describe your consent chain: what operators and any captured bystanders sign, and how consent records map to delivered episodes. 15. What happens to our task definitions, environment setups, and prompts after the engagement ends?

Red Flags: When to Walk Away

A red flag in robotics data procurement is any vendor behavior that predicts unrecoverable problems after contract signature. These are the ones we treat as disqualifying, based on deals we have watched from both sides:

  • Pricing available only after a discovery call. Opacity here usually means price is set by your budget, not their costs.
  • Refusal of a paid pilot with buyer-defined acceptance criteria. A vendor confident in their pipeline will take your money to prove it.
  • Sample data that is not from a production rig. Ask directly. Showcase rigs and production rigs can be different machines run by different people.
  • No stated time-sync tolerance. If they have never measured it, your training team will be the first to.
  • Proprietary delivery format with no export path. Lock-in plus audit difficulty in one package.
  • No consent documentation for operators or captured humans. This becomes your legal problem, not theirs.
  • Broad data reuse rights buried in the MSA. Your task distribution is competitive information.
  • “Unlimited scale” claims without rig counts. Throughput is rigs times shifts times yield. Anyone who will not show the multiplication is guessing.
  • No redelivery or credit policy for failed hours. QA without consequences is marketing.

Cost and Economics: What Robot Data Actually Costs

Robot training data pricing in 2026 clusters into ranges that depend on rig type, task complexity, and annotation depth, and any playbook needs those ranges to sanity-check quotes. The figures below are DexSet benchmarks and typical ranges we see across the market; treat them as calibration, not quotes.

Data type Typical market range (per hour) Main cost drivers Typical usable-hour yield we see
Bimanual teleoperation (ALOHA-class rig) $28–$60 Operator skill, task resets, rig count 80–90 percent
Humanoid on-robot teleoperation $150–$400 Robot cost and uptime, safety oversight, pilot skill 70–85 percent
Egocentric human video (mono to stereo + IMU) $15–$40 Participant recruiting, consent, headset hardware 80–90 percent
Exocentric multi-view capture $20–$50 Camera count, calibration, studio setup 80–90 percent
Dense annotation add-on +$8–$25 Label depth, audit sampling n/a
Independent QA add-on +$5–$12 Sampling rate, audit depth n/a

Two economics rules matter more than the sticker price:

Rule 1: price per usable hour, not per delivered hour. A $35/hr vendor at 70 percent yield costs you $50 per usable hour. A $44/hr vendor at 90 percent yield costs $48.89, arrives with less rework, and does not poison your training mix with borderline episodes.

Rule 2: model the pipeline, not the purchase. A 1,000-hour teleop program at $40/hr is $40,000 in data, but plan another 15 to 25 percent for integration, conversion, storage, and your own audit time if the vendor scores poorly on format compatibility and QA. Vendors who score 4 to 5 on criteria 1, 4, and 8 compress that overhead, which is usually worth more than a $5/hr discount.

The 50-Hour Paid Pilot Protocol

A pilot evaluation protocol is a fixed, paid, pre-contract engagement that measures whether a vendor’s production pipeline meets your acceptance spec, using metrics you define before the first hour is captured. We recommend 50 hours: large enough to expose process problems, small enough that a failed pilot costs weeks rather than quarters.

Protocol:

  • Fix the spec first. Write the acceptance spec (camera config, sync tolerance, task list, annotation rubric, delivery format) before contacting vendors. The pilot tests the vendor against the spec, not the spec against the vendor.
  • Pay for it. 50 hours at market rates is $1,500 to $4,500 for most modalities. Paying keeps the vendor’s incentives honest and gets you production treatment, not showcase treatment.
  • Demand production conditions. Same rigs, same operators, same QA pipeline that would run your annual contract. Put this in the pilot agreement.
  • Measure usable-hour yield. Run every delivered episode through your acceptance checks. Target: 85 percent or higher for teleop, 90 percent or higher for egocentric video.
  • Audit annotations on a sample. Independently re-label 5 percent of episodes. Target: 97 percent agreement or higher.
  • Train a policy and measure the delta. Fine-tune a fixed baseline (an ACT or diffusion policy head, or a small VLA) on your existing data alone, then on existing data plus pilot data. Evaluate both on the same fixed task set. The pilot passes only if success rate improves; flat or negative delta at 50 hours predicts flat or negative at 1,000.
  • Test the loader. Delivered data should load into your LeRobot or RLDS pipeline within one engineer-day. Log every schema surprise; each one recurs at scale.
  • Decide on numbers. Yield, audit accuracy, policy delta, loader time. Four numbers, agreed in advance, written into the pilot agreement.

Case Study: A Humanoid Foundation Model Team Rebuilds Its Buying Process

One humanoid foundation model team we work with came to us after a year of ad hoc purchasing: three vendors, three formats, no shared acceptance spec, and a training team spending roughly a day per delivery writing conversion scripts and triaging bad episodes. Their internal estimate was that a quarter of purchased hours never reached a training run.

They rebuilt procurement around the artifacts in this guide. The spec came first: stereo egocentric plus wrist cameras, sync under 10 ms, LeRobot delivery, a 60-task manipulation list. The RFP went to six vendors; four answered with numbers, two answered with adjectives and were dropped. The scorecard separated the four finalists cleanly, mostly on quality SLAs and licensing terms, where two vendors wanted broad reuse rights. Both finalists ran 50-hour paid pilots. One delivered 90 percent usable-hour yield and a measurable success-rate improvement on the fixed eval set; the other delivered 78 percent yield and failed the loader test.

The team signed with the first vendor. The numbers that mattered afterward: usable-hour yield across the first 1,000 contracted hours held within two points of the pilot, and data engineering time per delivery dropped from about a day to under an hour because the format and schema were locked before signature. No named clients, no invented revenue figures, just the pattern we see repeatedly: the pilot predicted production, because the pilot was run under production conditions.

Get the Template

The fastest way to apply this playbook is to not rebuild it. We packaged the full RFP question bank (the 15 questions above plus 20 more), the weighted scorecard as a spreadsheet with the math built in, the red flags checklist, and the pilot agreement language into a single download. Send the RFP as-is or strip it to the sections that match your program.

Related reading on this site: why procurement is the real bottleneck in physical AI, comparing data sourcing approaches and their costs, a humanoid team’s VLA data procurement case study, and five hidden challenges in robotics data RFPs.

Put the Playbook to Work

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

What is a robotics data buyer’s playbook?

A robotics data buyer’s playbook is a structured procurement process for robot training data, built around four artifacts: an RFP with specification-forcing questions, a weighted vendor scorecard, a red flags checklist, and a paid pilot protocol with predefined pass/fail metrics.

Five to eight. Fewer gives you no basis for comparison; more than eight means your spec is probably too vague to have filtered anyone out, and evaluation time grows linearly with responses.

In our benchmarks, mature teleoperation pipelines deliver 80 to 90 percent of hours passing a well-defined acceptance spec. Below 80 percent, rework and training-mix contamination usually erase any price advantage.

Paid. A paid 50-hour pilot ($1,500 to $4,500 at typical market rates) obligates the vendor to run production processes and lets you enforce buyer-defined acceptance criteria. Free samples are curated by definition.

Require native delivery in at least one of LeRobot, RLDS/TFDS, or documented HDF5, with MCAP or rosbag2 as options for ROS 2 stacks. Reject proprietary-only formats; they create lock-in and make independent audits harder.

Start from the DexSet weights (quality SLAs 15, modality coverage 12, calibration/sync 12) and shift weight toward whichever criterion caused your last data problem. Keep quality, calibration, and licensing as automatic-disqualification criteria regardless of weights.

Download the RFP Template + Vendor Scorecard (XLSX) and send it to your shortlist this week, or book a demo and we will walk you through how DexSet answers all 15 questions, numbers included.

Robot Training Data Costs and Pricing: The Complete 2026 Guide

Posted on by Sainath Gupta

In our first year we priced a 6,000-hour teleoperation program the way most of this market still prices: by the raw hour. We hit our quoted rate and still broke the client’s budget model, because 22 percent of captured episodes failed their acceptance spec and nobody’s plan had funded the recollection. We rebuilt our cost model around that miss, and this guide is the rebuilt model, published.

The mistake was possible because robot training data has no commodity unit yet. An “hour” of data can mean a raw teleop stream with 30 percent unusable episodes, or a QA-passed, annotated, deduplicated hour that trains a policy. Those two hours differ in cost by 2x or more, and vendors quietly quote whichever one makes their number look better.

So here is the thesis this guide argues from the first table to the last: the only honest unit for pricing robot training data is cost per usable hour, and every quote you receive should be converted into that unit before you compare anything. To make the conversion possible, you will get our first-hand cost-per-hour benchmarks for teleoperation, egocentric human video, and multi-view exocentric capture; rig capex figures anchored to public hardware like ALOHA and UMI; the QA rejection math that separates raw hours from usable hours; and a worked budget for a 10,000-hour VLA data program.

DexSet collects egocentric, exocentric, teleoperation, mono, and stereo data for physical AI teams. We run these rigs, staff these operators, and eat these QA rejections every week. Every number below is either our own operating benchmark or a cited public source.

TL;DR: Robot Training Data Costs at a Glance

  • Teleoperation data: $28 to $60 per raw hour (rig amortization + operator + QA), based on DexSet benchmarks.
  • Egocentric human video: $15 to $40 per hour, the cheapest scalable modality.
  • Multi-view exocentric capture: $20 to $50 per hour depending on camera count and calibration load.
  • Annotation passes: $8 to $25 per hour extra, on top of any capture modality.
  • Rig capex: ~$20k for an ALOHA-style bimanual station (per the ALOHA paper), ~$32k for Mobile ALOHA, under $1k per UMI-style handheld gripper by our build estimates.
  • QA rejection runs 10 to 30 percent in our pipelines, so always budget on cost per usable hour, not raw hour.
  • Public scale references: Open X-Embodiment aggregates 1M+ trajectories across 22 embodiments; DROID contains 76k episodes.

What Do Robot Training Data Costs Actually Include?

Robot training data cost is the fully loaded price of producing one hour of demonstration data that a robot learning pipeline can actually consume, covering hardware amortization, operator or collector labor, QA review, annotation, and delivery infrastructure. Most published debates skip half of these line items, which is why budgets built from a single “per hour” quote fall apart in month two.

A defensible cost model has five layers:

  • Capture hardware (capex). Teleop stations, headsets, camera arrays, grippers. Amortized over 12 to 24 months of use.
  • Collection labor (opex). Teleoperators, camera-wearing collectors, scene setup staff. Usually the largest single line.
  • QA and curation (opex). Human review, automated checks, rejection and recollection. The most commonly omitted layer.
  • Annotation (opex). Language instructions, segmentation, keyframes, success labels. Priced per pass.
  • Infrastructure (opex). Storage, upload bandwidth, format conversion, dataset versioning.

When a vendor quotes you a single number, ask which of these five layers it covers. In our experience the quoted number usually covers layers 1 and 2 and quietly excludes 3 through 5, which add 30 to 80 percent on top.

Core Modalities and What Each One Costs

A data modality is the combination of sensor viewpoint and control method used to produce demonstrations: teleoperation, egocentric human video, exocentric multi-view capture, and their mono or stereo variants. Each modality has a distinct cost structure because each one shifts spend between hardware, labor, and QA differently.

Teleoperation Data: $28 to $60 per Hour

Teleoperation data is demonstration data produced by a human directly controlling a robot, typically through a leader-follower arm setup or a VR interface, so the recorded actions are executable robot trajectories. It is the gold standard for imitation learning and VLA post-training because actions come out in the robot’s own action space, but it is also the most expensive modality per hour.

Our benchmarks across bimanual manipulation programs:

  • Rig amortization: $4 to $9 per hour (a $20k to $32k station amortized over 18 months of two-shift use, including maintenance and spare grippers).
  • Operator labor: $18 to $38 per hour depending on region, task dexterity, and whether the task needs trained specialists (cable routing and garment handling sit at the top of that range).
  • QA overhead: $6 to $12 per hour, covering episode review, rejection, and partial recollection.

Total: $28 to $60 per raw teleop hour. Long-horizon mobile manipulation lands at the top of the range; tabletop pick-and-place with experienced operators lands at the bottom.

The entity chain matters here: teleoperation feeds imitation learning methods like ACT, which the ALOHA project introduced, and imitation learning at scale is what current VLA models are built on. The ALOHA paper (Zhao et al., “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,” https://arxiv.org/abs/2304.13705) demonstrated that a roughly $20k bimanual rig could produce data good enough for fine manipulation, which reset industry assumptions about capture hardware pricing.

Egocentric Human Video: $15 to $40 per Hour

Egocentric data is first-person video captured from head-mounted or body-mounted cameras while a human performs tasks with their own hands, giving models human-level dexterity examples without any robot in the loop. It is the cheapest modality to scale because the “rig” is a wearable and the collector is doing a task they already know how to do.

Our cost structure:

  • Hardware amortization: $1 to $4 per hour. Headsets and head-mounted cameras cost $300 to $3,500 and survive thousands of capture hours.
  • Collector labor: $10 to $26 per hour.
  • QA overhead: $5 to $10 per hour. Egocentric QA is dominated by motion blur, gaze drift, and occlusion checks.

Total: $15 to $40 per hour. The catch is the embodiment gap: human hands are not robot grippers, so egocentric data usually pretrains representations rather than directly supervising actions. Datasets like Ego4D and EgoExo4D (https://arxiv.org/abs/2311.18259) established the research value of this modality; the commercial question is purely about cost-effective volume.

Multi-View Exocentric Capture: $20 to $50 per Hour

Exocentric data is third-person video captured from fixed or mobile external cameras observing a task from multiple calibrated viewpoints, which gives models scene context and cross-view consistency that a single egocentric stream cannot. Cost scales with camera count and, more painfully, with calibration and synchronization labor.

Our benchmarks: $20 to $50 per hour for 3 to 8 synchronized views, including calibration checks at every scene change. Stereo pairs add roughly 15 to 25 percent over mono at the same view count because of the extra calibration and QA burden, but they buy you metric depth, which matters for manipulation policies.

Annotation: $8 to $25 per Hour, on Top of Everything Above

Annotation cost is the incremental price of adding structured labels to captured data: language instructions, subtask segmentation, success and failure flags, object masks, or keyframe tags. It is always additive to capture cost, and it is where “cheap” datasets quietly become expensive.

Typical per-pass pricing from our pipelines:

  • Language instruction labeling: $8 to $12 per data hour
  • Subtask segmentation: $10 to $16 per data hour
  • Dense object masks or contact annotation: $18 to $25 per data hour

Comparison Table: Cost per Hour by Modality

Modality Raw cost/hour (DexSet benchmark) Hardware amortization share QA rejection rate Best suited for
Teleoperation (bimanual) $28 to $60 $4 to $9 10 to 25% VLA post-training, imitation learning
Egocentric human video $15 to $40 $1 to $4 15 to 30% Representation pretraining, hand priors
Exocentric multi-view (3 to 8 cams) $20 to $50 $3 to $7 10 to 20% Scene understanding, cross-view learning
Stereo add-on (vs mono) +15 to 25% +$1 to $2 +2 to 5 pts Depth-dependent manipulation
Annotation pass (language) +$8 to $12 n/a n/a Instruction-following VLAs
Annotation pass (dense masks) +$18 to $25 n/a n/a Grasp and contact modeling

Rig Economics: Capex Benchmarks You Can Verify

Rig capex is the upfront hardware cost of a capture station before a single hour of data exists, and it is the number that determines whether building in-house ever beats buying data. The public record here is unusually good, so you do not have to trust vendor hand-waving.

Rig Approx. capex Source Notes
ALOHA bimanual teleop station ~$20,000 ALOHA paper, https://arxiv.org/abs/2304.13705 Leader-follower arms, cameras, frame
Mobile ALOHA ~$32,000 Mobile ALOHA paper, https://arxiv.org/abs/2401.02117 Adds mobile base for whole-body tasks
DexSet production teleop cell $20,000 to $32,000 First-hand DexSet build costs ALOHA-class arms plus industrial cameras, lighting, sync hardware
UMI handheld gripper Under $1,000 per unit (our build estimate) UMI paper, https://arxiv.org/abs/2402.10329 Portable gripper with wrist camera; no robot needed at capture time
Egocentric headset kit $300 to $3,500 Consumer/enterprise hardware pricing Camera glasses to mixed-reality headsets

Two practical lessons from running these rigs:

First, capex is rarely the problem. A $26k teleop cell running two shifts amortizes to under $9 per hour within 18 months. Labor and QA dominate every mature program we run.

Second, UMI-style handheld grippers changed the low end of the market. Because the capture device is a portable gripper with a wrist camera rather than a full robot cell, collection can happen in real homes and kitchens at egocentric-like labor costs while still producing gripper-centric trajectories. The trade-off is a heavier post-processing and QA load to recover clean actions.

Cost per Usable Hour: The Number That Actually Matters

Cost per usable hour is the total program spend divided by the hours that survive quality assurance, and it is always higher than the quoted cost per raw hour. This is the single most important correction to apply to any vendor quote, including ours.

In DexSet pipelines, QA rejection runs 10 to 30 percent depending on modality and task difficulty. Episodes get rejected for dropped frames, desynchronized views, failed task completion, occluded end-effectors, or annotation mismatches. The math is unforgiving:

Usable-hour math. At $40 per raw teleop hour with a 25 percent rejection rate, your real cost is $40 / 0.75 = $53.33 per usable hour. A competitor quoting $36 per hour with an unmeasured 35 percent rejection rate is actually charging $55.38. The cheaper quote is the more expensive dataset.

Quoted raw $/hr Rejection rate True cost per usable hour
$30 10% $33.33
$30 30% $42.86
$40 15% $47.06
$40 25% $53.33
$55 10% $61.11

When you evaluate any provider, require three things in writing: the measured rejection rate on a comparable program, who pays for recollection of rejected episodes, and whether QA review labor is inside or outside the quoted rate. If a vendor cannot produce a rejection rate, they are not measuring quality.

Budgeting a Program: A Worked Example

A data budget is a forward plan that converts a target usable-hour count into total spend across capture, QA, annotation, and infrastructure. Here is a realistic model for a VLA team that needs 10,000 usable teleop hours with language annotation.

  • Target: 10,000 usable hours
  • Assumed rejection rate: 20 percent, so raw capture target = 12,500 hours
  • Blended teleop rate: $42 per raw hour = $525,000 capture
  • Language annotation at $10 per usable hour = $100,000
  • Storage, versioning, delivery at roughly 4 percent of capture = $21,000
  • Total: ~$646,000, or $64.60 per usable annotated hour

For scale context, Open X-Embodiment pooled more than 1 million trajectories across 22 robot embodiments from 21 institutions (https://arxiv.org/abs/2310.08864), and DROID contains 76,000 episodes collected across 52 buildings (https://arxiv.org/abs/2403.12945). Those datasets exist because no single lab could afford to collect that volume alone, which tells you what the market already knows: collection cost, not model architecture, is the binding constraint on physical AI progress.

Build vs Buy: When Each One Wins

The build-vs-buy decision compares the fully loaded cost of standing up your own capture operation against a vendor’s cost per usable hour at your required volume and quality bar. Neither answer is always right; the crossover depends on volume, duration, and how much operational pain you can absorb.

Build wins when you need under roughly 2,000 hours of highly proprietary, robot-specific data, you already own the robots, and engineering time is genuinely available. Buy wins when you need volume and velocity: a vendor already amortized the rigs, trained the operators past the learning curve (operator throughput improves 30 to 50 percent over their first 200 hours in our programs), and built the QA tooling you would otherwise write from scratch. Most funded teams land on a hybrid: build one internal cell for rapid task iteration, buy production volume.

Case Study Proof: A Humanoid Foundation Model Team

A humanoid foundation model team came to us with a $400k data budget, a quoted competitor rate of $35 per hour, and a plan for 11,400 hours. The quote excluded QA review and carried no measured rejection rate. On a 200-hour pilot we measured 28 percent rejection against their own spec, which repriced the competitor dataset at $48.60 per usable hour before annotation.

We restructured the program: tightened the task spec to cut ambiguity-driven rejections, moved 30 percent of volume to egocentric capture for representation pretraining, and reserved teleop for post-training data. Result: 9,800 usable hours delivered inside the original budget, with rejection stabilized at 12 percent by week six. The lesson is not that our rate was lower. It is that cost per usable hour, measured on a pilot, is the only number that predicted their final spend.

Why Most Vendors Hide Pricing, and Why We Publish It

Hidden pricing is a deliberate market structure in which vendors quote deal by deal to maximize price discrimination, and it survives because buyers lack a shared benchmark. Large annotation-era incumbents built their margins on this asymmetry, and robot data inherited the habit.

We publish our ranges because the buyers we want, Heads of Data who run pilots and measure rejection rates, are exactly the buyers opaque pricing repels. Transparent ranges cost us the occasional overpriced deal and win us every buyer who has been burned before. You should treat any vendor’s refusal to publish even a range as information about how they expect the negotiation to go.

Free Download: The Robot Data Cost Model and RFP Scorecard

We packaged the math in this guide into two working documents: a cost model spreadsheet with editable assumptions for rejection rate, shift count, and amortization period, and a 24-question RFP scorecard covering the five cost layers, QA measurement, and recollection liability. Both are free, no email gate on the scorecard.

Related Reading

Put the Numbers to Work

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

How much does robot training data cost per hour?

Based on DexSet’s operating benchmarks: teleoperation runs $28 to $60 per raw hour, egocentric human video $15 to $40, and multi-view exocentric capture $20 to $50. Annotation adds $8 to $25 per hour per pass. Divide any quoted rate by (1 minus the rejection rate) to get the true cost per usable hour.

The original ALOHA paper reports a bimanual rig built for roughly $20,000, and Mobile ALOHA extends it to whole-body mobile manipulation at roughly $32,000. Our production cells, with industrial cameras, lighting, and sync hardware added, land between $20,000 and $32,000.

In our pipelines, 10 to 30 percent of raw episodes fail QA, depending on modality and task complexity. Well-specified tabletop teleop can hold near 10 percent; long-horizon mobile tasks and loosely specified egocentric capture push toward 30 percent.

Building tends to win below roughly 2,000 hours of proprietary, robot-specific data when you already own robots and engineering time. Buying wins at volume because vendors have amortized rigs, trained operators, and existing QA tooling. Most teams run a hybrid.

Egocentric capture uses wearable cameras and human hands, so hardware costs hundreds to a few thousand dollars and collectors perform familiar tasks at natural speed. Teleoperation requires a $20k to $32k rig plus a trained operator, and outputs executable robot actions, which is what you pay the premium for.

Open X-Embodiment aggregates more than 1 million trajectories across 22 robot embodiments, and DROID contains 76,000 episodes. Both are useful pretraining anchors, but most teams still need proprietary data matched to their own embodiment and tasks.

Require the measured QA rejection rate on a comparable program, clarity on who pays for recollection, an itemized list of which cost layers the rate includes (hardware, labor, QA, annotation, infrastructure), and a paid pilot with your acceptance spec before any volume commitment.

Download the DexSet Robot Data Cost Model and RFP Scorecard, or book a 30-minute pricing walkthrough with our data operations team. We will run your task list through the same model we use internally and hand you the spreadsheet.

The Complete Guide to Exocentric & Multi-View Data for Robot Learning (2026)

Posted on by Sainath Gupta

You are three weeks from a milestone demo and the policy still drops the mug the moment the gripper crosses in front of it. The wrist camera looks perfect in the replay viewer, the demonstrations are clean, and none of it helps. The model never saw the scene from anywhere else, so the instant its one viewpoint goes blind, so does the policy.

The problem exists because most teams start with the camera that is easiest to mount, not the camera set that answers the questions their model will ask at inference time. A single egocentric or wrist view gives you fine-grained contact detail and nothing else: no scene context, no occlusion recovery, no spatial grounding for language-conditioned tasks. Fixing that after you have collected 400 hours of single-view teleoperation is expensive. Fixing it before you start costs a tripod and a calibration session.

The thesis of this guide is simple: camera geometry is a first-order training decision, not plumbing, and calibrated multi-view capture is the cheapest performance intervention most manipulation teams have not yet made. We will argue it three ways: with the published record (DROID, Ego-Exo4D, RoboMimic), with our own capture benchmarks and ablations, and with a worked customer example. This guide covers what exocentric and multi-view data actually is, which camera and rig configurations the major research datasets use, how to calibrate and synchronize multiple cameras without corrupting your dataset, and what all of it costs. The numbers come from our own capture operations at DexSet, where we run multi-camera teleoperation and human demonstration rigs daily for VLA and humanoid foundation model customers, plus the public dataset cards and papers we cite throughout.

TL;DR – Exocentric data is third-person footage of a robot or human performing a task; multi-view data captures the same episode from two or more calibrated, time-synchronized viewpoints at once. – The strongest public evidence for multi-view capture comes from DROID (two external stereo cameras plus a wrist camera on a Franka arm) and Ego-Exo4D (paired egocentric and exocentric video of skilled human activity). – Our benchmarks: a production-grade multi-view rig costs $2,500 to $12,000 to build, and calibrated multi-view capture runs $20 to $50 per hour depending on camera count, sync requirements, and QA depth. – The second camera delivers most of the benefit. In our ablations, going from one external view to two produced the largest jump in policy success; the fourth camera added little beyond storage cost. – Calibration drift and time sync are where multi-view datasets quietly die. Budget for recalibration checks every capture session, not every capture month.

What Is Exocentric Data for Robot Learning?

Exocentric data is visual data recorded from a third-person viewpoint that observes the robot or human demonstrator and the workspace from outside the body performing the task. A camera on a tripod behind the workbench, an overhead camera above a bin-picking cell, and a shoulder-height camera watching a humanoid fold towels are all exocentric views. The defining property is that the camera pose is independent of the actor’s motion.

Egocentric data is the complement: footage from the actor’s own perspective, such as smart glasses on a human demonstrator or a head-mounted camera on a humanoid. Wrist cameras sit in between; they move with the arm but do not share the actor’s gaze. Most serious manipulation stacks end up wanting both. The exocentric view supplies global scene context and object relationships, and the egocentric or wrist view supplies contact-level detail during grasps.

For a model, the practical difference shows up in failure modes. Policies trained only on exocentric views struggle with precise insertion because the interesting pixels are small and far away. Policies trained only on wrist views fail whenever the gripper blocks the object or the task requires reasoning about anything outside a 30 cm bubble. This is why datasets built for general-purpose policies pair the two rather than choosing.

What Is Multi-View Data?

Multi-view data captures a single episode from two or more cameras with known relative poses and aligned timestamps. Two properties separate a true multi-view dataset from a pile of videos that happen to point at the same table: calibrated extrinsics (each camera’s position and orientation relative to a shared frame, typically the robot base) and synchronization (frames across cameras correspond to the same instant, ideally within a few milliseconds).

Both properties are load-bearing. Without extrinsics, you cannot fuse views geometrically, project actions between frames, or train models that reason about 3D structure. Without sync, a policy learns from image pairs that show slightly different world states, which injects label noise you can never remove afterward. We reject capture sessions at DexSet when cross-camera timestamp skew exceeds 10 ms on manipulation tasks, because we have watched that skew turn into unexplainable policy failures downstream.

Multi-view is not the same as stereo. A stereo camera such as the ZED 2i is one viewpoint with two lenses on a fixed 12 cm baseline, built to estimate depth. A multi-view rig places whole cameras (mono, stereo, or RGB-D) at meaningfully different poses around the workspace. You can and often should combine them: two stereo cameras at different poses give you multi-view and per-view depth at once, which is exactly the configuration DROID chose.

What the Research Record Shows

The strongest public evidence for multi-view capture comes from three sources, and they agree with each other more than most robotics literature does.

DROID (arXiv:2403.12945, dataset) collected roughly 76,000 teleoperated episodes, about 350 hours, on Franka arms across 13 institutions and 564 distinct scenes. Every episode records two external ZED stereo cameras plus a wrist-mounted ZED Mini, with calibrated extrinsics. The authors made multi-view a hard requirement of the collection protocol, not an optional extra, because scene diversity only pays off if the model can see the scene.

Ego-Exo4D (arXiv:2311.18259, project page) is the largest paired ego-exo resource: over 1,200 hours of video of skilled human activity, captured simultaneously from Aria glasses on the participant and four or more stationary exocentric cameras, with camera calibration and time sync across the full rig. It exists because ego-only and exo-only datasets kept failing at cross-view tasks like translating “what I see” into “what the coach sees.” Robot learning inherits the same problem when transferring human video priors onto robot viewpoints.

Open X-Embodiment (arXiv:2310.08864, Hugging Face) aggregates over one million trajectories from 22 robot embodiments, and its per-dataset cards read like a natural experiment in camera configuration: some sources are wrist-only, some single-exo, some multi-view. Teams consuming it for VLA pretraining consistently report the friction of heterogeneous viewpoints, which is itself an argument for capturing calibrated multi-view from day one rather than harmonizing after the fact.

On the modeling side, the RoboMimic study (arXiv:2108.03298) found that observation space design, including which camera views feed the policy, materially changes imitation learning outcomes on the same demonstrations. Camera choice is a first-order hyperparameter, not plumbing.

Camera Hardware: What to Put on the Tripod

The camera decision is a trade between depth quality, shutter behavior, sync options, and price. These are the three units we deploy most, with the specs that actually matter for robot data capture.

Spec Intel RealSense D455 Stereolabs ZED 2i Luxonis OAK-D
Type Active IR stereo RGB-D Passive stereo + neural depth Stereo + RGB, on-device compute
Depth baseline 95 mm 120 mm 75 mm
Max RGB mode 1280x800 @ 30 fps 2208x1242 @ 15 fps (1080p @ 30) 4K RGB @ 30 fps (12 MP sensor)
Depth range (practical) 0.6 to 6 m 0.5 to 20 m (neural) 0.7 to 8 m
Shutter Global (depth), rolling RGB on some SKUs Rolling Global (OV9282 stereo pair)
IMU Yes Yes (9-DoF, barometer) Yes (on most variants)
Sync options External sync pin, multi-cam sync Timestamp-based, no genlock Hardware trigger via GPIO
Street price ~$420 ~$500 ~$250
Where it wins Tabletop manipulation, tight spaces Longer range, outdoor, mobile robots Budget rigs, embedded preprocessing

Two practical notes from our rigs. First, rolling shutter plus fast arm motion produces skewed geometry that quietly degrades any 3D supervision; if your task involves dynamic motion, weight global shutter heavily. Second, the D455 sync pin and the OAK-D hardware trigger let you drive frame capture from a shared signal, while the ZED relies on timestamp alignment, which is fine at 30 fps tabletop speeds and marginal for high-speed tasks.

Rig Geometry: Where the Cameras Go

Rig geometry is the arrangement of camera poses around the workspace, and it matters as much as the cameras themselves. The configurations below cover nearly everything we build.

Two external + wrist (the DROID pattern). One camera at roughly 45 degrees over each shoulder of the workspace, 0.8 to 1.2 m from the task center, plus a wrist camera. This is our default for tabletop manipulation. It gives occlusion recovery (when one external view is blocked, the other usually is not), stereo-of-stereos geometry for 3D checks, and contact detail from the wrist.

Overhead + front. An overhead camera looking straight down disambiguates object layout for pick-and-place and bin tasks; a front camera at torso height captures approach trajectories. Overhead mounts need rigid fixturing. A camera on a boom arm that sags 2 mm over a week of sessions will silently invalidate your extrinsics, which is one of the failure modes we cover in 5 Hidden Challenges in Exocentric & Multi-View Data.

Ego + exo paired (the Ego-Exo4D pattern). Glasses or head-mounted camera on the demonstrator plus stationary exocentric cameras. This is the configuration to choose when your training strategy includes human video pretraining, because it gives you the cross-view correspondence needed to transfer human priors to robot viewpoints.

Ring or arc arrays (4 to 8 cameras). Necessary for full-scene reconstruction, humanoid whole-body capture, or world-model training data. Expensive in sync engineering and storage, and in our experience unnecessary for most single-arm manipulation targets.

Calibration and Synchronization: The Unglamorous Core

Calibration is the process of estimating each camera’s intrinsics (lens and sensor parameters) and the rig’s extrinsics (relative camera poses, plus camera-to-robot-base transforms). Synchronization is making frame timestamps agree across cameras. Neither is hard the first time. Keeping both true across hundreds of capture hours is the actual job.

The standard toolchain is mature and free. OpenCV handles single and stereo calibration with checkerboard or ChArUco targets. Kalibr handles multi-camera and camera-IMU calibration, which you want the moment your rig has more than two cameras or a moving ego camera. ROS 2 ships camera_calibration in image_pipeline for in-place intrinsic calibration on live topics.

For sync, there are three tiers. Software timestamping (each camera stamps frames against system clocks) is free and drifts. PTP, the IEEE 1588 Precision Time Protocol, disciplines device clocks over Ethernet to sub-millisecond agreement and is the right answer for GigE machine vision cameras. Hardware genlock or trigger lines drive every sensor’s shutter from one signal and are the only way to get true same-instant exposure, which matters at high frame rates or with fast motion. Consumer RGB-D cameras mostly give you the first tier plus, on some units, a trigger pin; plan your rig around that constraint rather than discovering it later.

One more transform matters for robot data specifically: camera-to-robot-base, often called hand-eye calibration. Extrinsics between cameras tell you how views relate to each other; the camera-to-base transform tells you how all of them relate to the robot’s action space, which is what lets you project end-effector trajectories into any view or supervise 3D policies in the base frame. We estimate it by touching the arm’s end effector to known target points visible to the cameras, then verify against commanded poses. Skip this and your multi-view dataset is geometrically consistent with itself and disconnected from the robot.

Our operational rule: capture a 20-second calibration verification clip (a ChArUco board swept through the shared view volume) at the start of every session, and gate ingestion on reprojection error staying under 0.5 px and cross-camera skew under 10 ms. It costs two minutes per session and has saved entire capture weeks.

Cost and Economics: What Multi-View Actually Costs

Multi-view capture costs break into rig build (one-time) and operated capture (per hour). The figures below are our own benchmarks from rigs we run at DexSet; treat them as typical ranges, not quotes.

Configuration Rig Build Cost Operated Capture Cost Typical Use
1 external RGB-D + wrist (baseline) $2,500 to $4,000 $20 to $28 / hr Prototyping, single-task policies
2 external stereo + wrist (DROID-style) $4,500 to $7,000 $26 to $38 / hr VLA training data, tabletop manipulation
Ego + 3 exo paired (Ego-Exo4D-style) $6,000 to $9,500 $32 to $44 / hr Human demo capture, cross-view learning
6-camera arc, hardware-triggered $9,000 to $12,000 $40 to $50 / hr Humanoid whole-body, reconstruction, world models

Rig build includes cameras, mounts and fixturing, sync hardware, a capture workstation, and the first calibration. Operated capture includes the operator, session calibration checks, QA review, and annotation-ready packaging; it excludes task design and annotation itself. Storage is the line item teams forget: a 4-camera rig at 1080p30 generates roughly 0.8 to 1.5 TB per capture day depending on codec, and multi-view triples or quadruples whatever single-view budget you had.

The marginal-value curve is the key economic fact. In our ablations on tabletop pick-place and insertion tasks, adding the second external view to a wrist-only setup produced the largest single improvement in policy success. The third camera helped mainly on occlusion-heavy tasks. The fourth was rarely distinguishable from noise while adding about 25 percent to storage and QA cost. Buy the second camera before you buy anything else; justify the fourth with an experiment, not a hunch. We break the per-configuration trade-offs down further in Comparing Exocentric & Multi-View Approaches: Pros, Cons & Costs.

Case Study: Multi-View at Scale for a VLA Team

A humanoid foundation model team came to us with a policy that plateaued at roughly 60 percent success on cluttered tabletop tasks, trained on single-view teleoperation data. Their failure analysis pointed at occlusion: success dropped sharply whenever the target object was blocked from the lone camera during the approach.

We rebuilt their capture around a DROID-style rig: two external stereo cameras plus wrist, hardware-checked calibration each session, sub-10 ms sync, and a QA gate on reprojection error. Over eight weeks we delivered several hundred hours of calibrated multi-view episodes across their task list. Retrained on the multi-view data with the same architecture and episode count, the policy’s success on the occlusion-heavy split improved by double digits, and their engineers stopped hand-labeling “camera blocked” failure cases entirely. The full pipeline detail is in How We Scaled Multi-View Data for a VLA Model, and the strategic version of the argument is in Why Multi-View Data Is the Biggest Bottleneck in Physical AI.

Downloadable: The Multi-View Rig RFP Scorecard

If you are evaluating data vendors or scoping an internal rig, the questions that separate real multi-view capability from marketing are specific: How are extrinsics verified per session? What is your cross-camera timestamp skew tolerance and how is it measured? What reprojection error gates ingestion? What is the storage format and per-hour deliverable size? We have packaged these into a one-page RFP scorecard with scoring weights you can hand to procurement. Download it below; it is the same rubric we hold our own rigs to.

Put This Guide to Work

If your current dataset is single-view and your policy failures cluster around occlusion or spatial grounding, the fix is a capture decision, not an architecture search. Download the Multi-View Rig RFP Scorecard to evaluate vendors or your own rig plan, or book a demo to see calibrated multi-view episodes from our production rigs, including sample data you can load the same day.

Request a Quote

Frequently Asked Questions

What is exocentric data in robot learning?

Exocentric data is third-person visual data captured by cameras positioned outside the robot or demonstrator, observing the actor and workspace from fixed external viewpoints. It provides global scene context that egocentric and wrist cameras cannot, and it pairs with those views in most modern manipulation datasets.

Two calibrated external views plus a wrist camera cover most manipulation use cases; this is the configuration DROID used across 76,000 episodes. In our ablations the second external camera delivers the largest gain, while a fourth camera rarely improves policy success enough to justify its storage and QA cost.

Stereo is one viewpoint with two lenses on a fixed short baseline, designed to estimate depth. Multi-view places separate cameras at meaningfully different poses around the workspace with calibrated extrinsics. A rig can be both, for example two ZED 2i stereo cameras at different positions.

Based on our benchmarks at DexSet, multi-view rig builds run $2,500 to $12,000 depending on camera count and sync hardware, and operated calibrated capture runs $20 to $50 per hour including session calibration checks and QA. Storage adds roughly 0.8 to 1.5 TB per capture day for a 4-camera 1080p30 rig.

Three tiers: software timestamping (free, drifts), PTP / IEEE 1588 clock discipline over Ethernet (sub-millisecond, right for GigE machine vision cameras), and hardware genlock or trigger lines (true same-instant exposure). For tabletop manipulation at 30 fps, keep cross-camera skew under 10 ms and verify it every session.

Ego-Exo4D is the largest, with over 1,200 hours of simultaneously captured ego (Aria glasses) and exo (stationary camera) video of skilled human activity, with calibration and sync across the rig. In robot-collected data, DROID pairs a wrist (near-ego) view with two calibrated external stereo views.

VLA models benefit from multi-view data because language-conditioned tasks require scene-level grounding and manipulation requires contact-level detail, which no single viewpoint provides. The RoboMimic study showed observation space choices materially change imitation learning outcomes, and Open X-Embodiment’s heterogeneous camera setups are a recurring friction point for teams pretraining VLAs.

Comparing Exocentric & Multi-View Data Approaches for Robot Learning: Pros, Cons & Costs

Posted on by Sainath Gupta

A buyer on a scoping call last month put the field’s confusion into one sentence: everyone tells him to collect multi-view data, and nobody will tell him which multi-view. He was right to push. “Multi-view” describes at least five distinct capture strategies with different rig costs, different failure modes, and different value per training hour, and the honest answer, the thesis of this post, is that the right configuration is determined by your task list and training strategy, not by your budget or by whichever public dataset you read about first. Choosing the wrong one is not a small mistake. A team that builds a six-camera arc when their tasks needed a wrist camera and one tripod has burned rig budget, tripled their storage bill, and slowed capture throughput for nothing.

The confusion is understandable. Public datasets each embody one choice without explaining the alternatives: DROID picked two external stereo cameras plus wrist, Ego-Exo4D picked glasses plus stationary exo arrays, and Open X-Embodiment inherited whatever its 22 source labs happened to mount. The papers report what was captured, not the decision tree.

This post is that decision tree. We compare the five approaches we quote and build most often at DexSet, with honest pros, cons, and cost ranges from our own rigs, and end with a matrix mapping task types to configurations. For the underlying camera specs, calibration toolchain, and sync engineering, see the pillar guide: The Complete Guide to Exocentric & Multi-View Data for Robot Learning.

Key Takeaways – Five capture approaches dominate: single exo + wrist, DROID-style (2 exo + wrist), ego+exo paired human capture, dense arrays (4-8 cameras), and sim-rendered multi-view. – DROID-style is the default for tabletop manipulation and VLA training data: $4,500 to $7,000 rig, $26 to $38 per operated hour in our benchmarks. – Ego+exo paired capture is the only approach that supports human-video pretraining with cross-view transfer; it costs more in sync engineering than in cameras. – Sim-rendered views are nearly free per view but inherit the sim-to-real gap; they complement real capture, they do not replace it.

The Five Approaches, Defined

A capture approach is the combination of camera count, camera placement, actor type (robot or human), and sync method used to record training episodes. The five that cover almost every real program:

  • Single exocentric + wrist. One fixed external camera plus a wrist camera on the robot. The minimum viable multi-view setup.
  • DROID-style: two exocentric stereo + wrist. Two external stereo cameras (ZED 2i class) at distinct poses plus a wrist camera, calibrated extrinsics, as used across DROID’s 76,000 episodes (arXiv:2403.12945).
  • Ego + exo paired human capture. Glasses or head-mounted camera on a human demonstrator plus stationary exocentric cameras, the Ego-Exo4D pattern (arXiv:2311.18259).
  • Dense array (4-8 cameras). Hardware-triggered ring or arc around the workspace for reconstruction-grade coverage.
  • Sim-rendered multi-view. Arbitrary virtual cameras rendered from simulation, optionally mixed with real data.

Master Comparison Table

Approach Rig Build Capture Cost / Hr Sync Difficulty Occlusion Coverage Human-Video Pretraining Main Risk
Single exo + wrist $2,500 to $4,000 $20 to $28 Low Partial No Blind spots remain
DROID-style (2 exo + wrist) $4,500 to $7,000 $26 to $38 Moderate Good No Calibration upkeep
Ego + exo paired $6,000 to $9,500 $32 to $44 High (moving ego cam) Good Yes Ego-exo time alignment
Dense array (4-8 cams) $9,000 to $12,000 $40 to $50 High (trigger/genlock) Excellent No Storage, diminishing returns
Sim-rendered multi-view Compute only ~$1 to $5 equivalent None Perfect Limited Sim-to-real gap

Rig and capture figures are DexSet benchmarks, including session calibration checks and QA; sim figures are rough GPU-time equivalents.

Where Each Approach Wins and Loses

Single exo + wrist earns its place as a starting point. Pros: cheapest real multi-view, simple calibration (one extrinsic pair), enough to break the wrist-only occlusion ceiling for many tasks. Cons: one blocked view and you are back to single-view; no view redundancy for QA cross-checks. We recommend it for prototyping and single-task policies, and we recommend planning the mount points for camera two on day one.

DROID-style is the workhorse, and not by accident. Two external views mean occlusion of one is usually covered by the other; three total views give the RoboMimic-style observation flexibility that lets ML teams ablate view combinations later (arXiv:2108.03298). Cons: per-session calibration verification becomes mandatory, because three cameras drift three ways. In our operations the added QA overhead is roughly 5 percent of session time. This is what we quote when a VLA team asks for a default.

Ego + exo paired solves a different problem: it is the only configuration that produces the ego-exo correspondences needed to pretrain on human demonstration video and transfer to robot viewpoints, the exact gap Ego-Exo4D was built to close. Pros: human demonstrators are fast and cheap per episode; the data doubles as a bridge to large human-video corpora. Cons: the ego camera moves, so extrinsics to the world frame change every frame and must be recovered via SLAM or the glasses’ own tracking; time alignment between glasses and fixed cameras is the hardest sync problem on this list. Choose it when your training strategy explicitly includes human video.

Dense arrays buy near-complete coverage and reconstruction-grade geometry for humanoid whole-body work and world-model data. The cons compound quietly: hardware triggering or genlock is effectively mandatory, storage runs 3 to 4 TB per capture day at 1080p30 in our pipelines, and, in every ablation we have run on single-arm manipulation, cameras five through eight never moved the success metric. Buy this coverage for reconstruction, not for policy learning on tabletop tasks.

Sim-rendered multi-view costs almost nothing per additional view, which is genuinely useful for view-invariance augmentation and architecture prototyping. But every rendered view inherits the simulator’s gap in contact dynamics, materials, and lighting. Teams in the Open X-Embodiment consortium (arXiv:2310.08864) mix sim and real rather than substituting one for the other, and that matches our experience: sim views stretch a real multi-view dataset, they do not replace it.

Decision Matrix: Match the Approach to the Program

Your Situation Recommended Approach
Prototyping one task, tight budget Single exo + wrist, mounts pre-planned for a second exo
Training VLA / manipulation foundation data at scale DROID-style (2 exo + wrist)
Pretraining on human demonstrations or video Ego + exo paired
Humanoid whole-body, reconstruction, world models Dense array, hardware-triggered
Need view diversity beyond rig budget DROID-style real capture + sim-rendered augmentation

One category the table cannot capture: switching costs. Moving from single-exo to DROID-style mid-program is cheap if the mount points and calibration workflow were planned for it, and painful if they were not, because your existing episodes and your new episodes will differ in geometry and your training pipeline has to reconcile them. Moving from robot-only capture to ego+exo is a bigger jump; it changes your demonstrator pool, your sync architecture, and your annotation scheme at once. Teams that expect to make either move should write the target configuration into their schema now, even if the extra cameras arrive next quarter.

Two cross-cutting rules. First, whatever you choose, log extrinsics and sync metadata into every episode; the approach you pick today is the aggregation problem someone inherits in two years. Second, ablate before you scale: run 20 hours in the candidate configuration, train, and let the success metric pick the rig.

Turn the Matrix Into a Procurement Rubric

If you are scoping a capture program or comparing vendors, download the Multi-View Rig RFP Scorecard. It turns this decision matrix into weighted evaluation questions on calibration verification, sync tolerances, and deliverable formats, the same rubric we hold our own rigs to.

Request a Quote

Frequently Asked Questions

What is the cheapest way to get multi-view robot data?

A single external camera plus a wrist camera, at roughly $2,500 to $4,000 for the rig and $20 to $28 per operated capture hour in DexSet benchmarks. It breaks the wrist-only occlusion ceiling for many tasks but leaves blind spots a second external view would cover.

For VLA training data and tabletop manipulation at scale, usually yes: the second external view covers occlusions the first misses and enables view ablations later. The premium over single-exo is about $2,000 to $3,000 in rig cost and $6 to $10 per hour.

No. Rendered views are nearly free and useful for view-invariance augmentation, but they inherit the simulator’s gaps in contact dynamics, materials, and lighting. Production programs mix sim views with real calibrated capture rather than substituting.

When your training plan includes learning from human demonstration video. Paired capture, as in Ego-Exo4D, provides the cross-view correspondences needed to transfer first-person human priors to third-person robot viewpoints.

Task-dependent, but in our single-arm manipulation ablations, cameras beyond the third stopped moving policy success while adding roughly 25 percent storage and QA cost per view. Dense arrays of 4 to 8 cameras are justified for reconstruction and whole-body humanoid work, not tabletop policies.

The Complete Guide to Egocentric Data Collection for Robotics (2026)

Posted on by Sainath Gupta

3,670 hours. That is the complete Ego4D corpus, the largest first-person video dataset ever assembled (arXiv:2110.07058), and it amounts to roughly seven months of one person’s waking life. The robot foundation models expected to generalize across every kitchen, warehouse, and workbench on earth are drawing from a first-person data supply about that size, while their language-model cousins trained on trillions of tokens. The shelf is not thin. It is nearly bare, and teleoperation refills it at a few hundred action-labeled hours per rig per year.

The gap exists because robots perceive the world from their own body. A vision-language-action (VLA) model driving a humanoid needs to learn from footage that looks like what its head camera will actually see: hands entering the frame from below, objects at counter height, occlusions caused by the manipulator itself. That viewpoint is called egocentric, and until recently there was no scaled, systematic way to collect it. Our thesis, argued with numbers throughout this guide: egocentric capture is the only collection method that scales to foundation-model volumes, but it earns that scaling only when modality mix, viewpoint geometry, and annotation depth are derived from the training mechanism rather than from a hardware catalog.

This guide covers the full stack: what egocentric data collection for robotics actually is, the modalities that matter (mono, stereo, depth, IMU, gaze, hand pose), the hardware options from a $500 Quest 3 to Aria Gen 2 research glasses, how the data plugs into policy training, and what it costs per hour. We include the benchmark numbers we use internally at DexSet, because pricing opacity is the single biggest complaint we hear from buyers.

DexSet supplies egocentric, exocentric, teleoperation, mono, and stereo data to physical AI teams. We have built and rebuilt the capture rigs, the QA pipelines, and the annotation stacks described below, and most of the numbers in this guide come from our own production logs.

TL;DR: Key Takeaways – Egocentric data collection captures first-person visual and sensor streams from a camera mounted at the head or chest of a human (or robot), matching the viewpoint a robot policy will see at inference time. – It is the most scalable source of manipulation pretraining data: a human wearing glasses collects demonstrations 3 to 5 times faster than a teleoperator on a bimanual rig, at roughly one third to one half the cost per hour in our benchmarks. – Hardware ranges from ~$500 (Meta Quest 3, GoPro head mounts) to research-grade Aria Gen 2 glasses with calibrated multi-camera, IMU, eye tracking, and on-device machine perception. – Egocentric human video does not replace teleoperation; the strongest results (EgoMimic, co-training pipelines behind modern VLAs) combine both. The action gap between human hands and robot grippers is the core technical problem. – Our production benchmarks: raw egocentric capture runs $15 to $22 per hour; fully annotated (hand pose, object tracks, temporal segmentation) runs $30 to $40 per hour. Teleoperation runs $28 to $60 per hour depending on rig and task complexity.

What Is Egocentric Data Collection for Robotics?

Egocentric data collection for robotics is the practice of recording synchronized video and sensor streams from a first-person viewpoint, typically a head-mounted or chest-mounted camera worn by a human demonstrator, to train robot perception and control models. The defining property is viewpoint: the camera sees the scene the way an embodied agent sees it, with the demonstrator’s own hands and workspace in frame.

Three properties separate egocentric robotics data from ordinary first-person video:

  1. Sensor completeness. A YouTube cooking clip is RGB only. A robotics-grade egocentric recording carries calibrated camera intrinsics and extrinsics, IMU streams for ego-motion, and often stereo pairs or depth so that 3D structure can be recovered.
  2. Action recoverability. The footage must support extraction of what the hands did: 3D hand pose, object 6-DoF tracks, contact events. Without recoverable actions, egocentric video is only useful for representation pretraining, not policy learning.
  3. Task intent. Recordings are organized into episodes with defined start states, goals, and outcomes, mirroring how robot demonstration datasets like those in Open X-Embodiment are structured (arXiv:2310.08864).

The reference datasets here are Meta’s Ego4D, 3,670 hours of daily-life egocentric video across 74 locations (arXiv:2110.07058), and EgoExo4D, which pairs egocentric and exocentric views of skilled activities with dense annotations (arXiv:2311.18259). Both were built for video understanding research; robotics teams now treat them as the template for what scaled first-person capture looks like.

Egocentric vs. Exocentric: Why Viewpoint Determines Value

Exocentric data is footage captured from an external, third-person viewpoint, such as a tripod camera watching a workbench, while egocentric data is captured from the agent’s own point of view. The distinction matters because a policy trained purely on third-person views must solve an extra correspondence problem at deployment: mapping an external observation of a scene onto its own body frame.

The relationship chain that matters for buyers runs like this: egocentric human video teaches visuomotor priors, teleoperation (through systems like ALOHA, arXiv:2304.13705) provides robot-embodiment action labels, imitation learning consumes both, and VLA models such as OpenVLA (arXiv:2406.09246) and π0 (arXiv:2410.24164) sit at the top of the stack. EgoExo4D demonstrated why you often want both viewpoints of the same episode: the exocentric view disambiguates whole-body motion that the egocentric camera cannot see.

In our pipelines, paired ego-exo capture adds roughly 20 to 30 percent to per-hour cost (a second calibrated camera, cross-view sync, extra QA) and is worth it for whole-body humanoid work. For tabletop manipulation, egocentric plus a single fixed reference camera is usually sufficient.

Core Modalities in Egocentric Capture

A modality is one synchronized sensor stream within a recording, and the modality mix determines both the cost of capture and what training objectives the data can support. The five that come up in nearly every RFP we see:

Mono RGB. A single color stream is the cheapest to capture and the only modality most internet-scale pretraining uses. Sufficient for representation learning and video prediction, insufficient on its own for metric 3D.

Stereo RGB. Two horizontally offset cameras allow metric depth recovery through disparity. Stereo is the workhorse for manipulation because grasp points need metric accuracy. Devices like the Intel RealSense D435i and D455 provide hardware-synced stereo pairs plus an onboard IMU; the D455’s wider baseline (95 mm vs. 50 mm) improves depth accuracy at counter-to-room distances.

Depth. Active or computed depth gives per-pixel range directly. Active IR depth degrades in sunlight and on reflective surfaces, which is why most of our outdoor captures rely on passive stereo instead.

IMU. Accelerometer and gyroscope streams recover head motion, enabling ego-motion compensation and SLAM. Project Aria glasses carry two IMUs precisely because ego-motion is that important for downstream 3D reconstruction (projectaria.com).

Gaze and hand pose. Eye tracking (available on Aria) reveals attention targets before the hand moves, and 3D hand pose is the raw material for retargeting human demonstrations to robot grippers. These are the modalities that convert “video” into “demonstration.”

Hardware: The 2026 Egocentric Rig Landscape

An egocentric capture rig is the wearable hardware package (cameras, IMU, compute, mounting) used to record first-person data, and rig choice is the largest single driver of both data quality and program cost. The four setups we run or evaluate most often:

Rig Approx. Hardware Cost Sensors Video Spec (Typical Capture Config) Calibration Best For
Meta Quest 3 (Passthrough Capture) ~$500 Stereo RGB passthrough, IMU, hand tracking 1280×1280 per eye class, 30 fps effective capture Factory, limited access Budget hand-tracked demos, teleop UI doubling as capture
Aria Gen 2 Research Glasses Research program device (not retail) RGB + mono SLAM cameras, 2 IMUs, eye tracking, spatial mics, on-device hand tracking RGB up to 8 MP class, SLAM cams at high frame rate Full factory calibration + MPS services Research-grade egocentric corpora, gaze + hand pose at scale
GoPro Head/Chest Mount $350–$550 Mono RGB (wide FOV), IMU Up to 5.3K, we typically run 4K/60 or 2.7K/120 Self-calibrated (checkerboard) High-volume, low-cost mono capture; harsh environments
RealSense D435i/D455 Helmet Rig (Custom) $700–$1,200 built Stereo IR + RGB, active depth, IMU 848×480 depth at 90 fps or 1280×720 at 30 fps, RGB 1080p Manual, per-rig Metric depth for manipulation, sim-to-real alignment

Three field notes from running these at volume:

  • Quest 3 is underrated as a capture device because its hand tracking gives you approximate 3D hand pose for free, but passthrough capture access is constrained and image quality trails dedicated cameras.
  • Aria Gen 2 is the quality ceiling. Factory-calibrated multi-camera plus eye tracking plus machine perception services means far less post-processing on our side. Access runs through Meta’s research program rather than retail channels, which affects fleet scaling plans.
  • GoPro rigs win on ruggedness and unit economics. The cost is downstream: no depth, so you pay in annotation and 3D lifting compute instead of hardware.

How Egocentric Data Trains Robot Policies

Egocentric data enters robot learning through three mechanisms: representation pretraining, action retargeting, and co-training with robot demonstrations. Understanding which mechanism you are buying data for should drive every spec decision.

Representation pretraining. Visual encoders pretrained on large egocentric corpora like Ego4D transfer to manipulation tasks better than encoders trained on third-person or object-centric images, because the visual statistics (hands, near-field objects, ego-motion blur) match deployment. This is the lowest-risk use of egocentric data: mono RGB is enough, and annotation requirements are light.

Action retargeting. Human hand trajectories extracted from egocentric video are mapped onto robot end-effectors, turning passive video into pseudo-demonstrations. This requires recoverable 3D hand pose, which is why gaze-and-hand-instrumented devices matter. EgoMimic (arXiv:2410.24221) showed that egocentric human data captured on Aria glasses, combined with robot data, improves manipulation policies over robot data alone.

Co-training. Modern VLA training mixes robot episodes (teleop, in formats like the LeRobot dataset standard, github.com/huggingface/lerobot) with human egocentric episodes in one curriculum. The human data supplies breadth of scenes and objects; the robot data anchors the action distribution to the target embodiment. Cross-embodiment training in Open X-Embodiment established the pattern that heterogeneous data mixtures beat single-source datasets, and egocentric human video is the cheapest heterogeneity you can add.

The failure mode to respect: the embodiment gap. Human wrists have degrees of freedom robot grippers lack, human reach and eye height differ from most robot platforms, and human demonstrators exploit compliance no rigid arm has. Data collection protocols can shrink this gap (constrained grasps, robot-plausible motion instructions, matched camera height), and we bake those constraints into our capture scripts.

Collection Approaches Compared: In-House, Crowdsourced, Vendor

A collection approach is the operational model used to produce the data: who wears the rig, who designs the tasks, and who owns QA. Most teams land on one of three models, and the trade-offs are stable across every program we have run.

Approach Cost per Finished Hour (Our Benchmarks) Throughput Ramp Quality Control Where It Breaks
In-house Capture Team Typically well above vendor rates once salaries, rig fleet, and management overhead are loaded in; often roughly double Slow: 2–3 months to steady state Tight, iterative Scaling past ~10 collectors; hiring drag
Crowdsourced / Distributed Low headline rate, before rejection Fast but noisy Weak; rejection rates at or beyond the top of our 10 to 30 percent planning band are common in our audits Calibration, sync, task compliance
Specialist Vendor (DexSet Model) $15–$40 fully QA’d, annotation-dependent 2–4 weeks to first delivery Contractual, sampled + automated Task designs needing daily iteration with your researchers

The honest read: in-house wins when your task distribution changes weekly and researchers need to redesign protocols on the fly. A vendor wins when the task list is stable and the bottleneck is volume with consistent QA. Crowdsourcing looks cheap until you price the rejection rate and the engineering time spent triaging unsynced, uncalibrated footage.

Cost and Economics: What Egocentric Data Actually Costs

The cost of egocentric data is best expressed as dollars per finished, QA-passed hour at a defined annotation depth, because raw capture is a minority of total program cost. Competitors rarely publish numbers, so here are ours. These are current DexSet benchmark ranges, stated as typical figures we see across programs, not quotes:

Line Item Typical Range (Per Finished Hour) Notes
Raw Egocentric Capture (mono/stereo, IMU, episode structure) $15–$22 Collector time, rig amortization, upload, storage
+ Temporal Annotation (task/step segmentation, outcome labels) +$5–$8 Largely tooling-assisted
+ 3D Hand Pose + Object Tracks +$8–$12 The expensive layer; drives the $30–$40 fully-annotated figure
Paired Ego + Exo Capture +20–30% on capture line Second camera, cross-view sync, extra QA
Teleoperation (for comparison) $28–$60 Rig and task complexity dependent; bimanual fine manipulation sits at the top

Two planning rules of thumb from our production logs:

  • Budget 15 to 25 percent of hours for QA failure. Motion blur, dropped IMU packets, and off-task episodes are facts of life. Vendors should absorb this; if you collect in-house, plan for it.
  • Annotation depth should follow the training mechanism. If you are pretraining encoders, do not pay for hand pose. If you are retargeting actions, hand pose is the whole point. We regularly see RFPs over-specified by $10+ per hour because annotation depth was copied from a paper rather than derived from the training plan.

One more line item buyers forget: storage and delivery. Stereo capture at 30 fps with IMU sidecars generates roughly 50 to 120 GB per hour depending on resolution and compression, so a 5,000-hour corpus is a few hundred terabytes before derivatives. Cloud egress on a corpus that size is real money, which is why we quote delivery format and transfer method inside the per-hour price rather than as a surprise on the final invoice. Ask any vendor to do the same.

At these rates, a 5,000-hour egocentric corpus with full annotation lands between $150K and $200K. The equivalent volume via bimanual teleoperation would run $140K to $300K and take 3 to 5 times as long on a comparable rig fleet, which is the arithmetic behind the current industry shift toward human egocentric pretraining with a smaller teleop fine-tuning set.

Case Study Proof: 4,000 Hours for a Humanoid VLA Team

A humanoid foundation model team came to us with a stalled co-training experiment: their teleop corpus was high quality but topped out near 400 hours, and scaling it 10x on their own rigs would have taken most of a year. We scoped a 4,000-hour egocentric program across kitchen, warehouse shelving, and assembly-bench task families, captured on stereo rigs at camera heights matched to their robot’s head frame, with hand pose and object tracks on the 30 percent of hours their researchers flagged as retarget-critical.

Delivery ran 14 weeks. Their team reported that co-training on the mixed corpus improved task success on unseen-object manipulation evaluations relative to their robot-only baseline, consistent with the direction published in EgoMimic-style co-training work. The full breakdown of task families, QA gates, and the capture protocol is in the case study blog this pillar links to below.

Related reading from this series: – Why Egocentric Data Collection for Robotics Is the Biggest Bottleneck in Physical AI – Comparing Egocentric Data Collection Approaches: Pros, Cons and Costs – Case Study: Scaling Egocentric Data Collection for a VLA Model – 5 Hidden Challenges in Egocentric Data Collection

Download: The Egocentric Data RFP Template

An RFP template turns this guide into a procurement tool: it lists the 40 questions we believe every buyer should ask a data vendor, covering rig specs, calibration evidence, sync tolerances, QA sampling methodology, annotation rubrics, pricing structure, and data licensing. We built it from the RFPs we answer, including the questions we wish more buyers asked. Download it, delete our name from the header if you like, and send it to every vendor on your shortlist including us.

Put These Numbers to Work

If you are scoping an egocentric data program this quarter, two options. Download the RFP template and pressure-test every vendor with it, or book a 30-minute demo and we will walk you through sample episodes from our stereo and Aria-class rigs, including the QA reports we ship with every batch. Either way, you leave with real numbers instead of a sales deck.
Request a Quote

Frequently Asked Questions

What is egocentric data collection for robotics?

Egocentric data collection for robotics is the recording of synchronized first-person video and sensor streams (RGB, stereo, depth, IMU, gaze, hand pose) from head- or chest-mounted rigs, structured into task episodes, to train robot perception and manipulation models.

Egocentric data captures a human performing tasks with their own hands from a first-person camera, while teleoperation data captures a robot performing tasks under human control, with exact joint-space action labels. Egocentric data is cheaper and faster to scale; teleoperation data matches the robot embodiment exactly. Most modern VLA pipelines use both.

Common rigs include Meta Quest 3 (~$500, stereo passthrough and hand tracking), Aria Gen 2 research glasses (calibrated multi-camera, IMU, eye tracking), GoPro head or chest mounts ($350 to $550, mono RGB), and custom helmet rigs built around Intel RealSense D435i or D455 stereo depth cameras.

In DexSet’s benchmarks, raw QA-passed egocentric capture runs $15 to $22 per hour, and fully annotated data with hand pose and object tracks runs $30 to $40 per hour. Teleoperation data runs $28 to $60 per hour for comparison.

No. Egocentric human video scales pretraining and improves generalization, but the embodiment gap between human hands and robot grippers means policies still need robot-embodiment data (teleoperation or autonomous rollouts) for reliable control. Research such as EgoMimic supports combining both.

It depends on the mechanism: encoder pretraining benefits from thousands of hours of lightly annotated video, while retargeting pipelines often start showing gains with hundreds of hours of densely annotated, task-matched capture combined with a robot demonstration set.