Skip to main content

Dexset

dexset
Menu
Request a Demo

Tag: DROID

Four Ways to Buy Robot Training Data, Compared: Pros, Cons, and Costs

Posted on by Sainath Gupta

Taped above a monitor in a buyer’s office we visited last year was a coffee-stained, three-page acceptance spec, its 10 ms sync tolerance circled twice in red pen. That battered document was doing more procurement work than the forty-page RFP folder on the shelf beside it, because it forced every vendor quote onto the same axes. It also marked its owners as unusual: teams rarely frame “how are we going to buy this” as a decision at all. They email two vendors someone met at CoRL, pick the cheaper quote, and only discover they chose a procurement approach when it fails.

The failure is predictable because each buying approach has a known cost structure and a known blind spot. An informal purchase is fast and blind. A full RFP is thorough and slow. A pilot-first approach measures what matters but covers one vendor at a time. Open datasets are free and almost never match your embodiment or task distribution.

This post compares the four approaches on speed, cost, and risk, with the math that lets you pick deliberately. It draws on the same scorecard and pilot protocol as our full Robotics Data Buyer’s Playbook, which is where the reusable templates live.

At DexSet we respond to all four buying styles weekly, so we see their outcomes from the supplier side: which approaches produce clean contracts and which produce disputes about what “an hour of data” was supposed to mean.

Key Takeaways

  • There are four common procurement approaches: informal purchase, full RFP + scorecard, pilot-first, and open-data-plus-top-up. Each has a distinct cost and risk profile.
  • Informal buying is cheapest to run ($0 process cost) and most expensive to survive: yield surprises routinely add 20 to 40 percent to effective cost.
  • The RFP + scorecard + pilot combination costs roughly 3 to 5 weeks and $1,500 to $4,500 in pilot fees, and is the only approach that measures quality before annual commitment.
  • Open datasets (Open X-Embodiment, DROID, Ego4D) are excellent for pretraining and mixing, but embodiment and task mismatch means most teams still purchase targeted data on top.

Approach 1: Informal Purchase

An informal purchase is vendor selection without a written specification, scoring method, or pilot: the buyer requests quotes, reviews sample clips, and signs with the most convincing option. It is how most first data purchases happen, and it is defensible exactly once, at very small volume, when you are still learning what to specify.

Pros: fastest path to first data (days, not weeks); no process overhead; fine for exploratory volumes under 20 hours.

Cons: samples are curated, so latent defects (sync offsets, calibration drift) go undetected; quotes are not comparable because no shared spec exists; no contractual yield commitment, so failed hours are your loss; format surprises arrive with the first delivery.

Cost profile: zero process cost up front. In our benchmarks, yield surprises and conversion work typically add 20 to 40 percent to effective per-usable-hour cost versus a piloted vendor. At 1,000 hours, that is $8,000 to $16,000 of avoidable spend on a $40/hr program.

Approach 2: Full RFP with Weighted Scorecard

A full RFP approach sends a written acceptance spec and a fixed question set to five to eight vendors, then scores responses on weighted criteria before any commitment. This is classic procurement discipline adapted to robot data: quality SLAs, calibration and sync specs, throughput evidence, pricing transparency, licensing terms.

Pros: quotes become comparable because everyone bids the same spec; weak vendors self-eliminate (in our experience, roughly a third of recipients answer with adjectives instead of numbers); the scorecard creates an audit trail for the decision; licensing and consent problems surface before signature.

Cons: takes two to four weeks; still paper-based, so a vendor can score well and underdeliver; overkill below roughly $25,000 in annual data spend.

Cost profile: the process costs internal time only, typically 20 to 30 person-hours across spec writing, scoring, and reconciliation. It buys you comparability and eliminates the worst outcomes, but on its own it does not measure production quality.

Approach 3: Pilot-First

A pilot-first approach skips broad solicitation and goes straight to a 50-hour paid pilot with one or two candidate vendors, judged on predefined metrics: usable-hour yield, annotation audit accuracy, policy success delta on a fixed eval set, and loader time into LeRobot or RLDS. It optimizes for measured evidence over paper promises.

Pros: measures the only thing that matters, production output; small, fixed downside ($1,500 to $4,500 per pilot at market rates); fast when you already know the credible vendors; the policy-delta test catches defects no document review can.

Cons: covers only the vendors you pilot, so a better option may never be evaluated; sequential pilots take longer than parallel paper scoring; requires you to have a stable eval task set and baseline policy, which very early teams may lack.

Cost profile: $3,000 to $9,000 to pilot two vendors, plus about one engineer-week for evaluation. Expensive compared to reading PDFs, cheap compared to one bad quarter of deliveries.

Approach 4: Open Data Plus Targeted Top-Up

The open-data approach builds the base training mix from public corpora, then purchases only the targeted data the public sets cannot provide. The public layer is genuinely strong now: Open X-Embodiment spans over one million episodes across 22 embodiments (arxiv.org/abs/2310.08864), DROID adds 76,000 diverse teleop episodes (arxiv.org/abs/2403.12945), and Ego4D provides thousands of hours of egocentric human video (arxiv.org/abs/2110.07058), most of it accessible through Hugging Face dataset cards and RLDS tooling.

Pros: near-zero acquisition cost for pretraining scale; well-documented formats (RLDS, LeRobot conversions); community-validated quality.

Cons: embodiment mismatch (your gripper, camera placement, and control rates differ from the source robots); task distribution rarely matches your product; licenses vary and some restrict commercial use, so legal review is not optional; fine-tuning still demands in-domain demonstrations, which puts you back in one of the first three approaches for the data that moves your metrics most.

Cost profile: storage and engineering only for the public layer, then standard market rates ($28 to $60 per teleop hour, $15 to $40 per egocentric hour in our benchmarks) for the top-up volume, which is typically 10 to 30 percent of total hours but drives most of the task-specific performance.

Side-by-Side Comparison

The four approaches differ most in where they spend money: process time up front, or rework after delivery.

Approach Time to contract Process cost Quality measured before commitment? Typical effective cost penalty vs piloted baseline Best for
Informal purchase 3–10 days ~$0 No +20–40% Exploratory buys under 20 hours
Full RFP + scorecard 2–4 weeks 20–30 person-hours Partially (paper only) +5–15% Annual spend above $25k, multiple candidate vendors
Pilot-first 2–3 weeks per vendor $1,500–$4,500 per pilot Yes Baseline Teams with stable eval tasks and known vendor shortlist
Open data + top-up 1–2 weeks (legal + integration) Engineering time Yes for public layer, no for top-up unless piloted Depends on top-up approach Pretraining scale plus targeted fine-tuning

The pattern most mature buyers converge on is a hybrid: RFP to filter the field, scorecard to rank it, pilot to verify the winner, open data underneath it all as the pretraining base. That sequence is exactly what the Robotics Data Buyer’s Playbook packages, including the scorecard weights and pilot pass/fail thresholds.

One sequencing note from the supplier side: run the approaches in that order, not in parallel. Teams that pilot before writing a spec end up measuring vendors against criteria invented after the data arrived, which makes the results unarguable in exactly the wrong way; nobody can agree what a pass looks like. Teams that RFP without a spec get six incomparable quotes and mistake the spread for market variance. The spec is upstream of everything, takes about a week to write, and is the only artifact in the process that costs nothing but attention.

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

Which robot data procurement approach is cheapest overall?

The RFP-plus-pilot hybrid, once volume passes roughly $25,000 per year. Informal buying has the lowest process cost but the highest effective cost, because yield surprises add 20 to 40 percent on typical programs.

Not fully. Open X-Embodiment, DROID, and Ego4D are strong pretraining bases, but embodiment and task mismatch means fine-tuning still needs in-domain demonstrations, typically 10 to 30 percent of total hours purchased to spec.

For exploratory volumes under about 20 hours, where the goal is learning what to specify rather than feeding a production training run. Anything feeding a release model deserves at least a pilot.

Send the RFP to five to eight vendors, score all responses, then pilot the top one or two. Piloting more than two rarely changes the decision and doubles the evaluation load.

Four numbers agreed in advance: usable-hour yield (target 85 percent or higher), annotation accuracy on an independently re-labeled 5 percent sample (97 percent or higher), policy success delta on a fixed eval set, and loader time into your training format (one engineer-day or less).

Why Exocentric & Multi-View Data Is the Biggest Bottleneck in Physical AI

Posted on by Enoch Pakanati

Language models got their training data for free: by the time the first large transformer trained, the internet had already spent decades producing trillions of tokens describing every topic from millions of viewpoints. Physical AI inherits no such gift. Robot data must be manufactured episode by episode, and most of it has been manufactured the cheap way: one camera, no calibration record, no synchronization guarantees. Compute got cheaper, architectures converged, and the gap between what models could absorb and what capture pipelines produce kept widening.

The pattern shows up concretely in teams we work with. One VLA group spent six weeks tuning architectures against a plateau. Bigger backbone, longer action chunks, better augmentation. The success rate on cluttered scenes moved two points. Then they looked at their data and found the real ceiling: every one of their 90,000 episodes was recorded from a single camera, and 40 percent of failures happened in the exact frames where that camera could not see the target object.

This post makes the case that calibrated exocentric and multi-view capture, not model design, is the binding constraint on manipulation performance right now, and shows what closing the gap costs. The evidence comes from public datasets (DROID, Open X-Embodiment, Ego-Exo4D), the RoboMimic observation-space study, and our own capture benchmarks at DexSet, where multi-view teleoperation rigs are what we run every day.

Key Takeaways – Single-view episodes cap policy performance on occlusion-heavy tasks regardless of architecture; the failure is in the data, not the model. – DROID made two external stereo views plus wrist a hard protocol requirement across 76,000 episodes; Open X-Embodiment’s viewpoint chaos shows what happens without such a standard. – The bottleneck is operational, not scientific: calibration drift, sync skew, and 3-4x storage are why teams default to one camera. – Our benchmarks: DROID-style rigs cost $4,500 to $7,000 to build and $26 to $38 per operated capture hour; that premium is small against a wasted training run.

Why Single-View Data Caps Policy Performance

A single-view dataset gives the policy exactly one projection of the world per timestep, so any state the camera cannot resolve is unlearnable. Occlusion is the obvious case: the gripper approaches, the object disappears behind it, and the policy is now acting on memory and hope. Less obvious is spatial grounding. Language-conditioned instructions like “put the mug behind the plate” require scene geometry a close-cropped wrist view never encodes.

The RoboMimic study (arXiv:2108.03298) quantified the general point years ago: on identical demonstrations, changing the observation space, including camera views, materially changed imitation learning outcomes. Observation design is a first-order variable. Yet teams routinely treat it as fixed plumbing while sweeping learning rates for a month.

We see the ceiling directly in our ablations at DexSet. On tabletop pick-place and insertion tasks, moving from wrist-only to wrist plus one calibrated external view produced the largest single jump in success rate we have measured from any data intervention. Adding a second external view helped again on occlusion-heavy tasks. No optimizer change came close.

What the Big Datasets Already Decided

The major post-2023 collection efforts treat multi-view as a requirement, and that consensus is evidence in itself. DROID (arXiv:2403.12945) recorded roughly 76,000 Franka episodes across 564 scenes with two external ZED stereo cameras and a wrist ZED Mini on every single episode, extrinsics calibrated. The protocol did not permit single-view shortcuts, because the authors understood that scene diversity is worthless if the model cannot see the scene.

Ego-Exo4D (arXiv:2311.18259) went further for human demonstration data: over 1,200 hours of skilled activity captured simultaneously from Aria glasses and multiple stationary exocentric cameras, synchronized and calibrated, precisely so models can learn correspondences between first-person and third-person views. Anyone planning to pretrain robot policies on human video needs that pairing.

Open X-Embodiment (arXiv:2310.08864) is the counterexample that proves the rule. It aggregates over a million trajectories from 22 embodiments, and its camera configurations are heterogeneous: wrist-only here, one exo view there, different poses everywhere, extrinsics often missing. Teams pretraining VLAs on it spend real engineering time coping with viewpoint inconsistency. The lesson is not that aggregation is bad; it is that viewpoint standards are cheap at capture time and expensive to retrofit.

The Real Bottleneck Is Operational, Not Scientific

The reason most data is still single-view is not ignorance; it is that multi-view capture is an operations problem disguised as a shopping list. Buying three cameras takes an afternoon. Keeping their extrinsics valid, their clocks aligned, and their output QA’d across hundreds of sessions is the part that defeats teams.

Three costs dominate:

  • Calibration maintenance. Extrinsics drift when mounts get bumped, booms sag, or thermal cycles shift fixtures. Without per-session verification, drift silently corrupts weeks of data. Our gate is a 0.5 px reprojection error check on a ChArUco sweep at every session start.
  • Synchronization. Software timestamps drift across devices; PTP (IEEE 1588) or hardware trigger lines fix it, but only if someone engineers and monitors the sync path. We reject sessions with more than 10 ms cross-camera skew on manipulation work.
  • Storage and throughput. A 4-camera 1080p30 rig produces 0.8 to 1.5 TB per capture day in our pipelines. Multiply your single-view storage budget by three or four, then add QA review time.

None of this is research. All of it is why the bottleneck persists.

What Closing the Gap Costs

The honest comparison is single-view capture cost versus multi-view capture cost versus the cost of the training runs and engineering time the single-view ceiling wastes. Our benchmark numbers:

Item Single-View (Wrist or 1 Exo) Multi-View (2 Exo + Wrist) Delta
Rig build $1,500 to $2,500 $4,500 to $7,000 +$3,000 to $4,500 one-time
Operated capture $18 to $25 / hr $26 to $38 / hr +$8 to $13 / hr
Storage per capture day ~0.3 TB ~1.0 TB ~3x
Occlusion-heavy task ceiling Hard cap, architecture-independent Removed The point

For a 500-hour dataset, the multi-view premium lands around $4,000 to $6,500 in capture plus the one-time rig delta. One senior engineer spending six weeks fighting a data-imposed plateau costs more, and one full retraining run on data you have to recollect anyway costs far more. The full cost model, camera comparisons, and rig geometry options are in our pillar guide: The Complete Guide to Exocentric & Multi-View Data for Robot Learning.

How to Scale Multi-View Capture Without Drowning

Scaling multi-view data means industrializing the boring parts. The checklist we run internally:

  • Standardize one rig geometry (we default to DROID-style: two external stereo, one wrist) so calibration procedures and QA gates are identical across stations.
  • Gate every session on a two-minute calibration verification clip; reject on reprojection error > 0.5 px or sync skew > 10 ms.
  • Automate extrinsics logging into the episode metadata, so every frame carries its camera poses forever.
  • Budget storage at 3-4x single-view and decide codec and retention policy before capture starts, not after the first full disk.
  • Ablate camera count on your own tasks before scaling past three views; in our experience the fourth camera rarely earns its cost.

Run the Failure Analysis Before the Next Sweep

If your policy metrics have plateaued and your dataset is single-view, run the failure analysis before the next architecture sweep: tag failures by whether the target was visible at decision time. If occlusion dominates, the fix is capture. Book a demo and we will walk you through calibrated multi-view sample episodes from our production rigs, with the calibration and sync metadata included.

Request a Quote

Frequently Asked Questions

Why is multi-view data considered the bottleneck in physical AI?

Because model architectures and compute have outpaced data quality: policies trained on single-view episodes hit occlusion and spatial-grounding ceilings that no architecture change removes, and calibrated multi-view capture is operationally hard enough that most existing datasets never provided it.

In DexSet benchmarks, a DROID-style rig costs $4,500 to $7,000 versus $1,500 to $2,500 for single-view, and operated capture runs $26 to $38 per hour versus $18 to $25. Storage roughly triples.

Standardize one rig geometry across stations, gate every session on calibration and sync checks, embed extrinsics in episode metadata, budget storage at 3-4x single-view, and ablate camera count on your own tasks before adding a fourth view.

DROID enforced two external stereo views plus wrist across 76,000 episodes; Ego-Exo4D paired ego and exo video across 1,200+ hours for cross-view learning; RoboMimic showed observation space choices materially change imitation outcomes; Open X-Embodiment shows the integration cost when viewpoint standards are absent.

No. In our ablations the second view delivers the largest gain, a third helps on occlusion-heavy tasks, and a fourth is rarely distinguishable from noise while adding roughly 25 percent to storage and QA cost.

The Complete Guide to Exocentric & Multi-View Data for Robot Learning (2026)

Posted on by Sainath Gupta

You are three weeks from a milestone demo and the policy still drops the mug the moment the gripper crosses in front of it. The wrist camera looks perfect in the replay viewer, the demonstrations are clean, and none of it helps. The model never saw the scene from anywhere else, so the instant its one viewpoint goes blind, so does the policy.

The problem exists because most teams start with the camera that is easiest to mount, not the camera set that answers the questions their model will ask at inference time. A single egocentric or wrist view gives you fine-grained contact detail and nothing else: no scene context, no occlusion recovery, no spatial grounding for language-conditioned tasks. Fixing that after you have collected 400 hours of single-view teleoperation is expensive. Fixing it before you start costs a tripod and a calibration session.

The thesis of this guide is simple: camera geometry is a first-order training decision, not plumbing, and calibrated multi-view capture is the cheapest performance intervention most manipulation teams have not yet made. We will argue it three ways: with the published record (DROID, Ego-Exo4D, RoboMimic), with our own capture benchmarks and ablations, and with a worked customer example. This guide covers what exocentric and multi-view data actually is, which camera and rig configurations the major research datasets use, how to calibrate and synchronize multiple cameras without corrupting your dataset, and what all of it costs. The numbers come from our own capture operations at DexSet, where we run multi-camera teleoperation and human demonstration rigs daily for VLA and humanoid foundation model customers, plus the public dataset cards and papers we cite throughout.

TL;DR – Exocentric data is third-person footage of a robot or human performing a task; multi-view data captures the same episode from two or more calibrated, time-synchronized viewpoints at once. – The strongest public evidence for multi-view capture comes from DROID (two external stereo cameras plus a wrist camera on a Franka arm) and Ego-Exo4D (paired egocentric and exocentric video of skilled human activity). – Our benchmarks: a production-grade multi-view rig costs $2,500 to $12,000 to build, and calibrated multi-view capture runs $20 to $50 per hour depending on camera count, sync requirements, and QA depth. – The second camera delivers most of the benefit. In our ablations, going from one external view to two produced the largest jump in policy success; the fourth camera added little beyond storage cost. – Calibration drift and time sync are where multi-view datasets quietly die. Budget for recalibration checks every capture session, not every capture month.

What Is Exocentric Data for Robot Learning?

Exocentric data is visual data recorded from a third-person viewpoint that observes the robot or human demonstrator and the workspace from outside the body performing the task. A camera on a tripod behind the workbench, an overhead camera above a bin-picking cell, and a shoulder-height camera watching a humanoid fold towels are all exocentric views. The defining property is that the camera pose is independent of the actor’s motion.

Egocentric data is the complement: footage from the actor’s own perspective, such as smart glasses on a human demonstrator or a head-mounted camera on a humanoid. Wrist cameras sit in between; they move with the arm but do not share the actor’s gaze. Most serious manipulation stacks end up wanting both. The exocentric view supplies global scene context and object relationships, and the egocentric or wrist view supplies contact-level detail during grasps.

For a model, the practical difference shows up in failure modes. Policies trained only on exocentric views struggle with precise insertion because the interesting pixels are small and far away. Policies trained only on wrist views fail whenever the gripper blocks the object or the task requires reasoning about anything outside a 30 cm bubble. This is why datasets built for general-purpose policies pair the two rather than choosing.

What Is Multi-View Data?

Multi-view data captures a single episode from two or more cameras with known relative poses and aligned timestamps. Two properties separate a true multi-view dataset from a pile of videos that happen to point at the same table: calibrated extrinsics (each camera’s position and orientation relative to a shared frame, typically the robot base) and synchronization (frames across cameras correspond to the same instant, ideally within a few milliseconds).

Both properties are load-bearing. Without extrinsics, you cannot fuse views geometrically, project actions between frames, or train models that reason about 3D structure. Without sync, a policy learns from image pairs that show slightly different world states, which injects label noise you can never remove afterward. We reject capture sessions at DexSet when cross-camera timestamp skew exceeds 10 ms on manipulation tasks, because we have watched that skew turn into unexplainable policy failures downstream.

Multi-view is not the same as stereo. A stereo camera such as the ZED 2i is one viewpoint with two lenses on a fixed 12 cm baseline, built to estimate depth. A multi-view rig places whole cameras (mono, stereo, or RGB-D) at meaningfully different poses around the workspace. You can and often should combine them: two stereo cameras at different poses give you multi-view and per-view depth at once, which is exactly the configuration DROID chose.

What the Research Record Shows

The strongest public evidence for multi-view capture comes from three sources, and they agree with each other more than most robotics literature does.

DROID (arXiv:2403.12945, dataset) collected roughly 76,000 teleoperated episodes, about 350 hours, on Franka arms across 13 institutions and 564 distinct scenes. Every episode records two external ZED stereo cameras plus a wrist-mounted ZED Mini, with calibrated extrinsics. The authors made multi-view a hard requirement of the collection protocol, not an optional extra, because scene diversity only pays off if the model can see the scene.

Ego-Exo4D (arXiv:2311.18259, project page) is the largest paired ego-exo resource: over 1,200 hours of video of skilled human activity, captured simultaneously from Aria glasses on the participant and four or more stationary exocentric cameras, with camera calibration and time sync across the full rig. It exists because ego-only and exo-only datasets kept failing at cross-view tasks like translating “what I see” into “what the coach sees.” Robot learning inherits the same problem when transferring human video priors onto robot viewpoints.

Open X-Embodiment (arXiv:2310.08864, Hugging Face) aggregates over one million trajectories from 22 robot embodiments, and its per-dataset cards read like a natural experiment in camera configuration: some sources are wrist-only, some single-exo, some multi-view. Teams consuming it for VLA pretraining consistently report the friction of heterogeneous viewpoints, which is itself an argument for capturing calibrated multi-view from day one rather than harmonizing after the fact.

On the modeling side, the RoboMimic study (arXiv:2108.03298) found that observation space design, including which camera views feed the policy, materially changes imitation learning outcomes on the same demonstrations. Camera choice is a first-order hyperparameter, not plumbing.

Camera Hardware: What to Put on the Tripod

The camera decision is a trade between depth quality, shutter behavior, sync options, and price. These are the three units we deploy most, with the specs that actually matter for robot data capture.

Spec Intel RealSense D455 Stereolabs ZED 2i Luxonis OAK-D
Type Active IR stereo RGB-D Passive stereo + neural depth Stereo + RGB, on-device compute
Depth baseline 95 mm 120 mm 75 mm
Max RGB mode 1280x800 @ 30 fps 2208x1242 @ 15 fps (1080p @ 30) 4K RGB @ 30 fps (12 MP sensor)
Depth range (practical) 0.6 to 6 m 0.5 to 20 m (neural) 0.7 to 8 m
Shutter Global (depth), rolling RGB on some SKUs Rolling Global (OV9282 stereo pair)
IMU Yes Yes (9-DoF, barometer) Yes (on most variants)
Sync options External sync pin, multi-cam sync Timestamp-based, no genlock Hardware trigger via GPIO
Street price ~$420 ~$500 ~$250
Where it wins Tabletop manipulation, tight spaces Longer range, outdoor, mobile robots Budget rigs, embedded preprocessing

Two practical notes from our rigs. First, rolling shutter plus fast arm motion produces skewed geometry that quietly degrades any 3D supervision; if your task involves dynamic motion, weight global shutter heavily. Second, the D455 sync pin and the OAK-D hardware trigger let you drive frame capture from a shared signal, while the ZED relies on timestamp alignment, which is fine at 30 fps tabletop speeds and marginal for high-speed tasks.

Rig Geometry: Where the Cameras Go

Rig geometry is the arrangement of camera poses around the workspace, and it matters as much as the cameras themselves. The configurations below cover nearly everything we build.

Two external + wrist (the DROID pattern). One camera at roughly 45 degrees over each shoulder of the workspace, 0.8 to 1.2 m from the task center, plus a wrist camera. This is our default for tabletop manipulation. It gives occlusion recovery (when one external view is blocked, the other usually is not), stereo-of-stereos geometry for 3D checks, and contact detail from the wrist.

Overhead + front. An overhead camera looking straight down disambiguates object layout for pick-and-place and bin tasks; a front camera at torso height captures approach trajectories. Overhead mounts need rigid fixturing. A camera on a boom arm that sags 2 mm over a week of sessions will silently invalidate your extrinsics, which is one of the failure modes we cover in 5 Hidden Challenges in Exocentric & Multi-View Data.

Ego + exo paired (the Ego-Exo4D pattern). Glasses or head-mounted camera on the demonstrator plus stationary exocentric cameras. This is the configuration to choose when your training strategy includes human video pretraining, because it gives you the cross-view correspondence needed to transfer human priors to robot viewpoints.

Ring or arc arrays (4 to 8 cameras). Necessary for full-scene reconstruction, humanoid whole-body capture, or world-model training data. Expensive in sync engineering and storage, and in our experience unnecessary for most single-arm manipulation targets.

Calibration and Synchronization: The Unglamorous Core

Calibration is the process of estimating each camera’s intrinsics (lens and sensor parameters) and the rig’s extrinsics (relative camera poses, plus camera-to-robot-base transforms). Synchronization is making frame timestamps agree across cameras. Neither is hard the first time. Keeping both true across hundreds of capture hours is the actual job.

The standard toolchain is mature and free. OpenCV handles single and stereo calibration with checkerboard or ChArUco targets. Kalibr handles multi-camera and camera-IMU calibration, which you want the moment your rig has more than two cameras or a moving ego camera. ROS 2 ships camera_calibration in image_pipeline for in-place intrinsic calibration on live topics.

For sync, there are three tiers. Software timestamping (each camera stamps frames against system clocks) is free and drifts. PTP, the IEEE 1588 Precision Time Protocol, disciplines device clocks over Ethernet to sub-millisecond agreement and is the right answer for GigE machine vision cameras. Hardware genlock or trigger lines drive every sensor’s shutter from one signal and are the only way to get true same-instant exposure, which matters at high frame rates or with fast motion. Consumer RGB-D cameras mostly give you the first tier plus, on some units, a trigger pin; plan your rig around that constraint rather than discovering it later.

One more transform matters for robot data specifically: camera-to-robot-base, often called hand-eye calibration. Extrinsics between cameras tell you how views relate to each other; the camera-to-base transform tells you how all of them relate to the robot’s action space, which is what lets you project end-effector trajectories into any view or supervise 3D policies in the base frame. We estimate it by touching the arm’s end effector to known target points visible to the cameras, then verify against commanded poses. Skip this and your multi-view dataset is geometrically consistent with itself and disconnected from the robot.

Our operational rule: capture a 20-second calibration verification clip (a ChArUco board swept through the shared view volume) at the start of every session, and gate ingestion on reprojection error staying under 0.5 px and cross-camera skew under 10 ms. It costs two minutes per session and has saved entire capture weeks.

Cost and Economics: What Multi-View Actually Costs

Multi-view capture costs break into rig build (one-time) and operated capture (per hour). The figures below are our own benchmarks from rigs we run at DexSet; treat them as typical ranges, not quotes.

Configuration Rig Build Cost Operated Capture Cost Typical Use
1 external RGB-D + wrist (baseline) $2,500 to $4,000 $20 to $28 / hr Prototyping, single-task policies
2 external stereo + wrist (DROID-style) $4,500 to $7,000 $26 to $38 / hr VLA training data, tabletop manipulation
Ego + 3 exo paired (Ego-Exo4D-style) $6,000 to $9,500 $32 to $44 / hr Human demo capture, cross-view learning
6-camera arc, hardware-triggered $9,000 to $12,000 $40 to $50 / hr Humanoid whole-body, reconstruction, world models

Rig build includes cameras, mounts and fixturing, sync hardware, a capture workstation, and the first calibration. Operated capture includes the operator, session calibration checks, QA review, and annotation-ready packaging; it excludes task design and annotation itself. Storage is the line item teams forget: a 4-camera rig at 1080p30 generates roughly 0.8 to 1.5 TB per capture day depending on codec, and multi-view triples or quadruples whatever single-view budget you had.

The marginal-value curve is the key economic fact. In our ablations on tabletop pick-place and insertion tasks, adding the second external view to a wrist-only setup produced the largest single improvement in policy success. The third camera helped mainly on occlusion-heavy tasks. The fourth was rarely distinguishable from noise while adding about 25 percent to storage and QA cost. Buy the second camera before you buy anything else; justify the fourth with an experiment, not a hunch. We break the per-configuration trade-offs down further in Comparing Exocentric & Multi-View Approaches: Pros, Cons & Costs.

Case Study: Multi-View at Scale for a VLA Team

A humanoid foundation model team came to us with a policy that plateaued at roughly 60 percent success on cluttered tabletop tasks, trained on single-view teleoperation data. Their failure analysis pointed at occlusion: success dropped sharply whenever the target object was blocked from the lone camera during the approach.

We rebuilt their capture around a DROID-style rig: two external stereo cameras plus wrist, hardware-checked calibration each session, sub-10 ms sync, and a QA gate on reprojection error. Over eight weeks we delivered several hundred hours of calibrated multi-view episodes across their task list. Retrained on the multi-view data with the same architecture and episode count, the policy’s success on the occlusion-heavy split improved by double digits, and their engineers stopped hand-labeling “camera blocked” failure cases entirely. The full pipeline detail is in How We Scaled Multi-View Data for a VLA Model, and the strategic version of the argument is in Why Multi-View Data Is the Biggest Bottleneck in Physical AI.

Downloadable: The Multi-View Rig RFP Scorecard

If you are evaluating data vendors or scoping an internal rig, the questions that separate real multi-view capability from marketing are specific: How are extrinsics verified per session? What is your cross-camera timestamp skew tolerance and how is it measured? What reprojection error gates ingestion? What is the storage format and per-hour deliverable size? We have packaged these into a one-page RFP scorecard with scoring weights you can hand to procurement. Download it below; it is the same rubric we hold our own rigs to.

Put This Guide to Work

If your current dataset is single-view and your policy failures cluster around occlusion or spatial grounding, the fix is a capture decision, not an architecture search. Download the Multi-View Rig RFP Scorecard to evaluate vendors or your own rig plan, or book a demo to see calibrated multi-view episodes from our production rigs, including sample data you can load the same day.

Request a Quote

Frequently Asked Questions

What is exocentric data in robot learning?

Exocentric data is third-person visual data captured by cameras positioned outside the robot or demonstrator, observing the actor and workspace from fixed external viewpoints. It provides global scene context that egocentric and wrist cameras cannot, and it pairs with those views in most modern manipulation datasets.

Two calibrated external views plus a wrist camera cover most manipulation use cases; this is the configuration DROID used across 76,000 episodes. In our ablations the second external camera delivers the largest gain, while a fourth camera rarely improves policy success enough to justify its storage and QA cost.

Stereo is one viewpoint with two lenses on a fixed short baseline, designed to estimate depth. Multi-view places separate cameras at meaningfully different poses around the workspace with calibrated extrinsics. A rig can be both, for example two ZED 2i stereo cameras at different positions.

Based on our benchmarks at DexSet, multi-view rig builds run $2,500 to $12,000 depending on camera count and sync hardware, and operated calibrated capture runs $20 to $50 per hour including session calibration checks and QA. Storage adds roughly 0.8 to 1.5 TB per capture day for a 4-camera 1080p30 rig.

Three tiers: software timestamping (free, drifts), PTP / IEEE 1588 clock discipline over Ethernet (sub-millisecond, right for GigE machine vision cameras), and hardware genlock or trigger lines (true same-instant exposure). For tabletop manipulation at 30 fps, keep cross-camera skew under 10 ms and verify it every session.

Ego-Exo4D is the largest, with over 1,200 hours of simultaneously captured ego (Aria glasses) and exo (stationary camera) video of skilled human activity, with calibration and sync across the rig. In robot-collected data, DROID pairs a wrist (near-ego) view with two calibrated external stereo views.

VLA models benefit from multi-view data because language-conditioned tasks require scene-level grounding and manipulation requires contact-level detail, which no single viewpoint provides. The RoboMimic study showed observation space choices materially change imitation learning outcomes, and Open X-Embodiment’s heterogeneous camera setups are a recurring friction point for teams pretraining VLAs.

Comparing Exocentric & Multi-View Data Approaches for Robot Learning: Pros, Cons & Costs

Posted on by Sainath Gupta

A buyer on a scoping call last month put the field’s confusion into one sentence: everyone tells him to collect multi-view data, and nobody will tell him which multi-view. He was right to push. “Multi-view” describes at least five distinct capture strategies with different rig costs, different failure modes, and different value per training hour, and the honest answer, the thesis of this post, is that the right configuration is determined by your task list and training strategy, not by your budget or by whichever public dataset you read about first. Choosing the wrong one is not a small mistake. A team that builds a six-camera arc when their tasks needed a wrist camera and one tripod has burned rig budget, tripled their storage bill, and slowed capture throughput for nothing.

The confusion is understandable. Public datasets each embody one choice without explaining the alternatives: DROID picked two external stereo cameras plus wrist, Ego-Exo4D picked glasses plus stationary exo arrays, and Open X-Embodiment inherited whatever its 22 source labs happened to mount. The papers report what was captured, not the decision tree.

This post is that decision tree. We compare the five approaches we quote and build most often at DexSet, with honest pros, cons, and cost ranges from our own rigs, and end with a matrix mapping task types to configurations. For the underlying camera specs, calibration toolchain, and sync engineering, see the pillar guide: The Complete Guide to Exocentric & Multi-View Data for Robot Learning.

Key Takeaways – Five capture approaches dominate: single exo + wrist, DROID-style (2 exo + wrist), ego+exo paired human capture, dense arrays (4-8 cameras), and sim-rendered multi-view. – DROID-style is the default for tabletop manipulation and VLA training data: $4,500 to $7,000 rig, $26 to $38 per operated hour in our benchmarks. – Ego+exo paired capture is the only approach that supports human-video pretraining with cross-view transfer; it costs more in sync engineering than in cameras. – Sim-rendered views are nearly free per view but inherit the sim-to-real gap; they complement real capture, they do not replace it.

The Five Approaches, Defined

A capture approach is the combination of camera count, camera placement, actor type (robot or human), and sync method used to record training episodes. The five that cover almost every real program:

  • Single exocentric + wrist. One fixed external camera plus a wrist camera on the robot. The minimum viable multi-view setup.
  • DROID-style: two exocentric stereo + wrist. Two external stereo cameras (ZED 2i class) at distinct poses plus a wrist camera, calibrated extrinsics, as used across DROID’s 76,000 episodes (arXiv:2403.12945).
  • Ego + exo paired human capture. Glasses or head-mounted camera on a human demonstrator plus stationary exocentric cameras, the Ego-Exo4D pattern (arXiv:2311.18259).
  • Dense array (4-8 cameras). Hardware-triggered ring or arc around the workspace for reconstruction-grade coverage.
  • Sim-rendered multi-view. Arbitrary virtual cameras rendered from simulation, optionally mixed with real data.

Master Comparison Table

Approach Rig Build Capture Cost / Hr Sync Difficulty Occlusion Coverage Human-Video Pretraining Main Risk
Single exo + wrist $2,500 to $4,000 $20 to $28 Low Partial No Blind spots remain
DROID-style (2 exo + wrist) $4,500 to $7,000 $26 to $38 Moderate Good No Calibration upkeep
Ego + exo paired $6,000 to $9,500 $32 to $44 High (moving ego cam) Good Yes Ego-exo time alignment
Dense array (4-8 cams) $9,000 to $12,000 $40 to $50 High (trigger/genlock) Excellent No Storage, diminishing returns
Sim-rendered multi-view Compute only ~$1 to $5 equivalent None Perfect Limited Sim-to-real gap

Rig and capture figures are DexSet benchmarks, including session calibration checks and QA; sim figures are rough GPU-time equivalents.

Where Each Approach Wins and Loses

Single exo + wrist earns its place as a starting point. Pros: cheapest real multi-view, simple calibration (one extrinsic pair), enough to break the wrist-only occlusion ceiling for many tasks. Cons: one blocked view and you are back to single-view; no view redundancy for QA cross-checks. We recommend it for prototyping and single-task policies, and we recommend planning the mount points for camera two on day one.

DROID-style is the workhorse, and not by accident. Two external views mean occlusion of one is usually covered by the other; three total views give the RoboMimic-style observation flexibility that lets ML teams ablate view combinations later (arXiv:2108.03298). Cons: per-session calibration verification becomes mandatory, because three cameras drift three ways. In our operations the added QA overhead is roughly 5 percent of session time. This is what we quote when a VLA team asks for a default.

Ego + exo paired solves a different problem: it is the only configuration that produces the ego-exo correspondences needed to pretrain on human demonstration video and transfer to robot viewpoints, the exact gap Ego-Exo4D was built to close. Pros: human demonstrators are fast and cheap per episode; the data doubles as a bridge to large human-video corpora. Cons: the ego camera moves, so extrinsics to the world frame change every frame and must be recovered via SLAM or the glasses’ own tracking; time alignment between glasses and fixed cameras is the hardest sync problem on this list. Choose it when your training strategy explicitly includes human video.

Dense arrays buy near-complete coverage and reconstruction-grade geometry for humanoid whole-body work and world-model data. The cons compound quietly: hardware triggering or genlock is effectively mandatory, storage runs 3 to 4 TB per capture day at 1080p30 in our pipelines, and, in every ablation we have run on single-arm manipulation, cameras five through eight never moved the success metric. Buy this coverage for reconstruction, not for policy learning on tabletop tasks.

Sim-rendered multi-view costs almost nothing per additional view, which is genuinely useful for view-invariance augmentation and architecture prototyping. But every rendered view inherits the simulator’s gap in contact dynamics, materials, and lighting. Teams in the Open X-Embodiment consortium (arXiv:2310.08864) mix sim and real rather than substituting one for the other, and that matches our experience: sim views stretch a real multi-view dataset, they do not replace it.

Decision Matrix: Match the Approach to the Program

Your Situation Recommended Approach
Prototyping one task, tight budget Single exo + wrist, mounts pre-planned for a second exo
Training VLA / manipulation foundation data at scale DROID-style (2 exo + wrist)
Pretraining on human demonstrations or video Ego + exo paired
Humanoid whole-body, reconstruction, world models Dense array, hardware-triggered
Need view diversity beyond rig budget DROID-style real capture + sim-rendered augmentation

One category the table cannot capture: switching costs. Moving from single-exo to DROID-style mid-program is cheap if the mount points and calibration workflow were planned for it, and painful if they were not, because your existing episodes and your new episodes will differ in geometry and your training pipeline has to reconcile them. Moving from robot-only capture to ego+exo is a bigger jump; it changes your demonstrator pool, your sync architecture, and your annotation scheme at once. Teams that expect to make either move should write the target configuration into their schema now, even if the extra cameras arrive next quarter.

Two cross-cutting rules. First, whatever you choose, log extrinsics and sync metadata into every episode; the approach you pick today is the aggregation problem someone inherits in two years. Second, ablate before you scale: run 20 hours in the candidate configuration, train, and let the success metric pick the rig.

Turn the Matrix Into a Procurement Rubric

If you are scoping a capture program or comparing vendors, download the Multi-View Rig RFP Scorecard. It turns this decision matrix into weighted evaluation questions on calibration verification, sync tolerances, and deliverable formats, the same rubric we hold our own rigs to.

Request a Quote

Frequently Asked Questions

What is the cheapest way to get multi-view robot data?

A single external camera plus a wrist camera, at roughly $2,500 to $4,000 for the rig and $20 to $28 per operated capture hour in DexSet benchmarks. It breaks the wrist-only occlusion ceiling for many tasks but leaves blind spots a second external view would cover.

For VLA training data and tabletop manipulation at scale, usually yes: the second external view covers occlusions the first misses and enables view ablations later. The premium over single-exo is about $2,000 to $3,000 in rig cost and $6 to $10 per hour.

No. Rendered views are nearly free and useful for view-invariance augmentation, but they inherit the simulator’s gaps in contact dynamics, materials, and lighting. Production programs mix sim views with real calibrated capture rather than substituting.

When your training plan includes learning from human demonstration video. Paired capture, as in Ego-Exo4D, provides the cross-view correspondences needed to transfer first-person human priors to third-person robot viewpoints.

Task-dependent, but in our single-arm manipulation ablations, cameras beyond the third stopped moving policy success while adding roughly 25 percent storage and QA cost per view. Dense arrays of 4 to 8 cameras are justified for reconstruction and whole-body humanoid work, not tabletop policies.