Skip to main content

Dexset

dexset
Menu
Request a Demo

Tag: Egocentric Data

Comparing Data Capture Hardware for Embodied AI Approaches: Pros, Cons and Costs

Posted on by Sainath Gupta

Our prediction, held at roughly 80 percent confidence: within two years, no serious VLA training stack will rely on a single capture approach, and the teams still arguing egocentric versus teleoperation will have lost the argument to “both.” That is an opinion, clearly flagged; the evidence behind it is not. Three camps currently give three confident, incompatible answers to the data collection question: strap cameras on humans, build teleop rigs, or ring the workspace with depth cameras. Each camp has papers proving its approach works, and none of the papers tell you what each approach costs per usable hour, or which one fits the model you are actually training.

The argument persists because these approaches produce different data, not the same data at different prices. That is the thesis this comparison argues: the four approaches are complements delivering different supervision signals, so the choice follows from your training objective. Egocentric human video has no robot actions in it. Teleoperation produces perfect action labels at real-time speed and real-time cost. Exocentric camera arrays see everything except the first-person view your humanoid will actually have. Choosing between them by hardware price alone is a category error.

This post compares the four capture approaches side by side: what hardware each needs, what data each yields, what each costs on our floors, and a decision matrix for matching approach to training objective. It condenses the hardware detail from our full guide to data capture hardware for embodied AI.

I run teleoperation systems at DexSet, where all four approaches operate in production, so the pros and cons below come from shift logs rather than spec sheets.

Key Takeaways – The four capture approaches are egocentric wearables, teleoperation, handheld demonstration devices (UMI-style), and multi-view exocentric arrays. They produce different modalities and are complements, not substitutes. – Egocentric capture is the cheapest per hour (a ~$400 GoPro seat) but carries no robot action labels; teleoperation is the most expensive but yields training-ready observation-action pairs. – Teleop hardware spans two orders of magnitude: ~$300 GELLO leader arms to $20,000+ ALOHA-class bimanual stations, with our delivered benchmarks at $28-60 per hour. – Most VLA training stacks want a blend: broad egocentric pretraining data plus targeted teleop fine-tuning data plus a small exo set for evaluation.

Approach 1: Egocentric Wearable Capture

Egocentric capture records first-person video and motion from devices worn by humans doing real tasks, no robot required. It is the approach behind Ego4D and Ego-Exo4D (arxiv.org/abs/2311.18259), and the standard source of large-scale pretraining data for humanoid perception.

Hardware options: GoPro head mounts (~$400, 5.3K/60, ~150 g), Meta Quest 3 (~$500, adds hand tracking and 6DoF pose), Apple Vision Pro ($3,499, adds eye tracking and LiDAR), and Project Aria research glasses (research access, best-in-class sensor sync at ~75 g).

Pros: lowest cost per hour of any approach; scales with people instead of robots; captures true task diversity in homes, kitchens, and warehouses; light devices sustain 6+ hour shifts.

Cons: no robot action labels, so it cannot train a policy by itself; consumer devices have weak cross-device sync; gaze and hand pose require the expensive end of the hardware range.

Approach 2: Teleoperation Capture

Teleoperation capture records a human operator controlling a real robot, producing synchronized observation-action pairs, which is exactly the supervision imitation learning consumes. The chain is direct: teleoperation produces demonstrations, demonstrations train ACT- and diffusion-policy-style models, and those methods feed VLA systems like OpenVLA and pi-0.

Hardware options: ALOHA-class leader-follower stations (~$20,000+ per bimanual cell, the platform behind ACT, arxiv.org/abs/2304.13705), GELLO printed leader arms (~$300 per arm on robots you already own, arxiv.org/abs/2309.13037), and VR teleop with a Quest 3 or Vision Pro streaming head and hand pose to the robot.

Pros: training-ready action labels at capture time; joint-space leader arms give fine bimanual control; output drops straight into LeRobot or HDF5 episode formats.

Cons: cost scales with robots, not just people; one real-time hour per data hour per rig; resets and calibration eat 30 to 40 percent of every shift; operator skill affects data quality.

Approach 3: Handheld Demonstration Devices

Handheld demonstration devices are robot-free grippers, UMI being the reference design, that a human carries through tasks while a wrist-mounted camera records; robot actions are recovered afterward via SLAM (arxiv.org/abs/2402.10329). The approach splits the difference between egocentric video and teleoperation.

Hardware: a 3D-printed UMI-style gripper plus a fisheye GoPro, roughly $400 per unit.

Pros: near-egocentric cost with usable action supervision; natural human motion, no retargeting latency; trivially parallel, ten grippers cost less than one robot arm.

Cons: no proprioception or force signal; action quality depends on SLAM post-processing, which becomes its own QA pipeline; single-gripper design limits bimanual tasks.

Approach 4: Multi-View Exocentric Arrays

Multi-view exocentric capture surrounds a workspace with synchronized third-person cameras, recording the scene from four to twelve angles at once. DROID standardized this pattern across labs (arxiv.org/abs/2403.12945), and it is the default source of evaluation and cross-view data.

Hardware options: Intel RealSense D435i (~$300-350) or D455 (~$420-450, wider baseline, hardware sync pin), Stereolabs ZED 2i (~$500, neural depth, needs an NVIDIA GPU per node), Luxonis OAK-D (~$250-350, on-device compute).

Pros: metric depth and full-scene coverage; hardware sync solves the alignment problem RealSense-style; fixed rigs need no operator wearables at all.

Cons: not the robot’s viewpoint; per-studio infrastructure (sync wiring, lighting, terabyte-per-shift storage) roughly doubles the camera bill; useless for capturing task diversity outside the cell.

What the Landmark Datasets Chose

The published datasets that shaped this field each committed to one of these approaches, and their choices are useful evidence. Ego4D collected thousands of hours of unscripted daily activity on head-mounted cameras because egocentric scale was the objective and action labels were not. ALOHA and Mobile ALOHA (arxiv.org/abs/2401.02117) went all-in on leader-follower teleoperation because fine bimanual action supervision was the entire point. DROID distributed identical RealSense-based exocentric rigs to dozens of labs to get multi-view consistency across sites. And Ego-Exo4D deliberately paired Aria glasses with stationary cameras, an explicit bet that ego plus exo beats either alone.

None of these teams picked their hardware by price. They picked by the supervision signal their research needed, then engineered the cost down. That is the right order of operations for a production pipeline too.

The Master Comparison Table

Approach Typical Hardware Capex per Seat/Cell Data Yielded Action Labels Our Delivered Cost Benchmarks Best For
Egocentric wearable GoPro ~$400, Quest 3 ~$500, Vision Pro $3,499, Aria (research) $400-4,000 First-person RGB, IMU, pose, gaze (device-dependent) None Lowest per hour (one third to one half of teleop) Pretraining scale, humanoid viewpoint priors
Teleoperation GELLO ~$300/arm, ALOHA ~$20k station, VR teleop $500-3,499 + robot $600-30,000 Obs-action pairs, proprioception, 50 Hz control Native, highest quality $28-60/hr Policy fine-tuning, bimanual manipulation
Handheld (UMI-style) Printed gripper + GoPro, ~$400 $400-600 Wrist-cam RGB + SLAM-recovered trajectories Recovered, no force/proprio Between egocentric and teleop Cheap single-arm demos at volume
Exocentric array 4-8x D455/ZED 2i/OAK-D $2,500-12,000 per studio Multi-view RGB-D, synced None (pairs with teleop) Studio amortizes across every episode captured inside it Evaluation, cross-view consistency, world models

Decision Matrix: Matching Approach to Training Objective

The right capture approach is determined by what your model consumes, and the mapping is more stable than hardware fashion.

  • Pretraining a humanoid perception backbone: egocentric wearables first. Buy GoPro seats for volume; add Aria or Vision Pro only where gaze labels earn their cost.
  • Fine-tuning a VLA policy on manipulation tasks: teleoperation. GELLO on existing arms if you own robots; an ALOHA-class cell if bimanual dexterity is the product.
  • Thousands of single-arm demos on a budget: UMI-style handheld grippers, with a budgeted SLAM QA pass.
  • Evaluation sets and world-model data: an exo array of hardware-synced D455s wrapped around one teleop cell, so every episode is captured ego and exo simultaneously.
  • All of the above on a deadline: that blend is precisely what a data vendor amortizes across clients; the build-versus-buy arithmetic is in the full hardware guide.

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

Which robot data collection approach is cheapest?

Egocentric wearable capture. A ~$400 GoPro head-mount seat produces first-person video at one third to one half the per-hour cost of teleoperation in our benchmarks, but it yields no robot action labels.

For policy training, yes. Teleoperation is the only approach that natively records synchronized observation-action pairs with proprioception, which is the supervision imitation learning and VLA fine-tuning require.

UMI is a handheld, robot-free gripper design with a wrist-mounted fisheye GoPro; actions are recovered by SLAM in post-processing. Use it when you need thousands of single-arm demonstrations at near-egocentric cost and can accept the absence of force and proprioception data.

For evaluation and cross-view generalization, a small one helps. Wrapping four to six hardware-synced RealSense D455s around one teleop cell gives you paired ego-exo episodes, the structure Ego-Exo4D demonstrated matters for viewpoint generalization.

Yes, and most serious VLA stacks do: broad egocentric data for pretraining, teleop episodes for fine-tuning, and exo views for evaluation. The practical requirement is one shared episode format and honest per-source metadata.

[Download Sample Data] from all four capture approaches, or start with the complete guide to data capture hardware for embodied AI.

The Complete Guide to Data Capture Hardware for Embodied AI (2026)

Posted on by Sainath Gupta

A Head of Data on a scoping call last quarter asked us which single capture device her team should standardize on, and for the first time in months, we had no quick answer. We had run every device she named in production. The problem was the question: it has no device-shaped answer, because a capture device is not a purchase, it is a commitment. The hardware you choose locks your pipeline into what that device can timestamp, sync, and survive, and teams that fail on data usually failed at that moment, months before anyone touched a model. That is this guide’s thesis: capture hardware is a data-strategy decision, judged on seven axes and on cost per delivered hour, never on spec sheets.

The reason the question stumps buyers is that capture hardware for embodied AI sits in a gap between consumer electronics and lab equipment. A Quest 3 was built for gaming, not for exporting synchronized RGB, IMU, and hand pose at training quality. A RealSense was built for prototyping, not for 40 units running ten hours a day. Vendor spec sheets answer none of the questions that matter for foundation model training, so teams learn the hard lessons after the purchase order clears.

This guide gives you the full picture: every major hardware category (headsets, research glasses, action cameras, stereo depth cameras, teleoperation rigs, handheld grippers, tactile sensors), compared on the seven axes that decide whether your dataset trains a VLA model or fills a hard drive. You will leave with real prices, a cost-per-hour model, and a decision matrix mapped to the three capture strategies that matter: egocentric human data, teleoperation, and multi-view exocentric capture.

At DexSet we run these devices in production. Our teams operate teleop cells, headset fleets, and multi-camera capture studios that deliver egocentric, exocentric, mono, and stereo data to robotics foundation model teams. Every number in this guide comes from our own rigs, published research, or a linked primary source.

TL;DR: Key Takeaways – Data capture hardware for embodied AI spans four categories: wearable egocentric devices ($350 to $3,499), stereo depth cameras ($250 to $500 per unit), teleoperation rigs ($300 per arm to $20,000+ per station), and tactile sensors. – The seven evaluation axes are price, resolution/fps, depth quality, sync capability, wearability, SDK maturity, and data formats. Sync capability is the axis teams most often ignore and most often regret. – A Meta Quest 3 at ~$500 covers most teleoperation UI needs; the Apple Vision Pro at $3,499 earns its price only for eye tracking and high-resolution passthrough research. – Teleop capex is lower than most CTOs assume: a GELLO leader arm costs roughly $300 per arm, against $20,000+ for a full ALOHA-class bimanual station. – In our production benchmarks, delivered teleop data runs $28 to $60 per hour depending on rig and task complexity; egocentric human data runs meaningfully cheaper per hour but trains different capabilities.

What Is Data Capture Hardware for Embodied AI?

Data capture hardware for embodied AI is the set of physical devices, wearables, cameras, teleoperation interfaces, and sensors, used to record the visual, spatial, inertial, tactile, and action data that trains robots and vision-language-action (VLA) models. It is the acquisition layer of the robot learning stack: everything between the physical world and your training dataset.

The category matters because embodied AI models learn from demonstrations, not from text scraped off the internet. A VLA model like OpenVLA (arxiv.org/abs/2406.09246) or Physical Intelligence’s pi-0 (arxiv.org/abs/2410.24164) consumes synchronized streams of camera frames, proprioception, and actions. The hardware that records those streams sets a hard ceiling on model quality. No amount of post-processing recovers depth a sensor never measured or a timestamp a device never wrote.

Core Concepts and Data Modalities

Data modalities are the distinct sensor streams a capture device records, and every hardware decision starts by listing which modalities your model actually needs. The main ones for embodied AI are RGB video, stereo pairs, depth, IMU, eye gaze, hand and body pose, robot proprioception, and tactile signal.

Mono vs. Stereo RGB

Mono RGB is a single camera stream; stereo RGB is a calibrated pair that allows depth to be recovered by triangulation. Mono is cheaper and lighter and dominates large egocentric datasets like Ego4D (arxiv.org/abs/2110.07058). Stereo gives your model, or your post-processing pipeline, metric 3D structure, which matters for manipulation policies that must reason about distance to a grasp point. On our rigs, stereo roughly doubles storage and calibration workload, so we specify it only when the downstream model consumes it.

Depth

Depth is a per-pixel distance measurement, produced either by active sensors (structured light, time-of-flight) or by stereo matching. Active depth from a RealSense-class sensor works well indoors at 0.3 to 6 meters but degrades in sunlight and on reflective surfaces. Neural stereo depth (ZED 2i class) extends range but requires a GPU at capture time or in post.

IMU and Time Synchronization

An IMU (inertial measurement unit) records acceleration and angular velocity, typically at 200 Hz to 1,000 Hz, and it is only useful if its clock aligns with the camera clock. Cross-device sync is the silent killer of embodied AI datasets: a 40 ms offset between video and proprioception is invisible in playback and fatal for action-labeled training data. Devices differ enormously here. Project Aria timestamps all sensors on a shared clock; a GoPro gives you a self-consistent file but no native way to sync with a robot’s ROS 2 clock without an external signal.

Proprioception and Action Data

Proprioception is the robot’s own joint state (positions, velocities, torques), and action data is the command stream sent to the robot. These come from the robot or the teleoperation interface, not from cameras, which is why teleop rigs are capture hardware in their own right. Imitation learning methods like ACT, introduced with the ALOHA rig (arxiv.org/abs/2304.13705), train directly on this stream. The entity chain to keep straight: teleoperation produces demonstrations, demonstrations feed imitation learning, imitation learning trains the VLA policies that run on robots.

Tactile

Tactile sensing records contact geometry and force at the gripper, usually via vision-based sensors in the GelSight style, where a camera watches a deformable gel pad. Tactile is the newest modality in production pipelines and the least standardized; we cover the hardware below.

The Four Hardware Categories

Capture hardware for embodied AI falls into four categories: wearable egocentric devices, stereo and depth cameras for exocentric capture, teleoperation rigs, and handheld or tactile interfaces. Most production pipelines combine at least two.

Category 1: Wearable Egocentric Devices

Wearable egocentric devices are head-mounted cameras and headsets that record the world from a human’s first-person viewpoint, the same viewpoint a humanoid robot’s head camera will have. This category powers egocentric pretraining datasets like Ego-Exo4D (arxiv.org/abs/2311.18259) and doubles as the operator interface for immersive teleoperation systems like Open-TeleVision (arxiv.org/abs/2407.01512).

Device Price Sensors & Resolution Depth Sync Wearability (8h shift) SDK Maturity Native Data Formats
Meta Quest 3 ~$500 Color passthrough cameras, 6DoF inside-out tracking, hand tracking, IMU Passthrough depth estimate, not export-grade Good within headset; WebXR/OpenXR timestamps for teleop Moderate; 515 g, front-heavy after ~2h Strong (OpenXR, Meta SDK, large dev community) App-level capture; pose + hand joints via OpenXR
Apple Vision Pro $3,499 High-res micro-OLED displays, high-quality color passthrough, eye tracking, hand tracking, LiDAR LiDAR scene depth Excellent within device; ARKit timestamps Poor for long shifts; 600-650 g, external battery Strong but restrictive (visionOS, ARKit); raw camera access gated ARKit poses, hand skeletons; limited raw sensor export
Project Aria / Aria Gen 2 Research access (not retail) RGB camera, 2 mono SLAM cameras, eye tracking cameras, spatial mics, dual IMUs (Gen 2 adds PPG, improved compute) No active depth; SLAM-derived Best in class; all sensors on shared clock, VRS format Excellent; ~75 g glasses form factor Good (Project Aria Tools, MPS cloud services) VRS files, MPS trajectories, gaze (projectaria.com)
GoPro head mount (Hero-class) ~$400 Single wide FOV RGB, 5.3K/60 or 4K/120, IMU telemetry None Weak across devices; GPMF telemetry internal only Good; ~150 g with mount Moderate (GPMF parsers, no live SDK) MP4 + GPMF metadata track

The pattern to notice: the $3,499 device is not seven times better than the $500 device for data capture. Vision Pro wins when you need eye gaze and the highest-fidelity passthrough for perception research or premium teleop. Quest 3 wins on fleet economics: at our scale, one damaged Vision Pro costs the same as seven Quest 3 replacements. Aria wins on data quality per gram but is research-access only, which makes it a partnership decision rather than a purchasing decision. The GoPro remains the workhorse for raw egocentric video volume, and it is the camera inside the UMI gripper stack discussed below.

Category 2: Stereo Depth Cameras for Exocentric Capture

Stereo depth cameras are fixed or tripod-mounted sensors that record a scene from outside the actor, providing the third-person views used in multi-view exocentric datasets and in robot workspace monitoring. Datasets like DROID (arxiv.org/abs/2403.12945) standardized on this class of sensor.

Device Price RGB / Depth Resolution & FPS Depth Quality Sync Capability SDK Maturity Data Formats
Intel RealSense D435i ~$300-350 Depth 1280x720 @ up to 90 fps; RGB 1080p @ 30; IMU Good 0.3-3 m; global-shutter depth sensors Hardware sync pin across units Mature (librealsense, ROS 2 wrapper) ROS 2 bags, .bag/.ply, raw streams
Intel RealSense D455 ~$420-450 Depth 1280x720 @ 90 fps; wider 95 mm baseline; global-shutter RGB; IMU Better than D435i at 4-6 m range Hardware sync pin Mature (same stack) Same as D435i
Stereolabs ZED 2i ~$500 Stereo 2K @ 15, 1080p @ 30, 720p @ 60; IMU, barometer, magnetometer Neural stereo depth, usable 0.3-20 m Software sync; multi-cam via ZED Hub Mature but GPU-dependent (ZED SDK, needs NVIDIA) SVO recordings, ROS 2 wrapper
Luxonis OAK-D ~$250-350 4K RGB + 2x mono stereo pair; on-device compute Good short-range stereo depth Hardware FSYNC on some variants Growing (DepthAI, ROS 2 driver) DepthAI streams, ROS 2

For multi-view exo capture, the deciding axis is sync. RealSense’s hardware sync pin lets you drive eight cameras off one trigger, which is why it still dominates academic capture cells despite Intel’s on-again-off-again commitment to the product line. ZED 2i buys you longer range and better outdoor behavior at the cost of an NVIDIA GPU per capture node. OAK-D is the value play when you want on-device neural inference to filter or annotate at capture time.

Category 3: Teleoperation Rigs

Teleoperation rigs are the leader-follower arm pairs, exoskeletons, and VR-based control systems through which a human operator produces robot action data. This is the category where hardware choice most directly sets your cost per demonstration.

System Price What It Is Action Fidelity Operator Fatigue SDK / Ecosystem Data Formats
ALOHA / ALOHA 2 rig ~$20,000+ per bimanual station Two leader + two follower arms, joint-space mapping, 50 Hz High; fine bimanual tasks Moderate; seated, arms supported Strong research ecosystem (ACT, Mobile ALOHA, LeRobot support) HDF5 episodes; LeRobot dataset format
GELLO ~$300 per arm (plus follower robot) 3D-printed, Dynamixel-based kinematic-twin leader arm (arxiv.org/abs/2309.13037) High for the matched follower Low-moderate Open source, LeRobot-compatible Joint streams to ROS 2 / LeRobot
UMI gripper ~$400 build (printed gripper + GoPro) Handheld gripper with wrist-mounted fisheye GoPro; robot-free demonstrations (arxiv.org/abs/2402.10329) Good; SLAM-recovered actions, no proprioception Low; natural human motion Open source, active community MP4 + GPMF, post-processed to episodes
VR teleop (Quest 3 / Vision Pro + arms) $500-$3,499 + robot Headset streams head/hand pose to robot controller (Open-TeleVision style) Good; depends on retargeting Moderate; passthrough latency drives fatigue Growing; OpenXR + custom stacks Pose streams + robot logs
Tactile (DIGIT / GelSight-style) A few hundred dollars per fingertip (DIGIT-class); research-grade GelSight units cost thousands Vision-based tactile sensor in gripper fingertips Adds contact-rich signal N/A (robot-mounted) Moderate (open DIGIT designs; PyTouch-era tooling) Compressed tactile image streams

The $20,000 ALOHA figure and the $300 GELLO figure describe different points on the same curve, not competing products. ALOHA is a complete bimanual station including four arms. GELLO is a leader-arm design you bolt onto follower robots you already own. UMI removes the robot from data collection entirely, which is why it produces the cheapest demonstrations and the weakest proprioception. Every serious pipeline we run mixes at least two of these.

Category 4: Multi-View Exocentric Studios

A multi-view exocentric studio is an array of four to twelve synchronized cameras surrounding a workspace, recording the same manipulation from multiple third-person angles. Ego-Exo4D paired Aria glasses with four to five stationary GoPros per scene precisely because models trained on both viewpoints generalize better than either alone. Budget $2,500 to $6,000 per studio in cameras and mounts, then double it for the sync, lighting, and storage infrastructure that spec sheets never mention.

Decision Matrix: Which Hardware for Which Use Case

The right hardware follows from the capture strategy, and there are three dominant strategies: egocentric human data, teleoperation, and multi-view exocentric capture.

Use Case First Choice Budget Alternative Premium Option Avoid
Egocentric human data at scale (pretraining) GoPro head mounts (~$400/operator) Used Hero-class cameras Project Aria via research partnership Vision Pro (cost, battery, comfort)
Egocentric with gaze + trajectories Project Aria / Aria Gen 2 Quest 3 with logging app Aria Gen 2 GoPro (no gaze, no shared clock)
Immersive teleoperation UI Quest 3 (~$500) Quest 3 Vision Pro (eye tracking, passthrough fidelity) GoPro (not a display)
Bimanual fine manipulation demos ALOHA-class station (~$20k) GELLO leaders on existing arms (~$300/arm) ALOHA 2 with tactile fingertips UMI (no bimanual joint coupling)
Cheap single-arm demonstrations at volume UMI grippers (~$400 each) GELLO + one follower arm ALOHA station Vision Pro teleop (capex per seat)
Multi-view exo studio 4-8x RealSense D455, hardware-synced OAK-D array ZED 2i array with GPU nodes Unsynced consumer webcams
Contact-rich tasks DIGIT-class fingertips on any rig above Single tactile finger + RGB GelSight-style research sensors Skipping tactile, then labeling contact by eye

Cost, Economics, and ROI

Capture hardware economics are dominated by throughput and operator time, not by device price, so the correct unit of analysis is cost per delivered data hour. This is the section competitors leave out, so here are the numbers we actually use.

Capex per seat, typical ranges:

  • Egocentric human capture seat: $400 to $700 (camera, mount, spare batteries, storage cards)
  • VR teleop seat: $1,200 to $5,500 (headset, workstation, networking; excludes the robot)
  • GELLO teleop seat: $600 to $1,000 on top of an existing follower arm
  • ALOHA-class bimanual cell: $22,000 to $30,000 fully commissioned
  • Exo studio (6 cameras): $2,500 to $6,000 in sensors, roughly the same again in sync, mounting, lighting, and storage

Cost per delivered hour, our production benchmarks: teleoperation data lands at $28 to $60 per hour depending on rig type, task complexity, and QA depth. Egocentric human data typically lands at one third to one half of that, because a human wearing a camera needs no robot, no retargeting stack, and far less resets-per-hour overhead. These are the typical ranges we see across our own pipelines; treat them as planning numbers, not quotes.

The ROI logic is straightforward. A $20,000 ALOHA cell running two shifts amortizes to under $2 per data hour within a year; the operators cost ten times that. Buying cheaper hardware that cuts operator throughput by 20 percent is a false economy. Buying $3,499 headsets when $500 ones sustain the same demonstrations per hour is the opposite mistake.

Case Study: A Humanoid Foundation Model Team’s Capture Stack

The proof is in a deployment, so here is an anonymized configuration from our own work. A humanoid foundation model team came to us needing three data types simultaneously: egocentric human video for pretraining, bimanual teleop demonstrations for fine-tuning, and multi-view exo recordings for evaluation.

We stood up three lines. Line one: 30 GoPro head-mount seats for kitchen and warehouse egocentric capture, delivering several hundred hours per week. Line two: four ALOHA-class stations plus six GELLO-equipped single-arm cells, producing task-labeled demonstrations in LeRobot format (github.com/huggingface/lerobot). Line three: a six-camera RealSense D455 studio, hardware-synced, wrapped around one teleop cell for paired ego-exo evaluation episodes.

Two hardware lessons generalized. First, the D455’s wider baseline halved our depth QA rejections at 2 to 4 meters compared with the D435i units we started with. Second, moving VR teleop from Vision Pro pilots to Quest 3 fleet units cut capex per seat by 85 percent with no measurable drop in demonstrations per operator-hour on pick-and-place classes of tasks. The Vision Pros stayed in service for the subset of tasks where gaze labels earned their cost.

Get the RFP Scorecard

A hardware decision this size deserves a structured evaluation, so we built one you can reuse. The DexSet Data Capture Hardware RFP Scorecard is a one-page matrix that scores any device or vendor on the seven axes in this guide (price, resolution/fps, depth quality, sync, wearability, SDK maturity, data formats) plus delivery terms and QA guarantees. Teams use it to compare quotes on equal footing instead of vendor spec sheets. Download it below, and if you want the comparison done for you, that is literally our job.

Related reading from this series:

Next Step

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

What is data capture hardware for embodied AI?

Data capture hardware for embodied AI is the set of wearable cameras, headsets, depth sensors, teleoperation rigs, and tactile sensors used to record the synchronized visual, inertial, action, and contact data that trains robot policies and vision-language-action models.

Only for specific needs. The Vision Pro’s eye tracking and passthrough fidelity justify its $3,499 price for gaze-labeled data and premium teleoperation research. For fleet teleop and general egocentric capture, the ~$500 Quest 3 delivers comparable demonstrations per operator-hour at one seventh the capex.

Between $300 and $30,000 depending on class. A GELLO leader arm costs roughly $300 per arm on top of a follower robot you already own; a UMI handheld gripper builds for about $400; a complete ALOHA-class bimanual station runs $20,000 or more fully commissioned.

Egocentric capture records from the actor’s first-person viewpoint using head-mounted devices; exocentric capture records the actor from external, third-person cameras. Datasets like Ego-Exo4D pair both because policies trained on combined viewpoints generalize better.

Because embodied AI models train on aligned observation-action pairs. A clock offset of tens of milliseconds between camera, IMU, and robot proprioception silently corrupts action labels, and it cannot be fixed after the fact without shared timestamps or an external sync signal.

For hardware-synchronized multi-view rigs, the Intel RealSense D455 (~$420-450) is the default choice because of its sync pin, 95 mm baseline, and mature ROS 2 support. The ZED 2i suits longer-range or outdoor scenes if each node has an NVIDIA GPU.

DexSet already runs the rigs in this guide at production scale. [Download the RFP Scorecard] to evaluate vendors on equal terms, or [Book a Demo] to see sample egocentric, teleop, and multi-view datasets from these exact devices. [Download Sample Data] is one click away.

Case Study: How We Scaled Egocentric Data Collection for Robotics for a VLA Model

Posted on by Sainath Gupta

Picture yourself as the head of data at a humanoid startup: your researchers have just validated the experiment that could anchor the next funding round, co-training human egocentric video with your teleop corpus improves policy generalization, and then you run the arithmetic on your own collection capacity and find the experiment wants ten times more human data than your team can produce this year. That is exactly where one humanoid foundation model team (unnamed here by agreement) was standing when they contacted us. Their result was in line with what EgoMimic-style research had reported (arXiv:2410.24221); their teleop dataset was around 400 hours and growing slowly.

Why was a well-funded team stuck? Because collection was competing with research for the same people. Every hour their engineers spent managing capture logistics was an hour not spent on training runs, and their pilot attempts at crowdsourced collection had returned footage with unusable sync and a rejection rate far beyond anything a program can absorb. Familiar story; we hear a version of it most months.

This post is the anonymized production log of what happened next: how we scoped, staffed, and delivered 4,000 QA-passed hours of egocentric data in 14 weeks, what it cost per hour, where we lost yield, and the three decisions that mattered most. Numbers are real from our program tracking, rounded to protect the client. The thesis the whole log supports: scaled collection is a manufacturing problem, and the returns came from three protocol decisions made before hour one, not from capture volume.

Key Takeaways – 4,000 QA-passed egocentric hours delivered in 14 weeks across three task families (kitchen, warehouse shelving, assembly bench) for a humanoid VLA co-training pipeline. – Blended cost landed in the $22 to $34 per hour band: mono-heavy capture for breadth, stereo for depth-critical tasks, dense hand-pose annotation on only the 30 percent of hours flagged retarget-critical. – First-pass QA yield started at 71 percent in week one and reached 86 percent by week six after protocol fixes; the delta was worth roughly $18K. – Camera height matched to the robot’s head frame and constrained “robot-plausible” motion instructions were the two protocol choices the client’s researchers valued most. – The client reported improved success on unseen-object manipulation evaluations versus their robot-only baseline after co-training on the mixed corpus.

The Brief: What “Scaled” Actually Meant

Scaling egocentric data collection means hitting a volume target at a defined quality bar and annotation depth on a fixed calendar, and all four variables were specified before we captured an hour. The brief we agreed with the client’s head of data:

  • Volume: 4,000 QA-passed hours (not raw hours; the distinction is the whole game).
  • Task families: kitchen manipulation, warehouse shelf pick-and-place, small-parts assembly bench, weighted 45/35/20.
  • Viewpoint constraint: camera height within 10 cm of the robot’s head frame; field of view overlapping the deployment camera’s.
  • Annotation: temporal task segmentation on everything; 3D hand pose and object tracks only on episodes their researchers flagged as retarget-critical, which settled at about 30 percent of hours.
  • Format: episode-structured delivery compatible with their LeRobot-based ingestion (github.com/huggingface/lerobot).

That last 30 percent decision deserves emphasis. Dense annotation on all 4,000 hours would have added roughly $10 per hour across the corpus, $40K of spend their training plan did not need. Annotation depth should follow the training mechanism; this program is why we repeat that line so often.

The Setup: Rigs, People, Protocol

Our production setup for this program combined two rig families, 26 trained collectors, and a task protocol written jointly with the client’s researchers in week zero. Rig split:

  • GoPro-class mono head rigs at 4K/60 for the breadth corpus, where the consumer was encoder pretraining and temporal understanding.
  • RealSense D455 helmet rigs for depth-critical assembly and shelving tasks, where metric 3D fed their grasp analysis.

The protocol work is what separated this from “wearing cameras at work.” Every task became a card: start state photo, goal condition, allowed object set, retry rule, and, critically, motion constraints. Collectors were instructed to keep grasps within the robot’s gripper envelope (no five-finger tricks a parallel-jaw gripper cannot imitate), avoid motions outside the robot’s reach profile, and re-stage rather than improvise when a scene drifted off-spec. We borrowed episode discipline from how public robot datasets are structured; if it cannot be expressed as a clean episode, the way entries in Open X-Embodiment are (arXiv:2310.08864), it does not ship.

The Numbers: Timeline, Yield, and Cost

Program metrics are the honest core of any case study, so here is the phase table straight from our tracking, rounded:

Phase Weeks Hours Delivered (QA-Passed) First-Pass Yield Notes
Pilot + protocol iteration 1 to 2 120 71% Blur and sync failures dominated rejections
Ramp 3 to 6 900 78% rising to 86% Fixed exposure presets; re-trained 4 collectors
Steady state 7 to 12 2,400 85 to 88% ~420 hours/week across 26 collectors
Completion + re-delivery 13 to 14 580 87% Backfilled task-family gaps flagged by client

Cost landed where the pilot predicted: mono breadth hours in the low $20s per QA-passed hour, stereo depth-critical hours with dense annotation in the low-to-mid $30s, blending to $22 to $34 across the corpus. For context on how those bands compare across rigs and vendors, the economics section of our pillar guide has the full tables: The Complete Guide to Egocentric Data Collection for Robotics.

Where did the 12 to 29 percent of rejected hours go? Motion blur in fast kitchen tasks (fixed with shutter-priority presets), IMU packet loss on two aging rigs (retired), off-protocol grasps (re-training), and a stubborn tail of episodes with ambiguous outcomes (protocol clarification). Yield improvement from 71 to 86 percent was worth roughly $18K over the program, which is why we staff a protocol engineer for the first six weeks rather than treating capture as a solved problem.

The Three Decisions That Mattered

Looking back through the retro notes, three decisions carried most of the outcome:

1. Matching viewpoint geometry before hour one. Camera height within 10 cm of the robot’s head frame and overlapping FOV meant the client’s encoder saw deployment-like images from the first batch. This costs nothing at capture time and is nearly impossible to fix afterward.

2. Robot-plausible motion constraints. Instructing collectors to stay inside the gripper envelope narrowed the embodiment gap at the source instead of hoping retargeting would absorb it. Their researchers called this the highest-value protocol clause.

3. Two-tier annotation. Dense hand pose on 30 percent of hours, selected by the researchers who consumed it, kept the budget aligned with the training plan and cut roughly $40K of unneeded spend.

An honorable mention: weekly delivery instead of end-of-program delivery. The client’s researchers trained on batch one while we captured batch four, surfaced a task-family gap in week five, and we corrected course while the collectors were still staffed. On a single-delivery program, that gap would have been discovered after demobilization, when fixing it costs triple.

The Outcome, Stated Carefully

The client reported improved success rates on unseen-object manipulation evaluations after co-training on the mixed corpus versus their robot-only baseline; direction consistent with published ego-plus-robot co-training results, and we will not dress it in precise percentages that are theirs to publish, not ours. What we can state as first-hand fact: the corpus passed their ingestion validation on first delivery, the program hit its calendar, and they extended into a second phase covering exocentric paired capture.

Opinion, clearly flagged: I think the 10x-human-to-robot-hours ratio this team converged on will look conservative within two years. Human egocentric capture scales with people; teleop scales with robot fleets, and people are the cheaper fleet.

Running This Play for Your Team

If your co-training experiments are data-starved, the fastest way to de-risk a program like this is a scoped pilot: 50 to 100 hours against your protocol, with QA reports attached. Book a scoping call and bring your task list; we will bring the yield curves from programs like this one.

Request a Quote

Frequently Asked Questions

How long does it take to collect 4,000 hours of egocentric robotics data?

In this program, 14 weeks end to end with 26 trained collectors, including a two-week pilot and protocol iteration phase. Steady-state throughput ran about 420 QA-passed hours per week.

The blended rate landed between $22 and $34 per QA-passed hour: mono breadth capture in the low $20s, stereo depth-critical hours with dense hand-pose annotation in the low-to-mid $30s.

Match camera height and field of view to the robot’s head frame, constrain grasps to the robot’s gripper envelope, keep motions inside its reach profile, and structure everything as clean episodes with defined start states and outcomes.

This program started at 71 percent first-pass yield and stabilized at 85 to 88 percent by week six. Budget for 15 to 25 percent rejection if you are starting fresh.

The client reported improved success on unseen-object manipulation evaluations after co-training the mixed corpus versus their robot-only baseline, consistent with published results such as EgoMimic. Exact figures are theirs to publish.

Comparing Egocentric Data Collection for Robotics Approaches: Pros, Cons and Costs

Posted on by Sainath Gupta

There is no best egocentric capture rig, and shopping for one is how robotics teams burn their first data budget. I have built or debugged every rig family in this post, and the pattern behind the expensive mistakes I get called in to fix is always the same: a team copied another lab’s hardware without copying the training objective that made the hardware correct. That is the thesis this comparison defends: rig choice is a downstream consequence of your training mechanism, and any comparison that ranks rigs without naming the mechanism is selling you something.

The confusion is understandable. The hardware landscape moved fast: research glasses like Aria Gen 2 appeared alongside consumer devices like the Meta Quest 3 that happen to make decent capture rigs, while GoPros and RealSense cameras have been quietly strapped to helmets for years. Each option makes a different trade between image quality, sensor completeness, cost, and how much post-processing pain you inherit.

This post compares the four egocentric rig families we run in production, plus teleoperation and simulation as the two alternatives buyers always ask about. You get pros, cons, our per-hour cost benchmarks, and a decision matrix that maps rigs to training objectives.

At DexSet we operate all of these rigs daily across egocentric, exocentric, and teleoperation programs, so the failure modes below are ones we have paid for personally.

Key Takeaways – Rig choice should follow training mechanism: mono action cams for encoder pretraining, instrumented glasses for action retargeting, stereo depth helmets for metric 3D, teleop for embodiment-matched fine-tuning. – Our finished-hour cost benchmarks: GoPro mono $15 to $20, Quest 3 $18 to $26, RealSense helmet $20 to $30, Aria-class glasses $25 to $40, teleoperation $28 to $60. – Cheap capture is not cheap data. Uncalibrated mono footage shifts cost from hardware to annotation and 3D lifting compute. – Simulation complements rather than replaces real capture; the sim-to-real gap in contact-rich manipulation remains the limiting factor. – Pilot 50 to 100 hours on your shortlisted rig and train on it before committing to a full program.

What Counts as an Egocentric Collection Approach?

An egocentric collection approach is the combination of a wearable sensor rig, a capture protocol, and a post-processing pipeline that together produce first-person training episodes. The rig gets the attention, but all three layers set your cost. A $500 headset with a sloppy protocol produces more expensive data than a disciplined GoPro program, once you price the rejected hours.

For orientation across the whole field of modalities, hardware, and economics, see our pillar: The Complete Guide to Egocentric Data Collection for Robotics. This post narrows to the comparison question.

The Master Comparison Table

Approach Hardware Cost Sensors You Get Finished Cost/Hr (Our Benchmarks) Biggest Strength Biggest Weakness
GoPro head/chest mount $350 to $550 Mono RGB (wide FOV), IMU $15 to $20 Cheapest scale; rugged No depth, no hand pose; annotation-heavy
Meta Quest 3 ~$500 Stereo passthrough, IMU, hand tracking $18 to $26 Built-in 3D hand pose Constrained capture access; middling image quality
RealSense D435i/D455 helmet $700 to $1,200 built Stereo IR + RGB, active depth, IMU $20 to $30 Metric depth for manipulation Custom build; per-rig calibration burden
Aria-class research glasses Research program access Multi-camera RGB + SLAM, 2 IMUs, eye tracking, hand tracking $25 to $40 Factory calibration, gaze, richest streams Fleet scaling gated by program access
Teleoperation (ALOHA-class) ~$20K+ per station Robot proprioception + cameras, exact actions $28 to $60 Perfect embodiment match Throughput ceiling; capex and lab ops
Simulation Compute cost Anything you render $1 to $5 equivalent Infinite variation, free labels Sim-to-real gap in contacts and materials

Costs are DexSet production benchmarks for QA-passed hours at typical annotation depths; your protocol and rejection rate will move them.

GoPro Rigs: The Volume Play

A GoPro-based rig is a mono wide-angle action camera on a head or chest mount, and it remains the highest-throughput, lowest-cost way to collect egocentric video. We run 4K/60 or 2.7K/120 depending on task speed. The cameras survive kitchens, warehouses, and weather, batteries swap in seconds, and collectors need five minutes of training.

The costs arrive downstream. Mono RGB has no metric depth, so any 3D understanding must be lifted with structure-from-motion or learned depth, and hand pose must be estimated rather than measured. For encoder pretraining on thousands of hours, that trade is usually correct; the Ego4D corpus proved how far lightly instrumented first-person video can go (arXiv:2110.07058). For action retargeting, it is usually wrong.

Choose when: you need thousands of hours for representation learning and can tolerate estimated 3D. Avoid when: your pipeline consumes metric hand trajectories.

Quest 3: The Accidental Capture Device

The Meta Quest 3 is a ~$500 consumer VR headset whose passthrough cameras and native hand tracking make it a surprisingly capable egocentric rig. The hand tracking is the point: you get approximate 3D hand pose at capture time, free, on hardware a college student can buy retail. Teams also reuse the same headset as a teleoperation interface, which keeps the hardware pool simple.

The weaknesses are real. Access to raw passthrough streams is constrained by platform policy, effective image quality trails dedicated cameras, and wearing a headset for long capture shifts changes how people move; collectors are measurably more hesitant in headset than in glasses, and that hesitancy shows up in the motion statistics of your dataset.

Choose when: hand pose matters, budget is tight, and sessions are short. Avoid when: you need long natural sessions or top-tier image quality.

RealSense Helmet Rigs: Metric Depth on a Budget

A RealSense helmet rig mounts an Intel D435i or D455 stereo depth camera on headgear, giving hardware-synced stereo, active depth, and an onboard IMU for roughly $700 to $1,200 built. The D455’s 95 mm baseline improves depth accuracy at room distances over the D435i’s 50 mm; we pick per task family. When your consumers need metric 3D (grasp point estimation, sim scene reconstruction, depth-conditioned policies), this is the price-performance sweet spot.

You pay in operations. These are custom builds, so calibration is on you, mounts loosen, IR projectors misbehave in sunlight, and USB bandwidth is a daily negotiation. Budget real engineering time or the depth streams silently degrade.

Choose when: metric depth is a hard requirement. Avoid when: you cannot staff rig maintenance.

Aria-Class Glasses: The Quality Ceiling

Aria Gen 2 research glasses are purpose-built egocentric capture devices with calibrated multi-camera arrays, dual IMUs, eye tracking, and on-device machine perception (projectaria.com). Data arrives factory-calibrated with services for SLAM and hand tracking, which deletes whole stages of your post-processing pipeline. EgoExo4D was collected on Aria hardware (arXiv:2311.18259), and retargeting work like EgoMimic used Aria capture as its human-data source (arXiv:2410.24221). Collectors also behave naturally in glasses, which matters more than teams expect.

The constraint is access: these are research program devices rather than retail products, so fleet scaling depends on program terms rather than a purchase order.

Choose when: you need retargeting-grade data with gaze and hand pose at research quality. Avoid when: your plan requires buying 200 units next month.

Teleop and Sim: The Two Non-Egocentric Alternatives

Teleoperation and simulation are the alternatives every budget conversation reaches, and both are complements rather than substitutes. Teleoperation through ALOHA-class rigs (arXiv:2304.13705) is the only approach that produces exact robot-embodiment actions, which is why fine-tuning sets are teleop even when pretraining is egocentric; the LeRobot ecosystem has standardized much of this tooling (github.com/huggingface/lerobot). Simulation generates unlimited labeled variation at $1 to $5 per equivalent hour, and remains weakest exactly where manipulation is hardest: contact dynamics, deformables, and material appearance.

Decision Matrix: Match the Rig to the Mechanism

Your Training Objective First Choice Second Choice
Visual encoder pretraining at scale GoPro mono Quest 3
Action retargeting to grippers Aria-class glasses Quest 3
Depth-conditioned manipulation RealSense helmet Aria-class glasses
Embodiment-matched fine-tuning Teleoperation (no substitute)
Domain randomization / rare events Simulation Sim + real mix

Choosing From Here

The full economics, QA gates, and rig field notes behind this comparison are in the pillar guide linked above. If you would rather see the data than read about it, book a demo and we will put sample episodes from three different rigs side by side on a call.

Request a Quote

Frequently Asked Questions

Which egocentric capture rig is cheapest per hour?

GoPro-based mono rigs are cheapest in our benchmarks at $15 to $20 per finished hour, but they shift cost into annotation and 3D lifting because they capture no depth or hand pose.

Yes, for short sessions where 3D hand pose matters and budgets are tight. Its native hand tracking provides approximate pose at capture time, though passthrough access limits and image quality keep it below dedicated rigs for long natural capture.

When downstream consumers need metric 3D: grasp point estimation, scene reconstruction for sim, or depth-conditioned policies. RealSense D435i/D455 helmet rigs are the common budget path; Aria-class glasses the research-grade one.

No. Simulation adds cheap variation and labels but still misses contact dynamics and material realism in manipulation, so real egocentric and teleop data remain necessary anchors.

Derive the choice from your training mechanism: mono for encoder pretraining, instrumented glasses or Quest 3 for retargeting, stereo helmets for metric depth, teleoperation for embodiment-matched fine-tuning. Pilot 50 to 100 hours before scaling.

Why Egocentric Data Collection for Robotics Is the Biggest Bottleneck in Physical AI

Posted on by Enoch Pakanati

In the kitchen bay of our capture floor, a collector claps once in front of her helmet camera, waits for the sync check to pass on the wall monitor, and begins unloading a dishwasher using only grasps a parallel-jaw gripper could copy. By the end of her shift she will have produced more usable first-person manipulation data than one of our bimanual teleoperation stations yields in a week. Nothing about that scene is exotic. The exotic thing is how rare it still is.

It is rare for a structural reason. Text and images were exhaust from human activity; someone else already paid to produce them. First-person manipulation data is not exhaust. Nobody films their hands doing the dishes with calibrated stereo cameras and synchronized IMUs, so every hour has to be manufactured deliberately, by someone wearing a rig or driving a teleoperation station, and then inspected before it is fit to train on. The largest open robot manipulation dataset, Open X-Embodiment, took 21 institutions pooling 22 embodiments to reach roughly a million trajectories (arXiv:2310.08864), while language models trained on trillions of tokens.

This post argues one thesis: the binding constraint on physical AI right now is the supply of QA-passed egocentric hours, not compute and not architecture. You will get the arithmetic behind that claim, our cost benchmarks for the three ways teams try to break the constraint, and a checklist for scaling collection without torching your budget.

We spend our days at DexSet manufacturing exactly this data: egocentric, exocentric, and teleoperation streams for VLA and humanoid teams. The numbers below come from our production logs, not analyst reports.

Key Takeaways

  • Robot learning is data-bound: internet-scale text corpora dwarf the largest robot manipulation datasets by many orders of magnitude, and almost no internet video is first-person with recoverable actions.
  • Teleoperation alone cannot close the gap. A well-run bimanual station yields roughly 15 to 25 usable episode-hours per week; at $28 to $60 per hour, 10,000 hours is a multi-year, seven-figure program.
  • Egocentric human capture shifts the curve: $15 to $40 per hour in our benchmarks, 3 to 5x the collection rate per operator, and it scales with people instead of robot fleets.
  • The constraint is not raw hours, it is QA-passed hours. Plan for 15 to 25 percent of captured footage to fail quality gates.
  • Scaling works when you treat collection as a manufacturing process: fixed protocols, instrumented rigs, sampled QA, and annotation depth matched to the training objective.

Why Data, Not Compute, Is the Binding Constraint

The binding constraint in a training pipeline is the input you cannot buy more of at market price, and for physical AI that input is embodiment-relevant data. Compute is rentable by the hour. Architectures are published within months of working. But there is no marketplace where you can order 10,000 hours of first-person manipulation with hand pose annotations for next quarter, because until recently the industry that manufactures it did not exist.

Look at what the flagship models actually consumed. RT-2 leaned on internet-scale vision-language pretraining precisely because robot data was scarce (arXiv:2307.15818). OpenVLA trained on 970K robot episodes from Open X-Embodiment (arXiv:2406.09246). π0 combined internet-scale pretraining with a proprietary dexterous manipulation corpus (arXiv:2410.24164). The pattern across all three: model capability tracked data supply, and every team supplemented public data with expensive private collection.

Meanwhile the largest first-person video corpus in existence, Ego4D, holds 3,670 hours (arXiv:2110.07058). Impressive for a research effort, and still less footage than a single mid-size YouTube channel uploads in a year. For robotics, most of it lacks the calibration and action annotations that policy learning needs. The shelf is nearly empty. Whoever wants the data has to make it.

The Teleoperation Ceiling

The teleoperation ceiling is the practical throughput limit of collecting robot demonstrations through human-controlled rigs, and it is lower than most roadmaps assume. ALOHA-style bimanual stations (arXiv:2304.13705) made teleop dramatically cheaper to build; the ALOHA rig famously targeted a ~$20K bill of materials against six-figure predecessors. What they did not change is the physics of throughput.

In our teleop operations, a trained operator on a healthy rig produces 15 to 25 usable episode-hours per week once you subtract resets, calibration, failed episodes, and operator fatigue. Push operators harder and quality drops before throughput rises; fine manipulation under fatigue produces exactly the jerky, hesitant trajectories you do not want a policy imitating.

Now run the arithmetic for a 10,000-hour corpus, which is a modest target by foundation model standards:

ApproachCost per QA-passed hour (our benchmarks)Realistic throughputTime to 10,000 hoursProgram cost
Teleoperation, 5 bimanual rigs$28 to $6075 to 125 hrs/week1.5 to 2.5 years$280K to $600K
Teleoperation, 20 rigs$28 to $60300 to 500 hrs/week5 to 8 months$280K to $600K + fleet capex
Egocentric human capture, 20 collectors$15 to $40400 to 700 hrs/week4 to 6 months$150K to $400K

The 20-rig teleop row hides the real problem: rig fleets need robot hardware, lab space, maintenance engineers, and operator pipelines. Twenty humans wearing $500 to $1,200 capture rigs need a protocol document and a QA queue. That asymmetry is the entire argument for egocentric capture as the scaling layer, with teleop reserved for the embodiment-matched fine-tuning set. Systems like EgoMimic have shown the combined recipe outperforms robot data alone (arXiv:2410.24221).

Why Egocentric Capture Is Hard to Scale Anyway

Scaling egocentric collection means turning a research-style capture activity into a manufacturing process with yield metrics, and most attempts stall on the same four problems.

Yield, not volume. Raw hours are easy; QA-passed hours are the product. Across our programs, 15 to 25 percent of captured footage fails gates for motion blur, dropped IMU packets, sync drift, or off-protocol behavior. Teams that do not measure yield discover it at training time, which is the most expensive place to discover it.

Calibration debt. Every uncalibrated GoPro hour you collect today is an hour of 3D lifting problems you inherit later. Devices like Aria carry factory calibration and machine perception services (projectaria.com); consumer rigs need per-unit calibration routines and periodic re-checks, and skipping them is the most common false economy we see.

Annotation as the cost center. Capture at $15 to $22 per hour feels cheap until hand pose and object tracks push finished cost to $30 to $40. The fix is not cheaper annotation; it is refusing to annotate hours your training plan does not need at that depth.

Protocol drift. Thirty collectors interpret “tidy the workspace” thirty ways. Without scripted task cards, fixed start states, and outcome labels, you end up with footage instead of episodes. The LeRobot dataset conventions (github.com/huggingface/lerobot) are a useful forcing function here: if a capture cannot be expressed as episodes with consistent schema, it probably was not collected with enough discipline.

A Scaling Checklist That Survives Contact With Reality

A scaling checklist is only useful if each item maps to a failure we have actually watched happen, so here is ours:

  • Define the training mechanism first (encoder pretraining, retargeting, co-training), then derive modalities and annotation depth from it.
  • Set a QA yield target (we hold vendors, including ourselves, to 80 percent+ first-pass) and sample-inspect every batch.
  • Match camera height and field of view to the deployment embodiment before collecting hour one.
  • Script tasks as episodes: start state, goal, outcome label, retry policy.
  • Calibrate on a schedule, not on failure.
  • Track cost per QA-passed hour weekly; it is the only number that predicts your budget.
  • Pilot 50 to 100 hours and train on them before committing to thousands.

If you want the full treatment of rigs, modalities, and economics behind this checklist, it lives in our pillar guide: The Complete Guide to Egocentric Data Collection for Robotics.

Frequently Asked Questions

Why is data the bottleneck in physical AI rather than compute?

Compute and architectures are purchasable and published, while first-person manipulation data with recoverable actions barely exists on the internet and must be manufactured deliberately. The largest robot datasets hold on the order of a million trajectories, orders of magnitude below the corpora that trained large language models.

In DexSet’s operations, a trained operator on a bimanual rig produces roughly 15 to 25 usable episode-hours per week after resets, failures, and fatigue. Scaling further requires more robot rigs, lab space, and maintenance, which is why teleop fleets scale slowly.

Our benchmarks put QA-passed egocentric capture at $15 to $40 per hour depending on annotation depth, against $28 to $60 per hour for teleoperation, with 3 to 5 times higher collection throughput per operator.

No. The embodiment gap between human hands and robot grippers means policies still need robot-embodiment demonstrations. The strongest published results co-train on both, as in EgoMimic.

Plan for 15 to 25 percent of raw captured footage to fail quality gates for blur, sync drift, sensor dropouts, or protocol violations.

From Bottleneck to Plan

If your roadmap has a data line item and a question mark next to it, start with the pillar guide above, then request a sample batch from us. We ship real episodes with real QA reports, and the pricing table is on the first page, not behind a discovery call.

Request a Quote

Teleop, Egocentric, Exocentric, or Handheld: Comparing Robot Training Data Approaches by Cost

Posted on by Sainath Gupta

In February, we priced the same 5,000-hour manipulation corpus three different ways for one buyer: all bimanual teleoperation at a blended $44 per hour, a 65/35 egocentric-to-teleop mix at just under $30, and a handheld-gripper-heavy plan in between. Same task list, same acceptance spec. The spread between the first two plans came to roughly $70,000.

I build teleop cells for a living, so this is not an argument against teleoperation. It is the thesis that quoting exercise made unavoidable: modality selection is a budgeting decision, and the right modality for each training objective is the cheapest one that actually satisfies it. For plenty of what your model needs to learn, the premium teleop hour is simply the wrong purchase.

Teams get this wrong in both directions. Some buy 20,000 hours of premium teleop and burn budget teaching their encoder what a kitchen looks like, a job $18-per-hour egocentric video does fine. Others go all-in on cheap human video and then discover their policy has beautiful representations and no idea how to move an actual gripper.

The reason the mistake is so common is that modality costs and modality capabilities are usually discussed separately. Cost tables live in procurement decks; capability arguments live in arXiv papers. This post puts them in one place, with DexSet’s operating benchmarks attached, so you can match each dollar to the learning objective it actually serves.

By the end you will have per-hour costs for four collection approaches, an honest pros-and-cons list for each, and a decision matrix that maps training objectives to the cheapest modality that satisfies them.

Key Takeaways

  • Teleoperation ($28 to $60/hr) is the only approach that outputs executable robot actions natively. Pay for it where action supervision matters.
  • Egocentric human video ($15 to $40/hr) is the cheapest volume play, best for representation pretraining, worst for the embodiment gap.
  • Multi-view exocentric capture ($20 to $50/hr) buys scene context and cross-view consistency; calibration labor is its hidden cost.
  • UMI-style handheld grippers (under $1k per device by our estimates) collect gripper-centric data at near-egocentric labor cost, with heavier post-processing.
  • QA rejection (10 to 30 percent in our pipelines) and annotation ($8 to $25/hr per pass) apply to all four. Compare on cost per usable hour.

The Four Approaches, Defined

A data collection approach is the pairing of a capture device with a control source: robot teleoperation, head-mounted egocentric capture, calibrated exocentric camera arrays, or handheld instrumented grippers. Everything else, mono versus stereo, camera count, annotation depth, is a variation within these four.

Teleoperation: $28 to $60 per Hour

Teleoperation data is produced by a human driving a real robot through leader-follower arms or a VR interface, so every recorded frame pairs observations with executable actions in the robot’s own action space. The ALOHA project showed this could be done on a roughly $20k bimanual rig (https://arxiv.org/abs/2304.13705), and Mobile ALOHA extended it to whole-body mobile tasks on roughly $32k of hardware (https://arxiv.org/abs/2401.02117).

Pros

  • Native action labels; feeds imitation learning and VLA post-training directly
  • Matches your exact embodiment, gripper, and camera placement
  • Long-horizon, contact-rich tasks are demonstrable at production quality

Cons

  • Highest labor cost: trained operators at $18 to $38 per hour, plus rig amortization and QA
  • Throughput capped by operator skill and episode reset time
  • Data is embodiment-specific; switching robots strands some of its value

Egocentric Human Video: $15 to $40 per Hour

Egocentric data is first-person video from a head-mounted camera while a human performs tasks with their own hands, capturing human-level dexterity with no robot in the loop. Ego4D and EgoExo4D (https://arxiv.org/abs/2311.18259) made the research case; the commercial case is pure economics, since the collector works at natural speed on tasks they already know.

Pros

  • Cheapest per hour; scales to thousands of hours quickly
  • Enormous task and scene diversity, including real homes
  • Strong pretraining signal for visual encoders and hand-object interaction priors

Cons

  • No robot actions; the embodiment gap means it rarely supervises control directly
  • QA rejection skews high (motion blur, gaze drift, occlusion), 15 to 30 percent in our pipelines
  • Needs retargeting or paired data to transfer to a gripper

Multi-View Exocentric Capture: $20 to $50 per Hour

Exocentric data is third-person video from multiple calibrated, synchronized cameras observing the same task, giving models scene-level context and cross-view consistency that neither ego nor teleop streams provide alone. Stereo pairs add 15 to 25 percent over mono at the same view count and buy metric depth in return.

Pros

  • Full-scene coverage; occlusions in one view are recovered in another
  • Calibrated multi-view supports 3D reconstruction and world-model training
  • Pairs well with egocentric streams (the EgoExo4D recipe)

Cons

  • Calibration and synchronization labor at every scene change is the silent budget eater
  • Fixed arrays limit scene diversity; mobile arrays raise cost
  • Still no action labels without a paired control source

Handheld Instrumented Grippers (UMI-style): Near-Egocentric Cost, Gripper-Centric Output

Handheld gripper capture uses a portable, wrist-camera-equipped gripper operated by a human, producing gripper-centric trajectories without any robot present at collection time. The UMI paper (https://arxiv.org/abs/2402.10329) defined the category; our build estimate is under $1,000 per device including the camera.

Pros

  • Capex is trivial next to a $20k to $32k teleop cell
  • Collection happens anywhere a person can walk, at near-egocentric labor rates
  • Output is closer to robot action space than raw human video

Cons

  • Heavier post-processing to recover clean actions (SLAM drift, kinematic mismatch)
  • Gripper form factor constrains which tasks are demonstrable
  • QA tooling for this modality is younger; expect iteration

Master Comparison Table

Approach DexSet cost/raw hr Capex per station Action labels QA rejection Best use
Teleoperation $28 to $60 $20k to $32k Native 10 to 25% VLA post-training, imitation learning
Egocentric video $15 to $40 $300 to $3.5k None 15 to 30% Encoder pretraining, dexterity priors
Exocentric multi-view $20 to $50 $5k to $15k (3 to 8 cams) None 10 to 20% Scene context, world models, 3D
Handheld gripper (UMI-style) $18 to $42 (our estimate) Under $1k/device Recoverable 15 to 25% Diverse-scene manipulation at low capex

Annotation is additive to every row: $8 to $12 per hour for language instructions, up to $18 to $25 for dense masks and contact labels. And every row’s real price is its cost per usable hour: divide by (1 minus rejection rate). The full math, with a budget spreadsheet, lives in our robot training data costs and pricing guide.

Decision Matrix: Match the Dollar to the Objective

A modality decision matrix assigns each training objective the cheapest approach that actually satisfies it, instead of defaulting everything to the premium modality. Here is the one we use in scoping calls:

Your objective Buy this Not this Why
Post-train a VLA on your robot Teleoperation Egocentric You need native actions on your embodiment
Pretrain visual encoders at volume Egocentric Teleoperation Paying $42/hr for pixels is waste
Scene diversity across 100+ homes Handheld gripper or egocentric Fixed exo array Portability beats calibration
Depth-dependent manipulation Stereo exo + teleop Mono anything Metric depth earns its 15 to 25% premium
World-model or video-prediction training Exo multi-view + ego pairs Teleop only Cross-view consistency is the signal
Bimanual, contact-rich skills Teleoperation (ALOHA-class) Handheld gripper Two grippers, force-aware demos

The pattern behind the matrix: mix modalities and stage them. Open X-Embodiment’s 1M+ trajectories across 22 embodiments (https://arxiv.org/abs/2310.08864) and DROID’s 76k episodes (https://arxiv.org/abs/2403.12945) already prove cross-source data mixes train better generalists. Your budget should look like a portfolio, not a single line item. A 70/30 split of cheap pretraining hours to teleop post-training hours routinely cuts blended cost by a third in programs we run, with no loss on the action-supervised objectives.

Match the Modality to the Objective

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

What is the cheapest way to collect robot training data?

Egocentric human video, at $15 to $40 per hour in our benchmarks, since hardware is a wearable camera and collectors work at natural speed. It is cheapest per hour but supplies no robot actions, so it cannot carry a program alone.

When you need executable actions on your specific embodiment: VLA post-training, imitation learning for contact-rich or bimanual skills. For those objectives nothing cheaper substitutes, which is exactly why you should not spend teleop dollars on anything else.

They are production-useful with caveats. Capex is under $1,000 per device by our estimates and scene diversity is the widest of the four approaches, but plan for heavier post-processing and a 15 to 25 percent rejection rate while the QA tooling matures.

Stereo adds 15 to 25 percent to exocentric capture cost and pays for itself on depth-dependent manipulation tasks. For pure representation pretraining, mono volume usually beats stereo precision per dollar.

Convert everything to cost per usable hour: quoted rate divided by (1 minus the measured QA rejection rate), plus annotation per pass. Our pricing guide includes the worked tables.

Start with the Robot Training Data Costs and Pricing Guide for the full benchmark tables, then book a scoping call and we will run your task list through the decision matrix above.

Robot Training Data Costs and Pricing: The Complete 2026 Guide

Posted on by Sainath Gupta

In our first year we priced a 6,000-hour teleoperation program the way most of this market still prices: by the raw hour. We hit our quoted rate and still broke the client’s budget model, because 22 percent of captured episodes failed their acceptance spec and nobody’s plan had funded the recollection. We rebuilt our cost model around that miss, and this guide is the rebuilt model, published.

The mistake was possible because robot training data has no commodity unit yet. An “hour” of data can mean a raw teleop stream with 30 percent unusable episodes, or a QA-passed, annotated, deduplicated hour that trains a policy. Those two hours differ in cost by 2x or more, and vendors quietly quote whichever one makes their number look better.

So here is the thesis this guide argues from the first table to the last: the only honest unit for pricing robot training data is cost per usable hour, and every quote you receive should be converted into that unit before you compare anything. To make the conversion possible, you will get our first-hand cost-per-hour benchmarks for teleoperation, egocentric human video, and multi-view exocentric capture; rig capex figures anchored to public hardware like ALOHA and UMI; the QA rejection math that separates raw hours from usable hours; and a worked budget for a 10,000-hour VLA data program.

DexSet collects egocentric, exocentric, teleoperation, mono, and stereo data for physical AI teams. We run these rigs, staff these operators, and eat these QA rejections every week. Every number below is either our own operating benchmark or a cited public source.

TL;DR: Robot Training Data Costs at a Glance

  • Teleoperation data: $28 to $60 per raw hour (rig amortization + operator + QA), based on DexSet benchmarks.
  • Egocentric human video: $15 to $40 per hour, the cheapest scalable modality.
  • Multi-view exocentric capture: $20 to $50 per hour depending on camera count and calibration load.
  • Annotation passes: $8 to $25 per hour extra, on top of any capture modality.
  • Rig capex: ~$20k for an ALOHA-style bimanual station (per the ALOHA paper), ~$32k for Mobile ALOHA, under $1k per UMI-style handheld gripper by our build estimates.
  • QA rejection runs 10 to 30 percent in our pipelines, so always budget on cost per usable hour, not raw hour.
  • Public scale references: Open X-Embodiment aggregates 1M+ trajectories across 22 embodiments; DROID contains 76k episodes.

What Do Robot Training Data Costs Actually Include?

Robot training data cost is the fully loaded price of producing one hour of demonstration data that a robot learning pipeline can actually consume, covering hardware amortization, operator or collector labor, QA review, annotation, and delivery infrastructure. Most published debates skip half of these line items, which is why budgets built from a single “per hour” quote fall apart in month two.

A defensible cost model has five layers:

  • Capture hardware (capex). Teleop stations, headsets, camera arrays, grippers. Amortized over 12 to 24 months of use.
  • Collection labor (opex). Teleoperators, camera-wearing collectors, scene setup staff. Usually the largest single line.
  • QA and curation (opex). Human review, automated checks, rejection and recollection. The most commonly omitted layer.
  • Annotation (opex). Language instructions, segmentation, keyframes, success labels. Priced per pass.
  • Infrastructure (opex). Storage, upload bandwidth, format conversion, dataset versioning.

When a vendor quotes you a single number, ask which of these five layers it covers. In our experience the quoted number usually covers layers 1 and 2 and quietly excludes 3 through 5, which add 30 to 80 percent on top.

Core Modalities and What Each One Costs

A data modality is the combination of sensor viewpoint and control method used to produce demonstrations: teleoperation, egocentric human video, exocentric multi-view capture, and their mono or stereo variants. Each modality has a distinct cost structure because each one shifts spend between hardware, labor, and QA differently.

Teleoperation Data: $28 to $60 per Hour

Teleoperation data is demonstration data produced by a human directly controlling a robot, typically through a leader-follower arm setup or a VR interface, so the recorded actions are executable robot trajectories. It is the gold standard for imitation learning and VLA post-training because actions come out in the robot’s own action space, but it is also the most expensive modality per hour.

Our benchmarks across bimanual manipulation programs:

  • Rig amortization: $4 to $9 per hour (a $20k to $32k station amortized over 18 months of two-shift use, including maintenance and spare grippers).
  • Operator labor: $18 to $38 per hour depending on region, task dexterity, and whether the task needs trained specialists (cable routing and garment handling sit at the top of that range).
  • QA overhead: $6 to $12 per hour, covering episode review, rejection, and partial recollection.

Total: $28 to $60 per raw teleop hour. Long-horizon mobile manipulation lands at the top of the range; tabletop pick-and-place with experienced operators lands at the bottom.

The entity chain matters here: teleoperation feeds imitation learning methods like ACT, which the ALOHA project introduced, and imitation learning at scale is what current VLA models are built on. The ALOHA paper (Zhao et al., “Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware,” https://arxiv.org/abs/2304.13705) demonstrated that a roughly $20k bimanual rig could produce data good enough for fine manipulation, which reset industry assumptions about capture hardware pricing.

Egocentric Human Video: $15 to $40 per Hour

Egocentric data is first-person video captured from head-mounted or body-mounted cameras while a human performs tasks with their own hands, giving models human-level dexterity examples without any robot in the loop. It is the cheapest modality to scale because the “rig” is a wearable and the collector is doing a task they already know how to do.

Our cost structure:

  • Hardware amortization: $1 to $4 per hour. Headsets and head-mounted cameras cost $300 to $3,500 and survive thousands of capture hours.
  • Collector labor: $10 to $26 per hour.
  • QA overhead: $5 to $10 per hour. Egocentric QA is dominated by motion blur, gaze drift, and occlusion checks.

Total: $15 to $40 per hour. The catch is the embodiment gap: human hands are not robot grippers, so egocentric data usually pretrains representations rather than directly supervising actions. Datasets like Ego4D and EgoExo4D (https://arxiv.org/abs/2311.18259) established the research value of this modality; the commercial question is purely about cost-effective volume.

Multi-View Exocentric Capture: $20 to $50 per Hour

Exocentric data is third-person video captured from fixed or mobile external cameras observing a task from multiple calibrated viewpoints, which gives models scene context and cross-view consistency that a single egocentric stream cannot. Cost scales with camera count and, more painfully, with calibration and synchronization labor.

Our benchmarks: $20 to $50 per hour for 3 to 8 synchronized views, including calibration checks at every scene change. Stereo pairs add roughly 15 to 25 percent over mono at the same view count because of the extra calibration and QA burden, but they buy you metric depth, which matters for manipulation policies.

Annotation: $8 to $25 per Hour, on Top of Everything Above

Annotation cost is the incremental price of adding structured labels to captured data: language instructions, subtask segmentation, success and failure flags, object masks, or keyframe tags. It is always additive to capture cost, and it is where “cheap” datasets quietly become expensive.

Typical per-pass pricing from our pipelines:

  • Language instruction labeling: $8 to $12 per data hour
  • Subtask segmentation: $10 to $16 per data hour
  • Dense object masks or contact annotation: $18 to $25 per data hour

Comparison Table: Cost per Hour by Modality

Modality Raw cost/hour (DexSet benchmark) Hardware amortization share QA rejection rate Best suited for
Teleoperation (bimanual) $28 to $60 $4 to $9 10 to 25% VLA post-training, imitation learning
Egocentric human video $15 to $40 $1 to $4 15 to 30% Representation pretraining, hand priors
Exocentric multi-view (3 to 8 cams) $20 to $50 $3 to $7 10 to 20% Scene understanding, cross-view learning
Stereo add-on (vs mono) +15 to 25% +$1 to $2 +2 to 5 pts Depth-dependent manipulation
Annotation pass (language) +$8 to $12 n/a n/a Instruction-following VLAs
Annotation pass (dense masks) +$18 to $25 n/a n/a Grasp and contact modeling

Rig Economics: Capex Benchmarks You Can Verify

Rig capex is the upfront hardware cost of a capture station before a single hour of data exists, and it is the number that determines whether building in-house ever beats buying data. The public record here is unusually good, so you do not have to trust vendor hand-waving.

Rig Approx. capex Source Notes
ALOHA bimanual teleop station ~$20,000 ALOHA paper, https://arxiv.org/abs/2304.13705 Leader-follower arms, cameras, frame
Mobile ALOHA ~$32,000 Mobile ALOHA paper, https://arxiv.org/abs/2401.02117 Adds mobile base for whole-body tasks
DexSet production teleop cell $20,000 to $32,000 First-hand DexSet build costs ALOHA-class arms plus industrial cameras, lighting, sync hardware
UMI handheld gripper Under $1,000 per unit (our build estimate) UMI paper, https://arxiv.org/abs/2402.10329 Portable gripper with wrist camera; no robot needed at capture time
Egocentric headset kit $300 to $3,500 Consumer/enterprise hardware pricing Camera glasses to mixed-reality headsets

Two practical lessons from running these rigs:

First, capex is rarely the problem. A $26k teleop cell running two shifts amortizes to under $9 per hour within 18 months. Labor and QA dominate every mature program we run.

Second, UMI-style handheld grippers changed the low end of the market. Because the capture device is a portable gripper with a wrist camera rather than a full robot cell, collection can happen in real homes and kitchens at egocentric-like labor costs while still producing gripper-centric trajectories. The trade-off is a heavier post-processing and QA load to recover clean actions.

Cost per Usable Hour: The Number That Actually Matters

Cost per usable hour is the total program spend divided by the hours that survive quality assurance, and it is always higher than the quoted cost per raw hour. This is the single most important correction to apply to any vendor quote, including ours.

In DexSet pipelines, QA rejection runs 10 to 30 percent depending on modality and task difficulty. Episodes get rejected for dropped frames, desynchronized views, failed task completion, occluded end-effectors, or annotation mismatches. The math is unforgiving:

Usable-hour math. At $40 per raw teleop hour with a 25 percent rejection rate, your real cost is $40 / 0.75 = $53.33 per usable hour. A competitor quoting $36 per hour with an unmeasured 35 percent rejection rate is actually charging $55.38. The cheaper quote is the more expensive dataset.

Quoted raw $/hr Rejection rate True cost per usable hour
$30 10% $33.33
$30 30% $42.86
$40 15% $47.06
$40 25% $53.33
$55 10% $61.11

When you evaluate any provider, require three things in writing: the measured rejection rate on a comparable program, who pays for recollection of rejected episodes, and whether QA review labor is inside or outside the quoted rate. If a vendor cannot produce a rejection rate, they are not measuring quality.

Budgeting a Program: A Worked Example

A data budget is a forward plan that converts a target usable-hour count into total spend across capture, QA, annotation, and infrastructure. Here is a realistic model for a VLA team that needs 10,000 usable teleop hours with language annotation.

  • Target: 10,000 usable hours
  • Assumed rejection rate: 20 percent, so raw capture target = 12,500 hours
  • Blended teleop rate: $42 per raw hour = $525,000 capture
  • Language annotation at $10 per usable hour = $100,000
  • Storage, versioning, delivery at roughly 4 percent of capture = $21,000
  • Total: ~$646,000, or $64.60 per usable annotated hour

For scale context, Open X-Embodiment pooled more than 1 million trajectories across 22 robot embodiments from 21 institutions (https://arxiv.org/abs/2310.08864), and DROID contains 76,000 episodes collected across 52 buildings (https://arxiv.org/abs/2403.12945). Those datasets exist because no single lab could afford to collect that volume alone, which tells you what the market already knows: collection cost, not model architecture, is the binding constraint on physical AI progress.

Build vs Buy: When Each One Wins

The build-vs-buy decision compares the fully loaded cost of standing up your own capture operation against a vendor’s cost per usable hour at your required volume and quality bar. Neither answer is always right; the crossover depends on volume, duration, and how much operational pain you can absorb.

Build wins when you need under roughly 2,000 hours of highly proprietary, robot-specific data, you already own the robots, and engineering time is genuinely available. Buy wins when you need volume and velocity: a vendor already amortized the rigs, trained the operators past the learning curve (operator throughput improves 30 to 50 percent over their first 200 hours in our programs), and built the QA tooling you would otherwise write from scratch. Most funded teams land on a hybrid: build one internal cell for rapid task iteration, buy production volume.

Case Study Proof: A Humanoid Foundation Model Team

A humanoid foundation model team came to us with a $400k data budget, a quoted competitor rate of $35 per hour, and a plan for 11,400 hours. The quote excluded QA review and carried no measured rejection rate. On a 200-hour pilot we measured 28 percent rejection against their own spec, which repriced the competitor dataset at $48.60 per usable hour before annotation.

We restructured the program: tightened the task spec to cut ambiguity-driven rejections, moved 30 percent of volume to egocentric capture for representation pretraining, and reserved teleop for post-training data. Result: 9,800 usable hours delivered inside the original budget, with rejection stabilized at 12 percent by week six. The lesson is not that our rate was lower. It is that cost per usable hour, measured on a pilot, is the only number that predicted their final spend.

Why Most Vendors Hide Pricing, and Why We Publish It

Hidden pricing is a deliberate market structure in which vendors quote deal by deal to maximize price discrimination, and it survives because buyers lack a shared benchmark. Large annotation-era incumbents built their margins on this asymmetry, and robot data inherited the habit.

We publish our ranges because the buyers we want, Heads of Data who run pilots and measure rejection rates, are exactly the buyers opaque pricing repels. Transparent ranges cost us the occasional overpriced deal and win us every buyer who has been burned before. You should treat any vendor’s refusal to publish even a range as information about how they expect the negotiation to go.

Free Download: The Robot Data Cost Model and RFP Scorecard

We packaged the math in this guide into two working documents: a cost model spreadsheet with editable assumptions for rejection rate, shift count, and amortization period, and a 24-question RFP scorecard covering the five cost layers, QA measurement, and recollection liability. Both are free, no email gate on the scorecard.

Related Reading

Put the Numbers to Work

Ready to scope your data program? Talk to our team.

Request a Quote

Frequently Asked Questions

How much does robot training data cost per hour?

Based on DexSet’s operating benchmarks: teleoperation runs $28 to $60 per raw hour, egocentric human video $15 to $40, and multi-view exocentric capture $20 to $50. Annotation adds $8 to $25 per hour per pass. Divide any quoted rate by (1 minus the rejection rate) to get the true cost per usable hour.

The original ALOHA paper reports a bimanual rig built for roughly $20,000, and Mobile ALOHA extends it to whole-body mobile manipulation at roughly $32,000. Our production cells, with industrial cameras, lighting, and sync hardware added, land between $20,000 and $32,000.

In our pipelines, 10 to 30 percent of raw episodes fail QA, depending on modality and task complexity. Well-specified tabletop teleop can hold near 10 percent; long-horizon mobile tasks and loosely specified egocentric capture push toward 30 percent.

Building tends to win below roughly 2,000 hours of proprietary, robot-specific data when you already own robots and engineering time. Buying wins at volume because vendors have amortized rigs, trained operators, and existing QA tooling. Most teams run a hybrid.

Egocentric capture uses wearable cameras and human hands, so hardware costs hundreds to a few thousand dollars and collectors perform familiar tasks at natural speed. Teleoperation requires a $20k to $32k rig plus a trained operator, and outputs executable robot actions, which is what you pay the premium for.

Open X-Embodiment aggregates more than 1 million trajectories across 22 robot embodiments, and DROID contains 76,000 episodes. Both are useful pretraining anchors, but most teams still need proprietary data matched to their own embodiment and tasks.

Require the measured QA rejection rate on a comparable program, clarity on who pays for recollection, an itemized list of which cost layers the rate includes (hardware, labor, QA, annotation, infrastructure), and a paid pilot with your acceptance spec before any volume commitment.

Download the DexSet Robot Data Cost Model and RFP Scorecard, or book a 30-minute pricing walkthrough with our data operations team. We will run your task list through the same model we use internally and hand you the spreadsheet.

The Complete Guide to Egocentric Data Collection for Robotics (2026)

Posted on by Sainath Gupta

3,670 hours. That is the complete Ego4D corpus, the largest first-person video dataset ever assembled (arXiv:2110.07058), and it amounts to roughly seven months of one person’s waking life. The robot foundation models expected to generalize across every kitchen, warehouse, and workbench on earth are drawing from a first-person data supply about that size, while their language-model cousins trained on trillions of tokens. The shelf is not thin. It is nearly bare, and teleoperation refills it at a few hundred action-labeled hours per rig per year.

The gap exists because robots perceive the world from their own body. A vision-language-action (VLA) model driving a humanoid needs to learn from footage that looks like what its head camera will actually see: hands entering the frame from below, objects at counter height, occlusions caused by the manipulator itself. That viewpoint is called egocentric, and until recently there was no scaled, systematic way to collect it. Our thesis, argued with numbers throughout this guide: egocentric capture is the only collection method that scales to foundation-model volumes, but it earns that scaling only when modality mix, viewpoint geometry, and annotation depth are derived from the training mechanism rather than from a hardware catalog.

This guide covers the full stack: what egocentric data collection for robotics actually is, the modalities that matter (mono, stereo, depth, IMU, gaze, hand pose), the hardware options from a $500 Quest 3 to Aria Gen 2 research glasses, how the data plugs into policy training, and what it costs per hour. We include the benchmark numbers we use internally at DexSet, because pricing opacity is the single biggest complaint we hear from buyers.

DexSet supplies egocentric, exocentric, teleoperation, mono, and stereo data to physical AI teams. We have built and rebuilt the capture rigs, the QA pipelines, and the annotation stacks described below, and most of the numbers in this guide come from our own production logs.

TL;DR: Key Takeaways – Egocentric data collection captures first-person visual and sensor streams from a camera mounted at the head or chest of a human (or robot), matching the viewpoint a robot policy will see at inference time. – It is the most scalable source of manipulation pretraining data: a human wearing glasses collects demonstrations 3 to 5 times faster than a teleoperator on a bimanual rig, at roughly one third to one half the cost per hour in our benchmarks. – Hardware ranges from ~$500 (Meta Quest 3, GoPro head mounts) to research-grade Aria Gen 2 glasses with calibrated multi-camera, IMU, eye tracking, and on-device machine perception. – Egocentric human video does not replace teleoperation; the strongest results (EgoMimic, co-training pipelines behind modern VLAs) combine both. The action gap between human hands and robot grippers is the core technical problem. – Our production benchmarks: raw egocentric capture runs $15 to $22 per hour; fully annotated (hand pose, object tracks, temporal segmentation) runs $30 to $40 per hour. Teleoperation runs $28 to $60 per hour depending on rig and task complexity.

What Is Egocentric Data Collection for Robotics?

Egocentric data collection for robotics is the practice of recording synchronized video and sensor streams from a first-person viewpoint, typically a head-mounted or chest-mounted camera worn by a human demonstrator, to train robot perception and control models. The defining property is viewpoint: the camera sees the scene the way an embodied agent sees it, with the demonstrator’s own hands and workspace in frame.

Three properties separate egocentric robotics data from ordinary first-person video:

  1. Sensor completeness. A YouTube cooking clip is RGB only. A robotics-grade egocentric recording carries calibrated camera intrinsics and extrinsics, IMU streams for ego-motion, and often stereo pairs or depth so that 3D structure can be recovered.
  2. Action recoverability. The footage must support extraction of what the hands did: 3D hand pose, object 6-DoF tracks, contact events. Without recoverable actions, egocentric video is only useful for representation pretraining, not policy learning.
  3. Task intent. Recordings are organized into episodes with defined start states, goals, and outcomes, mirroring how robot demonstration datasets like those in Open X-Embodiment are structured (arXiv:2310.08864).

The reference datasets here are Meta’s Ego4D, 3,670 hours of daily-life egocentric video across 74 locations (arXiv:2110.07058), and EgoExo4D, which pairs egocentric and exocentric views of skilled activities with dense annotations (arXiv:2311.18259). Both were built for video understanding research; robotics teams now treat them as the template for what scaled first-person capture looks like.

Egocentric vs. Exocentric: Why Viewpoint Determines Value

Exocentric data is footage captured from an external, third-person viewpoint, such as a tripod camera watching a workbench, while egocentric data is captured from the agent’s own point of view. The distinction matters because a policy trained purely on third-person views must solve an extra correspondence problem at deployment: mapping an external observation of a scene onto its own body frame.

The relationship chain that matters for buyers runs like this: egocentric human video teaches visuomotor priors, teleoperation (through systems like ALOHA, arXiv:2304.13705) provides robot-embodiment action labels, imitation learning consumes both, and VLA models such as OpenVLA (arXiv:2406.09246) and π0 (arXiv:2410.24164) sit at the top of the stack. EgoExo4D demonstrated why you often want both viewpoints of the same episode: the exocentric view disambiguates whole-body motion that the egocentric camera cannot see.

In our pipelines, paired ego-exo capture adds roughly 20 to 30 percent to per-hour cost (a second calibrated camera, cross-view sync, extra QA) and is worth it for whole-body humanoid work. For tabletop manipulation, egocentric plus a single fixed reference camera is usually sufficient.

Core Modalities in Egocentric Capture

A modality is one synchronized sensor stream within a recording, and the modality mix determines both the cost of capture and what training objectives the data can support. The five that come up in nearly every RFP we see:

Mono RGB. A single color stream is the cheapest to capture and the only modality most internet-scale pretraining uses. Sufficient for representation learning and video prediction, insufficient on its own for metric 3D.

Stereo RGB. Two horizontally offset cameras allow metric depth recovery through disparity. Stereo is the workhorse for manipulation because grasp points need metric accuracy. Devices like the Intel RealSense D435i and D455 provide hardware-synced stereo pairs plus an onboard IMU; the D455’s wider baseline (95 mm vs. 50 mm) improves depth accuracy at counter-to-room distances.

Depth. Active or computed depth gives per-pixel range directly. Active IR depth degrades in sunlight and on reflective surfaces, which is why most of our outdoor captures rely on passive stereo instead.

IMU. Accelerometer and gyroscope streams recover head motion, enabling ego-motion compensation and SLAM. Project Aria glasses carry two IMUs precisely because ego-motion is that important for downstream 3D reconstruction (projectaria.com).

Gaze and hand pose. Eye tracking (available on Aria) reveals attention targets before the hand moves, and 3D hand pose is the raw material for retargeting human demonstrations to robot grippers. These are the modalities that convert “video” into “demonstration.”

Hardware: The 2026 Egocentric Rig Landscape

An egocentric capture rig is the wearable hardware package (cameras, IMU, compute, mounting) used to record first-person data, and rig choice is the largest single driver of both data quality and program cost. The four setups we run or evaluate most often:

Rig Approx. Hardware Cost Sensors Video Spec (Typical Capture Config) Calibration Best For
Meta Quest 3 (Passthrough Capture) ~$500 Stereo RGB passthrough, IMU, hand tracking 1280×1280 per eye class, 30 fps effective capture Factory, limited access Budget hand-tracked demos, teleop UI doubling as capture
Aria Gen 2 Research Glasses Research program device (not retail) RGB + mono SLAM cameras, 2 IMUs, eye tracking, spatial mics, on-device hand tracking RGB up to 8 MP class, SLAM cams at high frame rate Full factory calibration + MPS services Research-grade egocentric corpora, gaze + hand pose at scale
GoPro Head/Chest Mount $350–$550 Mono RGB (wide FOV), IMU Up to 5.3K, we typically run 4K/60 or 2.7K/120 Self-calibrated (checkerboard) High-volume, low-cost mono capture; harsh environments
RealSense D435i/D455 Helmet Rig (Custom) $700–$1,200 built Stereo IR + RGB, active depth, IMU 848×480 depth at 90 fps or 1280×720 at 30 fps, RGB 1080p Manual, per-rig Metric depth for manipulation, sim-to-real alignment

Three field notes from running these at volume:

  • Quest 3 is underrated as a capture device because its hand tracking gives you approximate 3D hand pose for free, but passthrough capture access is constrained and image quality trails dedicated cameras.
  • Aria Gen 2 is the quality ceiling. Factory-calibrated multi-camera plus eye tracking plus machine perception services means far less post-processing on our side. Access runs through Meta’s research program rather than retail channels, which affects fleet scaling plans.
  • GoPro rigs win on ruggedness and unit economics. The cost is downstream: no depth, so you pay in annotation and 3D lifting compute instead of hardware.

How Egocentric Data Trains Robot Policies

Egocentric data enters robot learning through three mechanisms: representation pretraining, action retargeting, and co-training with robot demonstrations. Understanding which mechanism you are buying data for should drive every spec decision.

Representation pretraining. Visual encoders pretrained on large egocentric corpora like Ego4D transfer to manipulation tasks better than encoders trained on third-person or object-centric images, because the visual statistics (hands, near-field objects, ego-motion blur) match deployment. This is the lowest-risk use of egocentric data: mono RGB is enough, and annotation requirements are light.

Action retargeting. Human hand trajectories extracted from egocentric video are mapped onto robot end-effectors, turning passive video into pseudo-demonstrations. This requires recoverable 3D hand pose, which is why gaze-and-hand-instrumented devices matter. EgoMimic (arXiv:2410.24221) showed that egocentric human data captured on Aria glasses, combined with robot data, improves manipulation policies over robot data alone.

Co-training. Modern VLA training mixes robot episodes (teleop, in formats like the LeRobot dataset standard, github.com/huggingface/lerobot) with human egocentric episodes in one curriculum. The human data supplies breadth of scenes and objects; the robot data anchors the action distribution to the target embodiment. Cross-embodiment training in Open X-Embodiment established the pattern that heterogeneous data mixtures beat single-source datasets, and egocentric human video is the cheapest heterogeneity you can add.

The failure mode to respect: the embodiment gap. Human wrists have degrees of freedom robot grippers lack, human reach and eye height differ from most robot platforms, and human demonstrators exploit compliance no rigid arm has. Data collection protocols can shrink this gap (constrained grasps, robot-plausible motion instructions, matched camera height), and we bake those constraints into our capture scripts.

Collection Approaches Compared: In-House, Crowdsourced, Vendor

A collection approach is the operational model used to produce the data: who wears the rig, who designs the tasks, and who owns QA. Most teams land on one of three models, and the trade-offs are stable across every program we have run.

Approach Cost per Finished Hour (Our Benchmarks) Throughput Ramp Quality Control Where It Breaks
In-house Capture Team Typically well above vendor rates once salaries, rig fleet, and management overhead are loaded in; often roughly double Slow: 2–3 months to steady state Tight, iterative Scaling past ~10 collectors; hiring drag
Crowdsourced / Distributed Low headline rate, before rejection Fast but noisy Weak; rejection rates at or beyond the top of our 10 to 30 percent planning band are common in our audits Calibration, sync, task compliance
Specialist Vendor (DexSet Model) $15–$40 fully QA’d, annotation-dependent 2–4 weeks to first delivery Contractual, sampled + automated Task designs needing daily iteration with your researchers

The honest read: in-house wins when your task distribution changes weekly and researchers need to redesign protocols on the fly. A vendor wins when the task list is stable and the bottleneck is volume with consistent QA. Crowdsourcing looks cheap until you price the rejection rate and the engineering time spent triaging unsynced, uncalibrated footage.

Cost and Economics: What Egocentric Data Actually Costs

The cost of egocentric data is best expressed as dollars per finished, QA-passed hour at a defined annotation depth, because raw capture is a minority of total program cost. Competitors rarely publish numbers, so here are ours. These are current DexSet benchmark ranges, stated as typical figures we see across programs, not quotes:

Line Item Typical Range (Per Finished Hour) Notes
Raw Egocentric Capture (mono/stereo, IMU, episode structure) $15–$22 Collector time, rig amortization, upload, storage
+ Temporal Annotation (task/step segmentation, outcome labels) +$5–$8 Largely tooling-assisted
+ 3D Hand Pose + Object Tracks +$8–$12 The expensive layer; drives the $30–$40 fully-annotated figure
Paired Ego + Exo Capture +20–30% on capture line Second camera, cross-view sync, extra QA
Teleoperation (for comparison) $28–$60 Rig and task complexity dependent; bimanual fine manipulation sits at the top

Two planning rules of thumb from our production logs:

  • Budget 15 to 25 percent of hours for QA failure. Motion blur, dropped IMU packets, and off-task episodes are facts of life. Vendors should absorb this; if you collect in-house, plan for it.
  • Annotation depth should follow the training mechanism. If you are pretraining encoders, do not pay for hand pose. If you are retargeting actions, hand pose is the whole point. We regularly see RFPs over-specified by $10+ per hour because annotation depth was copied from a paper rather than derived from the training plan.

One more line item buyers forget: storage and delivery. Stereo capture at 30 fps with IMU sidecars generates roughly 50 to 120 GB per hour depending on resolution and compression, so a 5,000-hour corpus is a few hundred terabytes before derivatives. Cloud egress on a corpus that size is real money, which is why we quote delivery format and transfer method inside the per-hour price rather than as a surprise on the final invoice. Ask any vendor to do the same.

At these rates, a 5,000-hour egocentric corpus with full annotation lands between $150K and $200K. The equivalent volume via bimanual teleoperation would run $140K to $300K and take 3 to 5 times as long on a comparable rig fleet, which is the arithmetic behind the current industry shift toward human egocentric pretraining with a smaller teleop fine-tuning set.

Case Study Proof: 4,000 Hours for a Humanoid VLA Team

A humanoid foundation model team came to us with a stalled co-training experiment: their teleop corpus was high quality but topped out near 400 hours, and scaling it 10x on their own rigs would have taken most of a year. We scoped a 4,000-hour egocentric program across kitchen, warehouse shelving, and assembly-bench task families, captured on stereo rigs at camera heights matched to their robot’s head frame, with hand pose and object tracks on the 30 percent of hours their researchers flagged as retarget-critical.

Delivery ran 14 weeks. Their team reported that co-training on the mixed corpus improved task success on unseen-object manipulation evaluations relative to their robot-only baseline, consistent with the direction published in EgoMimic-style co-training work. The full breakdown of task families, QA gates, and the capture protocol is in the case study blog this pillar links to below.

Related reading from this series: – Why Egocentric Data Collection for Robotics Is the Biggest Bottleneck in Physical AI – Comparing Egocentric Data Collection Approaches: Pros, Cons and Costs – Case Study: Scaling Egocentric Data Collection for a VLA Model – 5 Hidden Challenges in Egocentric Data Collection

Download: The Egocentric Data RFP Template

An RFP template turns this guide into a procurement tool: it lists the 40 questions we believe every buyer should ask a data vendor, covering rig specs, calibration evidence, sync tolerances, QA sampling methodology, annotation rubrics, pricing structure, and data licensing. We built it from the RFPs we answer, including the questions we wish more buyers asked. Download it, delete our name from the header if you like, and send it to every vendor on your shortlist including us.

Put These Numbers to Work

If you are scoping an egocentric data program this quarter, two options. Download the RFP template and pressure-test every vendor with it, or book a 30-minute demo and we will walk you through sample episodes from our stereo and Aria-class rigs, including the QA reports we ship with every batch. Either way, you leave with real numbers instead of a sales deck.
Request a Quote

Frequently Asked Questions

What is egocentric data collection for robotics?

Egocentric data collection for robotics is the recording of synchronized first-person video and sensor streams (RGB, stereo, depth, IMU, gaze, hand pose) from head- or chest-mounted rigs, structured into task episodes, to train robot perception and manipulation models.

Egocentric data captures a human performing tasks with their own hands from a first-person camera, while teleoperation data captures a robot performing tasks under human control, with exact joint-space action labels. Egocentric data is cheaper and faster to scale; teleoperation data matches the robot embodiment exactly. Most modern VLA pipelines use both.

Common rigs include Meta Quest 3 (~$500, stereo passthrough and hand tracking), Aria Gen 2 research glasses (calibrated multi-camera, IMU, eye tracking), GoPro head or chest mounts ($350 to $550, mono RGB), and custom helmet rigs built around Intel RealSense D435i or D455 stereo depth cameras.

In DexSet’s benchmarks, raw QA-passed egocentric capture runs $15 to $22 per hour, and fully annotated data with hand pose and object tracks runs $30 to $40 per hour. Teleoperation data runs $28 to $60 per hour for comparison.

No. Egocentric human video scales pretraining and improves generalization, but the embodiment gap between human hands and robot grippers means policies still need robot-embodiment data (teleoperation or autonomous rollouts) for reliable control. Research such as EgoMimic supports combining both.

It depends on the mechanism: encoder pretraining benefits from thousands of hours of lightly annotated video, while retargeting pipelines often start showing gains with hundreds of hours of densely annotated, task-matched capture combined with a robot demonstration set.