— Use Case · Human-Object Interaction
Teach models how people actually handle objects.
Grasping, repositioning, using, and releasing objects looks simple until a model has to do it. dexset builds human-object interaction datasets that capture grip, intent, sequence, and outcome across real variation.
Hand-object interaction · 21-pt pose
— Capabilities
What HOI datasets capture.
Grasp and grip variation
The same object handled with different grips, hand orientations, and approach angles across many people.
Hand-object contact labels
Contact points, grasp zones, and 21-point hand pose annotated through the interaction.
Intent and sequence
Temporal labels that separate reaching, grasping, using, and releasing into learnable phases.
Object diversity
Rigid, deformable, articulated, and irregular objects across sizes, textures, and weights.
Occlusion handling
Deliberate capture and annotation through self-occlusion — the frames where most datasets give up.
Outcome labels
Every interaction tagged with result: success, slip, mis-grip, or recovery.
— In Scope
Interaction types in scope.
— Compliance & Trust
Compliant by design, across every dataset.
From first consent form to final delivery, dexset workflows are built to meet data protection, security, and labor standards across the regions where we capture and the regions where our customers operate.
Informed consent
Every participant is briefed and signs a release before capture begins. Consent records attach to each clip, and withdrawal requests are honored across all delivered versions.
Privacy protection
Face blurring, anonymization, and exclusion zones are applied wherever required. We minimize personal data by design and never collect more than the task brief demands.
Data security
Footage is encrypted in transit and at rest, with role-based access controls, signed delivery URLs, and per-recipient transfer records on every dataset.
Regional compliance
Capture and delivery workflows are designed to align with GDPR and UK GDPR, CCPA/CPRA, PIPL, APPI, and LGPD requirements, with regional data-residency options where needed.
Traceability & audit
Source, consent, capture, environment, and annotation records persist for every dataset, supporting audits that run from raw footage to the exported training file.
Responsible sourcing
Capture operators, demonstrators, and annotators are fairly paid and work under documented, safe conditions — quality data should never come from exploitative labor.
— What Teams Say
Trusted by robotics teams.
★★★★★
“The egocentric hand-pose labels are the best we have evaluated. Our grasp success rate improved 18% after one fine-tuning round.”
Priya Raman
Manipulation Lead, AI Robotics Company
★★★★★
“Coverage scoring meant we knew exactly what variation we were missing before training, not after burning a GPU budget on it.”
Sofia Almeida
Perception Engineer, AI Robotics Company
★★★★★
“We sent one task brief and got back a dataset that loaded into our pipeline on the first try. Custom schema, zero rework.”
Marcus Chen
ML Infrastructure Lead, Independent Robotics Vendor
— FAQ
Frequently asked questions.
What makes a good human-object interaction dataset?
Variation with structure: the same objects handled with different grips, speeds, and approach angles by different people, labeled with contact points, hand pose, interaction phase, and outcome — so models learn the behavior, not one demonstrator’s habit.
Which objects can you cover?
Rigid, deformable, articulated, and irregular objects across sizes, textures, and weights — from cartons and tools to cables, fabric, and packaging. Object sets are specified in the task brief and tracked as a coverage axis.
How is hand pose annotated?
With 21-point hand keypoints tracked through the interaction, including through brief occlusions, paired with grasp-zone and contact labels. Confidence scores accompany every pose frame.
How do interaction phases get labeled?
Temporal segmentation separates reaching, grasping, manipulation, and release into distinct labeled phases, giving sequence models clean boundaries instead of one undifferentiated clip.
Need interaction data for your model?
Tell us the objects and the behaviors. We design the dataset around them.