USB Insertion
Insert the USB flash drive fully into the laptop's USB port.
Dexterous robotic manipulation
Datasets, benchmarks & tools for dexterous manipulation.
From task-oriented interaction to vision and touch.
DexJoCo2 / Isaac Sim
Dataset availableA VisuoTactile Dataset for Dexterous Robotic Manipulation
GitHub Dataset on Hugging FaceRecorded modalities
Current release
Generated demonstrations are derived from the human-teleoperated data.
More tasks. More demonstrations. DexJoCo2 continues to grow. Future releases will expand both task coverage and demonstration volume.
10 videos Pause, scrub, or view fullscreen
Insert the USB flash drive fully into the laptop's USB port.
Swipe the bank card through the POS terminal's card reader.
Use the power drill to drive the screw into the board.
Use the pipette to draw brown iodine solution from the source beaker and dispense it into the reaction beaker containing clear starch and vitamin C solution, turning the mixture dark blue-black.
Hold the medicine bottle and press down while turning the childproof cap to unscrew and remove it.
With the cabinet contents hidden from view, use touch alone to distinguish the corn from the apple. Take out the corn and leave the apple inside.
Enter 1, then 2, and press Enter on the safe's keypad. Turn the handle and open the unlocked door.
Use the hammer to drive the nail into the wooden board.
Fold the glasses, place them inside the glasses case, and close the lid.
Place the soda in the shopping basket, then lift the basket completely off the table and hold it up.
Swipe to explore 1 / 10
DexJoCo / MuJoCo
DexJoCo is a benchmark and toolkit for task-oriented dexterous manipulation in MuJoCo, covering tool use, bimanual coordination, long-horizon execution, reasoning, human demonstration collection, and modern VLA evaluation.
Achieving human-level manipulation requires dexterous hands and standardized evaluation. Existing dexterous benchmarks often lack realistic manipulator-hand setups, tasks that reveal the unique capabilities of dexterous hands over parallel grippers, reliable demonstration acquisition tools, and unified pipelines for evaluating modern VLA models.
DexJoCo addresses these gaps with 11 functionally grounded tasks built around the Franka Panda and Allegro Hand. It provides a low-cost motion-capture data collection system, 1.1K human demonstration trajectories, replay-based domain randomization, and evaluation support for modern imitation learning and VLA policies.
DexJoCo / MuJoCo
DexJoCo tasks are designed around functional interactions rather than isolated object relocation. Each task defines interactive objects and success constraints over execution order, object poses, articulated joint states, and contact, so completion requires meaningful progress toward an everyday objective.
The benchmark covers tool use, reasoning, bimanual coordination, and long-horizon execution. Its assets provide explicit visual interaction feedback, such as unlocking an iPad after entering a password, spraying water from a watering can, and waking a display by clicking a mouse.
Single-arm demos
Bimanual demos
DexJoCo provides a low-cost teleoperation system for efficient human demonstration collection. Rokoko Smartgloves capture hand motion without camera occlusion, while HTC Vive Trackers and Base Stations track wrist motion for Franka end-effector control in a unified setup of about $2,300 USD.
The software combines wrist tracking with retarget MLP, a lightweight self-supervised retargeting method that maps human fingertip poses to Allegro Hand joint configurations without paired human-robot annotations.
DexJoCo is designed as a low-cost system for human demonstration data collection.
The teleoperation system combines hand motion retargeting and wrist motion tracking. Because human and robotic hands have different structures, direct linear mapping is infeasible. DexJoCo uses retarget MLP to preserve fingertip motion directions, workspace coverage, pinch behavior, and collision avoidance for real-time Allegro Hand control.
DexJoCo collects 1.1K human demonstration trajectories across the 11 benchmark tasks. Each trajectory records rich observations, including third-person and wrist-mounted visual streams, object and robot states, TCP pose, and hand joint angles, while actions are represented as target absolute end-effector poses and hand joint angles.
The dataset can be converted into common formats such as LeRobot Dataset v3.0 and Diffusion Policy Zarr. DexJoCo then evaluates policies through constructed task environments and an asynchronous server-client deployment pipeline.
augmentation
DexJoCo supports domain randomization across all task scenarios. Object placement and table height are randomized for trajectory diversity, while third-person camera poses, lighting direction and color, and tabletop textures are randomized to evaluate visual robustness.
Camera poses are sampled on a spherical surface, then filtered to select viewpoints with minimal occlusion.
For lighting randomization, we follow a simple procedure inspired by our implementation. Each light in the scene is randomized in terms of its position, direction, and diffuse color to introduce diverse illumination conditions.
Table height is randomized together with object placement to broaden the state distribution of replayed trajectories.
For tabletop texture randomization, we sample textures from a pre-constructed texture library.
DexJoCo benchmarks ACT, Diffusion Policy, π0.5, and GR00T N1.5 under object-only and full visual randomization. The benchmark is challenging: visual randomization sharply reduces success rates, difficult bimanual tasks expose persistent failures, and precise interaction remains a key bottleneck.
DexJoCo citation
@misc{wang2026dexjocobenchmarktoolkittaskoriented,
title={DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo},
author={Hanwen Wang and Weizhi Zhao and Xiangyu Wang and Siyuan Huang and He Lin and Boyuan Zheng and Rongtao Xu and Gang Wang and Yao Mu and He Wang and Lue Fan and Hongsheng Li and Zhaoxiang Zhang and Tieniu Tan},
year={2026},
eprint={2605.16257},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2605.16257},
}