Skip to Content

DexReach

GPU-Accelerated Whole-Body Dexterous Grasp and Manipulation Planning

Clinton Enwerem, John S. Baras, & Calin Belta

University of Maryland, College Park

We developed DexReach, a GPU-accelerated Python package built on cuRobo for grasp and manipulation planning with a unified arm-hand configuration. The planner evaluates contact wrenches under friction and actuation limits, constructs whole-body motions from the evaluated grasp, and jointly plans object trajectories with fingertip and table-support forces. We evaluated complete manipulation sequences in Isaac Lab across arm-hand embodiments, object-waypoint tasks, and simultaneous dual-object manipulation.

Isaac Lab demonstrations with Franka-Ability and dual Franka-Ability, xArm7–LEAP pick-and-place, and G1–Dex3 box grasping and lifting. The Franka-Ability overview includes two-object manipulation, a 45° yaw target, and a 30° mid-path tilt target.

Overview Sources and Checksums

Grasp and Motion Planning

Arm-Hand Configuration

Let \(q_{\mathrm{arm}}\) and \(q_{\mathrm{hand}}\) denote the arm and hand joint coordinates, with \(n_{\mathrm{arm}}\) and \(n_{\mathrm{hand}}\) joint coordinates, respectively. Their configuration domains are \(\mathcal Q_{\mathrm{arm}}\subseteq\mathbb R^{n_{\mathrm{arm}}}\) and \(\mathcal Q_{\mathrm{hand}}\subseteq\mathbb R^{n_{\mathrm{hand}}}\). The combined configuration is

\[ q=(q_{\mathrm{arm}},q_{\mathrm{hand}}) \in\mathcal Q_{\mathrm{arm}}\times\mathcal Q_{\mathrm{hand}}. \]

In the release pipeline, we evaluate hand grasp candidates, solve arm inverse kinematics, and reevaluate contacts in the reached arm-hand configuration. An alternative optimizer updates the arm and hand coordinates jointly. Joint limits, contact conditions, and geometric collision checks constrain the configurations accepted for motion planning.

Contact and Task-Wrench Feasibility

For a grasp with \(k\) usable contacts and specified actuation limits, \(f\in\mathbb R^{3k}\) stacks the forces on the object, expressed in object coordinates. At contact \(i\), the set \(\mathcal F_i(\mu)\) requires a nonnegative inward normal force, a tangential force magnitude no larger than the friction coefficient \(\mu\) times that normal force, and any specified fingertip force cap. The grasp map \(G(q)\in\mathbb R^{6\times3k}\) maps these forces to a wrench \(w\in\mathbb R^6\), comprising force in newtons and moment in newton-meters about the object's center of mass, expressed in object coordinates. Let \(\tau(q,f)\in\mathbb R^{n_\tau}\) be the modeled joint-torque vector and \(\tau_{\max}\in\mathbb R^{n_\tau}\) its componentwise limits, both in newton-meters, where \(n_\tau\) is the number of torque-limited model coordinates. A prescribed task-wrench set \(\mathcal W_T\) is feasible in this static contact model when it lies in the admissible wrench set:

\[ \begin{aligned} \mathcal W_{\mathrm{adm}}(q;\mu) &=\left\{G(q)f:\ f_i\in\mathcal F_i(\mu)\ \text{for every }i, \quad |\tau(q,f)|\leq\tau_{\max}\right\},\\ \mathcal W_T&\subseteq\mathcal W_{\mathrm{adm}}(q;\mu). \end{aligned} \]

Reachability-Checked Grasp Synthesis

Contacts and task loads are evaluated in an arm-hand configuration.

Whole-Body Motion Planning

Approach and manipulation motions are constructed from the evaluated grasp.

Contact-Force Optimization

Object motion and fingertip and support forces are optimized jointly.

Simulation Evaluation

Sequence completion is measured under bounded actuation.

Grasp geometry, task-wrench feasibility, and motion planning are evaluated through complete manipulation sequences in Isaac Lab.

Whole-Body Motion and Contact Forces

The evaluated grasp configuration anchors the approach, insertion, close, lift, carry, place, release, and retreat motions. During held-object waypoint motion, the hand follows the prescribed object pose with a fixed object-relative hand transform and fixed finger joints. We check sampled robot configurations against the floor, table, robot geometry, and relevant objects, with object-collision checks determined by the motion phase. Unsuccessful motion candidates are rejected.

Force-aware planning couples the robot joint configuration with each object's pose and twist. Joint velocities and fingertip and table-support forces are the trajectory inputs. Discretized Newton–Euler equations relate these forces to object translation and rotation. The plan imposes scheduled open and closed contacts, friction constraints, and joint position, velocity, and torque bounds at trajectory knots. The default robot torque model is quasi-static; full robot dynamics are optional.

The executor tracks planned joint positions and velocities using position and velocity feedback with contact-torque feedforward. The feedforward contribution decreases during release. The reported force-aware experiments used a fixed execution window initialized from the measured object pose; the initial perturbation was known exactly from the simulator.

Embodiment Models

An embodiment model combines arm-hand kinematics, fingertip contact geometry, actuation limits, and finger couplings. The recordings show four arm-hand models grasping tabletop objects. Isaac Lab renders the robot and object poses recorded during each trial.

Recording Sources and Checksums

Watch the complete Franka-Ability soup-can sequence · Sequence Sources and Checksums

xArm7–LEAP Pick and Place

The xarm7_leap model pairs the seven-joint UFACTORY xArm7 with a right LEAP Hand with 16 actuated joints. We evaluated ten planned pick-and-place trajectories per object in Isaac Lab. The robot completed 10 of 10 trials with the tomato soup can, 7 of 10 with the sugar box, and 7 of 10 with the mustard bottle.

The xArm7 and LEAP Hand grasp, lift, transport, and place each YCB object before releasing it and withdrawing. The recording shows one completed Isaac Lab trial per object.

Recording Sources and Trial Outcomes

Simulation Evaluation

We ran Isaac Lab at 240 Hz for physics and 60 Hz for control. Joint torques were capped by rated effort where specified; otherwise, limits were derived from fingertip force caps and the planned contact Jacobian. Coupled fingers used mimic constraints, with no independent drive at follower joints. Robot-link gravity and articulation self-collision were disabled in these simulations; the planner evaluated geometric collision checks separately.

A complete manipulation sequence requires lift, carry, placement within the goal tolerances, release, object settling, cleared robot contact, and robot retreat. The goal-position tolerance is 0.02 m, and final settling drift is at most 0.005 m. Final pose checks assess tilt and, when prescribed, goal orientation. Both-object completion requires each object to complete its sequence.

Grasp Selection across Embodiments and Objects

For each embodiment-object case, we evaluated ten grasps selected by an in-place physics check, using three perturbation draws per grasp for 30 rollouts. Selection and evaluation used distinct random seeds. Object starts varied within a 5 mm radius and ±10° yaw, and friction coefficients ranged from 0.5 to 0.9. Soup-can and mustard-bottle masses ranged from 0.8 to 1.2 times nominal; cylinder, cube, and box masses remained fixed.

Complete manipulation sequences out of 30 rollouts per embodiment-object case, using ten physics-selected grasps and three perturbation draws per grasp.
EmbodimentCylinderCubeBoxSoup CanMustard Bottle
xArm7-L62920292730
Franka-LEAP302830158
Franka-Ability3028302730

Franka-LEAP completed all cylinder and box trials, but completed fewer soup-can and mustard-bottle sequences than xArm7-L6 and Franka-Ability in this evaluated grasp cohort.

Object Waypoints and Contact-Loss Monitoring

The single-object waypoint trials used Franka-Ability with a soup can and a 20 cm carry. Each stored plan executed once, with friction coefficient 0.7 and acceleration 2.0 m/s². The dual Franka-Ability cohort used serial per-object motions in a persistent scene at these friction and acceleration settings, with one rollout per plan. In separate contact-loss tests, we injected a hand opening during the Franka-Ability carry; control trials omitted this injected opening. The validator monitored contact loss during carry and stopped the task after detected loss.

Waypoint, serial dual-object, and contact-loss outcomes. Completion counts require the complete manipulation sequence; stop counts refer to injected-opening and control trials.
Task or TrialIsaac Lab Outcome
Place8/10 completed
Yaw, 45° about world z8/10 completed
Mid-path tilt, 30° about world x, returning upright7/9 completed
Serial dual Franka-Ability6/6 completed both objects; final object goal-position errors spanned 4 to 19 mm
Injected hand opening during carry5/5 trials stopped
Controls without injected opening0/5 trials stopped

Force-Aware Franka-LEAP Execution

In a separate force-aware evaluation, we reused ten Franka-LEAP soup-can grasps across three random seeds, with three perturbation draws per grasp per seed. Force planning used nominal friction and mass and retargeted the plan to the exact initial simulator pose. Evaluation perturbed object starts within a 5 mm radius and ±10° yaw, with friction from 0.5 to 0.9 and mass from 0.8 to 1.2 times nominal. Franka-LEAP completed the manipulation sequence in 76/90 rollouts.

Complete manipulation sequences for Franka-LEAP with the soup can under force-aware execution: ten fixed grasps with three perturbation draws per grasp per seed.
Perturbation SeedCompleted / Rollouts
200026/30
200126/30
200224/30
Pooled76/90

Simultaneous Dual-Object Manipulation

Simultaneous planning combines the left and right joint trajectories on a common clock. Each shared phase takes the longer of the per-arm durations, synchronizing the manipulation stages. The force-aware trajectory model includes both objects' states and their contact forces.

The A7-L6 bimanual robot manipulator carried a cylinder in its left hand and a soup can in its right. We paired one reused left-hand cylinder grasp with nine right-hand soup-can grasps, giving nine fixed grasp pairs. Each pair received three perturbation draws per seed across three seeds. Both object starts varied within 5 mm and ±10° yaw, and friction ranged from 0.5 to 0.9. Soup-can mass ranged from 0.8 to 1.2 times nominal; cylinder mass remained fixed. The force plan used friction 0.5 and soup-can mass 1.2 times nominal, with exact initial-pose retargeting and fixed-window execution.

Complete manipulation sequences under simultaneous force-aware execution by the A7-L6 bimanual robot manipulator: nine fixed grasp pairs with three perturbation draws per pair per seed.
Base Perturbation SeedCylinderSoup CanBoth Objects
200023/2722/2719/27
200121/2721/2718/27
200223/2718/2716/27
Pooled67/8161/8153/81

The both-object count requires completion by each object in a rollout. In a separate nominal schedule comparison, serial and simultaneous executions reused the per-arm grasps and motions. The simultaneous schedule lasted 8.93 s; executing the arm motions consecutively lasted 17.4 s.

Hand-Link Origin Separation

In a separate diagnostic, the requested distance between selected hand-link origins was 20 cm. The observed distance at the final trajectory configuration was 19.76 cm.

Bimanual Key Insertion

The left A7-L6 hand stabilized a free housing while the right hand lifted and inserted a free key. A 3 N lateral load acted on the housing during insertion.

Isaac Lab renders the arm, hand, key, and housing poses recorded during the MuJoCo trial.

Software Resources

The dexreach package is released under the Apache 2.0 license and requires a CUDA-capable environment with NVIDIA cuRobo for planning and Isaac Lab for simulation evaluation.

Citation

@software{enwerem2026dexreach,
  title   = {{DexReach}: {GPU}-{Accelerated} {Whole}-{Body} {Dexterous} {Grasp} and
             {Manipulation} {Planning}},
  author  = {Enwerem, Clinton and Baras, John S. and Belta, Calin},
  year    = {2026},
  version = {0.1.0},
  license = {Apache-2.0},
  url     = {https://clintonenwerem.com/dexreach/}
}