Grasp Planning Under Uncertainty#
A candidate approach specifies travel direction, palm roll, finger preshape, and joints to close. The contact planner enumerates these commands, simulates approach and closure, and ranks admitted candidates by modeled wrench capacity under pose and friction uncertainty. Records include nominal contact geometry and scoring samples. The calling application evaluates arm reachability and plans and executes the arm trajectory.
Run the Planner#
The CLI requires the native GraspIt! library, an exported hand XML, and
an object XML with their matching assets. For example, a hand exported
as my_hand and an object exported as object can use the paths below.
Save this configuration as plan.json, replacing the hand values with
the calibration for your model. The hand-frame values shown are the
dataclass defaults; the planner settings select a nominal example.
The RealHand L6 walkthrough supplies a concrete hand
calibration and Docker commands.
{
"hand": {
"approach_axis": [1.0, 0.0, 0.0],
"finger_axis": [0.0, 0.0, 1.0],
"grasp_center_mm": [40.0, 10.0, 93.0]
},
"planner": {
"n_pose_samples": 0,
"friction_samples": [0.5],
"tail_mass": 1.0
}
}
xvfb-run -a .venv/bin/python -I -m graspit_py.planning \
--robot output/hand/my_hand.xml \
--object output/object/object.xml \
--config plan.json \
--out grasps.json
This configuration selects nominal scoring at the supplied friction
coefficient. Omitting those planner overrides uses eight perturbed poses,
nine friction nodes, and tail mass 0.2. --robot, --object, and --out
are required; --config and integer --top-k are optional. Without
--top-k, the CLI writes all admitted candidates. A positive value
retains up to that many of the highest-ranked records; zero retains none.
Use a nonnegative value for --top-k. This command assumes the native
environment described in Installation; -I selects the
installed package instead of an unbuilt source copy.
The CLI constructs HandFrame and PlannerConfig directly from the
optional hand and planner JSON objects. Unknown nested fields raise
TypeError. Type annotations do not validate values, and the dataclasses
do not enforce all ranges. The tables give meaningful input requirements;
they identify additional runtime checks where present.
Set the Hand Frame#
The approach axis points toward the object in the palm frame. The planner projects the finger axis onto the plane perpendicular to it to define roll; both axes must be nonzero and nonparallel. The grasp center \(g\) is a hand-fixed point near the closing region. Let \(R\) map palm coordinates into the object frame, \(c\) be the object center of mass, \(d\) the unit travel direction, and \(s\) the standoff in millimeters. The starting palm translation is
Thus the grasp center starts at \(Rg+t=c-sd\), a distance \(s\) from \(c\). Roll rotates the hand about \(d\); zero roll depends on the planner’s reference-axis construction.
Approach geometry in the object frame. Standoff \(s\) spans the starting grasp center to \(c\) along \(d\); the palm origin is at \(c-sd-Rg\).#
|
Default |
Type and Units |
Valid Input and Meaning |
|---|---|---|---|
|
|
Length-three vector |
Nonzero, palm-frame approach direction. |
|
|
Length-three vector |
Nonzero and nonparallel to approach axis; defines roll. |
|
|
Length-three vector, mm |
Palm-frame grasp center; calibrate for the hand. |
Specify Candidate Commands#
Without explicit approaches, the planner samples Fibonacci-sphere travel directions and equally spaced rolls. Every approach is combined with each preshape and closing set. Preshapes use exported DOF order; coordinates outside the selected closing set remain at preshape during closure. Explicit preshapes can come from a low-dimensional posture subspace; see Ciocarlie and Allen.
|
Default |
Type |
Valid Input and Meaning |
|---|---|---|---|
|
|
Integer |
Positive direction count; used without |
|
|
Integer |
Positive roll count; rolls are equally spaced over 360 degrees. |
|
|
Array |
Rows |
|
|
Array |
Starting joint vectors of exported DOF length \(D\), within model limits. Null or empty uses |
|
|
Lists of integer indices |
Zero-based DOF indices within bounds; null closes all. |
Approach and Closure#
The planner advances the open hand along \(d\) in discrete increments, searching up to twice the initial standoff. When an increment would collide, it starts closure at the last collision-free travel distance minus clearance, clamped to nonnegative travel. GraspIt! advances the selected coordinates under autograsp. The planner admits closures with enough object contacts and applies the constraints below. The calling application verifies continuous hand and arm clearance during execution.
|
Default |
Type and Units |
Valid Input and Meaning |
|---|---|---|---|
|
|
Float, mm |
Positive starting distance from grasp center to object center. |
|
|
Float, mm |
Positive travel increment; zero or negative can prevent termination. |
|
|
Float, mm |
Nonnegative retreat from last-free placement. |
|
|
Integer |
Nonnegative minimum contact count for admission. |
|
|
Float, speed factor |
Passed as GraspIt! |
Contact and Environment Constraints#
The contact-box and minimum-contact predicates run in every closure, including perturbed trials. Support and keep-out predicates inspect final nominal hand vertices. The support test includes its boundary; the keep-out test rejects strictly interior vertices. These are geometric filters on the candidate bank.
|
Default |
Type and Units |
Valid Input and Meaning |
|---|---|---|---|
|
|
Length-three vector |
Null disables support filtering; otherwise supply a nonzero object-frame normal. |
|
|
Float, mm |
Every nominal hand vertex \(v\) must satisfy \(v^\top\hat n\ge\text{offset}\) for normalized support normal \(\hat n\). |
|
|
Array |
Ordered lower/upper object-frame corners; every contact must lie inside or on the box. |
|
|
Array |
Each box is |
Select a Wrench Score#
The default quality is Ferrari–Canny epsilon: the radius of an origin-centered ball in the normalized wrench hull. Its moment radius is the maximum contact distance from the center of mass, with a 0.01 mm floor. Task loading instead uses the largest common multiplier supported along specified load rays, with a 100 mm moment radius for both hull and rays. The theory guide defines the force budget, normalization, and strict-interior requirement for task capacity.
|
Default |
Type and Units |
Valid Input and Meaning |
|---|---|---|---|
|
|
Array |
Rows |
The planner normalizes each supplied force direction. For downward
gravity, [0, 0, 0, 0, 0, 1] specifies upward support through the
object-frame origin. Its moment about the center of mass depends on the
line of application. Load multipliers use the modeled normal-force
budget; calibrated physical forces are needed to express them in newtons.
Model Approach and Friction Uncertainty#
The planner closes at the nominal approach and at each perturbed pose. Translation and rotation-vector errors are componentwise isotropic Gaussian draws in the object frame. The perturbation rotation left-multiplies the palm rotation and rotates the travel direction. The seed and command determine each candidate’s stream independently of enumeration history.
|
Default |
Type and Units |
Valid Input and Meaning |
|---|---|---|---|
|
|
Integer |
Nonnegative perturbed-pose count; nominal pose is additional. |
|
|
Float, mm/component |
Nonnegative translation standard deviation. |
|
|
Float, degrees/component |
Nonnegative rotation-vector standard deviation. |
|
|
Sequence of floats |
Nonempty finite nonnegative grid; supplied order retained. Equal weights, no planner clipping. |
|
|
Float |
\(0<\text{tail\_mass}\le1\); checked by the risk function when scoring. |
|
|
Integer |
Integer seed accepted by NumPy’s random generator. |
Every realized contact set is scored at every friction coefficient.
A failed perturbed approach or closure contributes a zero row. The
planner ranks by the lower-tail average of all pose–friction entries,
including the nominal row with equal weight. This emphasizes adverse
low-capacity outcomes. Set tail_mass=1 for mean ranking. Nominal
scoring requires both n_pose_samples=0 and a single friction coefficient;
the configuration above selects that case. The theory guide
includes an executed fractional-tail example.
Interpret Candidate Records#
The CLI sorts candidates by decreasing score. Let \(D\) be the exported
DOF count, \(N\) the nominal contact count, \(P=\text{n\_pose\_samples}+1\),
and \(F\) the friction-grid length. Sample arrays have pose rows and
friction columns: the first row is nominal, and columns retain supplied
order. The nominal scalar uses column F//2, the upper middle column
for an even-length grid. It need not correspond to the mean friction
coefficient. Admission does not require a positive score.
Candidate JSON Field |
Shape or Type |
Meaning and Units |
|---|---|---|
|
|
Accepted nominal palm-to-object transform; translation mm. Python attribute: |
|
|
Nominal closed coordinates, rad, exported/native DOF order. Python attribute: |
|
\(N\) objects |
Each has |
|
Float |
Native simulator epsilon at nominal closure, using simulator normalization. |
|
Float |
|
|
|
Normalized Python epsilon values; failed perturbed rows are zero. |
|
Float |
Selected epsilon or task lower-tail mean; ranking scalar. |
|
Float |
Arithmetic mean of all epsilon entries. |
|
|
Unit travel direction, object frame. |
|
Float, degrees |
Roll about travel direction. |
|
|
Starting command coordinates. |
|
List of integers or null |
Zero-based closing indices; null means all DOFs. |
|
Float |
Epsilon lower-tail mean, populated also in task mode. |
|
|
Task-load multipliers; null without task loading. |
|
Float or null |
|
|
Float or null |
Arithmetic mean of task entries, or null. |
The top-level JSON contains these fields:
Field |
Interpretation |
|---|---|
|
Input XML paths. |
|
Hand-frame and planner settings. |
|
Elapsed runtime observed by the CLI, in seconds. |
|
Counts of nominal rejection causes; only incremented keys appear. |
|
|
|
Ranked candidate records described above. |
Rejection keys are approach_blocked_or_missed,
closure_start_collides, grasp_compute_failed, too_few_contacts,
contact_outside_region, below_support, and in_keepout.
Perturbed failures affect scoring arrays, not these counts.
native_epsilon and Python epsilon use distinct normalizations.
See MuJoCo Model Export for converting palm and joint
targets back to the source model.
The following abbreviated record comes from the L6 reference workflow. Values are rounded for display. Its scores belong to that workflow’s model and sampling settings, not to the nominal configuration above.
{
"dofs_rad": [
0.5847,
0.45,
0.612,
0.566,
0.641,
0.712
],
"native_epsilon": 0.122575,
"nominal_epsilon": 0.064501,
"score": 0.00895771
}
native_epsilon and nominal_epsilon report the native and Python
quality metrics under their respective wrench conventions. score averages the adverse tail of the sampled
Python quality values. The full record also carries the pose, contacts,
and sample arrays listed above.
Planned Grasps and Simulated Execution#
RealHand L6. Planned key grasp.#
LEAP Hand. Simulated cube hold.#
PSYONIC Ability Hand. Simulated sphere hold.#
The L6 panel shows the planned closure on the insertion key. The other panels show final states of recorded lift-and-hold simulations in MuJoCo, initialized from GraspIt! plans.