ParcelStow: Does Imitation Learning Preserve Robustness?

Matching expert task success at one operating condition does not establish robustness preservation as task conditions vary. ParcelStow evaluates the expert–learner performance difference across a controlled variation in execution timing. The difference can widen, narrow, or reverse sign.

A scalar speedup factor r divides selected phase durations: at r = 2, those phases take half as long. Acquisition and settling durations are fixed. We evaluate expert and learner policies across the same speedup factors on three simulated manipulation tasks.

Expert above ACT. Parcel insertion, upright placement, and keyed peg insertion at r = 2, played at 2× speed. Labels identify each recorded outcome. These clips depict individual trials; the table below reports evaluation-bank success counts.

MP4 · 1.4 MB GIF · 11.5 MB

Expert–ACT Comparisons

In the recorded evaluations, ACT matches or exceeds expert nominal task success on all three tasks. At the higher speedup factors shown below, the expert succeeds more often. For parcel insertion, this difference appears within the demonstrated speed range; for upright placement and keyed peg insertion, it appears beyond that range.

Task	Nominal `r` = 1		Higher `r`	Success at Higher `r`
Task	Expert	ACT	Higher `r`	Expert	ACT
Parcel Insertion	100/100	100/100	2	84/100	53/100
Upright Placement	185/200	194/200	2	105/200	24/200
Keyed Peg Insertion	182/200	191/200	1.5	150/200	3/200

Expert-minus-ACT differences at the highlighted factors are 31, 40.5, and 73.5 percentage points, with pointwise 95% paired bootstrap intervals [18, 44], [32, 49], and [67.5, 79.5]. Each task reports one trained ACT policy; training replications are reported individually in the records.

These evaluations use fixed state-based policies in Isaac Lab simulation. The release also contains Diffusion Policy and DAgger results, together with their pairing status and comparison limits. Explore the evaluation records.

Use the Benchmark

The benchmark includes 937 successful expert demonstrations and 970,565 control steps across the three tasks. Learned policies use 147-dimensional state observations and share a 16-dimensional joint-position action interface with the scripted experts at 50 Hz.

The frozen evaluation bundle contains 136 conditions from 20 source series and 23,200 episodes. Reproduce success counts and paired intervals on a CPU without running the simulator:

git clone https://github.com/coenwerem/parcelstow.git
cd parcelstow
python3 -m pip install numpy
python3 scripts/reproduce_manuscript.py --bootstrap \
  --output-dir outputs/reproduce/canonical

Reproduction Runbook Evaluate Your Policy Demonstrations & Checkpoints

Using ParcelStow?

Star on GitHub Like on Hugging Face

Citation

@misc{enwerem2026parcelstow,
  title = {Does Imitation Learning Preserve Temporal Robustness in
           Dexterous Manipulation? An Expert-Learner Comparison
           Across Task Execution Speeds},
  author = {Enwerem, Clinton and Baras, John S. and Belta, Calin},
  year = {2026},
  eprint = {2609.01453},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url = {https://arxiv.org/abs/2609.01453}
}