AgileX Mobile Cobot — Bimanual Mimicry
Abstract
This project demonstrates an AgileX mobile cobot platform configured for bimanual manipulation. In the demo one operator directly controls two arms while two additional objects (or proxy arms) mimic those movements, enabling coordinated tele-operation and synced manipulation tasks. The demo focuses on intuitive control, low-latency mirroring, and robust grasping of two arbitrary objects.
A mobile cobot platform carrying two manipulator arms performs synchronized bimanual tasks. An operator tele-operates both arms while two proxy objects mimic the motions, demonstrating coordinated grasping, low‑latency mirroring, and base repositioning for complex manipulation scenarios.
Key Features
- Tele-operated bimanual control (one operator controls two arms)
- Mirroring/mimic mode: two additional objects mimic the arm motions
- Low-latency state synchronization for coordinated manipulation
- Support for arbitrary graspable objects and basic force safety checks
Methodology
Operator inputs are captured from a master controller and mapped to dual-arm trajectories. A middleware layer handles inverse kinematics, collision-aware trajectory smoothing, and real-time mirroring to the two proxy objects. Visual feedback is provided through an onboard camera and an optional external monitor for the operator.
Future Work
- Curate and publish a reproducible dataset (ROS bags, camera/depth logs, joint states, and annotated grasps) under
assets/datafor training and replay - Provide preprocessing scripts (TFRecord/LMDB converters, normalization, augmentation) and clear dataset schema for downstream training
- Train diffusion-based motion synthesis models for generating realistic bimanual trajectories; include training configs, checkpoints, and sampling scripts
- Develop transformer-based models for multi-modal sensor fusion and teleoperation policies (training scripts, hyperparameters, and evaluation pipelines)
- Benchmark results and provide reproducible training/evaluation notebooks for comparing diffusion vs transformer approaches on the shared dataset