TriPilot-FF: Coordinated Whole-Body Teleoperation with Force Feedback
Zihao Li, Yanan Zhou, Ranpeng Qiu, Hangyu Wu, Guoqiang Ren, Weiming Zhi · 2026
Read TriPilot-FF: Coordinated Whole-Body Teleoperation with Force Feedback by Zihao Li, Yanan Zhou, Ranpeng Qiu, Hangyu Wu, Guoqiang Ren, Weiming Zhi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09888.txt.
Introduction
Mobile manipulators widen what robots can do, but whole-body teleoperation forces one operator to coordinate a wheeled base and two arms while watching obstacles and contact. Zihao Li and five co-authors present TriPilot-FF, an open-source whole-body teleoperation system for a bimanual mobile manipulator, in their 2026 paper. Its channel is a foot-operated pedal with lidar-driven haptics, paired with upper bimanual leader-follower control.
From proximity-to-obstacle signals along the commanded direction, a resistive pedal cue is rendered by the system using only a cheap mounted lidar. Commands steer toward safer behavior with no avoidance controller in play. Arm-side force reflection adds contact awareness, and live force and visual guidance of bimanual manipulability prompts base repositioning for better reach. The team also feeds teleoperation signals into an Action Chunking with Transformers policy with improved results, and releases the pedal design and software stack.
Catalog record
| CREATOR | Zihao Li, Yanan Zhou, Ranpeng Qiu, Hangyu Wu, Guoqiang Ren, Weiming Zhi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09888
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TriPilot-FF: Coordinated Whole-Body Teleoperation with Force Feedback Zihao Li, Yanan Zhou, Ranpeng Qiu, Hangyu Wu, Guoqiang Ren, Weiming Zhi Abstract Mobile manipulators broaden the operational envelope for robot manipulation. However, the whole-body teleoperation of such robots remains a problem: operators must coordinate a wheeled base and two arms while reasoning about obstacles and contact. Existing interfaces are predominantly hand-centric (e.g., VR controllers and joysticks), leaving foot-operated channels underexplored for continuous base control. We present TriPilot-FF , an open-source whole-body teleoperation system for a custom bimanual mobile manipulator that introduces a foot-operated pedal with lidar-driven pedal haptics, coupled with upper-body bimanual leader-follower teleoperation. Using only a low-cost base-mounted lidar, TriPilot-FF renders a resistive pedal cue from proximity-to-obstacle signals in the commanded direction, shaping operator commands toward collision-averse behaviour without an explicit collision-avoidance controller. The system also supports arm-side force reflection for contact awareness and provides real-time force and visual guidance of bimanual manipulability to prompt mobile base repositioning, thereby improving reach. We demonstrate the capability of TriPilot-FF to effectively “co-pilot” the human operator over long time-horizons and tasks requiring precise mobile base movement and coordination. Finally, we incorporate teleoperation feedback signals into an Action Chunking with Transformers (ACT) policy and demonstrate improved performance when the additional information is available. We release the pedal device design, full software stack, and conduct extensive real-world evaluations on a bimanual wheeled platform. The project page of TriPilot-FF is http://bit.ly/46H3ZJT . I Introduction Mobile manipulators are increasingly deployed in settings where value is created through whole-body interactions. Compared with fixed-base arms, these platforms offer substantially larger operational envelopes and the ability to recover from failed interactions by moving the body rather than re-grasping. However, this capability comes with a core systems challenge: whole-body control couples navigation, collision avoidance, and manipulation. Operators must continuously coordinate a mobile base and two arms while reasoning about obstacles, contact forces, reachability, and task intent. In typical interfaces, most control bandwidth is allocated to the hands, such as joysticks, while the feet are often unused. This design choice is at odds with human motor capabilities: feet can provide a robust, low-latency control channel that naturally complements hand-centric manipulation, especially for base motion. As a result, operators often lack an informative modality for continuous base control that reduces collisions and coordination burden. Additionally, autonomous robot policies are increasingly trained from large-scale real-world datasets [ 21 ] . Teleoperation is therefore not merely a fallback for autonomy, but is becoming a primary data acquisition and supervision mechanism for scalable robot learning. Interfaces that reduce operator workload, prevent common failures such as base collisions, and expose informative feedback channels can translate directly into cleaner demonstrations and training signals for policy learning. As a result, we are motivated by a design philosophy that treats whole-body teleoperation as an interactive setup, not simply a mapping from user inputs to robot actions. The operator is continuously balancing base safety, contact stability at the end-effectors, and kinematic feasibility of the bimanual configuration. Providing this information through on-board cameras alone is often insufficient; it forces the operator to mentally simulate constraints that the system can encounter in real time. A complementary approach is to render task-relevant constraints over visual inputs and, through haptic guidance, shape user commands without removing control authority or requiring an explicit autonomy stack. We present TriPilot-FF , an open-source whole-body teleoperation system for a custom bimanual mobile manipulator that operationalises this approach. Here, we reclaim the feet as a continuous base-control modality, augmented with lightweight, real-time guidance signals that reduce cognitive load while preserving intuitive operation. TriPilot-FF introduces a foot-operated pedal device that provides lidar-driven force feedback . Using only a low-cost base-mounted lidar, the system estimates proximity to obstacles in the commanded direction and renders an opposing resistive cue at the pedal. This physically shapes base commands toward collision-averse behaviour, allowing the operator to remain in control while receiving immediate, embodied feedback about imminent constraints. TriPilot-FF couples this with upper-body bimanual leader-follower teleoperation for intuitive arm control with arm-side force reflection to improve contact awareness during manipulation. To address reachability and coordination failures common in mobile bimanual tasks, the system additionally provides real-time force and visual guidance of bimanual manipulability, prompting operators with force cues when base repositioning is required to restore reach. These feedback signals act as a “co-pilot” for operators. Finally, we connect interface design to robot learning by incorporating feedback signals into an Action Chunking with Transformers (ACT) policy [ 29 ] . Teleoperation produces more than state-action pairs: it also yields structured resistive feedback that reflects task constraints. We demonstrate that including torque into the state information of the policy improves downstream performance, supporting the thesis that better teleoperation interfaces can be leveraged for better learning. illustrates a few of the autonomous and teleoperated tasks demonstrated in this work. Concretely, our technical contributions include: • TriPilot-FF: a whole-body teleoperation system coupling foot-operated base control with bimanual leader-follower arm teleoperation for mobile manipulation. • Lidar-driven pedal feedback: direction-conditioned resistive cues from obstacle proximity, biasing commands toward collision-averse behaviour. • Multi-modal whole-body guidance: arm-side force reflection for contact awareness and real-time bimanual manipulability guidance on the pedal to mitigate reachability-driven failures. • Autonomous robot executions by an augmented ACT formulation that incorporates collected force information. II Related Works Mobile and Whole-body Manipulation: Mobile manipulation has traditionally been approached via motion planning and control pipelines [ 23 , 19 , 8 , 22 ] , coordinating an arm mounted on a wheeled base through hand-crafted hierarchical systems. More recently, robot learning, most notably MobileALOHA [ 7 ] , has enabled hardware and data-collection setups for training whole-body policies. Earlier efforts also studied mobile manipulation data collection in simulation [ 25 ] , including task-specific settings such as door opening [ 27 ] . Beyond wheeled bases, prior work explores other mobile platforms, including quadrupeds with manipulators [ 6 , 30 ] . A related thread studies humanoid teleoperation [ 11 , 16 , 9 ] , including interfaces that provide visual feedback to the operator [ 3 ] . Our system targets bimanual mobile manipulators and is most closely related to MobileALOHA [ 7 ] and recent mobile manipulation systems [ 20 , 10 ] , with conceptual links to novel humanoid teleoperation hardware [ 1 ] . Unlike MobileALOHA, our operator need not be co-located with the robot, and receives both visual and force feedback across upper- and lower-limb control channels. Imitation Learning: Beyond the teleoperation interface, our work connects to imitation learning of visuomotor policies [ 15 ] , which seeks to map the robot’s sensory inputs directly to robot actions. Recent policy classes include diffusion models [ 4 ] , flow-based models [ 17 , 12 ] , and transformer-based policies [ 29 , 2 , 14 ] . Complementary to model design, several low-cost systems have been proposed for demonstration collection in imitation learning [ 5 , 26 , 18 ] . TriPilot is similarly low-cost, and enables the collection of high-quality data: we use it to train ACT [ 29 ] policies, condition them on additional force/torque signals logged by TriPilot-FF, and show that the resulting dataset supports data-efficient training. Fig. 2 : TriPilot-FF combines bimanual leader-follower arms