Robot Trajectron V3 A Probabilistic Shared Control Framework for SE(3) Manipulation

Jul 16, 2026

Teleoperating robotic arms for high-degree-of-freedom manipulation is cognitively demanding and error-prone, especially under low-bandwidth, noisy user interfaces. Robot Trajectron V3 (RT-V3) introduces a probabilistic shared control framework for SE(3) grasping tasks, framing shared control as Bayesian inference. A learned intent prior models user future trajectories conditioned on past robot dynamics and visual context, then combines it with a real-time command likelihood to infer and continuously refine posterior intent. Experiments show high trajectory-prediction accuracy, competitive reactive planning, and improved success rate, efficiency, and reduced physical and mental workload.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

Introduction

Challenges in teleoperation shared control

Environment complexity and grasp ambiguity

Low-bandwidth user input vs high-DoF control

Proposed approach: Robot Trajectron V3 (RT-V3)

Robot Trajectron V3: A Probabilistic Shared Control Framework for SE(3) Manipulation Pinhao Song 1 ,3 , Zhongxi Li2 ,3 , Ze Fu 1 ,3 , Federico Ulloa Rios 1 ,3 , Renaud Detry 1 ,2 ,3 arXiv :2607 .09315v1 [ cs .RO] 10 Jul 2026 Abstract—We aim to address the challenge of teleoperating robotic arms for high-degree-of-freedom (high-DoF) manipulation tasks, which is cognitively demanding and error-prone, particularly when relying on low-bandwidth interfaces. We propose Robot Trajectron V3 (RT-V3), a probabilistic shared control framework designed for SE(3) grasping tasks. RT-V3 formulates shared control as Bayesian inference by learning a prior over user intent and combining it with real-time user commands to estimate the posterior intent distribution. The prior models user intent as a distribution over future trajectories conditioned on past robot dynamics and visual scene context. The intent prior is parameterized by a transformer-based conditional generative model that reasons over point clouds and candidate grasp poses, together with a factorized translation-rotation representation that improves learning efficiency in high-dimensional action spaces. During execution, RT-V3 continuously estimates the posterior distribution over future trajectories by combining the learned intent prior with a user-command likelihood derived from the observed control input, enabling continuous intent refinement and shared assistance. Comprehensive experiments demonstrate that RT-V3 achieves high accuracy in trajectory prediction and competitive performance in reactive planning. Furthermore, realworld user studies indicate that RT-V3 significantly outperforms baseline methods in terms of success rate and efficiency, while substantially reducing the user’s physical and mental workload. Our code is available at [https://mousecpn.github.io/RTV3](https://mousecpn.github.io/RTV3) page/. I. INTRODUCTION Teleoperation of robotic arms has made significant progress in recent years, improving both real-time performance and usability. Modern teleoperation systems often rely on highbandwidth interfaces such as shadow arms [1], virtual reality devices [2], or motion capture systems [3], enabling users to control robots with high precision. Such systems provide an important pathway for assistive manipulation, allowing individuals to interact with and manipulate objects through robotic devices. However, these interfaces are often inaccessible to users with motor impairments. Instead, such users typically rely on low-bandwidth and noisy input devices, such as chin joysticks [4] or head joysticks [5] . Emerging neural interfaces [6], [7] further offer a promising alternative for users with severe motor impairments, but they remain limited by low-bandwidth and noisy control signals. Controlling highdegree-of-freedom (high DoF) robotic manipulators with these Supported by Interne Fondsen KU Leuven/Internal Funds KU Leuven. Partially supported by Flanders Make (strategic research centre for the manufacturing industry) . 1 KU Leuven, Dept. Mechanical Engineering, Research unit Robotics, Automation and Mechatronics, B-3000 Leuven, Belgium. Email: [firstname.lastname@kuleuven.be](firstname.lastname@kuleuven.be) 2 KU Leuven, Dept. Electrical Engineering, Research unit Processing Speech and Images, B-3000 Leuven, Belgium. 3Flanders Make@KU Leuven. interfaces is challenging. Performing manipulation tasks such as grasping objects from specific orientations often requires numerous commands, making teleoperation slow, cognitively demanding, and prone to error. Shared control addresses this challenge by combining user input with contextual information to infer user intent and assist task execution. By integrating human commands with environmental perception, shared control systems can reduce user effort while preserving user authority, enabling more efficient human–robot collaboration. Shared autonomy methods have demonstrated promising results in planar na

FAQs

What problem does Robot Trajectron V3 target in robotic teleoperation?

It targets the difficulty of teleoperating high-DoF robotic arms for manipulation and grasping under low-bandwidth, noisy interfaces, where tasks become slow, cognitively demanding, and error-prone.

How does RT-V3 implement shared control?

RT-V3 formulates shared control as Bayesian inference by learning a prior over user intent and combining it with a user-command likelihood to estimate a posterior distribution over future trajectories.

What information does RT-V3 use to form its intent prior?

The intent prior is modeled as a distribution over future trajectories conditioned on past robot dynamics and visual scene context, enabling context-aware grasp assistance.

Jul 16, 2026

Upload by Evangeline