Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learning Journal: Engineering Applications of Artificial Intelligence
Xiaoyi Hu, Yue Mao, Gang Wang, Qingdu Li, Jianwei Zhang, Yunfeng Ji · 2025
Read Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learning Journal: Engineering Applications of Artificial Intelligence by Xiaoyi Hu, Yue Mao, Gang Wang, Qingdu Li, Jianwei Zhang, Yunfeng Ji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01251.txt.
Introduction
Xiaoyi Hu and five co-authors teach a table tennis robot, inside a simulation, to catch balls that arrive with extreme spin. Their paper came out in 2025. Spin makes the ball swerve and bounce unpredictably, and most playing robots fail against it.
Training followed a curriculum, moving from easy balls to harder ones in stages. A physics analysis of spinning ball collisions produced realistic practice trajectories. The reward signal was built from defined states of the ball's flight, and abnormal rallies were filtered out so they could not distort learning. After training, a reality-to-simulation check tested whether the learned skill matched what real spinning balls do. The authors note their reward design should transfer to other repeating robot tasks.
Catalog record
| CREATOR | Xiaoyi Hu, Yue Mao, Gang Wang, Qingdu Li, Jianwei Zhang, Yunfeng Ji |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01251
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Catching Spinning Table Tennis Balls in Simulation with End-to-End Curriculum Reinforcement Learning Journal: Engineering Applications of Artificial Intelligence Xiaoyi Hu, Yue Mao, Gang Wang, Qingdu Li, Jianwei Zhang, Yunfeng Ji Abstract The game of table tennis is renowned for its extremely high spin rate, but most table tennis robots today struggle to handle balls with such rapid spin. To address this issue, we have contributed a series of methods, including: 1. Curriculum Reinforcement Learning (RL): This method helps the table tennis robot learn to play table tennis progressively from easy to difficult tasks. 2. Analysis of Spinning Table Tennis Ball Collisions: We have conducted a physics-based analysis to generate more realistic trajectories of spinning table tennis balls after collision. 3. Definition of Trajectory States: The definition of trajectory states aids in setting up the reward function. 4. Selection of Valid Rally Trajectories: We have introduced a valid rally trajectory selection scheme to ensure that the robot’s training is not influenced by abnormal trajectories. 5. Reality-to-Simulation (Real2Sim) Transfer: This scheme is employed to validate the trained robot’s ability to handle spinning balls in real-world scenarios. With Real2Sim, the deployment costs for robotic reinforcement learning can be further reduced. Moreover, the trajectory-state-based reward function is not limited to table tennis robots; it can be generalized to a wide range of cyclical tasks. To validate our robot’s ability to handle spinning balls, the Real2Sim experiments were conducted. For the specific video link of the experiment, please refer to: https://www.youtube.com/watch?v=uAx7eYAGylc. 1 INTRODUCTION In all ball sports, table tennis (Ping-Pong) is renowned for its spin. The high spin not only changes the trajectory of the table tennis ball but also makes it more difficult to hit it back to the opponent’s table. Due to the high cost involved in measuring the spin speed of table tennis balls, predicting the trajectory of spinning table tennis balls and formulating corresponding hitting policies have become extremely challenging tasks. Therefore, most table tennis robots today are unable to cope with high-speed spinning table tennis balls. Despite the difficulty in returning spinning table tennis balls, there has been a significant amount of research on the hitting policies for table tennis robots. At present, there are two methods to implement the hitting policy: the model-based hitting policy and the data-driven hitting policy. The main feature of the model-based hitting policy is to predict the table tennis trajectory by using the physical model and state estimation method. Then, the hitting action is made according to the prediction [ 1 ] , [ 2 ] , [ 3 ] . Although this policy has excellent interpretability, unprecise physical properties may degrade its performance. The data-driven policy utilize playing data to improve the performance of the table tennis robot. This training method encompasses supervised learning to improve the accuracy of trajectory prediction [ 4 ] , [ 5 ] , [ 6 ] and optimizing robot performance with control theory [ 7 ] . These methods mentioned above often require significant costs for data collection and labeling. Moreover, policies based on labeled data may not optimal. Therefore, a better solution is to enable robots to actively collect data and use it for self-updates and improvements. In recent years, with the rapid development of reinforcement learning (RL), using reinforcement learning in robotics has gradually become feasible. However, since this learning method requires a significant amount of trial and error to collect data, ensuring safety has been a concern [ 8 ] , [ 9 ] . To ensure the safety of table tennis robots in RL, many researchers have proposed various solutions. [ 10 ] proposed a robot driven by pneumatic muscles so that the robot can perform trials and errors without causing any damage even colliding with itself or external objects. To ensure that the actions performed by the trained robots are safe, [ 11 ] proposed that RL should only be used to determine the posture and velocity of the racket, while the control elements that may cause unexpected accidents, such as the racket position and joint angle, should be implemented by other safer methods. Despite addressing safety concerns, the extremely low data reuse in reinforcement learning leads to high training costs for robots in real-world scenarios [ 12 ] , [ 13 ] . This not only wastes lots of time but also causes fatigue in the mechanical structure of the robot [ 14 ] . Compared to training robots in reality, training robots in a simulation can significantly expedite the sampling efficiency through multi-environment. Additionally, with the help of Sim2Real, it becomes possible to transfer the learned behaviors of robots from the simulation to the real-world environment [ 15 ] . This approach accelerates data collection and reduces costs, making it a promising solution for safe and efficient robot learning in real world. Considering the factors mentioned above, the main focus of this paper is on how to train a table tennis robot in a simulation environment that can effectively catch spinning balls. Additionally, the feasibility of this approach will be demonstrated through Real2Sim experiments, which involve transferring the learned behaviors from simulation to the real world. The goal is to show that the training in the simulated environment can effectively equip the robot to handle spinning balls and perform well in real-world settings. 1. Curriculum Reinforcement Learning: Curriculum reinforcement learning is an special method in reinforcement learning that involves dividing a training task into multiple subtasks [ 16 ] , [ 17 ] . By employing this method, the inherent difficulty of the training task is reduced, leading to enhanced performance of the learning agent. Therefore, we introduce a new end-to-end table tennis robot training method on the basis of curriculum reinforcement learning. The curriculum reinforcement learning method employed in this study requires the robot to progressively learn three tasks: catching the ball, hitting the ball, and hitting the ball to the target point. 2. Analysis of Spinning Table Tennis Ball Collisions: When using Sim2Real for robot reinforcement learning, the performance of the robot is often affected by the accuracy of the simulation environment. In many simulation environments, trajectory deviations caused by high-speed spinning table tennis ball collisions are not considered. To address this issue, this study analyzes the velocity changes before and after the collision of spinning table tennis balls based on a physics model. The physics model is integrated with the existing simulation environment to enhance the accuracy of simulating spinning ball collisions. 3. Definition of Trajectory States: Since training an end-to-end table tennis robot solely using curriculum learning cannot fully achieve the target, a trajectory state definition method for table tennis rally is introduced in this study. During training for different tasks, the reward function varies based on the current trajectory state of the table tennis robot. Therefore, we propose a matrix-based approach to define the reward function, making the reward function for curriculum reinforcement learning more intuitive to comprehend. 4. Selection of Valid Rally Trajectories: To improve the learning efficiency and reduce the impact of irrelevant samples on the agent, this paper constructs a scheme to automatically generate valid spinning table tennis ball trajectories according to the definition of trajectory states. 5. Reality-to-Simulation (Real2Sim) Transfer: To