Quantifying and Visualizing Sim-to-Real Gaps: Physics-Guided Regularization for Reproducibility

Yuta Kawachi · 2025

Read Quantifying and Visualizing Sim-to-Real Gaps: Physics-Guided Regularization for Reproducibility by Yuta Kawachi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-23445.txt.

Introduction

Yuta Kawachi wrote a 2025 paper on the gap between simulated and real robot control. Training a controller in simulation is cheap, but it can fail on real hardware when the gap is wide. Randomizing the simulation is a common fix, yet it can make control too timid. Kawachi draws on classic feedback control for a new tack.

The method measures a robot's real response gains with simple tests. Training then penalizes a learned controller whose local responses stray from those gains. The controller also sees the current plant settings as it runs. On a cheap balancing robot, the result settled in hardware as fast as in simulation, with no lingering wobble.

Catalog record

CREATORYuta Kawachi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-23445

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Quantifying and Visualizing Sim-to-Real Gaps: Physics-Guided Regularization for Reproducibility Yuta Kawachi Abstract Simulation-to-real transfer using domain randomization for robot control often relies on low-gear-ratio, backdrivable actuators, but these approaches break down when the sim-to-real gap widens. Inspired by the traditional PID controller, we reinterpret its gains as surrogates for complex, unmodeled plant dynamics. We then introduce a physics-guided gain regularization scheme that measures a robot’s effective proportional gains ( k x , k v , k θ , k ω ) (k_{x},k_{v},k_{\theta},k_{\omega}) via simple real-world experiments. Then, we penalize any deviation of a neural controller’s local input–output sensitivities from these values during training. To avoid the overly conservative bias of naïve domain randomization, we also condition the controller on the current plant parameters. On an off-the-shelf two-wheeled balancing robot with a 110:1 gearbox, our gain-regularized, parameter-conditioned RNN achieves angular settling times in hardware that closely match simulation. At the same time, a purely domain-randomized policy exhibits persistent oscillations and a substantial sim-to-real gap. These results demonstrate a lightweight, reproducible framework for closing sim-to-real gaps on affordable robotic hardware. 1 Introduction Simulation-to-real (sim-to-real) transfer has achieved significant success on robotic platforms equipped with low-gear-ratio, backdrivable actuators. In these systems, such as small humanoids that learn to walk, reinforcement learning (RL) combined with domain randomization (DR) often closes the gap between virtual training and physical deployment [ 14 ] [ 1 ] [ 16 ] . By exposing the controller to modest parameter perturbations in simulation, these methods can operate on real hardware without detailed modeling of friction, backlash, or sensor delay [ 7 ] . However, this success does not readily extend to tasks with a wider sim-to-real gap [ 10 ] [ 15 ] . High-gear-ratio mechanisms [ 7 ] , tight object-insertion assemblies [ 11 ] , and aggressive maneuvers such as parkour jumps [ 2 ] or skateboarding [ 9 ] require additional hardware or training tricks. When the randomization range in simulation is enlarged by these requirements, 1. the probability that a sampled model matches the real plant decreases, which lowers the hit rate , and 2. the learned policy is marginalized over many inaccurate models, which yields the “overly conservative policies” problem [ 5 ] [ 6 ] . From a scientific standpoint, the RL+DR methodology is closer to gambling than engineering: an RL agent gets lucky if it converges to a policy that transfers. In industrial settings, in which reproducibility and certifiable performance are critical, depending on serendipity is unacceptable. Therefore, closing significant sim-to-real gaps requires more accurate knowledge of the physical system. A straightforward remedy is real-to-sim system identification [ 17 ] or iterative simulator refinement. Although this approach can improve fidelity, it demands extensive data collection limited by safety considerations and hardware wear. Furthermore, any unmeasured or hazardous operating regime remains poorly modeled, so the refined simulator can still diverge from reality when the robot executes aggressive motions. Another option is to abandon physics modeling altogether and gauge the hardware directly. The only practical controller family that is both human-interpretable and suitable for direct gauging is the classical proportional-integral-derivative (PID) controller. Nevertheless, PID control is inherently linear and cannot deliver the nonlinear performance required for wide-amplitude swing-up, contact-rich manipulation, or motion with large gear reductions. In this work, we adopt a different perspective. We reinterpret proportional gains, which are traditionally treated as human-tuned design knobs, as surrogates for the complex nature of unmodeled plant characteristics, encapsulating bandwidth, friction, and gear-train dynamics. We incorporate physics-guided gain regularization into the learning loop. We measure the robot’s effective proportional gains (k_{x},\,k_{v},\,k_{\theta},\,k_{\omega}) and penalize any deviation of a neural controller’s local input–output sensitivities from these hardware gains during training. This mechanism prevents the simulator from rewarding unrealistically large virtual gains and keeps the learned policy within the feasible actuation envelope of the real machine while maintaining the theoretical nonlinear control capability. To counteract the “overly conservative policies” that emerge from naïve DR, we introduce parameter-conditioned domain randomization . The current plant parameters are appended to the policy input so the network can specialize its output instead of collapsing toward a single compromise solution. The combination of physics-guided gain regularization and parameter conditioning delivers reproducible, high-performance control on an off-the-shelf two-wheeled balancing robot with a 110:1 gearbox, which demonstrates that lightweight physics guidance, rather than enlarging randomization ranges, is the practical key to affordable robotics. In summary, our contributions are as follows. 1. Physics-guided gain regularization : we formulate a loss that forces the input–output partial derivatives of the neural controller to match hardware gains, which prevents the simulator from rewarding unrealistically large virtual gains. 2. Parameter-conditioned domain randomization : by feeding the plant parameters into the policy, we remove the bias toward overly conservative behaviors. 3. Quantitative analysis of the sim-to-real gap : we visualize how unconstrained policies inflate equivalent gains and we compare both simulated and real responses, which clearly shows the sim-to-real discrepancy in naïve DR. 4. Hardware validation on a low-cost robot : We verified our method on a low-cost, off-the-shelf, two-wheeled, high-gear-ratio (110:1) balancing robot Balboa 32U4, which is considered to be a hard example for the naïve RL+DR approaches. 2 Method Real World: Proportional Gain Measurement Simple Motion Simulation: Train Physics-Guided RNN (Gain Regularization + Conditioned DR) Complex Motion Real World: Deploy Trained RNN Complex Motion Measured Gains Trained RNN Figure 1: Training Pipeline for RNN Controller with Physics-Guided Regularization 2.1 Modeling We consider a two-wheeled balancing robot as a surrogate for a low-cost robotics platform. We adopt a planar cart-pole model with no wheel slip. This simplification captures key nonlinear control challenges while enabling analytical modeling. The nonlinear state-space model is defined as follows [ 4 ] : \displaystyle\Delta \displaystyle=M(ml^{2}+J)-(ml\cos\theta)^{2}, (1) \displaystyle\dot{x} \displaystyle=v, (2) \displaystyle\Delta\dot{v} \displaystyle=-(ml)^{2}g\sin\theta\cos\theta+ \displaystyle(ml^{2}+J)(f+ml\omega^{2}\sin\theta-D_{c}v)+mlD_{p}\omega\cos\theta, (3) \displaystyle\dot{\theta} \displaystyle=\omega, (4) \displaystyle\Delta\dot{\omega} \displaystyle=Mmgl\sin\theta- \displaystyle ml\cos\theta(f+ml\omega^{2}\sin\theta-D_{c}v)-MD_{p}\omega, (5) where x , v , \theta , \omega denote horizontal position, velocity, angle, and angular velocity. m , m_{c} are the pole and cart masses, M=m+m_{c} , l is pole length, J is its inertia, g is gravity, and D_{c} , D_{p} are viscous damping terms. We treat all friction as viscous as an engineering approximation. We use nominal parameters based on empirical tuning (Table 2 ). 2.2 Controller Design and Training We adopt a single-layer Elman RNN controller based on [ 8 ] to handle the plant’s nonlinear dynamics with minimal computation. It receives the plant state \mathbf{x}=[x,v,\theta,\omega]^{\top} and hidden state \mathbf{h} , and outputs force f : \displaystyle f=G(\mathbf{x},\mathbf{h})=c_{\rm out}\cdot C(\sigma(A\mathbf{h}+B\mathbf{x}+b_{1}))+b_{2}, (6) with trainable weights A,B,C,b_{1},b_{2} , and activation \sigma . The RNN hidden state \mathbf{h} is initialized to zeros. c_{\rm out} scales the NN output to match the force range. The matrix A is decomposed as in [ 8 ] to suppress the divergence of the training loss. Unlike deep RL, we exploit known dynamics and define direct losses for control objectives. The state tracking loss is \displaystyle{\mathcal{L}}_{s} \displaystyle=\mathbb{E}_{b,t}\Bigl|({\hat{s}}-s)/s_{\rm base}\Bigr| (7) where s\in\{x,v,\theta,\omega\} , \hat{s}=[0,0,0,0]^{\top} (means upright and stationary), The operator \mathbb{E}_{b,t} denotes the average over mini-batches and time. States are normalized by reference scales s_{\rm base} in the loss calculation. To stabilize training, we apply a continuous curriculum learning (CCL) strategy as shown in Table 1 : the initial training phase

Read the complete text