Quantification of Sim2Real Gap via Neural Simulation Gap Function
P Sangeerth, Pushpak Jagtap · 2025
Read Quantification of Sim2Real Gap via Neural Simulation Gap Function by P Sangeerth, Pushpak Jagtap (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17675.txt.
Introduction
Sangeerth and Jagtap present a paper from twenty twenty-five on measuring the gap between a math model and a near-real simulator. Controllers built on a math model often fail in the real world. The unmodeled gap between model and reality causes those failures. Their gap function puts a formal number on that mismatch. With it, model-based tools can design controllers with a promise of safe transfer. They train the gap function with a neural network on finite data points.
Formal guarantees cover the whole state space, even unseen points. Data come from high-grade simulators tuned to match reality closely. Two case studies, a wheeled robot and a pendulum, show the approach. The authors aim to bridge simulation and real deployment.
Catalog record
| CREATOR | P Sangeerth, Pushpak Jagtap |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-17675
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quantification of Sim2Real Gap via Neural Simulation Gap Function P Sangeerth, Pushpak Jagtap Abstract In this paper, we introduce the notion of neural simulation gap functions, which formally quantifies the gap between the mathematical model and the model in the high-fidelity simulator, which closely resembles reality. Many times, a controller designed for a mathematical model does not work in reality because of the unmodelled gap between the two systems. With the help of this simulation gap function, one can use existing model-based tools to design controllers for the mathematical system and formally guarantee a decent transition from the simulation to the real world. Although in this work, we have quantified this gap using a neural network, which is trained using a finite number of data points, we give formal guarantees on the simulation gap function for the entire state space including the unseen data points. We collect data from high-fidelity simulators leveraging recent advancements in Real-to-Sim transfer to ensure close alignment with reality. We demonstrate our results through two case studies - a Mecanum bot and a Pendulum. 1 Introduction Plenty of literature is available for systems whose precise mathematical models are known. However, in real-world scenarios, getting the exact mathematical model of the system is not always possible. Hence, the controllers developed using the approximate mathematical model of the real world do not necessarily need to work on the original system. This introduced the trend of designing data-driven control algorithms that do not use the actual model of the system but demand vast data of the states, input, and output of the system to design a controller to satisfy the desired specifications. [ 1 ] contains several ways of identifying the system from the input and output data without guarantees. Several papers, including [ 2 , 3 , 4 , 5 ] use the Gaussian process (GP) where the system dynamics is unknown, and estimated dynamics is used to design controllers with probabilistic guarantees by collecting data from the system. Data-driven techniques similar to the ones mentioned in [ 6 , 7 , 8 ] for control Lyapunov functions, control barrier functions, and robust control techniques. However, collecting real-world data is challenging due to the significant time and cost involved. The advent of simulators addressed the problem of time by parallelizing the data collection process.The advancements in sensor technology have improved simulation software, reducing the gap between real-world and simulated environments [ 9 , 10 ] . Advanced simulators like ADAMS-Simulink [ 11 ] , Unreal physics engine [ 12 ] , Metamoto [ 13 ] , NVIDIA’s Drive Constellation 3 3 3 https://resources.nvidia.com/en-us-auto-constellation/drive-constellation , CarMaker 4 4 4 https://ipg-automotive.com/en/products-solutions/software/carmaker/ , and OpenDRIVE 5 5 5 https://www.asam.net/standards/detail/opendrive/ enhance simulation fidelity using advanced sensor models, physics engines, and uncertainty models, providing valuable data for developing controllers with data-driven approaches. Several studies, such as [ 14 , 15 , 16 , 17 ] , concentrate on bridging the gap between simulation environments and reality within the Sim2Real context. In [ 16 ] , the author quantifies the Sim2Real gap as a numerical value with a probabilistic guarantee. In [ 17 ] , the authors quantified the Sim2Real gap as a function dependent on both state and input with 100\% guarantees. However, their approach assumes that this function can be expressed as a linear combination of predefined basis functions, with weight parameters determined through an optimization process similar to regression. The limitation of this method is that if the chosen structure of the simulation gap function is incorrect, the results may become overly conservative. To address this issue, we propose a more flexible approach by modeling the simulation gap function using a neural network, which can better capture complex, nonlinear relationships without requiring a predefined structure. Once this simulation gap is determined, we leverage existing model-based control techniques to synthesize a controller that operates seamlessly with the high-fidelity simulator model. We demonstrate this claim with two physical case studies: a Mecanum bot (simulator model in Gazebo) and a Pendulum (simulator model in Pybullet). 2 System Description and Problem Definition Notations. We denote sets of non-negative integers, positive real numbers, and non-negative real numbers, respectively, by \mathbb{N}:=\{0,1,2,3,\dots\} , \mathbb{R}^{+} , and \mathbb{R}^{+}_{0} . The absolute value of x\in\mathbb{R} is denoted by |x| . The Euclidean norm of x\in\mathbb{R}^{n} is represented by \lVert x\rVert . The symbol {\mathbb{R}^{n}} is used to denote an n -dimensional Euclidean space. A column vector x\in\mathbb{R}^{n} is denoted by x\hskip-1.99997pt=\hskip-1.99997pt[x_{1};x_{2};\cdots;x_{n}] . For a matrix A\in\mathbb{R}^{m\times n} , A^{\top} denotes its transpose. The set of diagonal matrices of dimension n is denoted by \mathcal{D}^{n} and \mathcal{D}_{\geq 0}^{n} is the set of diagonal matrices with non-negative entries. 2.1 System Description In this paper, we consider a nominal mathematical model for the system represented by a discrete-time system, derived through the discretization of conventional continuous-time models [ 18 ] , with a sampling time \tau\in\mathbb{R}^{+} , expressed as {\Sigma}\!:x(k+1)={f}_{\tau}(x(k),u(k)),\quad k\in\mathbb{N}, (1) where x=[x_{1};x_{2};\cdots;x_{n}]\in X\subset\mathbb{R}^{n} is the state vector in a bounded state set X , u=[u_{1};u_{2};\cdots;u_{m}]\in U\subset\mathbb{R}^{m} is the input vector in a bounded input set U , and {f}_{\tau}:\textit{X}\times\textit{U}\rightarrow\textit{X} is the known transition map which is assumed to be locally Lipschitz continuous to guarantee the uniqueness and existence of the solution [ 19 ] . Controllers designed for exact mathematical models often underperform in practice. However, advances in sensor technology now provide accurate real-world state measurements, enhancing our understanding of real systems. This leads to the development of high-fidelity simulators like Gazebo [ 20 ] , CARLA [ 21 ] , Webots [ 22 ] , and Pybullet [ 23 ] which closely replicate real systems using sensor models, physics engines, and uncertainty models. We represent the real-world control system evolution in these simulators by an unknown discretized map with the same discretization parameter \tau as \hat{\Sigma}\!:x(k+1)=\hat{f}_{\tau}(x(k),u(k)),\quad k\in\mathbb{N}, (2) where \hat{f}_{\tau}:{X}\times{U}\rightarrow\textit{X} represents the transition map of the high-fidelity simulator model, which is unknown. For brevity, the f_{\tau} and \hat{f}_{\tau} will be represented as f and \hat{f} in the further discussions. 2.2 Problem Formulation Now, we outline the problem considered in this work. Problem 2.1 Given the nominal mathematical model {\Sigma} and the high-fidelity simulator \hat{\Sigma} , we aim (i) to formally quantify a simulation gap between the mathematical model and the high-fidelity simulator model, represented by a function \gamma(x,u)\!:X\times U\rightarrow{\mathbb{R}^{+}_{0}}^{n} such that for all x\in X, and u\in U , |\hat{f}_{i}(x,u)-{f}_{i}(x,u)|\leq\gamma_{i}(x,u) (3) for all i\in\{1,2,\ldots,n\} and (ii) to design a controller using the nominal mathematical model and the neural simulation gap \gamma(x,u):=[\gamma_{1}(x,u);\gamma_{2}(x,u);\ldots;\gamma_{n}(x,u)] and to enforce desired specifications in the high-fidelity simulator {\hat{\Sigma}} . 3 Neural Simulation Gap Function An unpreventable gap exists between the mathematical model and the high-fidelity simulators. This section focuses on measuring this gap using a neural simulation gap function. To compute the simulation gap function, we first reformulate the conditions mentioned in ( 3 ) as a robust optimization problem (ROP) as follows: \displaystyle\min_{\gamma_{i}\in\mathcal{H},\eta_{i}\in\mathbb{R}} \displaystyle\eta_{i} (4) \displaystyle\text{s.t.} \displaystyle\gamma_{i}(x,u)\leq\eta_{i},\quad\forall x\!\in\!X,\forall u\!\in\!U, \displaystyle|\hat{f_{i}}(x,u)-{f_{i}}(x,u)|\!-\!\gamma_{i}(x,u)\!\leq\!0,\quad\forall x\!\in\!X,\forall u\!\in\!U, where i\in I , I:=\{1,2,\dots,n\} and \mathcal{H}:=\{g\,\big|\,g\!:X\times U\rightarrow\mathbb{R}_{0}^{+}\} is a functional space containing the set of all possible functions. For brevity, whenever we use the subscript i , the full set I is referred to unless specified otherwise. One can readily observe considerable challenges