Lipschitz-Based Robustness Certification for Recurrent Neural Networks via Convex Relaxation
Paul Hamelbeck, Johannes Schiffer · 2025
Read Lipschitz-Based Robustness Certification for Recurrent Neural Networks via Convex Relaxation by Paul Hamelbeck, Johannes Schiffer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-17898.txt.
Introduction
Paul Hamelbeck and Johannes Schiffer certify how much a recurrent net can amplify input noise. Their 2025 paper deals with nets that read data in sequence, used in control tasks where safety counts. A Lipschitz bound caps that gain, so a proved bound gives a safety claim.
Their method, RNN-SDP, turns layer links into a convex maths task and solves it for a bound. Known input limits can be fed in to tighten the bound a bit. Tests on a fake tank system gave bounds close to measured ones. Start-state error stood out as a factor folk often underrate.
Catalog record
| CREATOR | Paul Hamelbeck, Johannes Schiffer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-17898
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Lipschitz-Based Robustness Certification for Recurrent Neural Networks via Convex Relaxation Paul Hamelbeck, Johannes Schiffer Abstract Robustness certification against bounded input noise or adversarial perturbations is increasingly important for deployment recurrent neural networks (RNNs) in safety‑critical control applications. To address this challenge, we present RNN-SDP, a relaxation based method that models the RNN’s layer interactions as a convex problem and computes a certified upper bound on the Lipschitz constant via semidefinite programming (SDP). We also explore an extension that incorporates known input constraints to further tighten the resulting Lipschitz bounds. RNN-SDP is evaluated on a synthetic multi-tank system, with upper bounds compared to empirical estimates. While incorporating input constraints yields only modest improvements, the general method produces reasonably tight and certifiable bounds, even as sequence length increases. The results also underscore the often underestimated impact of initialization errors—an important consideration for applications where models are frequently re-initialized, such as model predictive control (MPC). I Introduction Neural networks (NNs) have demonstrated remarkable success across a wide range of applications, including computer vision [ 1 ] , natural language processing [ 2 ] , and control systems [ 3 ] , making them indispensable tools in modern machine learning and artificial intelligence. However, their highly non-linear and high-dimensional structure often renders them opaque, leading to their characterization as black-box models: capable of impressive performance, but offering limited interpretability or formal guarantees regarding their behavior [ 4 ] . This opacity becomes particularly problematic in safety-critical domains, such as autonomous driving, medical diagnostics, or large-scale control systems, where even small input perturbations can result in unpredictable or unsafe outcomes [ 5 ] . In such settings, formal verification and robustness analysis are essential to ensure reliability, stability, and trust in NN-based decision systems [ 4 ] . In response to these challenges, substantial efforts have been made to reduce noise sensitivity, while maintaining the predictive capabilities NNs as well as to develop tools for rigorously assessing their robustness. Techniques such as adversarial training [ 6 ] and regularization [ 7 ] aim to reduce a network’s sensitivity to input perturbations and thereby increas robustness against noise and adversarial attacks. While these robustness-enhancing techniques play a critical role during training, they must be complemented by verification tools that can formally assess the model’s behavior across a range of inputs. Such verification techniques typically fall into one of three categories: a) reachability-based , b) optimization-based , or c) relaxation-based verification methods, each offering a different balance of precision, scalability, and generality. Reachability-based Reachability-based verification methods aim to compute the full set of possible outputs \mathcal{Y} that a NN can produce for all inputs \mathcal{X} within a given region. This is typically achieved through layer-by-layer propagation of the input set, which can be computationally expensive due to the exponential growth in the number of activation regions as the network depth increases. Methods that do not introduce simplifications yield exact reachable sets without over-approximation, but their scalability is limited to small networks [ 8 ] . To address this, many approaches incorporate set-based relaxations or over-approximations, which significantly reduce computational complexity and make the analysis tractable for larger models [ 9 ] . Additionally, reachability-based methods as a whole are generally limited to networks with ReLU or other piecewise-linear activation functions, as these preserve the geometric structure necessary for sound and efficient set propagation [ 4 ] . Optimization-based Optimization-based verification methods attempt to either find a counterexample, that is within the set \mathcal{U} that produces an output outside the safe region \mathcal{Y} [ 10 ] or identify the worst-case input that causes the most undesirable behavior, as defined by a given cost function [ 11 ] . These approaches commonly encode NNs as a set of mixed-integer linear constraints, enabling formal optimization. While such formulations can be solved using complete solvers like branch-and-bound, which can either find a violation or prove that none exists, they are typically limited to networks with piecewise-linear activation functions, such as ReLU or max-pooling [ 11 ] . Another commonly used solver, the satisfiability modulo theories (SMT) solver, can, in principle, also handle arbitrary non-linear activation functions (e.g., sigmoid or tanh). However in practice applications to networks with such activation functions remain limited as computational costs drastically increase and the completness of the approach is lost [ 12 ] . Relaxation-based Relaxation-based verification methods approximate the constraints imposed by the NN with looser, more tractable formulations, typically to enable scalable analysis [ 4 ] . Instead of precisely encoding every activation or network branch, these methods relax non-linear or combinatorial behaviors into convex or piecewise-linear forms. The result is typically a bound on the network’s output or sensitivity, rather than a definitive yes/no certificate. The central trade-off is between tightness of the bound and computational efficiency: coarser relaxations scale well but offer weaker guarantees, while tighter relaxations are more informative but expensive to compute. Prominent examples include: CROWN [ 13 ] , which propagates linear bounds layer-by-layer; ConvDual [ 14 ] , which uses dual feasible solutions to certify output ranges; and LipSDP [ 15 ] , which leverages a convex optimization framework, that yields certified upper bounds on the global Lipschitz constant on a feed-forwards neural network (FFNN) by approximating non-linear activation functions with quadratic constraints. The LipSDP approach is particularly appealing from a control-theoretic perspective, as the Lipschitz constant, which bounds the worst-case change in output relative to input variation, is a widely used robustness metric in many control frameworks, such as observer design [ 16 ] , and model predictive control (MPC) [ 17 ] . However, recent advances in system identification and control have seen a shift from FFNNs to recurrent neural networks (RNNs), due to their superior capacity for modeling temporal dynamics [ 18 , 19 , 20 ] . RNNs achieve this by incorporating an internal feedback loop, which enables the retention of historical information and facilitates sequential prediction. In contrast, FFNNs generate outputs based solely on the current input. While this internal recurrence enhances the representational capacity of RNNs for dynamic systems, it also introduces structural complexity that renders many existing verification techniques inapplicable. Bonassi et al. [ 18 ] specifically highlight the absence of a method akin to LipSDP, in the context of RNNs, noting its importance for control applications that demand certified robustness. Our Contributions We propose the RNN-SDP framework for tightly bounding the Lipschitz constant of RNNs, enabling robustness and stability analysis. Inspired by LipSDP [ 15 ] , we propose an SDP-based certifiaction techique tailored to RNNs, producing finite‑horizon input–output Lipschitz bounds that account for initialization errors The latter is particularly critical in applications with frequent re-initialization, which prevents the system from reaching a steady state, making initialization effects persistently relevant. Additionally, we present a framework to utilize known input constraints, which are commonly present in practical NN applications [ 21 ] , to further tighten the estimated bound of the Lipschitz constant. Our Approach By restricting the sequence length for predictions to an arbitrary finite length of N , we can transform the RNN into an equivalent FFNN via unrolling [ 22 ] . For this unrolled structure, we exploit the slope-restricted nature of non-linear activation functions to derive an