Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates

Amine Jaouadi, Sahel Ashhab · 2026

Read Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates by Amine Jaouadi, Sahel Ashhab (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19990.txt.

Introduction

Jaouadi and Ashhab tackle robust calibration of multi-qudit quantum gates, where higher-dimensional systems beat qubits in principle but suffer crowded spectra and weak controllability. Their hybrid method first designs high-fidelity control pulses for controlled-phase gates on two qutrits under a nominal model. Reinforcement learning then learns small residual corrections under static model mismatch, preserving gate quality across device variations. The learning stage complements rather than competes with optimal control, cutting sensitivity to parameter swings and making reinforcement learning a practical calibration ingredient.

Catalog record

CREATORAmine Jaouadi, Sahel Ashhab
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63722
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19990

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reinforcement Learning for Robust Calibration of Multi-Qudit Quantum Gates Amine Jaouadi, Sahel Ashhab Abstract Higher-dimensional quantum systems, such as qudits, offer architectural and algorithmic advantages over qubits, but their increased spectral crowding and limited controllability render high-fidelity quantum gates particularly challenging. We propose a hybrid optimization framework that integrates optimal control theory methods with contextual deep reinforcement learning to achieve robust controlled-phase gates on two qutrits. Optimal control is first used to design high-fidelity control pulses for a nominal system model. Reinforcement learning is then employed as a calibration stage that learns small residual corrections to these pulses in the presence of static model mismatch, thereby preserving good gate performance under realistic parameter uncertainties. By learning structured, low-dimensional residual corrections conditioned on device-specific parameter variations, reinforcement learning enhances the transfer robustness of nominally optimal but parameter-sensitive control solutions across ensembles of devices. Crucially, the reinforcement learning step in our framework does not compete with the optimal control step but provides the adaptability required for realistic hardware, systematically reducing the sensitivity to parameter fluctuations. Our results establish reinforcement learning as a practical and scalable ingredient for robust calibration of quantum gates in high-dimensional systems. I Introduction Qudits— d -level quantum systems with d>2 —are increasingly recognized as promising building blocks for quantum information processing, offering larger local Hilbert spaces, richer entangling operations, and potential resource savings in circuit depth and error correction Bullock2005 ; Lanyon2009 ; Chi2022 . In particular, qutrits ( d=3 ) have recently been explored in several hardware platforms, including trapped ions and superconducting circuits Bianchetti2010 ; Blok2021 ; Roy2023 ; Goss2022 ; Kononenko2021 ; Glaser2015 ; Morvan2021 ; Yurtalan2020 ; Ringbauer2022 . Superconducting transmons are especially attractive qutrit candidates because their weak anharmonicity naturally provides access to the first three energy levels with gate times comparable to qubit operations Basyildiz2025 . It should be noted that other superconducting qubit designs, such as the capacitively-shunted flux qubit, also allow access to higher energy levels You2007 . Recent experiments have demonstrated high-fidelity single-qutrit control, randomized benchmarking, and two-qutrit entangling gates in transmon devices Goss2022 ; Roy2023 ; Subramanian2023 ; Kononenko2021 . Nevertheless, engineering robust two-qutrit gates remains challenging: the richer level structure enhances controllability but also introduces additional leakage channels and strong sensitivity to device parameters such as transition frequencies and coupling strengths Goss2022 ; Subramanian2023 . Fabrication variability and slow drifts in these parameters further accentuate the need for automated calibration strategies beyond manual tune-up. Quantum optimal control theory (QOCT), and in particular gradient ascent pulse engineering (GRAPE) Heeres2017 ; Poggi2024 ; Jaouadi2013 ; Vranckx2013 , has emerged as a key tool for numerically designing high-fidelity gates in effective models, such as superconducting circuits Khaneja2005 ; Koch2022 ; Egger2013 ; Kelly2014 . GRAPE efficiently computes gradients of the fidelity with respect to the control waveform and has produced fast, high-fidelity two-qubit CZ gates Egger2013 ; Ashhab2012 ; Ghosh2013 and leakage-suppressed single-qubit operations Motzoi2009 ; Ashhab2022 . However, the performance of OCT solutions is fundamentally tied to model accuracy: deviations of the true device Hamiltonian from the nominal design, whether from fabrication spread or temporal drift, can degrade gate fidelity Koch2022 ; Egger2014 . Accordingly, robustness has become a key objective in multi-level superconducting quantum control Poggi2024 . In parallel, deep reinforcement learning (DRL) has shown promise as a largely model-free approach to quantum control, enabling the direct optimization of control pulses through reward feedback rather than explicit system modeling Bukov2018 ; Niu2019 ; Liu2025 ; Jaouadi2024 . These methods have been successfully implemented using algorithms such as Soft Actor–Critic (SAC) Haarnoja2018 , Twin Delayed Deep Deterministic Policy Gradient (TD3) Fujimoto2018 , Deep Deterministic Policy Gradient (DDPG) Lillicrap2015 , and Proximal Policy Optimization (PPO) Schulman2017 . Yet recent studies have highlighted intrinsic limitations of model-free DRL when applied directly to high-dimensional pulse synthesis. In particular, the action space in quantum control scales with the number of pulse segments, often reaching hundreds or thousands of degrees of freedom. Recent studies Li2025_RLfD report that standard DRL agents fail to discover high-fidelity pulses when learning from scratch in such high-dimensional spaces, stalling at low rewards and exhibiting unstable training unless strong priors or demonstrations are provided. This limitation becomes particularly clear for multi-qutrit systems, where leakage channels and spectral crowding vastly enlarge the control landscape. Complementary insights into the control of multi-level superconducting circuits have recently been provided in Basyildiz2025 , in which the authors analyzed fundamental speed limits for a two-qutrit controlled-phase gate implemented with superconducting qudits. Using gradient-based optimal control to construct time-optimal protocols together with analytical quantum speed limit arguments. They showed that even for idealized models the achievable gate performance is constrained by the underlying Hamiltonian structure and control resources. On the other hand, these results emphasize that, on a nominal device, gradient-based OCT can already saturate the accessible control landscape for two-qutrit gates, leaving little room for improvement by unguided learning-based approaches. These observations suggest that RL is unlikely to replace OCT for pulse synthesis on nominal, well-characterized devices, but may instead play a complementary role in addressing model mismatch and calibration challenges. From a practical perspective, repeated re-optimization of OCT pulses for each device instance or parameter drift can be computationally demanding and difficult to scale. This motivates approaches in which a high-quality nominal pulse is computed once, and subsequent corrections are obtained through lightweight calibration procedures rather than repeated high-dimensional optimization. Motivated by these challenges, we introduce a hybrid OCT+DRL framework in which OCT and DRL play complementary roles: OCT provides a nominal high-fidelity open-loop pulse, while DRL acts as a calibration stage that learns residual corrections to mitigate fidelity loss caused by static model mismatch. The novelty of our contribution can be summarized as follows: • DRL on top of OCT pulses. Rather than asking DRL to discover an optimal pulse in a high-dimensional space (a task known to be prohibitive Li2025_RLfD ), we compute a pair of nominal GRAPE pulses, computed once on the nominal Hamiltonian for a two-qutrit \mathrm{CZ}_{3} gate and train DRL agents to output low-dimensional, smooth residual corrections . This approach bypasses the failure mode of DRL in large action spaces. • Contextual bandit formulation of device-aware calibration. Each episode corresponds to a different noisy device instance, and the observation encodes normalized parameter deviations. The reward is the incremental fidelity gain over the OCT baseline , forcing the agent to learn robustness rather than re-learn the gate. • Cosine-basis parametrization. Residual corrections are parametrized in a truncated discrete cosine basis, drastically reducing action dimensionality and enforcing smoothness. • Comparison of multiple DRL algorithms. SAC, TD3, DDPG, and PPO are evaluated under identical conditions. Importantly, none of the agents outperform OCT on the nominal Hamiltonian—consistent with the failure of DRL in high-dimensional unconstrained optimization Li2025_RLfD . This observation further supports our interpretation of DRL as a calibration (i.e. fine-tuning) tool rather than a replacement for gradient-based pulse design. The paper is organized as follows. Section II introduces the effective two-qutrit model, the target controlled-phase gate, and the fidelity measure, together with an overview of the control architecture. Section III presents the GRAPE-based optimal control procedure and demonstrates its convergence on the nominal device. Section IV describes the residual reinforcement-learning framework, including

Read the complete text