Practical Insights into Fair Comparison and Evaluation Frame for Neutral-Atom Compilers
Emil Khusainov , Yanbin Chen 1,2 , Jonas Winklmann 3 , Helmut Seidl 1 , Christian B. Mendl 2 · 2026
Read Practical Insights into Fair Comparison and Evaluation Frame for Neutral-Atom Compilers by Emil Khusainov , Yanbin Chen 1,2 , Jonas Winklmann 3 , Helmut Seidl 1 , Christian B. Mendl 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25478.txt.
Introduction
Khusainov and four colleagues present a shared test frame for neutral-atom code tools. Neutral-atom qubits promise quantum machines, and toolchains matter for moving qubits and side-by-side gates. Test scores hinge on uneven tuning, move models, and fidelity sources, since tools are judged under mismatched rules. Their frame brings RSQASM, a record of mapped, routed, and timed circuits, with side-by-side gates and shuttling. Adapter scripts turn tool outputs into RSQASM. For HybridMapper, DasAtom, and Enola, claimed gaps shrink or vanish once uneven rules are gone.
Catalog record
| CREATOR | Emil Khusainov , Yanbin Chen 1,2 , Jonas Winklmann 3 , Helmut Seidl 1 , Christian B. Mendl 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56803 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25478
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Practical Insights into Fair Comparison and Evaluation Frame for Neutral-Atom Compilers Thanks: The research is funded by the Munich Quantum Valley (MQV), which is supported by the Bavarian state government with funds from the Hightech Agenda Bayern Plus, and by the German Federal Ministry of Research, Technology and Space (BMFTR) under the funding program Quantum Technologies - From Basic Research to Market under contract number 13N16087. We are also grateful to the Quantum Compiler team at the Leibniz Supercomputing Centre (LRZ) for many fruitful discussions. Emil Khusainov , Yanbin Chen 1,2 , Jonas Winklmann 3 , Helmut Seidl 1 , Christian B. Mendl 2 Abstract Neutral-atom quantum computing is among the most promising platforms for scalable quantum computation, and compilation toolchains are crucial for leveraging capabilities such as qubit shuttling and parallel gate execution. An important challenge, however, is that existing neutral-atom compilers are often evaluated using metrics computed over different parts of the toolchain and under non-equivalent assumptions. Consequently, fair quantification and comparison of compiler performance remain difficult. Reported metrics may depend on inconsistent transpilation optimization levels, different movement-duration models, different sets of considered fidelity sources, and even minor implementation bugs or undocumented representation choices. To address this problem, we present a unified and reproducible evaluation framework for neutral-atom compilers. Our framework introduces RSQASM (Routed and Scheduled QASM), a QASM-inspired post-compilation representation that captures mapped, routed, and scheduled circuits, including explicit parallel gate execution and shuttling operations. As part of the framework, we provide adapter scripts that translate existing compiler outputs and intermediate artifacts into RSQASM. As a case study, we compare three well-known neutral-atom compilation toolchains: HybridMapper, DasAtom, and Enola, motivated by the large performance differences reported in prior work. Using our framework and representation, we perform a new evaluation and show that several previously claimed performance gaps become substantially smaller and, in some cases, are not reproduced once evaluation inconsistencies are removed. I Introduction Neutral-atom quantum computing has emerged as one of the most promising directions toward scalable quantum computation. Its support for large atom arrays, qubit shuttling, and parallel gate execution makes it a natural target for specialized compilation toolchains [ 17 , 2 , 4 , 1 , 5 ] . Consequently, a growing number of neutral-atom compilers have been proposed to map, route, and schedule quantum circuits while exploiting these hardware-specific capabilities [ 12 , 6 , 14 , 16 ] . A persistent challenge in comparing these solutions is that existing compiler studies typically rely on toolchain-specific fidelity models and evaluation assumptions, which makes direct benchmark comparisons difficult. This broader problem has already been discussed in the quantum computing literature [ 9 ] . Our focus, however, is not cross-hardware benchmarking but evaluation consistency across neutral-atom compilers. In our previous hardware-agnostic benchmarking study [ 8 ] , we observed that existing neutral-atom compilers are often evaluated over different parts of the toolchain and under non-equivalent assumptions, which makes direct performance comparisons unreliable. The study identified several sources of distortion, including inconsistent fidelity accounting, different coherence and shuttling-time models, partially mismatched hardware parameterizations, and minor implementation bugs that materially affect the reported metrics. After aligning these assumptions and correcting the discovered issues, the more than 400x fidelity advantage between DasAtom and Enola on QFT30 previously reported by [ 6 ] was reduced to about 5.5x. These findings exposed a broader unresolved problem: neutral-atom compilation still lacks a common post-compilation representation and a controlled, reproducible evaluation methodology. However, our previous study was primarily diagnostic: it showed that obtaining even a roughly fair comparison required substantial manual alignment of architectures, metrics, and tool outputs, together with non-trivial pre-processing, mid-processing, and post-processing steps. Even under this effort, small inaccuracies could still remain and materially affect the conclusions. This suggests that fair evaluation of neutral-atom compilers cannot rely on case-by-case manual alignment and ad hoc corrections alone. Instead, it requires a reusable post-compilation interface and a unified evaluation pipeline that make assumptions explicit, normalize compiler outputs systematically, and support controlled comparisons across toolchains. To address this problem, we develop a unified evaluation framework that turns such ad hoc comparisons into a systematic and reusable process. As illustrated in Fig. 1 , the proposed workflow decouples compiler-specific outputs from the evaluation stage by translating them into a common post-compilation representation. The evaluation stage applies a consistent set of metric definitions and hardware assumptions to compute fidelity-related quantities, including decoherences, movements, and gates fidelities, as well as the resulting approximate success probability. This design enables controlled, reproducible, and fair comparisons across different neutral-atom compilers. Furthermore, the framework provides a practical interface for integrating future toolchains into the same evaluation pipeline. The main contributions of this work are summarized as follows: • We introduce a unified representation for mapped, routed, and scheduled neutral-atom quantum circuits, derived from the QASM syntax used in Qiskit • We propose a benchmarking methodology based on relevant and observable sources of fidelity and success probability, inspired by existing distinct methodologies of considered prior compilers, as illustrative examples. In addition, we define a compact and easy-to-parse format for target hardware descriptions • We demonstrate the applicability of the framework by extracting intermediate representations and outputs from selected compilers, translating them into the proposed format, and evaluating them under a unified methodology. The evaluation pipeline automates the core stages, enabling reproducible experiments and allowing users to evaluate arbitrary circuits across different compilers in a consistent manner. Furthermore, we identify and address implementation inconsistencies in existing compilers that could lead to inconsistent evaluation results, and incorporate these corrections into the evaluation workflow. We expect the proposed framework to reduce inconsistencies in existing evaluation pipelines and to enable fairer comparisons across neutral-atom compiler toolchains. Moreover, the proposed format, RSQASM, captures the key post-compilation aspects of neutral-atom circuits and can serve as a common interface for future toolchains. Figure 1 : Overview of the proposed benchmarking framework. Different compilers produce compiled circuits and target hardware descriptions in the proposed format, abstracting away their internal representations. The unified evaluation module then processes this representation to compute fidelity-related metrics, as well as approximate success probability, enabling consistent and fair comparison II Preliminaries Neutral-atom quantum computing architectures can address qubit connectivity constraints not only through logical SWAP operations, but also through the physical movement of atoms, commonly referred to as shuttling. Furthermore, both gate execution and shuttling operations can be performed in parallel. We assume familiarity with the common physical and architectural constraints of neutral-atom quantum computing platforms [ 2 , 17 ] . To leverage these hardware capabilities, existing compilers adopt different mapping, routing, and scheduling strategies. In particular, routing strategies may rely on logical SWAP operations, physical qubit movements (shuttlings), or a combination of both (hybrid). These approaches are commonly referred to as SWAP-based, move-based, and hybrid routing, respectively. However, most of these compilers also rely on compiler-specific architectural abstractions and evaluation models, leading to substantial inconsistencies in reported benchmarking results. Consequently, fair comparison of compilation approaches remains difficult. In addition, tool-specific evaluation pipelines may obscure implementation bugs, as such issues can remain hidden under compiler-dependent assumptions and therefore compromise accurate cross-compiler assessment. III Method This section describes the proposed evaluation pipeline, the common representations used by the framework, the evaluator itself