MLIR-Smith: A Novel Random Program Generator for Evaluating Compiler Pipelines

Berke Ates, Filip Dobrosavljević, Theodoros Theodoridis, Zhendong Su · 2026

Read MLIR-Smith: A Novel Random Program Generator for Evaluating Compiler Pipelines by Berke Ates, Filip Dobrosavljević, Theodoros Theodoridis, Zhendong Su (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02218.txt.

Introduction

Compiler bugs hide in optimization pipelines. The MLIR infrastructure is flexible, and that flexibility defeats older random program generators built for fixed languages like Csmith. Berke Ates and three co-authors introduce MLIR-Smith. It is a random program generator designed for MLIR's dialect system.

Feeding generated programs into cross-tool testing uncovered multiple real bugs. The pipelines span MLIR, LLVM, DaCe, and DCIR. That is the practical proof of a testing tool. The generator's design respects MLIR's flexibility. It does not hard-code one dialect set. Beyond the bugs found, the paper argues broad randomized testing belongs in compiler engineering generally. MLIR-Smith is offered as a template for future tools in other flexible compiler worlds.

Catalog record

CREATORBerke Ates, Filip Dobrosavljević, Theodoros Theodoridis, Zhendong Su
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-02218

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MLIR-Smith: A Novel Random Program Generator for Evaluating Compiler Pipelines Berke Ates, Filip Dobrosavljević, Theodoros Theodoridis, Zhendong Su Abstract . Compilers are essential for the performance and correct execution of software and hold universal relevance across various scientific disciplines. Despite this, there is a notable lack of tools for testing and evaluating them, especially within the adaptable Multi-Level Intermediate Representation (MLIR) context. This paper addresses the need for a tool that can accommodate MLIR’s extensibility, a feature not provided by previous methods such as Csmith. Here we introduce MLIR-Smith, a novel random program generator specifically designed to test and evaluate MLIR-based compiler optimizations. We demonstrate the utility of MLIR-Smith by conducting differential testing on MLIR, LLVM, DaCe, and DCIR, which led to the discovery of multiple bugs in these compiler pipelines. The introduction of MLIR-Smith not only fills a void in the realm of compiler testing but also emphasizes the importance of comprehensive testing within these systems. By providing a tool that can generate random MLIR programs, this paper enhances our ability to evaluate and improve compilers and paves the way for future tools, potentially shaping the wider landscape of software testing and quality assurance. 1. Introduction The performance of programs is significantly enhanced by compiler optimizations, which employ a variety of techniques such as dead code elimination, loop unrolling, and common sub-expression elimination ( 16 ) . The effectiveness of these optimizations and the correctness of compilers can be evaluated through fuzz and differential testing using randomly generated programs. This approach was demonstrated by Yang et al. ( 24 ) using Csmith, a tool specifically designed for generating random C programs. The Multi-Level Intermediate Representation (MLIR) ( 11 ) project was initiated with the objective of mitigating the intricacies involved in compiler construction. MLIR serves as a unified infrastructure designed to alleviate software fragmentation, enhance compilation performance on heterogeneous hardware, and drastically cut down the expenditure involved in the development of domain-specific compilers. Moreover, it provides assistance in establishing connections between pre-existing compilers. Polygeist ( 15 ) possesses the ability to produce MLIR code from C code. However, given its novelty, its robustness is not yet fully reliable for randomly generated C code. Additionally, MLIR is capable representing operations that surpass the instruction set of the C programming language. This presents a challenge as there is currently no dedicated tool for generating random MLIR programs, which is essential for comprehensive testing and evaluation of MLIR-based compilers and optimizations. Figure 1 . Overview of MLIR-Smith. To overcome this challenge, we introduce MLIR-Smith, a novel random program generator for MLIR. MLIR-Smith facilitates the generation of random MLIR programs, enabling a thorough evaluation of the optimization capabilities of MLIR-based compilers and ensuring the correctness and effectiveness of the compiled code. We demonstrate the utility of MLIR-Smith by implementing a test harness that uses dead code elimination (DCE) techniques to evaluate the optimization capabilities of MLIR ( 11 ) , LLVM ( 10 ) , DaCe ( 3 ) , and DCIR ( 2 ) . DCE is a well-established method for analyzing the optimization efficiency of compilers and identifying missed opportunities for code optimization ( 23 ) . In addition, we use MLIR-Smith to uncover bugs in MLIR, LLVM, DaCe, and DCIR through differential testing. The paper makes the following contributions: • We design and implement MLIR-Smith, a random program generator that utilizes a minimal interface with operations, specifically for MLIR. • We address the current limitations in MLIR testing infrastructure that inhibit effective compiler testing. • We demonstrate the effectiveness of MLIR-Smith through a series of evaluations, showcasing its potential to enhance the field of compiler research and testing. 2. Background We begin by discussing the foundational approaches that our work is built upon, specifically the MLIR ( 11 ) and DaCe ( 3 ) frameworks, the DCIR pipeline ( 2 ) , and Csmith ( 24 ) . 2.1. MLIR LLVM ( 10 ) and JVM ( 12 ) are examples of traditional compiler systems that rely on a single level of abstraction. These frameworks translate programs from one language into an intermediate representation (IR), where all necessary transformations, improvements, and verifications are made. The goal of MLIR ( 11 ) is to make the creation and interoperability of many IRs simpler by providing a framework that enables numerous levels of abstraction to coexist. An intermediate representation in MLIR is called a dialect . Specialized operations and data types are available in dialects, along with conversion points for lowering from higher-level IRs and converting between IRs of equal level. Although MLIR comes with a number of standard dialects, users are free to create their own unique dialects. The majority of dialects provide lowering passes one level down, reducing redundant code in compiler infrastructure development. The expanding use of MLIR by front end and back end projects highlights the significance of this technology for compilers. For instance, the front end project Flang ( 14 ) converts FORTRAN code into MLIR dialects. On the back end, a recent performance assessment by Katel et al. ( 9 ) compared MLIR to CUBLAS ( 17 ) to demonstrate that MLIR can produce GPU code that is competitive with well-known libraries for matrix-matrix multiplications. Due to MLIR’s rising popularity, there is a high demand for reliable testing tools, which are crucial for ensuring the correct implementation of dialects. 2.2. DaCe A multitude of optimization techniques are designed to improve data movement, yet many are specifically tailored to distinct hardware systems or individual hardware configurations. Concurrently, the requirement for high-level programs to account for data movement often leads to an increase in code complexity. By providing a Stateful DataFlow multiGraph (SDFG) ( 3 ) representation, the DaCe programming framework provides a solution to this issue. The main goal of this IR is to comprehend and enhance data flow. DaCe offers front ends that convert Python, Octave, or C programming code into the SDFG IR. Additionally, it provides a transformation API on the IR that efficiently divides the responsibilities of the performance engineer and developer. Applications in a variety of fields have benefited from the use of DaCe, including weather and climate models ( 6 ; 4 ) , sparse linear algebra in quantum transport simulation ( 25 ) , graph analytics ( 3 ) , and comprehensive neural network optimization in deep learning ( 21 ) . The DaCe framework’s optimization method starts with an SDFG simplification pass that widens pure data-flow areas and eliminates superfluous memory allocation, much like the compiler’s -O1 flag. After that, automatic heuristic optimizations ( -O2 ) and sometimes manual transformation application by performance engineers are used. The SDFG IR is a control-flow graph, represented by a state machine, consisting of data-flow acyclic multigraphs. SDFGs distinguish between the usage of data in computational nodes (referred to as tasklets ) and the use of data containers and data movement (expressed as data-flow edges). As a result, the IR may describe data mobility directly, and it can even use data dependencies to decide the execution order. Explicit control-flow is only employed in cases, where inferring data-flow is not possible. However, passes that are often implemented by control-centric IRs, such as loop-invariant code motion

Read the complete text