Stencil Computations on Cerebras Wafer-Scale Engine

Elia Belli and Daniele De Sensi · 2026

Read Stencil Computations on Cerebras Wafer-Scale Engine by Elia Belli and Daniele De Sensi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07954.txt.

Introduction

Elia Belli and Daniele De Sensi port a classic scientific kernel, stencil computation, to the Cerebras Wafer-Scale Engine, an AI chip with massive on-chip memory and parallelism. Stencils drive fluid dynamics and climate codes but are memory-bound on GPUs, starved by the trip to off-chip memory. The wafer-scale engine's distributed on-chip memory promises to remove that wall.

Their CStencil framework implements two-dimensional stencils on the WSE-3, benchmarked fairly against a state-of-the-art GPU solver ported to the same precision on an A100. The wafer engine's mesh and on-chip storage largely eliminated the memory bottleneck, delivering speedups up to several hundredfold. A roofline analysis confirmed the implementation saturates the machine's resources.

Catalog record

CREATORElia Belli and Daniele De Sensi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07954

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Stencil Computations on Cerebras Wafer-Scale Engine Elia Belli and Daniele De Sensi Abstract Stencil computations are a fundamental kernel in scientific computing, critical for simulations in domains such as fluid dynamics and climate modeling. However, these computations are often memory-bound on traditional High-Performance Computing architectures like GPUs, struggling against the ”Memory Wall”. Simultaneously, the rise of AI-oriented hardware, such as the Cerebras Wafer-Scale Engine, offers massive core parallelism and high-bandwidth on-chip memory, though typically optimized for lower-precision workloads. This work investigates the viability of bridging this divergence by mapping stencil algorithms onto the Cerebras WSE-3. The study introduces CStencil, a novel framework designed to implement two-dimensional stencil computations on the WSE-3. To ensure a rigorous and fair performance evaluation, the research also adapts ConvStencil, a state-of-the-art GPU stencil solver, porting it from its original double-precision design to single-precision for execution on an NVIDIA A100 GPU. Experimental results show that the WSE-3’s distributed SRAM and mesh interconnect effectively eliminate the off-chip memory bottlenecks common in GPU implementations. CStencil achieves speedups of up to 342× over the adapted ConvStencil version. A roofline model analysis further confirms that CStencil saturates the available compute and memory resources, demonstrating that the WSE dataflow architecture can be successfully repurposed for traditional scientific algorithms. These findings highlight the potential of the WSE-3 to deliver hardware utilization levels unattainable on conventional systems, offering a promising path toward overcoming the memory limitations of current HPC architectures. I Introduction Graphics Processing Units (GPUs) have been the dominant architecture in High-Performance Computing (HPC) for the last decade, particularly for General-Purpose GPU (GPGPU) workloads. Concurrently, the rapid emergence of Artificial Intelligence (AI) has driven an exponential growth in computational demand, doubling every 3-4 months, a pace that far exceeds Moore’s Law. This growing demand has necessitated a shift away from solely scaling general-purpose hardware toward domain-specific accelerators (DSAs). The primary operations dominating AI, such as matrix multiplications (GEMM, SPGEMM), are now being accelerated by integrating specialized units, like NVIDIA’s Tensor Cores Units (TCUs) and AMD’s Matrix Cores, into established GPU architectures. A more radical strategy involves designing new hardware from the ground up, exemplified by architectures like Tenstorrent [ 24 ] , Cerebras [ 4 ] , SambaNova [ 20 ] , TPUs [ 12 ] and GraphCore [ 13 ] . For example, the Cerebras Wafer-Scale Engine (WSE), built on wafer-scale integration, features a massive array of specialized cores and a unique distributed memory model with unparalleled bandwidth, specifically designed for Machine Learning (ML) workloads. This trend toward highly specialized hardware has created a potential divergence: while AI achieves record performance, other computationally intensive fields, such as traditional scientific computing, have not been the primary target of these new architectures. However, these paths can be re-converged. Many core scientific algorithms, such as stencil computations fundamental to physics simulations, are operationally analogous to the kernels for which these AI accelerators are optimized. The central premise of this paper is that the performance benefits of mapping algorithms like stencils to AI-specific hardware can unlock new levels of efficiency by exploiting the accelerator’s high-bandwidth memory architecture. This strategy directly addresses the ”Memory Wall” that constrains performance on conventional systems. This work investigates this hypothesis by providing a detailed mapping of the 2D Stencil algorithm onto the Cerebras WSE-3 architecture, presenting CStencil as a novel implementation. We conduct a rigorous performance analysis against a state-of-the-art GPU implementation, ConvStencil [ 7 ] , including a comprehensive roofline analysis to isolate the performance-limiting bottlenecks across both architectures. Our results demonstrate that the memory architecture of the WSE-3 provides a significant advantage for memory-bound scientific workloads, pointing towards a re-convergence of specialized AI hardware and traditional HPC workloads. II Background and State-of-the-art II-A Stencil in Scientific Simulations Stencil computation is one of the seven critical numerical methods for scientific computing [ 2 ] , foundational to solving problems in heat diffusion [ 3 ] , wave propagation [ 1 ] , and fluid dynamics [ 23 ] . These phenomena are modeled by Partial Differential Equations (PDEs), which often lack analytical solutions (e.g., Navier-Stokes equations [ 8 ] ) and are impractical or too expensive to study purely through experimentation (e.g., wind tunnels). Scientific simulations, such as Computational Fluid Dynamics (CFD), address this by numerically solving PDEs through discretization. This process approximates the continuous domain with a discrete grid, transforming the PDE into an iterative system of equations solved on that grid. This iterative update is a stencil computation. A critical consideration in these simulations is numerical precision. Lower-precision formats (e.g., FP16 and BF16) common in AI are faster but can introduce accumulating rounding errors that prevent convergence or lead to numerical instability. Consequently, many sensitive scientific simulations require at least single-precision (FP32) for stability, and some non-negotiably require double-precision (FP64). Since the WSE supports up to single-precision (FP32), it can be a viable architecture for many of these stability-sensitive applications. II-B Stencil Computation A stencil computation iteratively updates the elements of a domain by applying a predefined pattern, or stencil kernel. At each iteration, a point is computed as a weighted function of its neighboring points. Stencils are characterized by three aspects: • Dimensionality : The spatial dimension of the domain (1D, 2D, or 3D). • Shape : Broadly categorized as Star Stencil (elements along axes) or Box Stencil (elements in a square/cube around a central point). • Radius : The spatial range of considered elements. The variety in stencil kernels, driven by the physics of the application, often necessitates specialized algorithms and automated code generators, such as DRStencil [ 25 ] , StencilPy [ 22 ] , GT4Py [ 19 ] and Devito [ 17 ] . We focus here on manually written 2-dimensional stencils, which are the most commonly studied and the dimensionality targeted by our implementation, CStencil. Fig. 1 : Examples of stencil kernels: Star2d-2r on the left and Box2d-2r on the right II-C Jacobi Method The Jacobi Method is the most common iterative algorithm for solving stencil systems. The update rule at each point uses values exclusively from the previous iteration, making the algorithm highly parallelizable. For an m\times n grid, the update rule for a generic 2D stencil at interior point u_{i,j} is: \displaystyle u_{i,j}^{(k+1)}=\sum_{n\in\mathcal{N}}w_{n}\cdot u_{n_{x},n_{y}}^{(k)} where k,k+1 are the previous and current iterations, \mathcal{N} represents the set of neighbors within the stencil footprint and w_{n} is the specific weight associated with the neighbor at relative coordinates (n_{x},n_{y}) . In parallel implementations, the domain is decomposed into tiles distributed across cores. Adjacent tiles must exchange boundary values, a communication pattern known as halo swap , though in practice this overhead is often limited, as only the boundary layer of each tile needs to be communicated. The deeper challenge is computational: stencil kernels typically have a low byte-to-flop ratio, making them memory-bound on classic multicore architectures. The Cerebras WSE’s massive on-chip fabric addresses both concerns, providing high-bandwidth memory access alongside efficient on-chip communication. II-D Related Work: GPU Stencil Implementations A current trend in HPC is the use of Tensor Core Units (TCUs) on GPUs to accelerate non-GEMM operations, including stencils. The stencil can be mapped onto a GEMM by unrolling the kernel into a vector, analogous to the im2col transformation [ 6 ]

Read the complete text