Numerical Kernels on a Spatial Accelerator: A Study of Tenstorrent Wormhole
Maya Taylor, Carl Pearson, Luc Berger-Vergiat, Giovanni Long, Jan Ciesko · 2026
Read Numerical Kernels on a Spatial Accelerator: A Study of Tenstorrent Wormhole by Maya Taylor, Carl Pearson, Luc Berger-Vergiat, Giovanni Long, Jan Ciesko (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23343.txt.
Introduction
Taylor and four co-authors test whether an AI chip can run classic science computing kernels. Tenstorrent's Wormhole is a spatial platform built to speed neural networks. The team ports three numerical kernels to it and composes them into a conjugate gradient solver. They tune the code for the chip's layout and compare it with Nvidia GPUs. The work exposes both porting pains and real openings. They conclude that AI accelerators deserve a serious look for CPU and GPU workloads. More effort should go into making them usable for scientists.
Catalog record
| CREATOR | Maya Taylor, Carl Pearson, Luc Berger-Vergiat, Giovanni Long, Jan Ciesko |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53417 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23343
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Numerical Kernels on a Spatial Accelerator: A Study of Tenstorrent Wormhole Thanks: This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Department of Energy Computational Science Graduate Fellowship under Award Number DE-SC0025528. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND#0000-XXXXX This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. We would like to thank the Sandia advanced architectures testbed program for providing computing resources for this publication. Additionally, would like to thank Felix LeClair, Nikola Cvetković, and Neil Sexton of Tenstorrent for their frequent technical discussions. CCS: Computing methodologies Parallel algorithms CCS: Computer systems organization Data flow architectures CCS: Mathematics of computing Solvers Maya Taylor, Carl Pearson, Luc Berger-Vergiat, Giovanni Long, Jan Ciesko Abstract . As AI accelerators gain prominence, their potential for traditional scientific computing workloads remains unclear. This paper explores Tenstorrent’s Wormhole architecture, a spatial computing platform designed for neural network acceleration, by implementing three numerical kernels and composing them into a conjugate gradient solver. We present architecture-specific optimizations for sparse numerical algorithms, evaluate their performance against Nvidia GPUs, and expose both challenges and opportunities in porting numerical methods to spatial architectures. Our results demonstrate that AI accelerators merit consideration for workloads traditionally dominated by CPUs and GPUs, and more work should be invested in understanding the capabilities of these architectures and making them accessible to the scientific computing community. 1. Introduction The advent of compute-intensive neural networks has driven proliferation of novel computer architectures designed to accelerate training and inference, including products developed by Cerebras, Graph Core, Groq, SambaNova, and Tenstorrent, among others. These architectures broadly feature small, simple processing elements (PEs) with single-instruction multiple-data (SIMD) parallelism for high-throughput math, small scratchpad memories, and a user-controlled network-on-chip (NoC). Differentiating factors include physical assembly techniques (e.g. Cerebras “Wafer-Scale Engine”), availability of global DRAM or high bandwidth memory (HBM) accessible to PEs, vectorization, and openness of toolchain. These have been referred to as “spatial” or “dataflow” architectures, with both terms emphasizing the concern of data placement and movement, demanding more explicit control over data and parallelism than CPUs and GPUs. Despite the resulting higher programming complexity of spatial architectures, it has proven tractable to manage data movement and parallelization (whether manually or automatically) for high performance execution, particularly in the context of regular dense-tensor operations present in neural networks. More recently, these architectures have also been used to demonstrate orders-of-magnitude acceleration for more traditional modeling and simulation (modsim) workloads with the combined, sustained efforts of a team of experts ranging from domain scientists to micro-architectural implementers ( 7 ; 10 ) . Despite the targeted successes of using these architectures, fundamental questions remain: • What kinds of numerical algorithms can be effectively implemented on these architectures? • Are past successes on a specific platform transferable to architectures with similar constraints? • Can these algorithms be expressed in a way familiar to traditional HPC (e.g., a CUDA-like massively-parallel offload model)? This work presents an initial effort to investigate these questions by implementing a selection of numerical kernels on Tenstorrent’s Wormhole architecture, in addition to a full Conjugate Gradient Solve implementation. We make four primary contributions: (1) Implementing and evaluating three key numerical kernels on Wormhole: basic vector arithmetic, global reduction, and stencil computations. (2) Demonstrating a modsim-relevant numerical algorithm (preconditioned conjugate gradient solve) at 32-bit precision on Wormhole for the first time. (3) Comparing performance achieved on Wormhole to Nvidia H100 via a GPU-style offload model. (4) Evaluating the additional performance achieved via a fused-kernel approach more suitable to Wormhole. Section 2 places this work in context of related work on spatial architectures. Section 3 describes the Wormhole architecture. The implementation and performance of basic vector arithmetic is described in Section 4 , global reduction in Section 5 , and stencil computations in Section 6 . Section 7 describes the implementation of the conjugate gradient solver using these building blocks. Section 7.2 provides an analysis of the performance results and Section 8 discusses future work. Finally, Sec. 9 concludes. 2. Background and Related Work The work presented in this paper builds on Brown and Barton ( 2 ) , which explores DDR access patterns and energy efficiency for a Jacobi iterative method on Tenstorrent’s earlier “Grayskull” architecture. Grayskull and Wormhole share many architectural features, including a grid of simple processing elements with small local SRAM and a similar programming model. Wormhole extends this with support for additional datatypes (including FP32), higher memory capacity, and optimizations for multi-device scaling ( 4 ) . Our work advances this earlier implementation with detailed analysis of key building blocks for numerical algorithms on Wormhole. We implement a more demanding iterative method with a 3D (rather than 2D) stencil pattern and stage intermediate values in on-chip SRAM rather than global DRAM for more effective use of the architecture. While Brown and Barton’s approach achieves performance similar to a single CPU core, our approach more closely matches datacenter GPU performance. Beyond Tenstorrent work, most prior work on modeling and simulation (modsim) algorithms for dataflow architectures targets Cerebras’ Wafer-Scale Engines (WSE). These works demonstrate that targeted implementations of specific algorithms can achieve excellent performance by exploiting hardware features, though this highlights the programmability challenges inherent to these platforms. Santos et al. ( 10 ) demonstrate superior strong scaling for molecular dynamics on the WSE. Sai et al. ( 8 ) present a matrix-free conjugate gradient solver for finite volume discretizations on Cerebras WSE CS-2, analogous to our work but facing different challenges on that platform. Tramm et al. ( 14 ) demonstrate Monte Carlo particle transport on Cerebras WSE at 32-bit precision with hardware-specific optimizations including hardware PRNGs and reduced-precision division. Related accelerator work includes Feldman et al. ( 5 ) , who propose a hardware accelerator for sparse iterative solvers with architectural features similar to spatial architectures. Other work explores algorithm primitives rather than full solvers. De Fine Licht et al. ( 3 ) address mapping large stencil programs to distributed spatial computing systems. Sai et al. ( 9 ) implement