AMBER: A Columnar Architecture for High-Performance Agent-Based Modeling in Python

Anh-Duy Pham · 2026

Read AMBER: A Columnar Architecture for High-Performance Agent-Based Modeling in Python by Anh-Duy Pham (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16292.txt.

Introduction

Anh-Duy Pham builds a fast agent model frame in Python with a column store. His 2026 preprint, named AMBER, deals with agent based models. Such models study group acts in nature and in social life. Python frames are easy to use but oft run slow at scale. Scale is just what group act studies need most.

AMBER stores agent traits in columns, not in lone rows. Column stores let code run the same step on all agents fast. Speed with ease of use would open big models to more teams. The frame aims to keep the Python feel users like. It seeks scale with no loss of that ease.

Catalog record

CREATORAnh-Duy Pham
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16292

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AMBER: A Columnar Architecture for High-Performance Agent-Based Modeling in Python Anh-Duy Pham Abstract Agent-based modeling (ABM) has emerged as an indispensable methodology for studying complex adaptive systems across the natural and social sciences. However, Python-based ABM frameworks face a fundamental tension between the accessibility that has made Python dominant in scientific computing and the performance requirements of large-scale simulations. This paper introduces AMBER, a framework that resolves this tension through a novel architectural approach: replacing the conventional object-per-agent representation with columnar state management using the Polars DataFrame library. We analyze the computational characteristics of both paradigms, present the architectural design of AMBER including its core abstractions, spatial environments, experiment management, and optimization capabilities. Empirical evaluation on three canonical benchmarks demonstrates that AMBER achieves speedups of 1.2 × \times to 93 × \times depending on workload characteristics, with the greatest advantages for models dominated by population-wide attribute operations. Memory profiling reveals 30–50% reduction in peak usage compared to object-oriented frameworks. Our results establish columnar state management as a viable architectural foundation for high-performance ABM in interpreted languages. 1 Introduction Agent-based modeling (ABM) provides a computational methodology for studying complex systems by simulating autonomous agents whose local interactions generate emergent macro-level phenomena ( 1 ) . The methodology has become essential across disciplines: epidemiologists model disease transmission through contact networks, ecologists simulate predator-prey dynamics and species distributions, economists study market formation and financial contagion, and social scientists investigate opinion dynamics and collective behavior. The power of ABM lies in its capacity to capture agent heterogeneity, bounded rationality, spatial structure, and adaptive behavior—phenomena that aggregate equation-based models cannot adequately represent. Python has emerged as the dominant platform for ABM in the research community. This dominance stems from Python’s gentle learning curve, which enables domain scientists to build models without extensive programming training; its rich ecosystem of libraries supporting the full research workflow from simulation to analysis; and its interpreted nature, which facilitates rapid prototyping and iterative model development. The frameworks Mesa ( 7 ) and AgentPy ( 4 ) have capitalized on these advantages to enable widespread ABM adoption. However, Python-based ABM frameworks share an architectural assumption that fundamentally limits their scalability. In these frameworks, each agent is represented as an independent Python object with state stored in instance attributes. While this representation is cognitively natural, it incurs computational overhead that grows linearly with population size. For simulations involving thousands to tens of thousands of agents—scales common in epidemiological, ecological, and economic modeling—this overhead becomes a significant barrier to research productivity, limiting parameter space exploration, reducing Monte Carlo sample sizes, and constraining model complexity. This paper presents AMBER (Agent-based Modeling with Blazingly Efficient Records), a framework that addresses this scalability challenge through a fundamental architectural innovation. Rather than distributing agent state across Python objects, AMBER centralizes state in a columnar DataFrame backed by Polars, a high-performance data processing library implemented in Rust ( 10 ) . This design enables vectorized operations that execute in compiled code while presenting programmers with familiar object-oriented abstractions. 1.1 Research Questions This work investigates three primary research questions. First, we examine architectural viability : can columnar state management serve as an effective foundation for ABM while preserving the expressiveness that makes Python ABM frameworks accessible? Second, we analyze performance characteristics : what speedups does the columnar approach achieve, and how do these vary across different workload characteristics? Third, we assess framework completeness : can a columnar architecture support the full range of ABM requirements including spatial environments, experiment management, and model calibration? 1.2 Contributions This work makes four principal contributions. First, we introduce columnar state management as an architectural alternative to object-oriented agent representation, analyzing the computational tradeoffs and identifying the conditions under which each approach excels. Second, we present AMBER, a comprehensive ABM framework implementing the columnar architecture with spatial environments (grid, continuous space, network), experiment management with parameter sweeps, and multiple optimization strategies including grid search, random search, Bayesian optimization, and SMAC integration. Third, we provide rigorous empirical evaluation on three canonical benchmarks, demonstrating speedups of 1.2 \times to 93 \times with detailed analysis of performance characteristics across agent population sizes and workload types. Fourth, we characterize the design space of ABM state management, providing guidance for researchers selecting frameworks and for developers designing next-generation ABM platforms. 2 Related Work 2.1 Python ABM Frameworks The Python ABM ecosystem has evolved through several generations of frameworks, each reflecting different design priorities. Mesa ( 7 ) established the dominant architectural pattern: agents as Python class instances with explicit step() methods, organized by a scheduler that controls activation order. Mesa introduced important patterns including datacollectors for systematic state recording and space modules for spatial structure. Its visualization server provides interactive model exploration. Mesa’s design prioritizes accessibility and flexibility over performance, making it well-suited for teaching and prototyping. AgentPy ( 4 ) refines Mesa’s approach with improved ergonomics. Its AgentList abstraction enables batch operations over agent collections, partially addressing performance concerns. AgentPy provides tighter integration with Jupyter notebooks and built-in sensitivity analysis tools. However, its fundamental architecture remains object-per-agent, limiting scalability. Melodie ( 14 ) separates models into distinct components—agents, environment, scenario, and calibrator—promoting modularity. It accelerates specific operations through Cython compilation, achieving partial performance improvements. ABSESpy ( 13 ) extends Mesa’s architecture with domain-specific abstractions for social-ecological systems modeling. SimPy ( 15 ) takes a fundamentally different approach, providing discrete-event simulation focused on resource queuing rather than population-level agent dynamics. While Python-based, SimPy addresses different modeling requirements than traditional ABM. Auxiliary tools complement these frameworks: PyCX ( 11 ) provides minimal implementations for teaching complex systems concepts, and PyNetLogo ( 5 ) bridges Python with NetLogo’s established ABM environment. 2.2 High-Performance ABM Frameworks Researchers facing performance limitations in Python frameworks typically migrate to implementations in compiled languages, accepting reduced accessibility in exchange for improved scalability. Repast ( 8 ) provides a mature Java-based platform with sophisticated scheduling, GIS integration, and distributed computing support. MASON ( 6 ) offers a lighter-weight Java framework emphasizing simplicity and checkpointing capabilities. Both achieve significant performance improvements over Python frameworks but require Java expertise. Agents.jl ( 3 ) exploits Julia’s just-in-time compilation to combine scripted development with compiled execution. Julia’s multiple dispatch enables elegant composable model components, though Julia’s smaller ecosystem can limit integration options. FLAME GPU 2 ( 9 ) achieves extreme performance through GPU acceleration, mapping agent operations onto CUDA kernels. This approach requires restructuring models to fit GPU-amenable patterns and demands CUDA programming expertise. NetLogo ( 12 ) remains influential for its visual development environment and domain-specific language, though its performance limits applicability to large-scale simulations. 2.3 Columnar Data Processing Columnar storage—organizing data by columns rather than rows—has revolutionized analytical database systems and data processing frameworks ( 2 ) . The approach provides several computational advantages relevant to ABM. Modern CPUs achieve peak performance when accessing contiguous memory. Columnar storage places all values of a single attribute in sequential memory locations, enabling efficient cache utilization when operations access one attribute across many records. Single Instruction Multiple Data (SIMD) instructions process multiple values simultaneously, and columnar arrays enable direct application of SIMD operations, with modern processors supporting 256-bit or 512-bit vectors. Columns of similar values compress more effectively than rows of heterogeneous

Read the complete text