Small models, big threats: Characterizing safety challenges from low-compute AI models
Prateek Puri · 2026
Read Small models, big threats: Characterizing safety challenges from low-compute AI models by Prateek Puri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21365.txt.
Introduction
Prateek Puri warns that small AI models pose safety risks that current policy largely ignores. AI brings clear gains to fields like medicine and materials work. Most governance so far, though, judges risk by the computing power behind a model.
Small models need little compute, so they slip under those thresholds. Yet they can still be capable, cheap to copy, and easy to misuse. The paper maps the challenges these models create for oversight. It argues that rules based on size alone will miss a growing class of threats. Policy, the author urges, should track what models can do, not just their cost.
Catalog record
| CREATOR | Prateek Puri |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21365
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Small models, big threats: Characterizing safety challenges from low-compute AI models Prateek Puri Abstract Artificial intelligence (AI) systems are revolutionizing fields such as medicine, drug discovery, and materials science; however, many technologists and policymakers are also concerned about the technology’s risks. To date, most concrete policies around AI governance have focused on managing AI risk by considering the amount of compute required to operate or build a given AI system. However, low-compute AI systems are becoming increasingly more performant - and more dangerous. Driven by agentic workflows, parameter quantization, and other model compression techniques, capabilities once only achievable on frontier-level systems have diffused into low-resource models deployable on consumer devices. In this report, we profile this trend by downloading historical benchmark performance data for over 5,000 large language models (LLMs) hosted on HuggingFace, noting the model size needed to achieve competitive LLM benchmarks has decreased by more than 10X over the past year. We then simulate the computational resources needed for an actor to launch a series of digital societal harm campaigns - such as disinformation botnets, sexual extortion schemes, voice-cloning fraud, and others - using low-compute open-source models and find nearly all studied campaigns can easily be executed on consumer-grade hardware. This position paper argues that protection measures for high-compute models leave serious security holes for their low-compute counterparts, meaning it is urgent both policymakers and technologists make greater efforts to understand and address this emerging class of threats. 1 Introduction Artificial intelligence (AI) technologies are enabling the widespread automation of information processing and reasoning tasks. Many anticipate these technologies will herald an era of unprecedented human productivity and economic output, while others are concerned they may be weaponized to cause large-scale societal harm. For example, researchers have investigated the extent to which advanced models, specifically large language models (LLMs), may facilitate synthetic biology attacks, compromise cybersecurity systems, and amplify the effects of disinformation campaigns ( 25 ; 26 ) . Numerous policy frameworks have been proposed to mitigate AI risks, many of which focus on monitoring or regulating access to compute ( 38 ) . For example, current US semiconductor export controls are designed, at least partially, to prevent the misuse of advanced models requiring high performance graphics processing units (GPUs), with US officials citing threats from ”both AI training and inference at scale” ( 14 ) . Similarly, The European Union (EU) AI Act designates 10 25 training floating point operations (FLOPs) as a threshold for systemic risk categorization and regulation ( 18 ) . However, training or deploying large models is not the only pathway to dangerous capabilities. Advancements in test-time compute, parameter quantization, agentic workflows, LLM tooling, and other techniques are rapidly diffusing capabilities from large AI systems into compact models that are easily deployable on consumer devices ( 43 ; 30 ; 31 ; 40 ) . Position : The swift compression of advanced AI capabilities into smaller, accessible, and easily deployable models poses significant security risks that current governance frameworks are not fully equipped to handle. We urgently encourage more researchers and policymakers to focus on developing innovative governance strategies specifically tailored to low-compute AI threats, as these threats are becoming increasingly severe, frequent, and difficult to detect. In this report, we quantitatively profile the rate at which open-source LLMs have become both more performant and more compute-efficient over time. Secondly, we outline how this shift has impacted the amount of compute resources needed for a single actor to execute a variety of societal harm campaigns. Next we profile the computational workloads of several academic and commercial AI use cases, demonstrating a high degree of overlap between the compute required by both. We discuss how this overlap, combined with the relatively modest amount of compute required to launch the studied societal harm campaigns, complicates existing AI risk mitigation frameworks centered around high-compute models. Lastly, we briefly highlight the promises and shortcomings of a set of proposed strategies for mitigating low-compute AI risks. As a clarification on terminology, we will refer to low-compute AI models in this report as those with ¡30B parameters, as these systems are increasingly deployable on low-cost hardware through parameter quantization and other inference optimization techniques. We will also use the term model compression to succinctly refer to the diffusion of AI capabilities into low-compute systems, although we acknowledge this phrase may have different meanings elsewhere in the literature. 2 Models are becoming more advanced at smaller sizes Market pressures are guiding models to become both more capable and more lightweight over time, while hardware improvements are reducing barriers to model deployment. We discuss both trends below while observing that if these trends continue to evolve, bad-faith and good-faith actors alike will be able to deploy increasingly sophisticated models with commonly accessible levels of compute. 2.1 Model miniaturization We download performance data from over 5,000 open-source LLMs hosted on the HuggingFace LLM leaderboard. Each model on the leaderboard has been evaluated against the Eleuther AI Language Model Evaluation Harness ( 17 ) , a suite of benchmark tests designed to probe language model abilities on diverse tasks such as common sense reasoning, mathematical abilities, and others. In this report, we define a LLM’s aggregate model performance, \alpha , as the mean score on the IFEval, BBH, MATH, GPQA, MUSR, and MMLU-PRO benchmark tests included within the Harness. 1 1 1 \alpha is bounded between [0%,100%], with 100% denoting a perfect score . For each graded model on the leaderboard, we extract the model size (FP16 precision), model performance ( \alpha ), and the date on which the model was created (Appendix A). In Figure 1 A, we plot the model size needed to obtain a given \alpha over time. Each scatter point represents the 25 th percentile model size value within the subset of models that surpass a given \alpha at a given date. We present five different curves corresponding to \alpha values of 30%, 35%, 40%, 45%, and 50%. For reference, the highest \alpha value listed on the HuggingFace leaderboard as of the writing of this report is 52%. We also fit a simple exponential decay curve to each set of data, which we present in the figure along with fit uncertainty bands for visual reference. As can be seen in Figure 1 A, across all performance levels, the model size needed to obtain a given benchmark score has dropped significantly over time, with the model size needed to obtain \alpha=0.35 falling by \sim 10X over the past year. 2 2 2 For robustness, we also recalculated these compression curves using each individual metric instead of the aggregate \alpha value and observed similar compression trends over time. Figure 1 : (a) Model size needed to obtain a given LLM benchmark score over time. Exponential curves are fit to the raw data and displayed along with the fit uncertainty bands. (b) Benchmark performance ( \alpha ) over time for three classes of Llama family models. Similarly, Figure 1 B displays how the \alpha of a model of a fixed size has increased over time. Here, we filter our LLM leaderboard dataset to focus on Meta’s Llama family of models, given their active use within the developer community. As can be