Learning with springs and sticks

Luis Mantilla Calderón, Alán Aspuru-Guzik · 2025

Read Learning with springs and sticks by Luis Mantilla Calderón, Alán Aspuru-Guzik (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19015.txt.

Introduction

A neural network is software, but a network of springs and sticks is a thing, and it can learn too. Luis Mantilla Calderón and Alán Aspuru-Guzik build such a machine in 2025. Sticks sketch the target curve in straight segments, while springs store the error as stretch. The whole frame then settles into its lowest energy shape. On regression tasks the contraption matched small neural networks.

Its physics carries a lesson: the drop in free energy as it learns relates to how well it can learn at all. When the environment's jiggle pushes that budget too far, a thermodynamic barrier stops learning cold. The authors offer the toy as a lens on learning itself. Understanding one physical learner, they suggest, illuminates the rest.

Catalog record

CREATORLuis Mantilla Calderón, Alán Aspuru-Guzik
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-19015

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning with springs and sticks Luis Mantilla Calderón, Alán Aspuru-Guzik Abstract Learning is a physical process. Here, we aim to study a simple dynamical system composed of springs and sticks capable of arbitrarily approximating any continuous function. The main idea of our work is to use the sticks to mimic a piecewise-linear approximation of the given function, use the potential energy of springs to encode a desired mean squared error loss function, and converge to a minimum-energy configuration via dissipation. We apply the proposed simulation system to regression tasks and show that its performance is comparable to that of multi-layer perceptrons. In addition, we study the thermodynamic properties of the system and find a relation between the free energy change of the system and its ability to learn an underlying data distribution. We empirically find a thermodynamic learning barrier for the system caused by the fluctuations of the environment, whereby the system cannot learn if its change in free energy hits such a barrier. We believe this simple model can help us better understand learning systems from a physical point of view. 1 Introduction The field of artificial intelligence (AI) has advanced rapidly in the past few years, and it has given us many tools that are becoming essential for our daily lives. However, training the large models that power these tools is very energy-intensive. Estimates of the CO 2 emissions caused by training and using language models are alarming [ 1 , 2 , 3 ] and are evidence of the need for more energy-efficient AI systems. Fortunately, multiple deep learning algorithms, such as diffusion models [ 4 , 5 ] , energy models [ 6 , 7 , 8 , 9 ] , and many other proposals [ 10 , 11 ] , are based on or inspired by physical processes. This relation can be leveraged to create specialized hardware that is better suited for training and running these models. Examples of these approaches include neuromorphic computing [ 12 , 13 , 14 , 15 , 16 ] which aims to simulate neural networks with hardware that mimics the dynamics of neurons and synapses, physical neural networks [ 17 , 18 , 19 ] which use non-conventional physical systems to parametrize and train neural networks, and thermodynamic computing [ 20 , 21 ] which is a recent proposal that utilizes guided stochastic processes to solve machine learning (ML) problems. There are many other ideas stemming from the classical and quantum domains [ 22 , 23 , 24 ] . The interplay and inspiration between physical processes and ML will hopefully help improve both the algorithms for AI and the hardware that runs them. In this paper, we propose a new machine learning algorithm that is based on a simple mechanical system composed of springs and sticks. Previously, the analogy between spring and stick systems and regression models has been discussed rather superficially [ 25 , 26 ] . Here, we build upon this analogy and create an ML algorithm that is universal and has a direct physical system associated with it. The springs and sticks (SS) model is a damped stochastic system that can perform regression over a dataset sampled from a smooth-enough distribution. Dissipative systems have been previously proposed as a way to perform computation [ 27 , 28 , 29 , 30 ] , and we follow this idea to engineer a dissipative universal learning machine. In Section 2 , we introduce the SS model, construct its Lagrangian, and obtain the equations of motion (EOMs) of such a system. In Section 3 , we perform simulations of the SS model for regression tasks, and show it can achieve similar performance to that of comparable multi-layer perceptrons (MLPs). In addition, we calculate the system’s change in thermodynamic free energy using the Jarzynski equality [ 31 ] and study its relation with the model’s performance. We show that the SS model cannot converge to a solution at small physical scales, where environment fluctuations are too large compared to the dynamics of the system. Finally, we conclude with future directions, including possible hardware implementations and interesting applications for the SS model. 2 The springs and sticks model In this section, we define the springs and sticks (SS) model and derive the equations of motion (EOMs) for the system. The SS model, shown in Fig. 1a , is a physical system composed of sticks and springs that can be used to solve regression problems. The system is defined on d+m dimensions, where d and m are the dimensions of the input space and output space, respectively. This system is composed of m\cdot N springs and N_{s}=\prod_{k=1}^{d}(N_{k}-1) sticks, where N is the number of data points and N_{k}-1 is the number of sticks along dimension k\leq d . By attaching springs from the data points to the grid of sticks and dissipating the elastic potential energy, the system relaxes to a minimum-energy configuration that equivalently minimizes a mean-squared error loss. The SS model can be used as a universal approximator of smooth functions via a piecewise-linear fit, and it can be used to solve regression problems on data \mathcal{D}=\{(\mathbf{u}_{i},\mathbf{y}_{i})\}_{i=0}^{N-1} where \dim(\mathbf{u}_{i})=d and \dim(\mathbf{y}_{i})=m . This model achieves an approximation error that scales as O(N_{s}^{-2}) for any smooth function with finite input and output dimensions. This error is equal to the integration error using the trapezoidal approximation of a function. We discuss universality with more detail and numerically show this error-scaling in Appendix A . (a) (b) Figure 1: Simulation of the SS system for simple regression problems. (a) Configuration of the model in a \mathbb{Z}_{5\times 5} lattice while fitting f(x,y)=x^{2}+x\cdot y^{2} with 80 datapoints sampled from (x,y)\sim\mathcal{U}([-\pi,\pi]^{2}) (b) Position of a stick extrema at different time steps for different hyperparameters (friction and temperature) while fitting f(x)=0 in a \mathbb{Z}_{2} lattice (one stick). The dashed line indicates the optimal position that minimize the springs’ potential energy. 2.1 Lagrangian of a general SS model To define the physical system of the SS model, consider a grid of sticks that are connected to the datapoints \mathcal{D} via springs as shown in Fig. 1a . The total Lagrangian of this system is the sum of the individual Lagrangians of each stick and spring [ 32 ] . A stick in free space has a translational kinetic energy and a rotational kinetic energy of K_{\text{tr}}=\frac{1}{2}M\dot{\mathbf{x}}_{\text{com}}^{2}=\frac{M}{8}(\dot{\mathbf{x}}_{0}+\dot{\mathbf{x}}_{1})^{2},\quad K_{\text{rot}}=\frac{1}{2}I\omega^{2}\approx\frac{M}{24}(\dot{\mathbf{x}}_{1}-\dot{\mathbf{x}}_{0})^{2} (1) where {\mathbf{x}}_{\text{com}} is the center of mass, \mathbf{x}_{0} and \mathbf{x}_{1} are the extrema of the stick, and I is its moment of inertia. Thus, for a grid of sticks where the extrema of neighboring sticks are held together, the total kinetic energy is K=\sum_{\text{sticks}}K_{\text{tr}}+K_{\text{rot}} since the kinetic energy is invariant under euclidean translations. In addition, the potential energy of a spring with rest length \ell_{0}=0 and spring constant k is given by U=\frac{1}{2}k\delta^{2} , where \delta is the displacement of the spring. Thus, attaching m springs between two points (\mathbf{u}_{i},y^{j}_{i}) and (\mathbf{u}_{i},\hat{y}^{j}) , one spring per coordinate j\in[m] , yields a potential energy of U_{i}=\frac{1}{2}k\sum_{j=0}^{m-1}(\hat{y}^{j}(\mathbf{u}_{i})-y^{j}_{i})^{2} per data point. Here, \hat{y}^{j}(\mathbf{u}_{i}) is the j -th coordinate of the predicted target of an input \mathbf{u}_{i} , and we will later define it for the SS model. Using these energies, we define

Read the complete text