MixFunn: A Neural Network for Differential Equations with Improved Generalization and Interpretability

Tiago de S. Farias, Gubio G. de Lima, Jonas Maziero, Celso J. Villas-Boas · 2025

Read MixFunn: A Neural Network for Differential Equations with Improved Generalization and Interpretability by Tiago de S. Farias, Gubio G. de Lima, Jonas Maziero, Celso J. Villas-Boas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22528.txt.

Introduction

Neural networks that solve differential equations often need huge numbers of parameters and stay hard to interpret. Tiago de Souza Farias and co-authors introduce MixFunn in this 2025 paper, a leaner network built from two new neuron types.

One neuron blends several shaped curves for flexibility, and the other adds a squared term that captures how inputs interact. Together they match or beat standard models with a tiny fraction of the parts. Tested on problems from mechanics, quantum physics, and fluid flow, the network stayed accurate and stretched better to new regions. Its solutions can also be read back as plain formulas.

Catalog record

CREATORTiago de S. Farias, Gubio G. de Lima, Jonas Maziero, Celso J. Villas-Boas
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22528

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MixFunn: A Neural Network for Differential Equations with Improved Generalization and Interpretability Tiago de S. Farias, Gubio G. de Lima, Jonas Maziero, Celso J. Villas-Boas Abstract We introduce MixFunn, a novel neural network architecture designed to solve differential equations with enhanced precision, interpretability, and generalization capability. The architecture comprises two key components: the mixed-function neuron, which integrates multiple parameterized nonlinear functions to improve representational flexibility, and the second-order neuron, which combines a linear transformation of its inputs with a quadratic term to capture cross-combinations of input variables. These features significantly enhance the expressive power of the network, enabling it to achieve comparable or superior results with drastically fewer parameters and a reduction of up to four orders of magnitude compared to conventional approaches. We applied MixFunn in a physics-informed setting to solve differential equations in classical mechanics, quantum mechanics, and fluid dynamics, demonstrating its effectiveness in achieving higher accuracy and improved generalization to regions outside the training domain relative to standard machine learning models. Furthermore, the architecture facilitates the extraction of interpretable analytical expressions, offering valuable insights into the underlying solutions. I Introduction Differential equations lie at the heart of modeling a vast array of physical systems, spanning disciplines from classical mechanics, describing the motion of planets, rockets, and collisions, to quantum mechanics, where they govern the dynamics of molecules and elementary particles. These equations not only provide a framework for understanding fundamental natural phenomena but also serve as powerful tools for simulating complex physical processes. In engineering, differential equations are equally essential, used to model systems in fluid dynamics, structural analysis, electrical circuits, and thermodynamics. Through these applications, we can gain deeper insights into system behavior, enabling more accurate predictions and optimizations in scientific and engineering contexts. Consequently, differential equations are indispensable to scientific inquiry and technological advancements, forming a foundational basis for many of the technologies we rely on today Ellahi et al. 2018 ; Betounes 2010 . A differential equation is a mathematical expression that involves the derivatives of a function with respect to its variables. Solving such equations entails finding the function that satisfies the equation for the given variables. The origins of differential equations trace back to the inception of calculus by Newton and Leibniz Archibald et al. 2004 . Over the centuries, extensive research has been conducted on methods to solve these equations Denis 2020 . Analytical techniques aim to derive exact or highly precise solutions in the form of mathematical functions. These solutions are particularly valuable as they allow for a deeper understanding of physical systems by enabling the analysis of the solution. However, these methods are limited in scope, as they can only be applied to relatively simple differential equations, which often do not represent the complexity of real-world physical phenomena Xu 2020 ; Ardourel and Jebeile 2017 . On the other hand, numerical methods offer a broader range of applicability by providing approximate solutions to more complex differential equations. These methods compute the values of the function at selected points within a domain of interest, making them suitable for various practical problems. Despite their versatility, numerical methods can be computationally expensive and often lack both the generalization capabilities and interpretability that are characteristic of analytical solutions. As a result, while they provide valuable numerical approximations, the underlying insights into the behavior of the system are often more challenging to extract Fadugba et al. 2020 ; Sapienza et al. 2024 . Artificial neural networks (ANNs) have emerged as one of the most influential and versatile machine learning algorithms, demonstrating remarkable success in solving various problems across different domains. From image recognition Krizhevsky et al. 2012a and natural language processing Vaswani et al. 2017a to protein folding Jumper et al. 2021 , neural networks have revolutionized these fields by providing highly accurate and scalable solutions Dargan et al. 2020 . The key to their effectiveness lies in their expressivity: Their capacity to represent complex functions and relationships within data Kurt Hornik and White 1989 . The expressivity of a neural network refers to the complexity it can model, which is closely tied to various factors such as the choice of activation functions, the depth and architecture of the network, and its arithmetic capabilities. More expressive networks, with the right design, can approximate highly complex functions, making them suitable for tackling traditionally difficult problems with analytical or numerical methods. However, this expressivity also introduces challenges, such as the need for extensive computational resources and the risk of overfitting when training on limited data. Thus, striking the right balance between network expressivity and generalization is a central concern in developing and applying neural networks across different fields. The design choices made when constructing a neural network, including the selection of activation functions, network architecture, and other hyperparameters, have a profound impact on the ability of the model to solve a given problem effectively. These choices influence the capacity of the model to learn complex patterns, generalize to unseen data, and converge efficiently during training. While some of these decisions can be guided by theoretical analyses, such as understanding the mathematical properties of activation functions or the expressivity of deep networks, they can also be informed by empirical observations Ithapu et al. 2017 ; Philipp 2021 ; Nikanjam and Khomh 2021 . In this work, we explore and combine two types of neurons for differential equations: the mixed-function and the second-order neurons. These neurons are designed to enhance the expressivity of neural networks in representing complex functions. These neurons form the foundation of two new models: the Mixed-Function Neural Network (MixFunn) and the Second-Order Mixed-Function Neural Network (Mix2Funn). By allowing each neuron to use multiple activation functions and incorporating relationships between inputs or other neurons, these models expand the arithmetic capabilities of standard networks. A key advantage of our proposed models is their ability to represent complex functions with a drastically reduced number of parameters, up to four orders of magnitude fewer than conventional approaches. This substantial reduction not only diminishes computational costs but also facilitates the extraction of interpretable analytical expressions, thereby enhancing generalization and interpretability. The second-order neurons in Mix2Funn, in particular, introduce interactions between inputs, further bolstering the capacity of a model to capture higher-order dependencies. We demonstrate that these models outperform standard neural networks in solving differential equations, particularly in applications drawn from classical mechanics, quantum mechanics, and fluid dynamics. The remainder of this article is organized as follows. In Section II, we review related papers that explore the application of neural networks to solving differential equations, highlighting their strengths, limitations, and key properties. Section III introduces our proposed methodology, detailing the design and functionality of the mixed-function and second-order neurons, as well as the regularization techniques that enhance the effectiveness of these models. In Section IV, we present and analyze the results of applying our proposed models to four distinct differential equation problems, discussing their performance and broader implications. Finally, in Section V, we provide concluding remarks, summarizing our contributions and outlining potential directions for future research. II Related work Physics-Informed Neural Networks (PINNs) Raissi et al. 2017a ; Raissi et al. 2017b ; Raissi et al. 2019 integrate differentiable programming with differential equations to approximate the solution of complex physical

Read the complete text