Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning

Mominul Rubel, Adam Meyers, Gabriel Nicolosi · 2025

Read Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning by Mominul Rubel, Adam Meyers, Gabriel Nicolosi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08759.txt.

Introduction

Mominul Rubel and 2 co-authors introduce the Fourier Learning Machine in a 2025 paper on neural networks for science. The design represents a Fourier series, a sum of waves, in many dimensions. Unlike older Fourier models, it keeps a complete separable basis.

A simple feedforward network with cosine activation learns frequencies, amplitudes, and phase shifts as trainable values. Those values map directly to Fourier coefficients, giving a clear link between the two forms. Tests cover benchmark differential equations and a family of optimal control problems. Performance matches or beats established models such as SIREN. The basis adapts to both periodic and nonperiodic functions.

Catalog record

CREATORMominul Rubel, Adam Meyers, Gabriel Nicolosi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE115 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-08759

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning Mominul Rubel, Adam Meyers, Gabriel Nicolosi Abstract We introduce the Fourier Learning Machine (FLM), a neural network (NN) architecture designed to represent a multidimensional nonharmonic Fourier series. The FLM uses a simple feedforward structure with cosine activation functions to learn the frequencies, amplitudes, and phase shifts of the series as trainable parameters. This design allows the model to create a problem–specific spectral basis adaptable to both periodic and nonperiodic functions. Unlike previous Fourier–inspired NN models, the FLM is the first architecture able to represent a complete, separable Fourier basis in multiple dimensions using a standard Multilayer Perceptron–like architecture. A one–to–one correspondence between the Fourier coefficients and amplitudes and phase-shifts is demonstrated, allowing for the translation between a full, separable basis form and the cosine phase–shifted one. Additionally, we evaluate the performance of FLMs on several scientific computing problems, including benchmark Partial Differential Equations (PDEs) and a family of Optimal Control Problems (OCPs). Computational experiments show that the performance of FLMs is comparable, and often superior, to that of established architectures like SIREN and vanilla feedforward NNs. Keywords: Nonharmonic Fourier Series, Multilayer Perceptron, Partial Differential Equations, Evolutionary Games, Rock–Paper–Scissors. 1 Introduction Approximating functions as a composition or weighted sum of simpler basis elements is a fundamental approach in both classical analysis and modern machine learning. Two typical examples of this idea are the Fourier series and artificial neural networks (NNs). The Fourier series decomposes a periodic function (or, for nonperiodic functions, their periodic extension) into a linear combination of a fixed and predetermined set of sinusoidal ( sines and cosines ) basis functions and gives an interpretable spectral representation of it. While nonperiodic functions can also be handled by defining a periodic extension over a finite interval, this often introduces discontinuities at the boundaries and leads to oscillatory behavior, known as the Gibbs phenomenon. Moreover, these Fourier bases are fixed and not adaptable to the specific features of the data. NNs, on the other hand, are computational models that approximate any function through compositions of parametrized nonlinear transformations of linear combinations of the data with (trainable) weights and biases. These parameters composing a NN are learned and adapted to the data during training (the process of optimizing the network parameters) using nonlinear optimization techniques. However, these weights and biases are numbers with little interpretation, and the network itself works as a black–box. Despite these differences, both approaches share the basic underlying principle of building complex functions from simpler components. Motivated by Joseph Fourier’s early 19^{th} century idea that any arbitrary function, regardless of its complexity or continuity, could be represented as a sum of sines and cosines , and by the successful application of NNs as universal function approximators since the late 20^{th} century, we propose in this paper a new Fourier–inspired NN model, and we shall name it the Fourier Learning Machine (FLM). Unlike typical black–box NN models, the FLM’s internal structure is built to directly represent an m- dimensional nonharmonic Fourier series. It achieves this by relying solely on the foundational components of a classical multilayer perceptron (MLP): namely, affine transformations followed by nonlinear activation functions (in the context of this paper, these being pure cosines ). This design choice eschews more complicated architectural features, such as multiplication nodes, recurrent relations or any other customized components, previously found in predecessor models. With hardwired integer weights in its input–to–hidden layer, the FLM represents an m- dimensional, full-basis classical (finite) Fourier series as a special case. In order to tackle the approximation of nonperiodic functions, we relax the integrality constraints of the frequency components, thus configuring the frequencies as learnable parameters of the FLM. This allows for the search for the sinusoidal basis functions that best represent the target map. While maintaining interpretability, FLMs build a function–specific spectral representation for a wide variety of functions, including nonperiodic ones. As such, our proposed model combines the interpretability of Fourier series with the data–driven nature of NNs (and, in particular, the simplicity of the MLP architecture), effectively representing a nonharmonic Fourier series [ 1 , 2 ] in which the basis functions are learned and adapted to the target function it is meant to approximate. The challenge of representing Fourier series with traditional MLP - like NNs can be understood by first considering a one - dimensional Fourier series on the interval [-\pi,\pi] (without loss of generality) in the cosine phase–shifted form, i.e. , \widehat{f}(t)=a_{0}+\sum_{n=1}^{N}A_{n}\cos(nt-\phi_{n}), where A_{n} and \phi_{n} denote the amplitude and the phase shift for the n^{th} frequency component, respectively. For such a form of the Fourier series, it is straightforward to see that the approximation \widehat{f} can be naturally represented by a single–layer NN with cosine activation functions. Nonetheless, generalizing this idea to higher dimensions requires extra care since, as we demonstrate in this paper, the NN structure then either departs too far from the MLP–like architecture or fails to represent an actual, truncated nonharmonic Fourier series. As a result, a certain degree of architectural ingenuity is required so that the model retains both a simple architecture and the integrity of the nonharmonic Fourier representation in higher dimensions. In this paper, we show that the proposed FLM does not only combine an interpretable structure with adaptable bases, but also achieves approximation accuracy comparable to, and in some cases exceeding, that of established NN architectures. Although we focus on using FLMs to solve Partial Differential Equations (PDEs) and an Optimal Control Problem (OCP), this model can be applied to a wide range of problems, such as signal and image reconstruction, spectral analysis, and surrogate modeling for high–dimensional systems, where both interpretability and adaptability are desired. The main contributions of this paper are: 1. We provide a comprehensive review of all previous attempts to construct such a function approximator, typically referred in the literature as “Fourier Neural Networks”, departing from foundational works [ 3 ] and covering up to recent architectures with trigonometric activation functions [ 4 ] . Within this context, we demonstrate that the FLM is not only architecturally simpler than previous models but also computationally competitive, often outperforming established, traditional NN models. 2. By defining the m –Lexi Sign Matrix, we show how the FLM, a feedforward NN, can simultaneously represent a full basis finite Fourier series in m –dimensions in both separable and cosine phase–shifted forms, with a one–to–one correspondence between Fourier coefficients, frequencies and phase–shift and the NN weights and biases. To the best of the author’s knowledge, this flexibility to represent a full–basis classical Fourier series in a MLP–like architecture, is unique in the literature. In the nonharmonic case, it is shown that such a representation feature is directly translated to a more computationally efficient parameter optimization when compared to other traditional and Fourier–based NN models. 3. Through a series of carefully designed computational experiments, we empirically demonstrate the FLM’s approximation capabilities for solving PDEs and optimization problems. In particular, we apply the FLM to the numerical solution of a multidimensional OCP arising from the evolutionary dynamics of a Rock–Paper–Scissors (odd–circulant) game, demonstrating its capability to solve complex, nonlinear problems. The remainder of the paper is organized as follows: Section 2 reviews related literature and

Read the complete text