Efficient Training for Optical Computing

Manon P. Bart, Nick Sparks, Ryan T. Glasser · 2025

Read Efficient Training for Optical Computing by Manon P. Bart, Nick Sparks, Ryan T. Glasser (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20833.txt.

Introduction

Manon Bart, Nick Sparks, and Ryan Glasser tackle a practical problem in optical computing: training the hardware takes too long. Optical processors use patterns of light to do math for machine learning, but adjusting their many settings is slow.

The team notes that the math inside these devices is built from a few simple kinds of steps, such as light spreading through free space and thin layers that shift the light. Past methods wasted effort because only a small share of the settings can change. The authors present a new training method that uses the structure of the device directly. It works out the needed changes for a whole layer at once, using a change of basis based on the Fourier transform and simple step-by-step multiplication. The result is a large cut in training time. The method can also shape light for other tasks.

Catalog record

CREATORManon P. Bart, Nick Sparks, Ryan T. Glasser
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-20833

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Training for Optical Computing Manon P. Bart, Nick Sparks, Ryan T. Glasser Abstract Diffractive optical information processors have demonstrated significant promise in delivering high-speed, parallel, and energy efficient inference for scaling machine learning tasks. Training, however, remains a major computational bottleneck, compounded by large datasets and many simulations required for state-of-the-art classification models. The underlying linear transformations in such systems are inherently constrained to compositions of circulant and diagonal matrix factors, representing free-space propagation and phase and/or amplitude modulation of light, respectively. While theoretically established that an arbitrary linear transformation can be generated by such factors, only upper bounds on the number of factors exist, which are experimentally unfeasible. Additionally, physical parameters such as inter-layer distance, number of layers, and phase-only modulation further restrict the solution space. Without tractable analytical decompositions, prior works have implemented various constrained minimization techniques. As trainable elements occupy a small subset of the overall transformation, existing techniques incur unnecessary computational overhead, limiting scalability. In this work, we demonstrate significant reduction in training time by exploiting the structured and sparse nature of diffractive systems in training and inference. We introduce a novel backpropagation algorithm that incorporates plane wave decomposition via the Fourier transform, computing gradients across all trainable elements in a given layer simultaneously, using only change-of-basis and element wise multiplication. Given the lack of a closed-form mathematical decomposition for realizable optical architectures, this approach is not only valuable for machine learning tasks but broadly applicable for the generation of arbitrary linear transformations, wavefront shaping, and other signal processing tasks. Decades of research has prefaced the ubiquity of machine learning in our present lives. Deep learning in particular has proven to be a multi-disciplinary tool, with research developments and industry applications in speech [ 1 ] and object recognition [ 2 ] , biomedical applications [ 3 , 4 ] , marketing and advertisement [ 5 ] , and stock market analysis [ 6 ] , among others. Despite the advancements in parallelism and processing power of hardware in recent years, modern technologies are approaching scaling limitations and drastic energy consumption due to the massive computing power and computational complexity needed for statistical inference [ 7 , 8 ] . These factors have serious environmental implications, especially during training and hyperparameter tuning [ 9 ] . Consequently, the field is increasingly shifting toward more sustainable, low-power approaches to machine learning. While early proposals of diffractive optical processors and machine learning [ 10 , 11 , 12 ] were largely overshadowed by the rise of deep neural networks (DNNs) coupled with rapid advances in graphical processing units (GPUs) [ 13 ] , renewed interest in optical processing can be attributed to the physical architecture’s low power consumption, high speed and bandwidth, and intrinsic parallelism [ 14 , 15 , 16 ] . While their are many ways to process optical information, such as hybrid opto-electronics, silicon photonic circuits, and diffractive surfaces or cascaded phase masks [ 17 , 18 , 19 , 20 ] , all of these processors leverage certain light matter interactions to perform linear transformations which are computationally expensive on current computing architectures [ 21 , 22 , 23 ] . Optical information processing has shown functionality in the case of free space communication [ 24 ] , optical imaging [ 25 ] , optical interconnections [ 26 ] , biomedical applications [ 27 , 28 , 29 ] , solving complex equations [ 30 , 31 ] , and demonstrations of quantum linear operations [ 32 , 33 ] , among others. Optical machine learning has matured in recent years with many analog optical neural network architectures such as single photon optical vector matrix multiplication, massively parallel diffractive neural networks, optical reservoir computing, and low to zero-power convolutional neural networks [ 34 , 35 , 36 , 37 , 16 , 38 ] . Diffraction-based optical architectures are particularly appealing for processing multi-dimensional data. This is due to the multiple degrees of freedom such as phase, amplitude [ 39 , 40 ] , polarization [ 41 ] , which, unlike integrated photonics, are retained at each layer, leading to the ability to recover information to mitigate errors in training and implementation [ 42 , 43 ] . Further, task specific processes which rely on real time correction suffer from low latency during pre-processing, whereas direct encoding can be done on free-space optical processors [ 25 , 44 , 45 ] . For diffraction-based optical processors, various physical architectures have been explored to implement linear transformations such as convolutions [ 37 , 46 ] or optical vector-matrix multiplication (OVMM) [ 19 , 47 ] . In recent years, cascaded phase masks or diffractive elements have gained significant traction as a scalable approach to machine learning tasks [ 22 , 23 ] . Information is typically encoded in the phase and amplitude channels of light and trainable phase masks, defined by a diagonal matrix, modulate the complex field of light at each layer. When cascaded with free-space propagation, represented as circulant matrices, these systems can realize arbitrary linear transformations [ 48 , 49 ] with extremely low power consumption [ 22 , 23 , 50 ] and non-destructive, unitary transformations that preserve optical information [ 51 ] . Unlike OVMM, where direct encoding of the matrix representing the linear transformation is encoded into a diffractive element [ 19 , 34 , 52 ] , decomposing a linear transformation into circulant and diagonal factors efficiently remains non-trivial [ 53 , 49 ] . Despite proof that a composition of circulant and diagonal matrices can generate an arbitrary linear transformation [ 48 ] , the generation of these matrices is constructive, and involves solving a structured system of polynomial equations [ 53 ] . An upper bound of 2\text{N}-1 alternating circulant and diagonal blocks for an exact decomposition of an \text{N}\times\text{N} linear transformation has been established, but this is experimentally and computationally prohibitive [ 53 ] . In addition, constraints such as task-specific parameters for physical setups (i.e. wavelength, sampling, inter-layer distances) as well as phase-only modulation and finite resolution make finding a suitable decomposition for a small number of circulant and diagonal factors a considerable challenge. For machine learning tasks, determination of an exact, unique decomposition is unnecessary, only requiring a sufficiently expressive optical architecture. Research has focused on strategies to address optical nonlinearities [ 54 , 55 , 56 , 57 ] and architectural complexity [ 58 , 39 , 40 ] in order to create models that approach the performance of state-of-the-art electronic machine learning models. A major bottleneck in scaling these efforts, however, remains training them. To address this, several strategies have been proposed using constrained minimization. A matrix pseudoinverse-based synthesis was utilized for single-layer complex linear transformations, but was not clearly generalizable to multi-layer architectures required for expressive linear transformations [ 22 ] . A finite difference method has been proposed; however, it requires performing forward propagation twice for each individual trainable parameter, which constrains the scalability that optical architectures offer [ 59 ] . Data-driven approaches using backpropagation, which calculate the gradients and error in trainable parameters with respect to a cost function and updates them to reach a local

Read the complete text