NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference

Marta Andronic and George A. Constantinides · 2025

Read NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference by Marta Andronic and George A. Constantinides (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00592.txt.

Introduction

Marta Andronic and George Constantinides present in 2025 a framework that assembles small neural units into bigger lookup-table neurons for chip inference. Table-based nets promise tiny delay on programmable chips for edge tasks. Wide inputs blow up table size, which forced past designs into sparse thin forms.

Mixed precision plus assembly from small units lifts connectivity while each table stays lean. Skip links across table blocks ease gradient flow in training. Scores on intrusion, digit, and jet tasks near match full network grades. Area-delay cost drops by up to eight times versus past table designs.

Catalog record

CREATORMarta Andronic and George A. Constantinides
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-00592

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference Marta Andronic and George A. Constantinides Abstract Efficient neural networks (NNs) leveraging lookup tables (LUTs) have demonstrated significant potential for emerging AI applications, particularly when deployed on field-programmable gate arrays (FPGAs) for edge computing. These architectures promise ultra-low latency and reduced resource utilization, broadening neural network adoption in fields such as particle physics. However, existing LUT-based designs suffer from accuracy degradation due to the large fan-in required by neurons being limited by the exponential scaling of LUT resources with input width. In practice, in prior work this tension has resulted in the reliance on extremely sparse models. We present NeuraLUT-Assemble, a novel framework that addresses these limitations by combining mixed-precision techniques with the assembly of larger neurons from smaller units, thereby increasing connectivity while keeping the number of inputs of any given LUT manageable. Additionally, we introduce skip-connections across entire LUT structures to improve gradient flow. NeuraLUT-Assemble closes the accuracy gap between LUT-based methods and (fully-connected) MLP-based models, achieving competitive accuracy on tasks such as network intrusion detection, digit classification, and jet classification, demonstrating up to 8.42 × 8.42\times reduction in the area-delay product compared to the state-of-the-art at the time of the publication. I Introduction Ultra-low latency NN inference has become instrumental in advancing fields such as particle physics, network security, and autonomous vehicles. In particle physics, machine learning (ML) models are essential for handling the immense data volumes generated by detectors. For instance, in the latest upgrade to the CMS trigger system at CERN’s Large Hadron Collider, the system processes data from six consecutive beam crossings, every 25 ns. The system must achieve nanosecond-level latency to be capable of accepting new data inputs continuously. Such real-time capabilities enable the collection of meaningful events that could otherwise be lost. In network security, ML-driven intrusion detection systems swiftly identify threats and anomalies, safeguarding critical infrastructure by delivering near-instant insights. Autonomous vehicles also rely on ML to interpret sensor data and make split-second navigation and safety decisions, particularly in complex or high-speed scenarios. However, due to the resource-constrained environments and strict latency requirements in these fields, deployed deep neural networks (DNNs) often fall short of the state-of-the-art accuracy in ML. FPGAs, with their highly customizable architecture, serve as a core platform for these applications, enabling optimized computation to meet stringent performance KPIs. Their reconfigurability supports rapid design iteration, making them ideal for applications that demand frequent model updates. Moreover, FPGAs perform many computations in parallel, significantly reducing processing time. Recent research in the field has focused on hardware-software co-design. Beyond designing efficient hardware, it is also important to adapt software for deployment efficiency. A growing area of research explores model architectures that map efficiently to hardware. For example, LUT-based approaches like NeuraLUT [ 1 ] , PolyLUT [ 2 ] , LogicNets [ 3 ] , NullaNet [ 4 ] , PolyLUT-Add [ 5 ] , AmigoLUT [ 6 ] . Other recent works have adopted decision tree-based approaches like TreeLUT [ 7 ] , or weightless neural networks [ 8 ] . Following [ 2 ] , in this paper we refer to lookup tables of arbitrary size as Logical-LUTs (L-LUTs), highlighting their ability to exceed the capacity of the Physical-LUTs (P-LUTs) on the FPGA. When an L-LUT requires more inputs than a P-LUT can handle, logic synthesis tools map it as a circuit of multiple interconnected P-LUTs. NeuraLUT, PolyLUT, LogicNets, and NullaNet encapsulate the entire computation of a neuron within a single L-LUT, creating a network of L-LUTs with no exposed datapaths. These prior works offer distinct trade-offs in model complexity and expressiveness. LogicNets and NullaNet excel in functional simplicity by using continuous piecewise linear functions. PolyLUT achieves similar accuracy with fewer piecewise regions (resulting in fewer L-LUTs) by employing continuous piecewise polynomial functions, delivering enhanced expressiveness but at the cost of increased training complexity. NeuraLUT trains a composition of multiple continuous piecewise linear functions, striking a balance between expressive power and reduced function complexity, leading to richer model representations for the same L-LUT complexity. However, for any of these approaches to be feasible, the lookup table size is constrained, which can limit the accuracy of these neural networks. PolyLUT-Add [ 5 ] takes a first step at trying to improve the connectivity of these networks by summing the results of multiple L-LUTs across the network. However, this approach utilizes LUTs for the implementation of the sum which are also restricted by their fan-in. AmigoLUT [ 6 ] creates ensembles of smaller LUT-based NNs, including NeuraLUT, to tackle the scalability. In this work, NeuraLUT-Assemble, we take a distinct approach to combat the challenge of the inherent limitation on the number of L-LUT inputs. We introduce a fully-parametrizable framework that assembles multiple NeuraLUT neurons as tree structures with larger fan-in, directly addressing the exponential scaling challenge. This customizable tool allows users to increase connectivity without fan-in restrictions by providing full control over the tree structure. The grouping of connections at the input of the tree structure is guided by the hardware-aware pruning strategy first introduced in the extended arXiv PolyLUT paper [ 9 ] . To enhance training stability, we propose a resource-efficient method that integrates skip-connections within the L-LUTs to ensure effective gradient flow in the individual L-LUTs or across the assembled tree structure. In summary, the novel contributions are as follows: • We introduce NeuraLUT-Assemble, an open-source 1 1 1 https://github.com/MartaAndronic/NeuraLUT toolflow that leverages the FPGA architecture by embedding dense, full-precision sub-networks within tree-structures of synthesizable Boolean lookup tables. • We develop a fully-parametrizable framework to increase connectivity by training larger fan-in tree structures of smaller L-LUT units, where connection grouping is determined post-initial training. • We develop a resource-efficient approach that embeds skip-connections within L-LUTs, promoting smooth gradient flow at training throughout the entire assembled tree structure. • We assess NeuraLUT-Assemble on three standard tasks used in the low-latency DNN research community: digit classification, jet substructure classification, and network intrusion detection. Our results show that compared to NeuraLUT, our method achieves the lowest area-delay product with up to 62\times reduction on MNIST combined with a higher test accuracy, and up to 26\times reduction on jet substructure for the same test accuracy. II Background Achieving real-time neural networks with low resource utilization and high accuracy requires reimagining model architecture to optimize performance on the target hardware. Recent approaches have emphasized hardware-software co-design, creating a tightly integrated cycle of model design, training, and deployment on custom hardware platforms. Co-design efforts for FPGA optimization, in particular, can be categorized by their primary learnable unit: DSP-based, XNOR-based, decision tree-based, differentiable LUT-based, and traditional LUT-based neural network architectures. II-A DSP-based architectures In the area of ultra-low latency, hls4ml [ 10 ] is a notable open-source framework created to enable machine learning inference on FPGAs, with a focus on low-latency applications. Duarte et al. [ 10 ] employ hls4ml to generate both fully-unrolled and rolled network architectures that target latency reduction. These designs, however, utilize high-precision networks, which results in considerable Digital Signal Processing (DSP) usage. Fahim et al. [ 11 ] further optimize hls4ml by incorporating techniques such as quantization-aware pruning. II-B XNOR-based architectures Ngadiuba et al. [ 12 ] use the

Read the complete text