Efficient and Interpretable Neural Networks Using Complex Lehmer Transform

Masoud Ataei, Xiaogang Wang · 2025

Read Efficient and Interpretable Neural Networks Using Complex Lehmer Transform by Masoud Ataei, Xiaogang Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15223.txt.

Introduction

Masoud Ataei and Xiaogang Wang design a clearer neural network in a 2025 paper. Most networks are black boxes: they score well but hide their reasons. Their network uses a new activation rule based on the Lehmer mean, a weighted average that can favor large or small inputs.

The rule adapts which features pass at each unit, and it extends to complex numbers, so phase in signals can be used rather than dropped. Math properties of both the real and complex forms are worked out. On standard test sets, the network matched strong rivals on accuracy while staying easier to read, a trade many fields would welcome.

Catalog record

CREATORMasoud Ataei, Xiaogang Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-15223

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient and Interpretable Neural Networks Using Complex Lehmer Transform Masoud Ataei, Xiaogang Wang Abstract We propose an efficient and interpretable neural network with a novel activation function called the weighted Lehmer transform. This new activation function enables adaptive feature selection and extends to the complex domain, capturing phase-sensitive and hierarchical relationships within data. Notably, it provides greater interpretability and transparency compared to existing machine learning models, facilitating a deeper understanding of its functionality and decision-making processes. We analyze the mathematical properties of both real-valued and complex-valued Lehmer activation units and demonstrate their applications in modeling nonlinear interactions. Empirical evaluations demonstrate that our proposed neural network achieves competitive accuracy on benchmark datasets with significantly improved computational efficiency. A single layer of real-valued or complex-valued Lehmer activation units is shown to deliver state-of-the-art performance, balancing efficiency with interpretability. keywords Lehmer Transform, Neural Network Interpretability, Complex-Valued Neural Networks, Nonlinear Activation Functions 1 Introduction We might have already entered the era of superintelligence, in which machines achieve better cognitive capabilities than humans. The current machine learning methods have made many revolutionary advances that were not even imaginable just a few years ago. The current deep learning models, while effective in capturing intricate feature interactions, often rely on numerous layers to achieve high performance with millions or even billions of parameters to capture complicated patterns. The black-box nature of the current machine learning architect has always been prohibitive for human reasoning to truly understand the nature of those millions of operations. This results in increased computational cost, diminished transparency, and challenges in understanding the underlying mechanisms driving their predictions. More importantly, this brings significant unforeseeable risks and uncertainties if we blindly trust the outcomes of those large models. For human intelligence to remain relevant in building machine learning models and to greatly reduce computational costs, we would need to construct neural network architectures that are not only efficient but also interpretable. Inspired by the Kolmogorov-Arnold representation theorem, a Kolmogorov-Arnold Network (KAN) has been previously proposed with a learnable node activation function Liu et al. (2024) . This network serves as an alternative to the traditional multi-layer perceptron architecture by leveraging the theorem’s elegant decomposition of high-dimensional functions. However, learning a high-dimensional function through a one-dimensional function can be challenging due to potential non-smoothness or fractal-like behavior in some mappings, which complicates optimization and interpretability. The KAN mitigates these issues by introducing a learnable activation function that adapts to the complexity of the target mappings. In this work, we take a different approach by employing a fixed nonlinear activation function within a single-layer architecture. This design not only simplifies the optimization process but also enhances interpretability and computational efficiency, offering a compelling alternative to KAN for applications requiring clarity and scalability. Specifically, we propose an efficient and interpretable neural network using a novel activation function called the Lehmer transform. The Lehmer transform, a parameterized family of means, provides a robust framework for summarizing and transforming features Ataei and Wang (2022) . Its ability to interpolate between different forms of mean behaviors allows it to emphasize small or large elements dynamically, making it a versatile tool for data representation. Building on its foundational mathematical properties, we extend the Lehmer transform into a weighted form, enabling adaptive feature importance through trainable weights. Additionally, we generalize the transform to the complex domain, introducing an oscillatory component that captures phase-sensitive dependencies and hierarchical relationships within data. The weighted form reflects the relative connection among inputs and also their absolute scale. It also enjoys perturbation and proportion invariance as well as smoothness and Schur convexity. The complex-valued weighted Lehmer transform introduces great flexibility by introducing an oscillatory component and enjoys differentiability in the complex domain. It also has a built-in smoothing or summarizing functionality for the inputs. These extensions form the foundation of the Lehmer Neural Network (LNN), a novel architecture that leverages the unique properties of the Lehmer transform. Central to the LNN are the Lehmer Activation Units (LAUs), which enable nonlinear summarization and phase-sensitive transformations through real-valued and complex-valued variants. The real-valued LAUs provide a parameterized mechanism for hierarchical feature aggregation, while the complex-valued LAUs further incorporate oscillatory contributions, enhancing their representational power for datasets with structured or phase-sensitive characteristics. With one single layer, this novel Neural Network achieves competitive accuracy with classical models on structured datasets such as Iris, Wine, and Wisconsin Breast Cancer, as well as high-dimensional datasets like MNIST. Empirical evaluations on these benchmark datasets also demonstrate the efficacy of LAUs in diverse contexts. These results highlight the ability of the LNN to achieve high performance with reduced architectural complexity, showcasing its efficiency and adaptability. To contextualize the development of our proposed neural network, it is helpful to examine recent advancements in interpretable neural networks. The increasing complexity and opacity of modern machine learning models have intensified the focus on interpretability, particularly in high-stakes domains such as healthcare and finance. Techniques such as attention mechanisms Bahdanau et al. (2015) and saliency maps Simonyan et al. (2013) have become prominent tools for providing insights into feature importance and decision-making processes. Recent advancements in self-explaining neural architectures embed interpretability directly into the network Alvarez-Melis and Jaakkola (2018) , ensuring transparency without compromising predictive accuracy. Concept-based methods Kim et al. (2018) and graph neural networks (GNNs) Kipf and Welling (2017) further extend these approaches, enabling interpretable learning in complex and structured data. These methods address the growing need for models that are both effective and explainable, particularly for ethical and transparent AI deployment Rudin (2019) ; Doshi-Velez and Kim (2017) . The opacity of traditional neural networks has also motivated the development of post-hoc explainability methods such as Layer-wise Relevance Propagation (LRP) Montavon et al. (2017) and Local Interpretable Model-agnostic Explanations (LIME) Ribeiro et al. (2016) . These techniques analyze feature importance after training, providing intuitive explanations for model behavior. Despite their widespread adoption, these approaches often fail to address the underlying training dynamics or ensure interpretability during the decision-making process. Recent research highlights the importance of integrating interpretability directly into the model architecture, exemplified by attention-based models Vaswani et al. (2017) and hybrid frameworks that combine interpretable components with high-performance black-box systems Gilpin et al. (2018) . This integration offers a promising path forward, balancing the trade-off between interpretability and predictive power Roscher et al. (2020) ; Tjoa and Guan (2020) . Simultaneously, complex-valued neural networks (CVNNs) have emerged as a powerful extension of traditional architectures, enhancing their ability to model oscillatory and phase-sensitive data. These networks leverage the complex plane for representation, capturing intricate dependencies in domains such as signal processing, radar imaging, and quantum mechanics Hirose and Yoshida (2012) ; Trabelsi et al. (2018) ; Du et al. (2023) . Advances in Wirtinger calculus Kreutz-Delgado et al. (2009) and complex-valued backpropagation Zhang and Li (2021) have supported the development of CVNNs, enabling efficient gradient-based optimization in the complex domain. Applications of CVNNs in time series prediction Schober et al. (2022) , image classification Chen and Qian (2020) , and polarimetric synthetic aperture radar (PolSAR) analysis Chen and Qian (2020) underscore their versatility and adaptability. By effectively modeling structured and periodic data, CVNNs demonstrate significant potential in capturing nuanced relationships that are difficult for real-valued networks to

Read the complete text