HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks
Boshko Koloski, Nada Lavrač, Blaž Škrlj · 2025
Read HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks by Boshko Koloski, Nada Lavrač, Blaž Škrlj (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-14346.txt.
Introduction
Boshko Koloski, Nada Lavrač, and Blaž Škrlj present a 2025 neural network design for tables that mix numbers with set labels. Such tables rule medicine, yet they often hold few rows with many columns, a hard shape for learning. Their design, HorNets, rests on a clipped polynomial-style switch. A routing part sends each input to the network part that fits its kind, with no told labels for the split.
The network can model part blends of traits or the whole space in one sweep. It reliably pulls out logic rules, even noisy twin-negation ones. On 14 real biomedical sets it topped rival scores. The team released the code and a fake-data maker under a free license. Small, messy tables gain a tool built for their shape.
Catalog record
| CREATOR | Boshko Koloski, Nada Lavrač, Blaž Škrlj |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-14346
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HorNets: Learning from Discrete and Continuous Signals with Routing Neural Networks Boshko Koloski, Nada Lavrač, Blaž Škrlj Abstract Construction of neural network architectures suitable for learning from both continuous and discrete tabular data is a challenging research endeavor. Contemporary high-dimensional tabular data sets are often characterized by a relatively small instance count, requiring data-efficient learning. We propose HorNets (Horn Networks), a neural network architecture with state-of-the-art performance on synthetic and real-life data sets from scarce-data tabular domains. HorNets are based on a clipped polynomial-like activation function, extended by a custom discrete-continuous routing mechanism that decides which part of the neural network to optimize based on the input’s cardinality. By explicitly modeling parts of the feature combination space or combining whole space in a linear attention-like manner, HorNets dynamically decide which mode of operation is the most suitable for a given piece of data with no explicit supervision. This architecture is one of the few approaches that reliably retrieves logical clauses (including noisy XNOR) and achieves state-of-the-art classification performance on 14 real-life biomedical high-dimensional data sets. HorNets are made freely available under a permissive license alongside a synthetic generator of categorical benchmarks. keywords Neural Networks, Categorical data, Machine Learning † † equal-contributors: These authors contributed equally to this work. † † equal-contributors: These authors contributed equally to this work. 1 Introduction In recent years, neural network models have shown great promise when modeling complex real-life data sets such as images, texts, and networks Dong et al. 2021 . Data sets are most frequently represented in the form of propositional tables. Bader et al. Bader et al. 2007 discussed the link between propositional logic and feedforward neural networks. They have successfully demonstrated that non-conventional activations taking values on the interval [-1,1] need to be adopted to model simple propositional expressions. They also discuss how such expressions can be composed into search trees and binary decision diagrams. Similar ideas were explored by Al Iqbal Al Iqbal 2011 , where decision trees were co-trained with neural networks to extract simple rules. The more recent work of Labaf et al. Labaf et al. 2017 also explored how background knowledge can be incorporated into the learning of logical expressions by approximating the decision process of neural networks with knowledge-aware logical rule learners. In parallel, this work is also inspired to a large extent by the recent progress in neuro-symbolic machine learning . For example, the works of Amizadeh et al. Amizadeh et al. 2020 explored the link between logical induction, coupled to low-level (neural) pattern recognition. Similarly, works like DeepProbLog Manhaeve et al. 2018 explore the language design required to bridge the two learning paradigms (i.e. extension of Prolog with a neural predicate). Such insights are key motivators of this study, as we show that neural networks can be used to produce, and operate directly with interpretable logical expressions. The main contributions of this paper are the following: • We propose HorNets (Horn Networks), a neural network architecture capable of operating with Horn clauses, that demonstrably achieves state-of-the-art performance on synthetic discrete/categorical and continuous data sets. • HorNets are based on polyClip activation function – an activation for which we show a probabilistic link to zero-order (propositional) logic. • To assess the HorNets capability to model logical gates, we conducted extensive benchmarks on synthetic data obtained by a generator implemented as part of this work. We demonstrate that HorNets neural network architecture achieves the best recall in terms of being able to model the most common logic gates (and with that, binary interactions). • Competitive performance of HorNets is demonstrated on more than ten real-life data sets from the biomedical domain, demonstrating it outperforms strong approaches such as gradient-boosted trees and selected AutoML-based classifiers. The rest of this paper is structured as follows. We begin by discussing the related work in Section 2 , followed by the proposed methodology and its theoretical implications in Section 3 . In Section 4 , we present the considered experimental setting (on synthetic data in Section 4.1 and on real data in Section 4.2 ) , followed by the results in Section 5 . Finally, in Section 6 , we discuss the achievements and drawbacks of this work and present future research directions. 2 Related work This section overviews some of the related work that impacted the development of HorNets. We begin by discussing binarized neural networks, followed by the body of literature focusing on neuro-symbolic learning . Exploration of how rule lists can be transformed into networks capable of classification was performed by ( Towell and Shavlik 1994 ) . Their networks could account for AND, OR and NOT connectives, offering expressions in full propositional logic. The subfield of studying neural networks that also resonates with the proposed approach concerns binary neural networks. For example, the work of Rastegari et al. Rastegari et al. 2016 demonstrates that a neural network capable of approximating the XNOR truth table offers sufficient performance on ImageNet, indicating high expressiveness of such networks – by being able to model XNOR, this was one of the first approaches that demonstrated capability to model general logical statements. Similar results were achieved on CIFAR-10 and MNIST data sets Courbariaux et al. 2015 . Binary neural networks were also shown as a promising variant of models that can be computed on specialized hardware such as FPGAs Moss et al. 2017 ; Sun et al. 2018 . Alternatively to considering neural networks that directly approximate logical expressions, such expressions can be generated before being fed into a neural network. This second branch of methods that inspired this work revolves around the notion of neuro-symbolic learning. This paradigm explores how neural network-based low-level recognition (e.g., vision) can be coupled with the notion of reasoning . Such ideas culminated into the concept known as deep relational machines Lodhi 2013 ; Dash et al. 2018 , offering a state-of-the-art performance for e.g., molecule classification tasks. DeepProbLog Manhaeve et al. 2018 combined the idea of probabilistic logic and deep learning, enabling probabilistic inference over deep latent structures, thus providing a tool for tasks such as distant supervision tasks. DeepProbLog Manhaeve et al. 2018 integrates probabilistic logic with deep learning, enabling inference over deep latent structures. This framework is ideal for tasks that require background knowledge for validation and utilize deep learning for representation learning in distant supervision scenarios. Contrary to the work of Bader et al. 2007 , where programs were extracted from feedforward neural networks, an inductive logic program was first used to construct features, which were used to train a deep neural network model. More recently, the link between deep learning and propositionalization was explored by Lavrač et al. 2020 , resulting in two fundamentally different ways to learn latent representations from a propositionalized relational database. Cropper and Dumančić 2022 Cropper and Dumančić 2022 recently a deeper overview on inductive logic programming and its connections to neuro-symbolic methods. The recent work on closing the loop between recognition and reasoning Li et al. 2020 introduced a grammar model as a symbolic prior to bridge neural perception and symbolic reasoning alongside a top-down, human-like induction procedure. This work demonstrated that such a combined approach significantly outperforms the conventional reinforcement learning-based baselines. The Microsoft