A Foundation Model for Zero-Shot Logical Rule Induction

Yin Jun Phua · 2026

Read A Foundation Model for Zero-Shot Logical Rule Induction by Yin Jun Phua (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04916.txt.

Introduction

Yin Jun Phua introduces a pretrained model that learns logical rules from data and can apply them to new tasks without retraining. Rule learning of this kind, called inductive logic programming, yields rules people can read. Older methods tie their learned settings to specific predicates, so each new task starts over.

His Neural Rule Inducer instead describes each literal by general statistics such as rates, entropy, and co-occurrence, which transfer across tasks. A slot-based decoder builds rules without imposing an arbitrary order, and training runs end to end on prediction accuracy. Tests cover rule recovery, noisy labels, and transfer to real benchmarks, with code and a checkpoint released.

Catalog record

CREATORYin Jun Phua
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.04916

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Foundation Model for Zero-Shot Logical Rule Induction Yin Jun Phua Abstract Inductive Logic Programming (ILP) learns interpretable logical rules from data. Existing methods are transductive: their learned parameters are bound to specific predicates and require retraining for each new task. We introduce Neural Rule Inducer (NRI), a pretrained model for zero-shot rule induction. Rather than encoding literal identities, NRI represents literals using domain-agnostic statistical properties such as class-conditional rates, entropy, and co-occurrence, which generalize across variable identities and counts without retraining. The model consists of a statistical encoder and a parallel slot-based decoder. Parallel decoding preserves the permutation invariance of logical disjunction; an autoregressive decoder would instead impose an arbitrary clause order. Product T-norm relaxation makes rule execution differentiable, allowing end-to-end training on prediction accuracy alone. We evaluate NRI on rule recovery, robustness to label noise and spurious correlations, and zero-shot transfer to real-world benchmarks, and we believe this work opens up the possibility of foundation models for symbolic reasoning. Code and the reference checkpoint are available at https://github.com/phuayj/neural-rule-inducer . An extended version with full appendices is available at https://arxiv.org/abs/2605.04916 . 1 Introduction Inductive Logic Programming (ILP) systems learn interpretable logical rules purely from examples. For example, an ILP system analyzing patient records with symptoms and diagnoses might induce a rule: “ (fever \land cough) \lor (chills \land body_aches) \to flu .” In high-stakes domains like healthcare and finance, black-box predictions are unacceptable. Transparent logical rules like this give us insights that we can actually trust and use. Traditional symbolic ILP methods like 19 ; 26 ; 31 ; 12 ; 20 ; 6 are sensitive to noise and often have to compromise between precision and coverage. Recently, differentiable approaches such as 7 ; 10 ; 13 have been proposed. These methods have proven to be more robust to noise but they are still transductive, where learned weights are tied to specific predicates. A model trained on family relationships ( Parent , Grandparent ) cannot be transferred to biology ( Protein , Enzyme ). This means that for every new dataset, we need to retrain the models from scratch. So how do we determine which variable (boolean attributes like fever , cough ) is predictive of a label (targets like flu )? We select variables by their statistical signatures rather than by name. Because these signatures are invariant to renaming and reordering and each literal is represented by a fixed-dimensional vector independent of N , they offer a plausible route to zero-shot transfer. A high class-conditional rate indicates a variable belongs in a rule, regardless of what it represents. In this paper we test this hypothesis by pretraining on synthetic DNFs and evaluating on held-out real-world tabular tasks without retraining. This approach requires addressing three challenges. First, variable-sized problems : generalization from fixed-size training data to problems with fewer or more variables. Second, inter-variable dependencies : individual statistics miss redundancy and complementarity (e.g., XOR patterns). Third, synthetic-to-real transfer : learning the abstract procedure of induction rather than overfitting to synthetic data. We present Neural Rule Inducer (NRI) , a foundation model 2 that consists of the following: • Literal Statistics Encoder encodes literals by statistical properties such as how often the literal is true and co-occurrences with other literals. • Parallel Slot-Based Decoder. This module synthesizes multiple clauses in parallel using learned slot queries. Compared to autoregressive models, this module preserves the permutation invariance of logical disjunction. • T-Norm Training. We use product t-norm relaxation to execute rules differentiably. This allows end-to-end training without explicit clause supervision. • Synthetic Data Training. We train entirely on randomly generated boolean formulas. This trains the model to recognize the induction procedure itself rather than domain-specific patterns, and removes the limitation on training data size. We show that a model trained entirely on synthetic boolean formulas can perform zero-shot rule induction on diverse real-world benchmarks. Figure 1 illustrates the overall architecture of NRI. While in this work, we focus on boolean variables, the statistical encoding framework can be extended to multi-valued and continuous domains via discretization or fuzzy predicates. Figure 1: Neural Rule Inducer takes an episode (X,Y) as input and calculates literal statistics. For each variable we calculate \phi(x_{i}) , \phi(\neg x_{i}) which consists of class-conditional rates ( P^{+} , P^{-} ), entropy ( H ), and co-occurrence strength ( C ). We then apply cross-attention over these statistics. The slot-based decoder produces K candidate clauses in parallel using learned literal gates z and clause gates w . By evaluating the produced rule with T-norm, we can perform end-to-end training. The rules are discretized to then produce an interpretable DNF rule. 2 Related Works Differentiable ILP. 7 proposed learning rules via gradient descent, but their method requires specifying rule templates and a fixed set of background predicates. NeuralLP 35 and DRUM 29 extended this to knowledge base reasoning using TensorLog. Neural Theorem Provers 28 introduced differentiable unification for end-to-end theorem proving. Logic Tensor Networks 30 use fuzzy semantics to integrate logical constraints into neural learning. DeepProbLog 18 extends probabilistic logic programming with neural predicates, enabling end-to-end learning of both neural and symbolic components. NeurASP 36 integrates neural networks with answer set programming, allowing neural network outputs to serve as probabilistic inputs to logic programs. GLIDR 13 generalized this to graph-like topologies, utilizing differentiable message passing to learn recursive and cyclic dependencies. Most differentiable ILP systems instantiate parameters against a fixed predicate or schema inventory, so transferring to a new dataset typically requires retraining. Classical symbolic systems such as FOLD-R++ 34 , which learns answer-set rules from mixed numerical and categorical data by top-down heuristic search, are similarly task-specific and are re-run from scratch on each new task. Phua and Inoue 25 proposed a model allowing zero-shot transfer learning. They addressed scaling issues by exploiting variable permutation symmetries. Our proposed method differs in mechanism where instead of utilizing the raw examples, we utilize statistical properties to handle missing and noisy data. Generative Neuro-Symbolic AI. LLM-based methods like ILP-CoT 23 and DeepSeek-Prover-V2 27 generate hypotheses by relying on knowledge encoded in language and human concepts. LINC 22 combines language models with first-order logic provers, using LLMs to translate natural language into formal logic for external verification. However, LLMs are not grounded in reality, they may use abstract symbols or notions that might not correspond to any measurable quantity in the real world. In contrast, our approach generates hypotheses directly from observed data and does not assume an explicit semantic model. TabPFN 11 demonstrated that transformers trained on synthetic tabular data can perform in-context learning for classification. While TabPFN produces black-box predictions, our approach outputs interpretable logical rules. 3 Background We want to learn a function f:(\mathcal{X},\mathcal{Y})\to\mathcal{R} that takes input examples \mathcal{X} (boolean variables) and labels \mathcal{Y} and outputs a logical hypothesis \mathcal{R} in Disjunctive Normal Form (DNF). Importantly, f should work regardless of how many variables N there are and what they represent. We should not need to retrain for each new problem. 3.1 Disjunctive Normal Form (DNF) DNF is a disjunction of conjunctions: \mathcal{R}=C_{1}\lor C_{2}\lor\dots\lor C_{K} , where each clause C_{k} is a conjunction of literals: C_{k}=l_{k,1}\land l_{k,2}\land\dots . A literal l_{j} is either a variable x_{i} or its negation \neg x_{i} .

Read the complete text