High Quality Embeddings for Horn Logic Reasoning \titletag
Yifan Zhang, Yasir White, Dean Clark, Joseph Sanchez, Jevon Lipsey, Ashely Hirst, Jeff Heflin · 2026
Read High Quality Embeddings for Horn Logic Reasoning \titletag by Yifan Zhang, Yasir White, Dean Clark, Joseph Sanchez, Jevon Lipsey, Ashely Hirst, Jeff Heflin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20467.txt.
Introduction
Zhang and six co-workers improve embeddings for Horn logic reasoning, where networks rank a reasoner's choices to speed search. Embeddings are numeric forms of logical statements, trained here with triplet loss on anchor, positive and negative examples. Their three ideas favor anchors with repeated terms, balance easy, medium and hard examples, and periodically stress the hardest cases. Tests across knowledge bases compare embeddings to find what suits each reasoning task.
Catalog record
| CREATOR | Yifan Zhang, Yasir White, Dean Clark, Joseph Sanchez, Jevon Lipsey, Ashely Hirst, Jeff Heflin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35965 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20467
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
High Quality Embeddings for Horn Logic Reasoning \titletag Yifan Zhang, Yasir White, Dean Clark, Joseph Sanchez, Jevon Lipsey, Ashely Hirst, Jeff Heflin Abstract Neural networks can be trained to rank the choices made by logical reasoners, resulting in more efficient searches for answers. A key step in this process is creating useful embeddings, i.e., numeric representations of logical statements. This paper introduces and evaluates several approaches to creating embeddings that result in better downstream results. We train embeddings using triplet loss, which requires examples consisting of an anchor, a positive example, and a negative example. We introduce three ideas: generating anchors that are more likely to have repeated terms, generating positive and negative examples in a way that ensures a good balance between easy, medium, and hard examples, and periodically emphasizing the hardest examples during training. We conduct several experiments to evaluate this approach, including a comparison of different embeddings across different knowledge bases, in an attempt to identify what characteristics make an embedding well-suited to a particular reasoning task. 1 Introduction Neural networks can be trained to rank the choices made by logical reasoners, resulting in more efficient searches for answers. Several researchers have explored whether machine learning can be used to improve the performance of first-order reasoning, similar to how AlphaGo used deep learning to improve AI game playing ( 11 ) . There have been some promising results, but there is still much to do. We hypothesize that the best way to make progress on the problem is to solve three subproblems: representation, learning strategy, and control strategy. To make the problem more amenable to study, we have restricted our initial investigation to the Horn fragment of first-order logic. This work contributes to a general neurosymbolic approach for logical reasoning that has three key components: 1) An embedding model that maps logical statements to vectors, 2) A scoring model that represents the likelihood that a path leads to a successful answer, and 3) A guided reasoner, where the neural model scores the choices of a backward-chaining reasoner. Our prior work focused on improving the scoring model and guided reasoner; here, we attempt different training methods for the embedding model to learn more useful embeddings and increase the overall performance. This paper extends the work previously presented in a workshop ( 15 ) . In particular, our contributions are 1) Three improvements to the process for learning an embedding model for logical statements, 2) An evaluation of the embeddings’ ability to improve the efficiency of reasoning using a downstream scoring model, and 3) An investigation into what makes a particular embedding useful across a variety of knowledge bases (KBs). 2 Background Neurosymbolic AI seeks “to integrate neural network-based methods with symbolic knowledge-based approaches” ( 10 ) . This can include a broad range of topics, from generating embeddings of knowledge graphs to training a neural network to predict whether one logical statement entails another. One of the earliest attempts to combine logic rules and neural networks was KBANN ( 12 ) . KBANN takes propositional Horn rules and directly encodes them into the neural network. 6 train a neural network that can perform inductive learning on first-order logic data. However, their architecture only allows the input of data representing a conjunction of atoms, and the output is a set of classes. There is no way to incorporate axioms into their reasoning. 7 trained a neural network to perform unification and apply a backward-chaining-like process to predict missing atoms in a KB. There are reasoners that attempt to leverage LLMs. AlphaGeometry generates proofs to solve Olympiad geometric problems ( 13 ) . It uses forward inference to find conclusions from a starting premise, then uses a language model to generate auxiliary points and retries the forward inference if the current proof space fails. Rather than being query-driven, AlphaGeometry deduces new statements exhaustively. ReProver, another LLM-based theorem prover for LeanDojo, selects premises from large math libraries ( 16 ) . ReProver scores the retrieved premises using an LLM. Inherent in using these approaches is having unlimited access to an LLM and the ability to fine-tune it to the needs of reasoning. Other researchers have used various representations for logical statements, particularly in the context of automated theorem proving. Jakubův and Urban use an approach based on term walks of length three ( 4 ) . They parse each logical statement and create a directed graph of it, then extract every sequence of three nodes from the graph and create a code for it. This code has one dimension for each possible sequence; thus for a vocabulary \Sigma , the vector must have {|\Sigma|}^{3} dimensions. This approach does not scale to KBs that have many constants. 2 propose a chain-based approach that starts with a graph similar to Jakubův and Urban, but extracts patterns from a clause that starts with predicates and ends with variables or constants. Each sequence is hashed using MD5, and then that value is further reduced to d dimensions by the modulo operator. Negative clauses are represented by the concatenation of an additional d dimensions. When creating patterns, all variables are replaced by the symbol “*”. This models the semantics that a variable can match with any term, but not that if the same variable appears multiple times in an expression, it must match the same term in every occurrence. Furthermore, the hashing and modulo operator addresses the scalability problem of termwalk, but it does mean statements are placed in latent space at random, as opposed to based on some inherent notion of similarity. 1 proposed that embeddings for logical atoms could be learned in a way that retains the semantics of variables. A core operation used by any first-order logic reasoning algorithm is unification, where \textnormal{Unify}(\alpha,\beta) returns a substitution \sigma such that \alpha\sigma=\beta\sigma or fails. A set of unifying atoms and another set of non-unifying atoms can be found procedurally. Using a neural network that optimizes triplet loss ( 14 ) , these atoms can be mapped into embeddings such that unifying atoms are near, and non-unifying atoms are far away. We describe Arnold and Heflin’s approach and our improvements in subsequent sections. 3 Learning to Improve Logical Inference Backward-chaining is an algorithm for Horn logic inference that operates by starting with a goal and systematically working backward through a series of rules and known facts to determine the conditions required to achieve the goal. Traditional backward chaining reasoners often rely on brute-force to explore potential solutions, which can lead to inefficiencies as the complexity and scale increases. Even relatively small knowledge bases of thousands of statements can result in searches of millions of nodes unless they have been carefully designed by a knowledge engineer. Our prior work looked at learning a scoring model to direct the search along promising paths and compared chainbased, termwalk, and unification as a means for solving this problem ( 5 ) . This work improves on the previous unification approach to learn meaningful embeddings for logical statements. Starting with a knowledge base of facts and rules, we use a forward-chaining reasoner to infer new facts from the existing information. From these new facts, we randomly substitute constants with variables and