On LLM-generated Logic Programs and their Inference Execution Methods

Paul Tarau 0000-0001-7192-9421 · 2025

Read On LLM-generated Logic Programs and their Inference Execution Methods by Paul Tarau 0000-0001-7192-9421 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09209.txt.

Introduction

Paul Tarau draws knowledge out of language models as logic programs in this 2025 paper. Models hold vast knowledge in compressed form from their training data. Logic programs state that knowledge as rules that sound reasoning can check.

Tarau elicits several forms: simple Horn clauses, dual clauses, fact triplets, and grammar rules. He then studies ways to run the programs. One method soft-matches facts against model text stored in a searchable database. Another uses graphics chips to speed up model building for large generated sets. The work aims to verify model output and stretch what it can infer.

Catalog record

CREATORPaul Tarau 0000-0001-7192-9421
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09209

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On LLM-generated Logic Programs and their Inference Execution Methods Paul Tarau 0000-0001-7192-9421 Abstract Large Language Models (LLMs) trained on petabytes of data are highly compressed repositories of a significant proportion of the knowledge accumulated and distilled so far. In this paper we study techniques to elicit this knowledge in the form of several classes of logic programs, including propositional Horn clauses, Dual Horn clauses, relational triplets and Definite Clause Grammars. Exposing this knowledge as logic programs enables sound reasoning methods that can verify alignment of LLM outputs to their intended uses and extend their inference capabilities. We study new execution methods for the generated programs, including soft-unification of abducible facts against LLM-generated content stored in a vector database as well as GPU-based acceleration of minimal model computation that supports inference with large LLM-generated programs. Keywords: LLM-generated logic programs; LLM-generated Definite Clause Grammars; LLM-generated relation graphs; soft-unification with abducible facts; GPU-supported evaluation of propositional Horn clause programs; visualization of LLM-generated relations. \lst@SaveOutputDef ‘_ \underscore@prolog 1 Introduction While the multi-step dialog model initiated by ChatGPT is now available from a few dozen online or locally run open source and closed source LLMs, it does not cover the need to efficiently extract salient information from an LLMs “parameter-memory” that encapsulates in a heavily compressed form the result of training the model on trillions of documents and multimodal data. Steps in this direction have been taken, relying on ground-truth involving additional information sources (e.g., collections of reference documents or use of web search queries). Among them, we mention work on improving performance of Retrieval Augmented Generation (RAG) systems [ 8 ] by recursively embedding, clustering, and summarizing chunks of text for better hierarchical LLM-assisted summarization [ 16 ] , multi-agent hybrid LLM and local computation aggregators [ 4 ] and deductive verifiers of chain of thoughts reasoning [ 10 ] . A more direct approach is recursion on LLM queries, by chaining the LLM’s distilled output as input to a next step and casting its content and interrelations in the form of logic programs, to automate and focus this information extraction with minimal human input [ 19 , 21 ] . Like in the case of typical RAG architectures [ 8 , 16 ] , this process can rely on external ground truth but it can also use new LLM client instances as “oracles” deciding the validity of the synthesized rules or facts. With focus on automation of this unmediated salient knowledge extraction from the LLM’s parameter memory and its encapsulation in the form of synthesized logic programming code, we will extend in this paper the work initiated in [ 19 , 21 ] with: • new LLM input-output chaining mechanisms • new types of generated logic programs • new relational representations elicited from LLM output steps • scalable execution mechanisms that accommodate very large logic programs at deeper recursion levels • soft-unification based execution of LLM-generated logic programs as a principled encapsulation of the RAG retrieval process The rest of the paper is organized as follows. Section 2 overviews the DeepLLM architecture described in [ 21 ] and its new extensions supporting the results in this paper. Section 3 overviews the generation of Horn clause programs with the online DeepLLM app. Section 4 explains the LLM-based generation of Dual Horn clause programs and their uses to explore counterfactual consequences and theory falsification, Section 5 introduces the use of DCG grammars as a representation of LLM-generated answer and follow-up question pairs. Section 6 describes fixpoint and GPU-supported minimal model computation for the generated programs. Section 7 describes relation-extraction and visualization from the minimal models of the LLM-generated propositional programs. Section 8 introduces the soft-unification based encapsulation of the information retrieval against facts extracted from authoritative document collections. Section 9 discusses related work and Section 10 concludes the paper. 2 Recursive exploration of LLM dialog threads Generative AI, with often human-like language skills is shifting focus from typical search engines to more conversational interactions. Yet, the challenge remains that humans must still process and verify this information, an often tedious task. Our answer to this, as implemented in the DeepLLM system is to automate the entire process. We start with a simple “initiator goal” and let the LLM dive recursively in its parametric memory and deliver a detailed answer focused on the initiator and the trace of this chain of steps summarized as the short term-memory maintained via its API. This automation also helps to minimize common issues like inaccuracies, made-up information, and biases that are often associated with LLMs. We refer to [ 21 , 19 ] for details of implementation of the DeepLLM system, as well as to its open-source code 1 1 1 https://github.com/ptarau/recursors/ and its online demo 2 2 2 https://deepllm.streamlit.app . The DeepLLM system’s active components (subclasses of the Agent class) are Interactors, Recursors, and Refiners: • Interactors manage input prompts and task breakdown • Recursors handle iterative exploration of subtasks • Refiners enhance clarity and relevance of LLM responses To validate its reasoning steps, the system also relies on stored knowledge resources: • Ground truth facts: sentences collected from online sources or local documents • Vector store: enabler of “semantic search” via embeddings of sentences Starting from a succinct prompt (typically a nominal phrase or a short sentence describing the task) an Interactor will call the LLM via its API, driven by a Recursor that analyzes the LLM’s responses and activates new LLM queries as it proceeds to refine the information received up to a given depth. Refiners are Recursor subclasses that rely on semantic search in an embeddings store containing ground-truth facts as well as on oracles implemented as specialized Interactors that ask the LLM for advice on deciding the truth of, or the rating of hypotheses. Besides returning a stream of answers, Recursors and Refiners compile their reasoning steps to a propositional Horn clause program available for inspection by the user or subject for execution and analysis with logic programming tools (in particular, with our model builder – a fast propositional Horn clause theorem prover). 3 Generating propositional Horn clause programs with the DeepLLM app Figure 1: DeepLLM app We refer to [ 19 ] for an extensive list of LLM-generated Horn clause programs. We will just briefly describe here the DeeLLM app (see Fig. 1 ) that we will use for generating our logic programs. In the case of the interaction shown in Fig. 1 , the initiator goal “computing stable models of answer set programs” starts the “scientific concept explorer” option and generates in the right side window a Horn clause program describing successive refinements of the initiator goal. The DeeLLM app is written with the Streamlit 3 3 3 https://streamlit.io/ webapp generator and offers the choice between GPT4, GPT3.5 or a local LLM, running as a server and supporting an OpenAI compatible API. It then lets the user choose between the Recursor, Advisor and Rater agents, providing for the latter a threshold level slider. The threshold informs the Rater oracle to accept or reject a generated rule head or fact (the higher the threshold the stricter the accept decision). Options to set the maximum recursion depth and activate relation extraction and visualization are also

Read the complete text