Pre-Training Meta-Rule Selection Policy for Visual Generative Abductive Learning

Yu Jin, Jingming Liu, Zhexu Luo, Yifei Peng, Ziang Qin, Wang-Zhou Dai, Yao-Xiang Ding, Kun Zhou · 2025

Read Pre-Training Meta-Rule Selection Policy for Visual Generative Abductive Learning by Yu Jin, Jingming Liu, Zhexu Luo, Yifei Peng, Ziang Qin, Wang-Zhou Dai, Yao-Xiang Ding, Kun Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06427.txt.

Introduction

Yu Jin and co-authors speed up a learning system that joins images with logic. Their 2025 paper works on visual generative abductive learning, where a model must induce rules from data while it learns to draw. Picking which rule templates to try eats most of the time when the symbol set is large.

Their fix pre-trains a policy that picks promising templates first, shrinking the search space. Training uses pure symbol data, so it costs little and skips raw images. A side gain surprised the team: the policy can also fix grounding errors it never saw in training. Tests show the full system learns faster with the policy in place.

Catalog record

CREATORYu Jin, Jingming Liu, Zhexu Luo, Yifei Peng, Ziang Qin, Wang-Zhou Dai, Yao-Xiang Ding, Kun Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-06427

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Pre-Training Meta-Rule Selection Policy for Visual Generative Abductive Learning Yu Jin, Jingming Liu, Zhexu Luo, Yifei Peng, Ziang Qin, Wang-Zhou Dai, Yao-Xiang Ding, Kun Zhou Abstract Visual generative abductive learning studies jointly training symbol-grounded neural visual generator and inducing logic rules from data, such that after learning, the visual generation process is guided by the induced logic rules. A major challenge for this task is to reduce the time cost of logic abduction during learning, an essential step when the logic symbol set is large and the logic rule to induce is complicated. To address this challenge, we propose a pre-training method for obtaining meta-rule selection policy for the recently proposed visual generative learning approach AbdGen ( Peng et al., 2023 ) , aiming at significantly reducing the candidate meta-rule set and pruning the search space. The selection model is built based on the embedding representation of both symbol grounding of cases and meta-rules, which can be effectively integrated with both neural model and logic reasoning system. The pre-training process is done on pure symbol data, not involving symbol grounding learning of raw visual inputs, making the entire learning process low-cost. An additional interesting observation is that the selection policy can rectify symbol grounding errors unseen during pre-training, which is resulted from the memorization ability of attention mechanism and the relative stability of symbolic patterns. Experimental results show that our method is able to effectively address the meta-rule selection problem for visual abduction, boosting the efficiency of visual generative abductive learning. Code is available at https://github.com/future-item/metarule-select . 1 Introduction Building neuro-symbolic visual generation model with logic symbol groundings, where the visual generative process is guided by logic rules, has been an emerging research topic recently ( Misino et al., 2022 ; Peng et al., 2023 ) . The symbol-grounded neural visual generator is appearing not only because the generation results are interpretable and stable, but also because they lead to strong rule-based generalization ability and capability of inducing logical generative rules from data. Among the initial researches, AbdGen ( Peng et al., 2023 ) promotes the idea of abductive learning ( Zhou, 2019 ; Dai et al., 2019 ) into visual generative learning, enjoying the advantage of doing neural visual generator learning, symbol grounding, and logic rule induction simultaneously from dataset with limited labeling information for symbol grounding. For further enabling AbdGen to handle large sets of logical symbols and complicated logic generation rules, which is common in visual generation tasks with complicated spacial and temporal dependencies, a major obstacle is the time cost of logical abduction during training. The logic abduction process of AbdGen is done by Prolog-based logic programming system, which is non-differentiable. Even though it is advantageous for its stability and interpretability, the efficiency is an issue especially compared with the neural module, which is differentiable and computationally efficient. The logical abduction process of AbdGen follows the framework proposed in MetaAbd ( Dai and Muggleton, 2020 ) , which involves both symbol grounding and rule induction. The learning process integrates symbol grounding abduction, and meta-interpretative learning ( Muggleton, 2017 ) , such that the induction of the rule is transformed into the search over the possible rule space induced by the meta-rule set. The size of the meta-rule set significantly affects the time cost of rule induction. The smaller the size, the more efficient the abduction process. On the other hand, for supporting inducing general logic rules without too much tweaking and hand-tuning, a general and larger-sized meta-rule set is preferrable. This trade-off is the main technical challenge to deal with for our work. In this paper, we propose a pre-training strategy for learning the meta-rule selection policy aiming at boosting the efficiency of logical abduction during AbdGen training. The pre-trained selection policy is the key to deal with the trade-off between generality and efficiency for meta-rule set. For AbdGen training, a general meta-rule set with complete meta-rules can be kept. Given a training instance which includes positive and negative cases of images without symbol grounding labels, the meta-rule selection policy chooses a small set of meta-rules for inputting into the logical reasoning module, hence boosting the abduction efficiency. Utilizing the embedding representation of symbol groundings in AbdGen model, our selection model further builds embedding representations of meta-rules, and utilizes the attention module for meta-rule selection, which can be effectively integrated with both neural model and logic reasoning system. Furthermore, benefited from the disentanglement of symbol and sub-symbol representations of AbdGen, the meta-rule selection policy can be pre-trained under pure symbolic data, which does not involve symbol grounding learning process, thus is of low time cost. Finally, a potentially technical issue for the pre-trained selection policy is that unseen symbol grounding errors can appear during the AbdGen training stage, especially at the beginning of training. While it is interesting to observe that the pre-trained selection policy has relatively strong toleration ability for the errors. This verifies the memorization ability of attention mechanism as well as the relative stability of symbolic patterns. Experimental results show that our method can be utilized to significantly reduce the time cost of abduction, even when the abductive learning task is unseen during the pre-training stage. 2 Related Work 2.1 Neuro-Symbolic Visual Generative Learning Visual generative learning has received great attention in recent years, in particular multi-modal generation such as text-to-vision generation ( Ramesh et al., 2021 ; Rombach et al., 2022 ; Saharia et al., 2022 ) . Despite the great success, the generation results suffer from hallucination, still lacking strong controllability, interpretability, and stability. One promising solution is to augment the generation process with symbolic modules to utilize their reasoning power instead of solely relying on black-box neural models. One possible solution is fully end-to-end differentiable neuro-symbolic models, which is still not quite common for generative models, but is common for addressing inference tasks such as visual question answering ( Yi et al., 2018 ; Hsu et al., 2024 ) . Such models are relatively efficient due to the fully differentiable properly. On the other hand, the neural part usually needs specific designs to achieve particular functionality. In comparison, traditional logic reasoning models usually enjoys the benefit of generality, while the efficiency is usually a major issue to utilize them in complicated tasks. While in our view, this issue can be addressed by guiding the search-based reasoning process based on the data-driven neural model. The method proposed in this work pursuits this idea. Building neuro-symbolic visual generative model is still an emerging research topic, in special logic-based models. One pioneer work is VAEL ( Misino et al., 2022 ) , which integrates the DeepProblog ( Manhaeve et al., 2018 ) framework with autoencoder visual generators, achieving impressive ability for visual objects generation based on logic rules, as well as strong generalization when the training and testing logic rules differ. On the other hand, AbdGen ( Peng et al., 2023 ) is based on the visual generative abductive learning framework, which enables both symbol grounding and rule induction, two central tasks in neuro-symbolic visual generation. Our work is built on AbdGen due to its strong disentanglement ability of symbolic and sub-symbolic representations and its effective and general logic induction functionality. 2.2 Meta-Rule

Read the complete text