GIFARC: Synthetic Dataset for Leveraging Human-Intuitive Analogies to Elevate AI Reasoning

Woochang Sim, Hyunseok Ryu, Kyungmin Choi, Sungwon Han, Sundong Kim · 2025

Read GIFARC: Synthetic Dataset for Leveraging Human-Intuitive Analogies to Elevate AI Reasoning by Woochang Sim, Hyunseok Ryu, Kyungmin Choi, Sungwon Han, Sundong Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20672.txt.

Introduction

Woochang Sim and co-authors present a 2025 arXiv paper on teaching AI systems to reason by analogy on abstract puzzles. The ARC test asks solvers to infer a pattern from a few examples, and leading models still score only forty to fifty-five percent. The team built GIFARC, a synthetic dataset of ARC-style tasks drawn from animated GIF images rich in everyday analogies.

Each task ships with its ground-truth analogy, mapping the visual change to a plain concept. The idea is to make the model name the analogy first, shrinking the search before pattern hunting starts. Experiments showed the guidance steered large models toward human-like analogic solving.

Catalog record

CREATORWoochang Sim, Hyunseok Ryu, Kyungmin Choi, Sungwon Han, Sundong Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-20672

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GIFARC: Synthetic Dataset for Leveraging Human-Intuitive Analogies to Elevate AI Reasoning Woochang Sim, Hyunseok Ryu, Kyungmin Choi, Sungwon Han, Sundong Kim Abstract The Abstraction and Reasoning Corpus (ARC) poses a stringent test of general AI capabilities, requiring solvers to infer abstract patterns from only a handful of examples. Despite substantial progress in deep learning, state-of-the-art models still achieve accuracy rates of merely 40–55% on 2024 ARC Competition, indicative of a significant gap between their performance and human-level reasoning. In this work, we seek to bridge that gap by introducing an analogy-inspired ARC dataset, GIFARC 1 1 1 Full dataset is available on Hugging Face at https://huggingface.co/datasets/DumDev/gif_arc . All 10,000 tasks generated with GIFARC are visualized in https://gifarc.vercel.app/ . . Leveraging large language models (LLMs) and vision-language models (VLMs), we synthesize new ARC-style tasks from a variety of GIF images that include analogies. Each new task is paired with ground-truth analogy, providing an explicit mapping between visual transformations and everyday concepts. By embedding robust human-intuitive analogies into ARC-style tasks, GIFARC guides AI agents to evaluate the task analogically before engaging in brute-force pattern search, thus efficiently reducing problem complexity and build a more concise and human-understandable solution. We empirically validate that guiding LLM with analogic approach with GIFARC affects task-solving approaches of LLMs to align with analogic approach of human. 1 Introduction Recent advances in AI have produced impressive results in specialized domains, ranging from image classification to natural language processing. Despite these achievements, many of these systems remain limited in their capacity for genuinely flexible, general-purpose reasoning – a core characteristic often associated with artificial general intelligence (AGI). In this context, the Abstraction and Reasoning Corpus (ARC) was introduced as a benchmark aiming to measure the ability of artificial general intelligence systems to abstract visual patterns and reason about them in a manner akin to human cognition [ 1 ] . ARC tasks are intentionally designed to challenge AI the ability of high-level abstract reasoning by requiring solutions derived from minimal few-shot examples. This requirement closely mirrors the cognitive flexibility inherent to human task-solving, thus making ARC an essential dataset for probing the fundamental limitations and capabilities of current AI systems in relation to achieving human-level AGI. Despite the excitement surrounding deep learning, current best-performing models in the 2024 ARC Prize competition left a significant gap compared to human-level proficiency [ 2 ] . While traditional deep neural networks typically excel with ample data, they struggle in example-deficient tasks that require understanding. This shortcoming underscores the limitations of purely data-driven approaches, especially when the underlying task involves rich forms of abstract and compositional reasoning [ 3 , 4 ] . As a result, ARC provides strong evidence that deep learning alone does not yet capture the breadth of human reasoning, pointing to critical shortcomings in its capacity for generalization, pattern inference, and systematic task-solving. Figure 1 : Illustration of how differently an agent solves ARC-style task when it is guided with or without analogic approach. In (a), an agent attempts to solve the ARC-style task in brute-force manner and is in risk to result logically complex solutions. In contrast, When the agent has been guided to think analogically with GIFARC dataset, as depicted in (b), it finds out a more concise and human-intuitive solution. Why does this gap exist? One crucial factor lies in how humans leverage analogies to map new visual or conceptual clues onto existing knowledge. When people see a grid transformation as in Figure 1 , they can instantly map this visual pattern onto familiar concepts such as gravity and color change. These priors, accumulated from a lifetime of diverse visual and conceptual experiences, provide immediate insight into the task constraints and sharply reduce the complexity of the tasks. Rather than searching blindly through every possible transformation, humans can reject lengthy or counterintuitive candidates and focus on the small subset of patterns consistent with their analogies. In contrast, deep learning models rarely have an explicit mechanism for analogy-making [ 5 ] . Even advanced neural networks typically rely on statistical correlations learned from large training sets, lacking the means to “tag” or “reframe” new tasks in terms of previously understood scenarios [ 6 ] . Consequently, these models miss a powerful heuristic: an analogy-driven search strategy that narrows down the solution space by linking visual abstractions to real-world events. Need of analogy-contained dataset. To help bridge this gap between human cognition and AI, the agent should obtain human-level amount of analogies in prior to any task-specific training. Since the models we aim to train are predominantly data-driven, the need for training dataset that consists of useful analogies naturally arise. The dataset we seek contains following criteria; (i) ARC-style examples : the dataset should contain analogies in forms of ARC-style examples. Since the agent is evaluated via ARC benchmark, the agent has the world perception in form of 2D grids. In addition, the analogies one needs to know should be expressible onto an ARC-style 2D grid. Thus, the analogy we intend to train should also be expressed and given in ARC-style 2D grid in order for the agents to understand and utilize during evaluation. (ii) Executable codes : In order for the agent to capture how to manipulate 2D grid in accordance to the analogy, analogy should also be expressed in executable codes. Therefore, the analogy of transformation should also be given as lines of executable codes. With those 2D grids and codes, the agent should be able to correlate ARC-style transformation with its underlying analogy and find it during evaluation. In this paper, we propose an analogy-inspired synthetic ARC dataset, GIFARC. Drawing on large language model (LLM) and vision-language model (VLM), we developed a framework to synthesize ARC-style tasks from online GIF images by extracting various visual patterns and the analogies they evoke. These GIFs encompass a range of scenarios (e.g., snowfall, blooming flowers, moving objects), each providing a different type of visual transformation. We encode these transformations in newly generated ARC-style tasks and, crucially, supply ground-truth analogy labels that clarify the underlying conceptual mapping (e.g., “water flow blocked by an obstacle”). Role of GIF as the basis of ARC-style task generation. Using GIFs as the starting ingredient of ARC-style tasks, our GIFARC synthesis pipeline takes great advantages in picking human-understandable and visualizable analogies, as GIFs are series of images that are intentionally drawn to visualize certain ideas. In addition, GIF also has advantages in holding a sufficiently small number of analogies. In terms of analogy extraction simplicity, it is critical for the analogies to be implied in the image series without any entanglement to one another. Longer image series such as videos are more likely to hold excessive number of analogies and consequently have a higher risk of entanglement in between. Thus, GIF is an adequate form of analogy-implying image series. These new ARC-style tasks generated from GIFs will possess a variety of intuitive analogies. We expect that the agent trained with this dataset can gain the grasp of analogical approaches and utilize them in the solving step. By incorporating diverse analogies into the dataset, we hope to facilitate models that can “think analogically” before searching for solutions, thereby reducing the complexity of task-solving. In addition,

Read the complete text