ARCTraj: A Dataset and Benchmark of Human Reasoning Trajectories for Abstract Problem Solving DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 9–13, 2026; Jeju, Korea ISBN: 979-8-4007-1454-2/2026/08 CCS: Computing methodolo

Sejin Kim, Hayan Choi, Seokki Lee, Sundong Kim · 2025

Read ARCTraj: A Dataset and Benchmark of Human Reasoning Trajectories for Abstract Problem Solving DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 9–13, 2026; Jeju, Korea ISBN: 979-8-4007-1454-2/2026/08 CCS: Computing methodolo by Sejin Kim, Hayan Choi, Seokki Lee, Sundong Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-11079.txt.

Introduction

Sejin Kim, Hayan Choi, and their colleagues present ARCTraj, a data set that records how people actually solve abstract visual puzzles step by step. Most puzzle data show only the start grid and the final grid, which hides the path a solver took. ARCTraj instead logs ordered, object-level actions as solvers turn inputs into outputs through a web interface.

The set holds about ten thousand solution paths across four hundred training tasks, each with task labels, time stamps, and success marks. A shared pipeline supports methods from reinforcement learning to sequence models. Studies of spatial picks, color choices, and strategy narrowing show the range of human reasoning in the set.

Catalog record

CREATORSejin Kim, Hayan Choi, Seokki Lee, Sundong Kim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-11079

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ARCTraj: A Dataset and Benchmark of Human Reasoning Trajectories for Abstract Problem Solving DOI: XXXXXXX.XXXXXXX Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 9–13, 2026; Jeju, Korea ISBN: 979-8-4007-1454-2/2026/08 CCS: Computing methodologies Cognitive science CCS: Computing methodologies Knowledge representation and reasoning Sejin Kim, Hayan Choi, Seokki Lee, Sundong Kim Abstract . We present ARCTraj, a dataset and methodological framework for modeling human reasoning through complex visual tasks in the Abstraction and Reasoning Corpus (ARC). While ARC has inspired extensive research on abstract reasoning, most existing approaches rely on static input-output supervision, which limits insight into how reasoning unfolds over time. ARCTraj addresses this gap by recording temporally ordered, object-level actions that capture how humans iteratively transform inputs into outputs, revealing intermediate reasoning steps that conventional datasets overlook. Collected via the O2ARC web interface, it contains around 10,000 trajectories annotated with task identifiers, timestamps, and success labels across 400 training tasks from the ARC-AGI-1 benchmark. It further defines a unified reasoning pipeline encompassing data collection, action abstraction, Markov decision process (MDP) formulation, and downstream learning, enabling integration with reinforcement learning, generative modeling, and sequence modeling methods such as PPO, World Models, GFlowNets, Diffusion agents, and Decision Transformers. Analyses of spatial selection, color attribution, and strategic convergence highlight the structure and diversity of human reasoning. Together, these contributions position ARCTraj as a structured and interpretable foundation for studying human-like reasoning, advancing explainability, alignment, and generalizable intelligence. 1. Introduction Understanding and modeling how humans reason and solve problems, rather than reproducing only their final answers, remains a central challenge in AI ( 10 ) . Such problem-solving involves conceptual abstraction, attention shifts, and the use of flexible strategies, yet these abilities remain difficult for machines to emulate ( 29 ; 14 ) . The Abstraction and Reasoning Corpus (ARC) ( 9 ) was introduced to benchmark these capabilities through grid-based tasks where solvers must infer rules from a small set of input-output examples. While the benchmark itself is well designed, most ARC research has focused on reproducing outputs rather than modeling reasoning. Early program synthesis approaches ( 3 ; 34 ; 26 ; 30 ; 11 ) struggled to infer solution strategies because the benchmarks provide limited guidance for program construction. To address this, several studies have attempted to extract auxiliary cues from input grids or from intermediate program execution results; however, these approaches have remained task-specific and brittle. More recently, LLM-based methods ( 33 ; 23 ; 28 ) have leveraged prior knowledge to generate diverse candidate programs, showing improved flexibility but still facing the fundamental challenge of an enormous search space without explicit reasoning guidance. In parallel, test-time training approaches ( 2 ; 12 ; 24 ) aim to enable on-the-fly adaptation during inference; however, their learning signals still rely on static supervision rather than the evolving structure of reasoning. As a result, existing ARC approaches (e.g., program synthesis, neuro-symbolic reasoning, and test-time learning) remain limited by their reliance on static supervision and their inability to capture the unfolding of human reasoning over time. To address this limitation, we present ARCTraj , a large-scale dataset of human reasoning trajectories collected while solving ARC tasks. Each trajectory captures a temporally ordered sequence of object-level actions (i.e., moving, rotating, and flipping objects) that transform an input grid into its correct output. By capturing these multi-step trajectories, ARCTraj provides explicit reasoning supervision that connects human perception, abstraction, and strategy formation, offering a structured alternative to unconstrained program search. These logs were collected via the O2ARC web interface ( 31 ) , which is designed to support natural human interaction with ARC problems. Each trajectory is annotated with metadata, including timestamps, task identifiers, and success labels, enabling both learning and analysis. Overview of the ARCTraj data collection process. Users solve ARC tasks through the O2ARC platform by interacting with grid-based objects. Their actions are recorded step-by-step to form semantically rich, temporally ordered trajectories. Figure 1 . Overview of the ARCTraj data collection process. Users solve ARC tasks through the O2ARC platform by interacting with grid-based objects. Their actions are recorded step-by-step to form semantically rich, temporally ordered trajectories. Overview of the ARCTraj data collection process. Users solve ARC tasks through the O2ARC platform by interacting with grid-based objects. Their actions are recorded step-by-step to form semantically rich, temporally ordered trajectories. Compared to existing human ARC datasets, ARCTraj offers several advantages. Whereas H-ARC ( 22 ) captures pixel-level edit logs and the ARC-Interactive-History-Dataset ( 32 ) records low-level cell and operation sequences, ARCTraj provides object-centric actions with consistent formatting and semantic structure across all 400 ARC-AGI-1 training tasks. This object-level abstraction reflects how humans group meaningful visual units and apply conceptual transformations rather than pixel-level edits. Such abstraction preserves strategic intent and eliminates redundant operations, resulting in trajectories that more accurately represent human reasoning. ARCTraj also defines a comprehensive reasoning pipeline that spans data collection, action abstraction, MDP formulation, and downstream learning. This unified framework integrates naturally with reinforcement learning, generative modeling, and sequential reasoning methods that require temporally structured supervision. Thus, ARCTraj serves not only as a dataset but also as a foundation for studying how reasoning behaviors can emerge from human trajectories. Ultimately, ARCTraj shifts the focus of ARC research from reproducing outputs to modeling reasoning itself. By learning from human trajectories, models can acquire structured strategies that reflect System 2 thinking ( 17 ) and generalize across domains, such as program synthesis, robotics, and data transformation. We expect ARCTraj to promote research on transferable reasoning strategies beyond training tasks. In this paper, we first introduce previous research (Sec. 2 ) and present the motivation and design of ARCTraj (Sec. 3 ). We then formalize how ARC solvers learn from human trajectories through a unified reasoning framework (Sec. 4 ). Next, we present auxiliary knowledge extracted from ARCTraj, including selection, color, and intention cues (Sec. 5 ). Finally, we demonstrate their applications in ARC solvers and discuss broader implications for reasoning alignment between humans and AI (Sec. 6 ). In summary, ARCTraj bridges a gap in the ARC domain by providing dynamic and interpretable records of human reasoning. It supports both behavioral studies and the development of cognitively inspired models, serving as a versatile resource for research on abstraction, planning, and generalization. 2. Related Work Abstraction and Reasoning Corpus (ARC) ARC ( 9 ) , also known as ARC-AGI, is a benchmark designed to test human-like generalization in abstract reasoning tasks. Each task consists of a few input–output grid pairs, requiring solvers to induce and apply conceptual transformations from limited examples. It has inspired research across multiple paradigms, including program synthesis ( 4 ; 6 ; 7 ) , neuro-symbolic reasoning ( 5 ; 25 ; 34 ) , and test-time training ( 2 ; 12 ; 24 ) , many of which were featured in the ARC Prize 2024 Technical Report ( 8 ) . However, ARC remains static; it provides only input–output pairs, offering no insight into the intermediate reasoning that drives human problem-solving. As a result, most studies evaluate solution generalization rather than the reasoning process itself. Understanding this process is crucial for modeling human-like abstraction. In this work, we use the term ARC

Read the complete text