Online library learning in human visual puzzle solving
Pinzhe Zhao ([email protected]), Emanuele Sansone, Marta Kryven, Bonan Zhao * · 2026
Read Online library learning in human visual puzzle solving by Pinzhe Zhao ([email protected]), Emanuele Sansone, Marta Kryven, Bonan Zhao * (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23244.txt.
Introduction
Zhao and three co-authors study how people build reusable helpers while solving visual puzzles. In their online experiment, players defined helper constructions that capture repeating structure. Early on, players made many helpers and favored completeness over speed. With practice, helper use grew more selective and cheaper. Helpers let players crack puzzles that were otherwise too hard or impossible. Modeling backs the story: a program induction model with library learning tracks decision times and effort well. Raw program length predicts failure instead.
Catalog record
| CREATOR | Pinzhe Zhao ([email protected]), Emanuele Sansone, Marta Kryven, Bonan Zhao * |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25782 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Online library learning in human visual puzzle solving Pinzhe Zhao ([email protected]), Emanuele Sansone, Marta Kryven, Bonan Zhao * Abstract When learning a novel complex task, people often form efficient reusable abstractions that simplify future work, despite uncertainty about the future. We study this process in a visual puzzle task where participants define and reuse helpers—intermediate constructions that capture repeating structure. In an online experiment, participants solved puzzles of increasing difficulty. Early on, they created many helpers, favoring completeness over efficiency. With experience, helper use became more selective and efficient, reflecting sensitivity to reuse and cost. Access to helpers enabled participants to solve puzzles that were otherwise difficult or impossible. Computational modeling shows that human decision times and number of operations used to complete a puzzle increase with search space estimated by a program induction model with library learning. In contrast, raw program length predicts failure but not effort. Together, these results point to online library learning as a core mechanism in human problem solving, allowing people to flexibly build, refine, and reuse abstractions as task demands grow. Keywords: compositional abstraction; online learning; library learning; problem-solving; program induction; computational modeling Introduction From knitting a sweater to building a cathedral, people often tackle complex tasks by creating and manipulating reusable structural units. While these units—intermediate representations or procedures—help manage cognitive load and enable the otherwise intractable solutions ( 13 ; 31 ; 12 ) , the decision of which units to build is surprisingly understudied ( 15 ) . Future demands are often uncertain, and constructing a new unit may incur costs in time, memory, and effort ( 29 ; 21 ) . If capturing the distribution of these units can be understood as a form of library learning ( 10 ) , then a central challenge to cognition is library learning but online: how do people build reusable abstractions on the fly? Studying online library learning poses several methodological challenges. First, the space of possible helpers is often large and open-ended, making it difficult to characterize meaningful abstractions ( 23 ) . Second, helper creation trades off expressivity against efficiency: too many helpers streamline individual solutions while increasing the cost of maintaining and selecting among them ( 29 ) . Third, behavioral tasks must be rich enough to elicit abstraction, yet structured enough to allow systematic analysis of learning trajectories. In this paper, we address these challenges using a novel visual puzzle paradigm. In this paradigm, participants can build and reuse geometric patterns while solving progressively more complex tasks, allowing us to study abstraction creation and reuse on the fly. We first motivate the task and its computational relevance, and then report an exploratory behavioral study examining how abstraction evolves over time. To foreshadow, we found that human cognitive computations scale with the size of the underlying search space of a program induction model with online library learning, but can not be predicted by the raw number of primitives encoding a given puzzle, suggesting that human problem solving relies on online learning of adaptive structural abstractions. Background Abstraction in problem solving has been studied extensively across multiple computational frameworks (e.g. 13 ; 12 ; 24 ) . Here we focus on program induction approaches applied to visual puzzles. Problem solving as program induction A common computational perspective on puzzle-solving is to formalize this problem as program induction – the process of inferring a program that generates the target solution from sparse input–output examples ( 28 ; 5 ; 17 ) . These programs specify generative compositional structures, capturing key properties of human cognition such as abstraction, hierarchy, and relational reasoning ( 12 ; 14 ) , across domains such as compositional reasoning ( 17 ) , few-shot learning ( 19 ; 18 ; 30 ) , and recently, geometric puzzles ( 16 ) . Despite their broad successes, program induction models depend critically on choice of representations and priors, raising the question of how representations themselves are learned, reused, and adapted across tasks. Human chess players, for example, rely on effective representations that organize experience into chunks to substantially reduce search ( 13 ) . By contrast, many existing program induction models assume fixed hand-specified representations ( 19 ; 11 ) , limiting their ability to scale as search spaces grow exponentially with task complexity. Library learning In response to these challenges, library learning in program induction models addresses how learners incrementally acquire reusable abstractions that support efficient problem solving across tasks ( 10 ) . At the computational level of analysis ( 20 ) , inducing a library that captures recurring structure in the environment enables compact and generalizable representations. At the algorithmic level, libraries compress future search by biasing it toward previously useful subroutines, reshaping the hypothesis space over time ( 31 ) . This perspective has been used to model abstraction ( 23 ) , skill acquisition ( 26 ) , and curriculum effects ( 16 ) , providing a formal account of how experience across tasks can accumulate into structured knowledge rather than isolated solutions. However, most existing library learning models rely on offline optimization, repeatedly re-solving all tasks to evaluate candidate libraries ( 10 ; 3 ; 6 ) . While such procedures are well suited for finding the computational-level solutions, they contrast sharply with the demands of everyday cognition: people rarely encounter identical tasks multiple times, and must operate under uncertainty about future tasks, making global optimization infeasible. Instead, individual cognition must address the library learning problem online , as experience unfolds. Visual puzzles To investigate online learning of complex structured representations, we adopt an experimental paradigm of a visual puzzle task. Visual puzzles provide a rich testbed for studying human problem solving because they combine perceptual reasoning, planning, and abstraction. Prior work has shown that such tasks reveal how people identify patterns, sequence actions, and transfer knowledge across related problems ( 7 ; 16 ) . They are especially useful for studying stepwise reasoning and the emergence of intermediate strategies, as they make the process-level solution observable and tractable for behavioral analysis ( 8 ) . Figure 1: Task interface. A. Starting view of a trial. Left: Total points so far and the target shape for this trial. Middle: Work space, matching the target shape with the provided Operations and Primitive shapes. Right: List of steps, each step corresponds to one line of program. Black borders are only added in the paper for illustration. B. Example preview. C. Example programs. Each line has a line number, a thumbnail of the pattern that line creates, and the corresponding program (operations over primitives, steps, or helpers). D. The initial helper space. E. Helper space with example helpers. Behavioral Paradigm: Pattern Builder We create a novel Pattern Builder Task (PBT) where people create progressively more complex visual patterns with access to creating reusable helpers that carry over to future tasks. Building patterns by composing programs In the PBT, participants see a target pattern on a 10\times 10 grid on the left side of the screen, and an empty canvas on the right (Figure 1 A). The goal is to recreate the target using a small set of geometric and transformation primitives, including five shapes: line_horizontal , line_vertical