Khai Loong Aw

∗ · 2026

Read Khai Loong Aw by ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10333.txt.

Introduction

Aw and eight colleagues propose the Zero-shot World Model, an idea for children's early grasp of the physical world. Children grasp depth, motion, and wholeness from very little data while applying them to new tasks. The model rests on three ideas: a sparse guesser split by time that splits looks from motion; zero-shot guesses through rough causal thought; and mixing thoughts into more complex skills. It learns from one child's life, rapidly gaining skill across physical benchmarks, with staged rise of powers and brain-like inner pictures.

Catalog record

CREATOR∗
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67003
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10333

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Khai Loong Aw ∗ Abstract Zero-shot World Models Are Developmentally Efficient Learners Khai Loong Aw , Klemen Kotar , Wanhee Lee , Seungwoo Kim , Khaled Jedoui , Rahul Venkatesh , Lilian Naing Chen , Michael C. Frank , Daniel L.K. Yamins Stanford University Corresponding author: [email protected] Abstract. Young children demonstrate early abilities to understand their physical world, estimating depth, motion, object coherence, interactions, and many other aspects of physical scene understanding. Children are both data-efficient and flexible cognitive systems, creating competence despite extremely limited training data, while generalizing to myriad untrained tasks — a major challenge even for today’s best AI systems. Here we introduce a novel computational hypothesis for these abilities, the Zero-shot Visual World Model (ZWM). ZWM is based on three principles: a sparse temporally-factored predictor that decouples appearance from dynamics; zero-shot estimation through approximate causal inference; and composition of inferences to build more complex abilities. We show that ZWM can be learned from the first-person experience of a single child, rapidly generating competence across multiple physical understanding benchmarks. It also broadly recapitulates behavioral signatures of child development and builds brain-like internal representations. Our work presents a blueprint for efficient and flexible learning from human-scale data, advancing both a computational account for children’s early physical understanding and a path toward data-efficient AI systems. Figure 1: Overview. ( A ) The Zero-shot Visual World Model (ZWM) framework has three design principles: temporally-factored prediction to flexibly separate appearance from dynamics; zero-shot extraction of visual-cognitive structures from the predictor through approximate causal inference; and composing extractors together to achieve increasingly complex inference abilities. ( B ) After self-supervised pretraining, ZWM can perform diverse visual-cognitive tasks zero-shot, i.e., without any additional training or examples. ( C ) We train ZWM with varying visual diets and single-child developmental curricula. ( D ) We evaluate BabyZWM’s task performance across training checkpoints (developmental trajectory) and the similarity of its internal representations with brain responses. By early childhood, humans demonstrate rich visual-cognitive abilities, exhibiting strong capacities on a diverse set of physical understanding tasks [ 57 , 4 , 88 , 89 , 13 , 90 ] . The mechanisms behind these abilities are remarkably powerful, in two related but distinct senses. First, they are data-efficient , in that children demonstrate these capacities despite highly limited “training data” afforded by the first-person experience of a single individual. Second, they are flexible , supporting performance of new task abilities from an existing general-purpose representation without task-specific examples (“zero-shot”), e.g., tracking motion, estimating depth, and intuitive physics. Motivated by the sophistication and early emergence of infants’ object and physical knowledge, some researchers have argued that babies have innate biases for visual cognition [ 88 , 89 , 13 , 90 ] . However, “innateness” can mean different things, e.g., the learning machinery (objectives, architectures, programs) or content (representational primitives and concepts). Here we ask: what ingredients are required to achieve data-efficient and flexible (zero-shot) visual cognition from early experience? We approach this question using computational models of visual learning and development. Inspired by neurophysiological observations [ 30 , 66 ] , deep neural networks (DNNs) have emerged as the most task-performant models for visual tasks as well as the most accurate models of neural responses across the visual cortex [ 104 , 42 , 103 , 58 , 11 ] and for human-like error patterns [ 80 ] . Initially, DNNs required supervision on large labeled datasets [ 64 , 22 ] and did not transfer broadly to downstream tasks. These issues motivated a shift to self-supervised models, which learn representations by grouping similar or temporally-proximate images [ 102 , 108 , 15 , 39 , 94 , 14 , 6 , 2 ] . Such self-supervised models also turn out to provide a description of neural and cognitive patterns in the primate visual system, rivaling or exceeding the accuracy of the earlier supervised systems [ 107 , 61 ] . However, from a developmental point of view, modern self-supervised learning methods are a “glass half full”. When trained on the visual data diet of real infants and children, they achieve substantial improvements compared to earlier methods such as predictive coding [ 107 , 75 , 70 ] , but still are far from matching human abilities and perform much worse than similar methods trained on curated, highly non-natural image databases such as ImageNet [ 75 , 107 , 86 , 76 , 69 ] . It has remained very difficult for even the best visual learning algorithms from the AI literature to efficiently extract powerful representations from natural data, perhaps due to camera motion/blur, occlusions, and the relatively low diversity of environments children encounter [ 18 , 19 , 93 , 85 , 105 ] . This ecological data learning gap is also strikingly present for natural language processing, where large language models (LLMs) need large-scale, highly-curated data to achieve linguistic competency [ 99 , 98 , 28 ] . Equally important, while current self-supervised visual models may learn useful representations, they cannot perform tasks directly in the flexible zero-shot manner of humans. Instead, for each downstream task, a separate readout must be trained using labeled data, making the overall pipeline for any given task (e.g., image segmentation) ecologically implausible. By contrast, in the language domain, modern LLMs can perform diverse tasks flexibly and zero-shot – although this behavior only emerges with extremely large amounts of data. The dual challenges of data efficiency and flexibility reflect our incomplete understanding of what inductive structure to build into our models to match human cognition. Here, we report substantial progress on the algorithmic foundations of data efficiency and flexibility, by building the Zero-shot Visual World Model (ZWM), a self-supervised neural network that flexibly performs a broad suite of visual-cognitive tasks zero-shot, i.e., without task-specific training/examples (Figure 1 B). ZWM is based on three key principles (Figure 1 A). First, ZWM’s underlying learned component is a sparse temporally-factored predictor model, a neural network that learns to make predictions given sparse and variable amounts of information. Second, after training, the model can be manipulated to yield a universal zero-shot prompting interface by comparing predictions under ground-truth inputs to predictions under minimally modified inputs – a form of approximate causal inference. Third, ZWM composes simple prompts into more complex queries – for example, simulating hypothetical motions of objects and then computing the resulting optical flow to segment those objects – thus, building a computational graph of visual representations that progressively extracts and integrates increasingly complex structure. Taken together, these components constitute a kind of data-driven world model that is able to forecast the effect of proxy actions on the visual scene. To test the developmental hypothesis of the ZWM framework and probe its data efficiency under naturalistic conditions, we leverage the BabyView dataset [ 69 ] , a set of egocentric video recordings from young children aged \sim 5 months to 5 years. Trained solely on 868 hours of BabyView videos ( \sim 3 months of waking experience), the BabyZWM model performs competitively with supervised, state-of-the-art models on challenging real-world datasets across a wide variety of visual cognitive tasks despite receiving no task-specific supervision or labeled probes. Beyond strong behavioral performance, evaluating

Read the complete text