Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence

Laura Cohen, Xavier Hinaut, Lilyana Petrova, Alexandre Pitti, Syd Reynal, Ichiro Tsuda · 2025

Read Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence by Laura Cohen, Xavier Hinaut, Lilyana Petrova, Alexandre Pitti, Syd Reynal, Ichiro Tsuda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07060.txt.

Introduction

Six authors, led by Laura Cohen, argue in 2025 that the limits of natural intelligence are the secret of its thrift. Babies learn speech, ideas, and movement from thin data on a tight energy budget. Modern AI, by contrast, burns vast power and data to reach its scores.

Narrow brain bandwidth forces short codes that still catch rich patterns, the authors show. Spiking signals, layered codes, and symbol like forms grow from that squeeze. The brain also drifts between passing states to fetch memories, and random projections let it learn fast from few cases. Drive from within and social feedback steer infant learning.

Catalog record

CREATORLaura Cohen, Xavier Hinaut, Lilyana Petrova, Alexandre Pitti, Syd Reynal, Ichiro Tsuda
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-07060

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence Laura Cohen, Xavier Hinaut, Lilyana Petrova, Alexandre Pitti, Syd Reynal, Ichiro Tsuda Abstract Natural intelligence (NI) consistently achieves more with less. Infants learn language, develop abstract concepts, and acquire sensorimotor skills from sparse data, all within tight neural and energy limits. In contrast, today’s AI relies on virtually unlimited computational power, energy, and data to reach high performance. This paper argues that constraints in NI are paradoxically catalysts for efficiency, adaptability, and creativity. We first show how limited neural bandwidth promotes concise codes that still capture complex patterns. Spiking neurons, hierarchical structures, and symbolic-like representations emerge naturally from bandwidth constraints, enabling robust generalization. Next, we discuss “chaotic itinerancy” , illustrating how the brain transits among transient attractors to flexibly retrieve memories and manage uncertainty. We then highlight reservoir computing, where random projections facilitate rapid generalization from small datasets. Drawing on developmental perspectives, we emphasize how intrinsic motivation, along with responsive social environments, drives infant language learning and discovery of meaning. Such active, embodied processes are largely absent in current AI. Finally, we suggest that adopting “less is more” principles – energy constraints, parsimonious architectures, and real-world interaction – can foster the emergence of more efficient, interpretable, and biologically grounded artificial systems. keywords:Energy constraints, Computational Limitations, Efficient Coding, Chaotic Itinerancy, Reservoir Computing, Active Learning, Artificial Intelligence 1 Introduction Physical limitations compel Natural Intelligence (NI) to continually surpass itself in order to survive and avoid collapse. It primarily does so by relying on simple yet effective strategies, allowing biological systems to achieve more with fewer resources. By contrast, modern Artificial Intelligence (AI) faces virtually no restrictions on data volume, time, computational power, or energy consumption, and relies on industrial scale engineering. While this approach has enabled AI to reach human-level performance in areas like language and cognition, it comes at a considerable cost. Moreover, as novel data becomes scarce, AI’s performance also nears a plateau [ JAI24 ] . In contrast, the infant brain transcends its inherent limitations, going beyond iterative learning and slow adaptation, despite the limited computational capabilities of its neurons. Its organizational structure supports rapid, flexible development, enabling it to withstand errors and crises. These are the properties of a complex system. Some of these unique traits of NI are captured by bio-inspired AI, while they are rarely found in classical Machine Learning and generative AI (Transformers). They also exhibit behaviors that are particularly challenging to replicate, such as abstract thinking, free will, intrinsic motivation, compositionality, structure learning, ad-lib interaction/coupling, and embodiment. These features are central to the core knowledge development of infants in language, planning, geometry, counting, music, and critical thinking [ SK07 ] . Here, we propose that Natural Intelligence (NI) leverages ecological principles to cope with limited resources under uncertain conditions. To paraphrase Jean Piaget’s citation, NI is not defined by what one knows, but rather by what one does when they don’t. Moreover, the constraints on capacity, resources, bandwidth, data, and time naturally underscore the need for efficiency and parsimony. This perspective on the limitations of intelligence echoes Fields Medal laureate Steve Smale’s designation of “What are the limits of intelligence, both artificial and human” , one of the key mathematical challenges for the 21st century [ SS98 ] . This work aligns with Smale’s call to explore those boundaries. Hence, in this paper, we seek out computational principles that, while remaining as simple as possible, can still capture the complex, efficient, and emergent behaviors underpinning NI and infant cognitive development, from the neuronal level to the organism as a whole. At the neuron level, we will start by showing how abstraction can be thought, and represented as low dimensional codes (part 1). Then, at the neural population level, we will present the concepts from dynamical systems of chaotic systems (part 2) and of reservoir computing (part 3). We will see how high-level abstractions can be bootstrapped with little data thanks to random projections as used in reservoir computing: indeed, randomly projecting inputs in high dimensional non-linear dynamics can help to bootstrap generalisation (rules) quickly and thus without the need for numerous data. We will show how chaos can be beneficial for memory retrieval in high-dimensional space, and for decision making under uncertainties. Following this, at the body scale, we will explore the mechanisms of active learning used to leverage the infant mind during physical and social interactions, optimizing convergence under high constraints (part 4). Then, in the discussion, we will address an open scope of the paradox and blessing of limitation. We argue that limitation is an intrinsic component of NI. It is remarkable that computers, digital processing and Information Theory started on the basis of limits of mathematical logic (Turing), computational resources (von Neumann) and physicality of channel communication (Shannon) to produce fantastic results. Current AI may reintroduce this principle almost forgotten into its roots, and refocus into NI, biology and physics. 2 Dealing with limited capacity, temporal constraints, and under uncertainty The way the human brain performs explicit cognitive processing stems from fundamental limitations 1 1 1 https://theconversation.com/chatgpt-ma-dit-que-lillusion-de-la-discussion-avec-lia-nous-mene-a-lerreur-238443 .First, we have a limitation in computing time: we sometimes need to adapt quickly to new situations. To do this, we have developed rapid learning capabilities whereby one or two examples are enough for us to learn continuously, to adapt to a new situation or to retain some of its characteristics. While the processing time of current AI (Large-Language models) appears very fast, it relies on heavy resources to make it work fast using parallel processing, and the massive learning stage can last for weeks.Second, we have a limitation in computing power. Our brain has major energy constraints that drastically limit our working memory, that is to say the number of information that we can process at the same time, or the operating frequency of our neurons. While modern computers, ever more efficient and faster, are breaking records in terms of computing power and storage capacity. The way our cognition has found to deal with this problem is to learn to break down complex problems into simpler subproblems to solve (such as memorizing intermediate steps instead of directly learning a complicated path), which has led us to organize our behavior in time and our knowledge into levels of abstraction. Current AI does not have this type of constraint, for example synthesizing thousands of texts. Efficient coding in the brain. Accordingly, the human brain must develop efficient strategies in order to transmit, preserve or exploit information continuously, robustly, and despite limitations of its neural material. Thus, instead of seeing limited computational capabilities as a problem or a constraint for the brain, we can think of it more as a design principle for parsimony and simplicity, much like the Occam razor principle: a heuristic to seek for minimalistic models with the least number of elements. This view is in line with Horace Barlow who proposed that the human brain exploits efficient codes to maximize information capacity of neurons, despite their intrinsic unreliabilities [ Bar61 ] . Taken isolatedly, the computational performance of neurons is very poor, but at the population level, their interaction rises their performance at a high level. As proposed

Read the complete text