Hippocampo-neocortical interaction as compressive retrieval-augmented generation Authors: Eleanor Spens, Neil Burgess Published: 2026 ABSTRACT AbstractMany aspects of learning, memory, and problem solving involve interplay between episodic (hippocampal) and semantic (neocortical) systems, but the neural mechanisms supporting this are unclear. We present a computational model in which sequential experiences are encoded in hippocampus in compressed form and replayed to train a neocortical generative network. This network captures the gist of specific episodes and extracts statistical patterns that generalise to new situations, enabling efficient reconstruction of the past and prediction of the future. The two systems interact during encoding, recall and problem solving, with the hippocampus retrieving relevant episodic information into working memory as a basis for generation using the ‘general knowledge’ of the neocortical network. We simulate this interaction as ‘retrieval-augmented generation’, with the addition of mechanisms to compress episodic memories into hippocampus and to consolidate them into neocortex. The model explains changes to memories over time, including schema-based distortions, and shows how episodic and semantic memory contribute to problem solving.Subject terms: Cognitive neuroscience, Human behaviour, Computational neuroscience Introduction Despite the success of the theorised distinction between episodic (hippocampal) and semantic (neocortical) systems, and long-standing ideas regarding the consolidation of memories from hippocampus to neocortex1–4, how both systems work together to support a wide variety of cognitive functions is unclear. These include recollection and problem solving, which rely on a variable mixture of specific episodic memories from the hippocampus and more general knowledge from the neocortex5,6. Here, we lay out a framework for how this co-operation occurs, and make use of recent advances in machine learning to simulate how the resulting system would perform these functions for comparison with human performance. This builds on many years of theoretical insights into the relationship between schemas and memory7–12, and their potential correspondence to large language models13,14. We explore three questions, and implement our hypothesised solutions as a computational model (see Table 1): How does episodic memory influence semantic memory? We hypothesise that a deep generative model in the neocortex is trained to predict its own inputs, using memories of episodic experiences replayed from the hippocampus (and during the experiences themselves). In this way, it develops generalisable (semantic) knowledge, enabling it to generate episodes from an initial input. With enough consolidation, the neocortex can approximately reconstruct memories without the hippocampus, allowing the redundant hippocampal trace to fade away. How does episodic memory influence semantic memory? We hypothesise that a deep generative model in the neocortex is trained to predict its own inputs, using memories of episodic experiences replayed from the hippocampus (and during the experiences themselves). In this way, it develops generalisable (semantic) knowledge, enabling it to generate episodes from an initial input. With enough consolidation, the neocortex can approximately reconstruct memories without the hippocampus, allowing the redundant hippocampal trace to fade away. How does semantic memory influence episodic memory? We hypothesise that the hippocampus, which must use neocortical representations for input and output, stores episodic memories in a highly compressed form, as a minimal subset of information from which the generative model can reconstruct each episode, combining both general schemas and episode-unique features. How does semantic memory influence episodic memory? We hypothesise that the hippocampus, which must use neocortical representations for input and output, stores episodic memories in a highly compressed form, as a minimal subset of information from which the generative model can reconstruct each episode, combining both general schemas and episode-unique features. How do semantic and episodic memory interact during recall and problem solving? We hypothesise that the hippocampus and neocortex work together to solve problems, with the hippocampus providing specific episodes and the neocortex generating predictions from semantic memory informed by these episodes. In other words, episodic contributions to problem solving correspond to the hippocampus retrieving relevant memories on which to base neocortical generation in working memory. Simply recalling a memory also requires neocortical predictions to reconstruct an event from its compressed hippocampal trace. How do semantic and episodic memory interact during recall and problem solving? We hypothesise that the hippocampus and neocortex work together to solve problems, with the hippocampus providing specific episodes and the neocortex generating predictions from semantic memory informed by these episodes. In other words, episodic contributions to problem solving correspond to the hippocampus retrieving relevant memories on which to base neocortical generation in working memory. Simply recalling a memory also requires neocortical predictions to reconstruct an event from its compressed hippocampal trace. Table 1Three questions about the interaction between episodic memory (EM), semantic memory (SM), and working memory (WM), our hypothesised solutions, how we implement our hypotheses computationally using large language models (LLMs) and retrieval-augmented generation (RAG), the relevant results, and relevant previous modelsQuestionOur hypothesisImplementation in modelSimulationsRelevant existing modelsHow does episodic memory influence semantic memory?SM develops through the consolidation of EM.General knowledge can be derived by prediction error minimisation on specific memories.Consolidation mechanism: reconstructed sequences from the model hippocampus are used to train the generative network representing the neocortex.The gist of events is consolidated into the model neocortex (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’ and ‘Modelling the interplay of semantic and episodic memory in consolidation’).Non-linguistic tasks can also be learned (‘Neocortical learning of sequential structure for problem solving’).Many models stemming from Complementary Learning Systems account3: SM gradually learned from EM.Consolidation of EM regulated for SM generalisation (e.g., ref. 30).Ref. 27 models consolidation as training of a generative network on replayed EM.How does semantic memory influence episodic memory?EM is compressed based on SM, with a minimal subset of neocortical representations indexed for approximate reconstruction of an event by the neocortex.The consolidation mechanism produces an ‘approximation of approximation’.Compression mechanism: each EM is stored as a gist vector, plus residual poorly predicted elements, from which the LLM approximately reconstructs the sequence via RAG.Consolidation mechanism: LLM is trained on memory reconstructed from gist and details, not the original sequence.Recall of recent memories as RAG (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’).The model displays a similar pattern of memory distortions to human data.Context maintenance and retrieval model115, building on ref. 110: semantic context encoded in hippocampus.Ref. 46 models EM encoding as ‘lossy compression’ using SM, explaining gist-based distortions.Refs. 43 and 27 models for simple non-sequential stimuli: EMs as latent code plus unpredictable details.Tolman-Eichenbaum machine22 for relational inference: new entities bound to positions in a familiar graph.SLIMM framework8: surprising EM content encoded more in hippocampus, while SM-congruent EM better remembered by cortex, thus a U-shaped relationship between SM-congruence and memory.Context compression LLM literature, e.g., ref. 33.How do semantic and episodic memory interact during recall and problem solving?EM and SM combine in working memory to solve problems.EM is retrieved from the hippocampus (‘in context’ memory), while SM makes predictions based on this (‘in weights’ memory).Retrieval-augmented generation (RAG): EM as in-context sequences retrieved associatively from the hippocampus, SM as in-weights information in the neocortex.Recall of recent memories as RAG (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’ and ‘Modelling the interplay of semantic and episodic memory in consolidation’).Episodic contributions to problem solving as RAG (‘Neocortical learning of sequential structure for problem solving’).Episodic contributions to reinforcement learning116: memories for specific episodes are combined with a general policy.Tolman-Eichenbaum machine22 for relational inference: solving new problems combines learned graph structure with arbitrary specifics.Ref. 13 reviews parallels between LLMs with external memory and human cognition.EM-LLM93 and others draw inspiration from episodic memory to help LLMs handle long contexts.Ref. 117 and others incorporate associative memory for ‘episodic’ information into the LLM itself. Three questions about the interaction between episodic memory (EM), semantic memory (SM), and working memory (WM), our hypothesised solutions, how we implement our hypotheses computationally using large language models (LLMs) and retrieval-augmented generation (RAG), the relevant results, and relevant previous models How does episodic memory influence semantic memory? SM develops through the consolidation of EM. General knowledge can be derived by prediction error minimisation on specific memories. The gist of events is consolidated into the model neocortex (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’ and ‘Modelling the interplay of semantic and episodic memory in consolidation’). Non-linguistic tasks can also be learned (‘Neocortical learning of sequential structure for problem solving’). Many models stemming from Complementary Learning Systems account3: SM gradually learned from EM. Consolidation of EM regulated for SM generalisation (e.g., ref. 30). Ref. 27 models consolidation as training of a generative network on replayed EM. How does semantic memory influence episodic memory? EM is compressed based on SM, with a minimal subset of neocortical representations indexed for approximate reconstruction of an event by the neocortex. The consolidation mechanism produces an ‘approximation of approximation’. Compression mechanism: each EM is stored as a gist vector, plus residual poorly predicted elements, from which the LLM approximately reconstructs the sequence via RAG. Consolidation mechanism: LLM is trained on memory reconstructed from gist and details, not the original sequence. Recall of recent memories as RAG (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’). The model displays a similar pattern of memory distortions to human data. Context maintenance and retrieval model115, building on ref. 110: semantic context encoded in hippocampus. Ref. 46 models EM encoding as ‘lossy compression’ using SM, explaining gist-based distortions. Refs. 43 and 27 models for simple non-sequential stimuli: EMs as latent code plus unpredictable details. Tolman-Eichenbaum machine22 for relational inference: new entities bound to positions in a familiar graph. SLIMM framework8: surprising EM content encoded more in hippocampus, while SM-congruent EM better remembered by cortex, thus a U-shaped relationship between SM-congruence and memory. Context compression LLM literature, e.g., ref. 33. How do semantic and episodic memory interact during recall and problem solving? EM and SM combine in working memory to solve problems. EM is retrieved from the hippocampus (‘in context’ memory), while SM makes predictions based on this (‘in weights’ memory). Recall of recent memories as RAG (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’ and ‘Modelling the interplay of semantic and episodic memory in consolidation’). Episodic contributions to problem solving as RAG (‘Neocortical learning of sequential structure for problem solving’). Episodic contributions to reinforcement learning116: memories for specific episodes are combined with a general policy. Tolman-Eichenbaum machine22 for relational inference: solving new problems combines learned graph structure with arbitrary specifics. Ref. 13 reviews parallels between LLMs with external memory and human cognition. EM-LLM93 and others draw inspiration from episodic memory to help LLMs handle long contexts. Ref. 117 and others incorporate associative memory for ‘episodic’ information into the LLM itself. We implement these ideas in a computational model in which compressed hippocampal memories relevant to a task are retrieved into working memory as context for a neocortical generative network, which is trained on replayed hippocampal memories through consolidation. This enables the generative network to develop generalisable semantic knowledge which supports problem solving, to reconstruct episodes efficiently from minimal hippocampal traces, and eventually to capture the facts of specific events. Our model reflects the complex interplay between episodic and semantic memory; semantic memory is acquired by learning to reconstruct episodic memories, but also shapes the representation of these episodes, both in terms of their initial encoding and their gradual schematisation. Here, working memory refers to the temporary retention of a limited amount of information15, including episodic memories retrieved from long-term memory into the ‘episodic buffer’16, which can accommodate considerably more than the four or so items that can be attended to at a given moment16,17. This is the ‘workspace’ into which episodic memories relevant to a task are retrieved from the hippocampus, upon which the neocortical network operates. We first consider simple narratives representing autobiographical episodes, and demonstrate how they are (i) encoded in compressed form in the hippocampus, (ii) retrieved and decompressed by the neocortical generative model, and (iii) consolidated into the neocortical generative model. We show that the model predicts changes to the content of memory over time, such as greater abstraction and the loss of specific detail, as seen in human memory18,19. We also show that the compression aspect of the model enables efficient storage and reconstruction, suggesting how rich and complex events can be stored over a lifetime without exceeding the limited hippocampal capacity. Next, we show how encoding and consolidation don’t just result in loss of detail, but systematically distort memories towards expectations in our model, matching experimental data20,21. Finally, we show how consolidation develops generalisable knowledge in the neocortical network22, allowing problem solving by combining this with relevant specifics, which are loaded into working memory from the hippocampus or ongoing experience. Our model works with both linguistic and non-linguistic stimuli, and is thus applicable to sequential experience of many kinds. Implementing the model via compressive retrieval-augmented generation Here, we use transformer-based large language models23 to simulate how neocortical networks combine with the hippocampus to extract semantic knowledge from sequential experiences, and to support memory and problem solving. Machine learning has shown how deep neural networks can act as task-general ‘foundation models’24. In particular, large language models (LLMs) demonstrate that complex sequential behaviours can develop as a byproduct of a simple ‘next item prediction’ task23,25, with distributed semantic representations slowly learned over time, in a way analogous to the development of semantic memory in neocortex26. (See ‘Large language models’ for further details.) Here we suggest that memory consolidation corresponds to the development of neural ‘foundation models’ in a similar way, with neocortex learning to reconstruct replayed memories via schemas (see ref. 27). Crucially, LLMs can memorise specific sequences as well as learning generalities28, meaning that both event-unique information and patterns across events can be captured within the same network. We model consolidation as the training of autoregressive sequence models, simulated using Mistral 7B29 and GPT-223, on sequences which represent replayed hippocampal memories. Consolidation can thus be thought of as teacher-student learning (see ref. 30). The objective during training is to predict the next item in sequences from the training data. Once trained, the network can continue an input sequence, or generate a new sequence from scratch, by iteratively predicting the next item from the items in the ‘prompt’ or ‘context’, which models the content of working memory. The hippocampus is modelled simply using modern Hopfield networks31, storing states which can be retrieved autoassociatively as well as the one-step transitions between them. Whilst not the focus of this paper, this illustrates the hypothesis that hippocampus and neocortex use very different mechanisms for sequence memory, though both bind together neocortical representations. We propose that the hippocampal ‘memory bank’ and neocortical generative network work together to enable problem solving, with the hippocampus retrieving memories relevant to a new task into working memory as context for the generative network. Specifically, we draw inspiration from retrieval-augmented generation (RAG32), which involves prompting LLMs with relevant data retrieved from external memory, as a model of drawing on relevant memories to inform predictions in a new situation (see also ref. 13). A typical task for RAG might be question answering based on a set of new facts which do not feature in the training data for the LLM. To ask the system a question, first the most relevant items are found in the external memory (using some similarity metric). A ‘prompt’ is then constructed, in which the retrieved items are given to the LLM together with the question (see Fig. 1). Summary of model. a Episodic memories are first encoded in the hippocampus, then replayed during rest to train a generative network (simulated by an LLM), which thereby acquires semantic knowledge. The hippocampus and neocortex interact to play complementary roles during problem solving, resembling ‘retrieval-augmented generation’ (RAG32). Relevant hippocampal traces are retrieved into working memory and used to ‘prompt’ the neocortical network. The hippocampus is modelled with modern Hopfield networks31 storing states and one-step transitions between them. b Sequences are encoded in the hippocampus in compressed form. Specifically, a vector from which the sequence can be approximately decoded by the neocortex is stored for each event, together with the subsequences that are most surprising given this compressed version of the event. c The neocortical and hippocampal networks work together to recall memories and solve problems via RAG, e.g., when encountering a new task which requires inferences to be drawn based on hippocampal memories. d Memories are replayed from the hippocampus during rest, and the generative network captures their statistics through prediction error minimisation. e RAG for LLMs. When the generative network (neocortex) is prompted with a query, related information is retrieved from external memory (hippocampus). The retrieval process finds stored information with similar vectors (produced by a ‘text embedding model’) to the query. A combined prompt is then created from the query and the retrieved information, improving the output from the generative network. f RAG with context compression using xRAG33. A small network, trained with the LLM, maps vectors from the text embedding model into a single token representation (a token is the atomic unit an LLM operates over, i.e., a ‘chunk’ of characters), which is combined with the tokens of the query to form the prompt for the LLM. Created in BioRender. Spens, E. (2026): https://BioRender.com/jvhli8u. We suggest that hippocampal-neocortical interaction resembles RAG, but crucially with additional compression and consolidation mechanisms. Memories in the hippocampus are stored in a compact conceptual form, as in the LLM literature on context compression (e.g., ref. 33). Compression is a natural consequence of schema learning; schemas develop in the neocortex over time, and thus feature in hippocampal traces which bind together neocortical representations. Consolidation then gradually trains the neocortical generative network on memories from the hippocampus. The conceptual representations in the hippocampus result from compressing experience in a way that is optimal for efficient reconstruction of memories by the neocortex, and all recall of hippocampal traces is retrieval-augmented generation, requiring the generative network to fill in the outline of a memory with its predictions. The ‘context’ (or ‘prompt’) provided to an LLM can be thought of as its ‘working memory’34, with the limited context length of an LLM representing the limited capacity of biological working memory. Note that the neocortical model’s outputs could be prompted by perception rather than memory using the same mechanism. Previous work There is a long history of examining how short, medium, and long-term stores support encoding, consolidation, and recall across the ‘lifespan’ of a memory (e.g., ref. 35). Crucially, semantic memory is thought to be extracted from episodic memory through consolidation2,3,36, learning statistical relationships from experience to aid prediction and thus survival37. But episodic memories also feature conceptual representations from semantic memory38,39; to use the terminology of Piaget40, the encoding of memories using existing schemas reflects ‘assimilation’, while the updating of schemas from memories reflects ‘accommodation’. Both semantic and episodic memory can interact within working memory when reconstructing what happened in the past or imagining what will happen in the future41, with episodic memory contributing specifics and semantic memory contributing generalisable knowledge42. The conception of memory as (re)constructed rather than veridical20 highlights the potential of recently developed deep generative networks to shed new light on these questions. The idea that memories are rapidly encoded by the hippocampus and then replayed over the course of systems consolidation to train a generative or predictive model of the world in the neocortex has received much recent interest. The combined system is thought to support multiple cognitive functions including episodic memory, semantic memory, imagination, and inference27,30,43–45. This provides a mechanistic account of the theory that episodic memories are reconstructions that are influenced by our beliefs, i.e., that recall involves ‘predicting’ the past46–48. However, previous work has focused on static patterns, whereas episodes are sequential, and arguably the most crucial function of semantic knowledge involves the prediction of what will come next. Memory helps to predict the future, not just recall the past37. Planning often requires anticipating the outcome of a possible action, e.g., the sequence of future states and rewards that would follow it. The mental simulation of future states to aid decision-making is known as ‘model-based’ planning49,50, which improves with consolidation51. We suggest that the generative network trained on memories can simulate events to support behaviour, such as when deciding between possible actions, or when inferring the next state given the sequence so far (see also ref. 52). Results In ‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’, we explore the compression, encoding, recall, consolidation, and forgetting of narrative events. We show how each event can be efficiently encoded in the hippocampus as a conceptual code plus surprising details, from which the neocortex reconstructs the event during recall via ‘retrieval-augmented generation’, with these reconstructions gradually consolidated into the neocortex, where interference from new consolidation produces forgetting. We show from simulations how this produces the changes observed in human memory over time. In ‘Modelling the interplay of semantic and episodic memory in consolidation’, we explore how the model captures the influence of semantic memory on episodic memory, and show that characteristic distortions towards the ‘priors’ of the network arise as a consequence, which captures gist-based distortions seen in human memory. In ‘Neocortical learning of sequential structure for problem solving’, we show that consolidation of memories into the neocortical network enables learning of shared structure, which can be used to solve new problems. We simulate human behaviour in two non-linguistic relational inference tasks, involving the prediction of new spatial or family relationships. We also show that human-like problem solving is supported by the mapping of new tasks to a learned representation in the hidden layers. We model the hippocampal network as a sequential variant of the modern Hopfield network (MHN) for recalling state n+1 given state n31,53, coupled with an MHN for retrieving state n from a noisy or partial version. Given a state as a cue, recall is simulated by finding the nearest stored state with the latter MHN and retrieving the rest of the sequence with the former. For further details, see ‘Modelling the hippocampus’. Replay is simulated by randomly sampling from the set of memory units corresponding to the first state of the sequence, and retrieving the rest of it. Sequences are repeated many times during training, reflecting offline replay rather than online learning from a single exposure. (Note that after confirming that the model hippocampus can recapitulate sequences in each task, we avoid unnecessary computational cost by using these sequences directly, unless stated otherwise below.) Modelling hippocampal-neocortical interaction as retrieval-augmented generation We begin by simulating the compression, encoding, and consolidation of memories, and the differing mechanisms for recall over the course of a memory’s lifespan. The compression and encoding of episodic memory In our model, memories are compressed for efficient storage54,55. Neocortex captures the gist of the episode, which is stored in the hippocampus together with unpredictable details. The stored hippocampal gist vector and additional details are unpacked by the neocortex into a full description of the episode during recall. This explains not only the presence of conceptual representations in the hippocampus38, but also the fact that memories are schematised from the time of encoding56,57. Retrieval-augmented generation is used to reconstruct memories or solve new problems based on this compressed information, with hippocampal traces retrieved based on relevance to the current task or prompt, and loaded into the context of the generative network. This minimises the hippocampal storage of redundant information that is already well-predicted by the generative network (see also ref. 27). In other words, this hippocampal representation allows the neocortical network to (re)generate experiences in a more efficient way, and the conceptual representations are optimised to enable generation. We simulate this by following recent LLM research on ‘context compression’, aimed at mitigating the practical problems of dealing with very long prompts by transforming them into vectors which capture their meanings33,58. Here we use the Mistral-7B-Instruct-v0.229 LLM to represent the neocortical network, together with pre-trained weights from ref. 33 to transform episodes into compressed xRAG vectors. The small compression network loosely represents the neocortex’s ability to perform dimensionality reduction59 and produce gist-like representations of events60 for hippocampal storage. This is one of several approaches to context compression which learn a compact vector representation of text (see also ref. 58), and does not require any special training of the LLM itself - just additional weights that map the vector representation of a sequence (i.e., the output of a ‘text embedding’ model) to a token in the input layer of the LLM (see Fig. 1f). Note that tokens are chunks of co-occurring characters that make up the basic units processed by an LLM. To illustrate how gist, detail, and semantic memory interact, we simulate the encoding and recall of narrative events (Fig. 2), using 500 stories from the ROC Stories dataset61. To encode a story in compressed form, its embedding is obtained from the text embedding model62, from which the xRAG vector is derived via the xRAG weights. Each story is encoded as its xRAG vector plus unpredictable details in the model hippocampus. To identify unpredictable details, all phrases are extracted from the text, then the perplexity (a measure of prediction error) of each phrase given the xRAG vector is calculated. In other words, for each phrase, the model is prompted with the xRAG vector and the phrase in question. Then the phrases are ranked in descending order of perplexity, with the top n phrases stored as the unpredictable details. See ‘The compression and encoding of episodic memory’ for further details. A model of hippocampal-neocortical interactions as retrieval-augmented generation with stories. a The neocortex generates a gist vector for the narrative, which is encoded in the hippocampus together with unpredictable details. b Given a query as the input to recall, the neocortex searches the hippocampus for relevant traces, e.g., by finding nearby gist vectors. c The generative network in the neocortex produces an ‘answer’ conditioned on the retrieved hippocampal trace(s). d Once a story is fully consolidated, the hippocampal trace is no longer needed to approximately reconstruct the event. e The stages of encoding and recall are shown for an example story from the ROC Stories dataset61, with Mistral-7B-Instruct-v0.2 used to simulate the neocortex, and xRAG vectors33 representing the conceptual gist. f–i Schematic of our model’s predictions compared to alternative baselines (including hippocampal forgetting, which is not simulated in the main results): memory is distorted from the outset, with the recalled version decreasing in similarity to the original experience (f) and increasing in similarity to priors (g). Compressed encoding within the hippocampus, through use of neocortical representations, explains distortions in newly encoded memories, and greater compression for more typical events, (h), while consolidation into the neocortex explains the increasing similarity to prior experience over time. Neocortex and hippocampus collaborate in problem solving, with neocortical contributions increasing and hippocampal contributions decreasing with consolidation (i). Created in BioRender. Spens, E. (2026): https://BioRender.com/e2y8plp. Given a query as the input to recall, the neocortex ‘searches’ the hippocampus for relevant traces. This is simulated by producing an xRAG vector for the query (as described above), and then by retrieving the trace(s) with the nearest xRAG vector(s) from the model hippocampus (see Supplementary Fig. S2, SI). The context given to the LLM consists of the xRAG vector(s), the details, and a prompt describing the task. Note that the xRAG vector only occupies the space of a single token in this input. The final recalled story is the output of the LLM. Figure 2e and Supplementary Table S1, SI, show examples of encoded and recalled stories. For instance, a story about a girl’s visit to an aquarium is encoded as its gist vector and a surprising detail (that the girl took a nap on the journey). The model neocortex reconstructs an approximation of the original story from the gist vector and the detail, with a few semantic intrusions added to the story, e.g., ‘Melody’s favourite part was the shark tank’; these intrusions are shaped by the expectations of the model neocortex, as in ref. 20. This hints at how gist, detail, and semantic memory can interact in the model during recall. The encoding of gist plus details was also compared to three baselines: encoding the story in full detail, encoding only the gist, and simply imagining the continuation (i.e., without using the model hippocampus at all). The mean size per memory (in tokens) and the accuracy of recall (the cosine similarity between the embeddings of the original and recalled story) were compared across these conditions, showing a trade-off between efficiency and accuracy in memory in Fig. 3a. The gist plus details encoding strikes a balance between these objectives. Encoding, consolidation and forgetting of narratives (ROC stories61). a The size of the memory trace stored in the model hippocampus (in tokens) and recall accuracy (cosine similarity between original and recalled stories’ embeddings with ‘all-MiniLM-L6-v2’118) for five variants. From left to right: simply imagining the event based on background knowledge, storing only the gist vector, storing the gist vector with one surprising detail, storing the gist vector with three surprising details, and storing the event in full. In (a–c) data are mean ± SEM across n = 500 stories. b–d How neocortical representations change over time. b The cosine similarity in embedding space between the neocortical memory and the original story (red), and between the neocortical memory and the replayed version (blue), which is itself a reconstruction of the compressed encoding. As the story is consolidated, the neocortex’s reconstruction moves closer to the encoded story in embedding space. c As new memories are consolidated into neocortex, forgetting occurs (we assume the hippocampal trace has faded away). Over the course of new memory consolidation (50 new memories per stage), the neocortex’s reconstruction moves away from the encoded story in embedding space. d Semantic memory accuracy (fraction of 50 questions for which the model’s answer is most similar to the correct answer) before and after consolidation. Source data are provided as a Source Data file. Modelling consolidation and forgetting In our model, sequential memories are initially encoded in the hippocampus in compressed form, reconstructed by the neocortex through ‘retrieval-augmented generation’32, and then gradually consolidated into a generative model in the neocortex, through prediction error minimisation on the replayed memories. This provides a mechanism for the training of neural ‘foundation models’24, which can both capture specific memories and learn more general schematic knowledge to enable problem solving. 500 stories from the ROC Stories dataset61 were encoded in the model hippocampus as described above (here in gist form without additional details for greater interpretability). To simulate consolidation, the memories were reconstructed from these traces, and the generative network was trained on the result (using low-rank adaptation63). As the story is consolidated, the neocortex’s reconstruction moves closer to the encoded story in embedding space (Fig. 3b), but distortions in the reconstruction are still observed. (See ‘Modelling the interplay of semantic and episodic memory in consolidation’ for a more extensive exploration of distortions in our model.) We also tested semantic memory for the content of the narratives by asking factual questions about 50 stories from the dataset, e.g., ‘Remember the story in which ‘Jane had recently gotten a new job.’ Why did Jane arrive late on her first day?’ Each prompt contains enough context to refer to a single relevant story, and the question has an answer in that story. Semantic memory is quantified as the fraction of questions for which the output is more similar to the correct answer than two alternatives (based on cosine similarity in embedding space). Performance increases with consolidation, showing that the network does not simply overfit to the exact text but can abstract semantic information from the story (Fig. 3d and Table 2). Table 2Examples of episodic and semantic memory over timea) Gist-based encoding, consolidation and forgetting of episodic memoryStageRecalled textOriginalJane had recently gotten a new job. She was nervous about her first day of work. On the first day of work, Jane overslept. Jane arrived at work an hour late. Jane did not make a good impression at her new job.Encoded (gist vector only)Jane had recently gotten a new job, but on her first day, she arrived late. She had overslept and missed her alarm. She then had trouble finding the office building and got lost. When she finally arrived, she was flustered and disorganised....Consolidation (early)Jane had recently gotten a new job. She was very excited about it and had been working hard to learn the new skills required for the job. However, she was also very nervous about the new job and was worried about making mistakes....Consolidation (late)Jane had recently gotten a new job. She was excited about the new opportunity and was looking forward to starting. However, on her first day, she arrived late. She had overslept and missed her alarm. She then had trouble finding the office...Forgetting (early)Jane had recently gotten a new job. She was very excited about it because it was a promotion and she had been working very hard to get it. She had been working long hours and had been very dedicated to her job. She had also been very ambitious...Forgetting (late)Jane had recently gotten a new job. She was working for a living now. She was saving up to buy a house. She knew that it would be a long process. She was determined to save as much as possible. She didn’t touch her savings...b) Semantic memory before and after consolidationQuestionRelevant storyCorrect answerBefore consolidationAfter consolidationRemember the story in which ‘Jane had recently gotten a new job.’ Why did Jane arrive late on her first day?Jane had recently gotten a new job. She was nervous about her first day of work. On the first day of work, Jane overslept. Jane arrived at work an hour late. Jane did not make a good impression at her new job.oversleptTraffic delay. Without additional context, this is a common reason for arriving late on the first day of a new job.She overslept.Remember the story in which ‘Ava was six years old.’ What did Ava want to learn to do?Ava was six years old. She wanted to learn to swim. Her mom took her to the pool. There, she patiently taught Ava the basics. Soon, Ava was paddling all by herself!swimAva wanted to learn to ride a bike.Ava wanted to learn to swim.Remember the story in which ‘Eric and his wife had a daughter named Meg.’ What happened to Eric’s wife?Eric and his wife had a daughter named Meg. Eric’s wife passed away. Eric and Meg were very sad. Eric met a woman and married her 5 years after his wife died. Meg is now happy with her new stepmother.passed awayThe story does not provide information on what happened to Eric’s wife.She passed away.a) Text for a randomly selected story across six stages: the original story from the ROC Stories61 dataset, the memory reconstructed from the compressed hippocampal trace, the neocortical memory early and late in consolidation, and the neocortical memory early and late in forgetting. The initially encoded hippocampal memory produces gist-based distortions, and the neocortex gradually learns this event through consolidation, further distorting it. The neocortical memory gradually fades as newly encoded memories interfere. In this simulation, only the gist (xRAG) vector is stored in the model hippocampus at initial encoding, increasing the bias towards prior experience, but in the full model, surprising details would also be encoded.b) Neocortical semantic memory for particular events is acquired through consolidation. Neocortex-only answers for selected semantic memory questions before and after ten epochs of consolidation of 500 ROC stories61. The model neocortex is prompted with the ‘Question’. Its answer is marked as correct if it is more similar to the ‘Correct answer’, as stated in the ‘Relevant story’, than the two incorrect answers (as measured by the similarity of their text embeddings). Note that in the full model, semantic memory would rely on RAG before consolidation, with the relevant event retrieved into the context from the hippocampus, but here only the neocortical memory is shown. Examples of episodic and semantic memory over time a) Text for a randomly selected story across six stages: the original story from the ROC Stories61 dataset, the memory reconstructed from the compressed hippocampal trace, the neocortical memory early and late in consolidation, and the neocortical memory early and late in forgetting. The initially encoded hippocampal memory produces gist-based distortions, and the neocortex gradually learns this event through consolidation, further distorting it. The neocortical memory gradually fades as newly encoded memories interfere. In this simulation, only the gist (xRAG) vector is stored in the model hippocampus at initial encoding, increasing the bias towards prior experience, but in the full model, surprising details would also be encoded. b) Neocortical semantic memory for particular events is acquired through consolidation. Neocortex-only answers for selected semantic memory questions before and after ten epochs of consolidation of 500 ROC stories61. The model neocortex is prompted with the ‘Question’. Its answer is marked as correct if it is more similar to the ‘Correct answer’, as stated in the ‘Relevant story’, than the two incorrect answers (as measured by the similarity of their text embeddings). Note that in the full model, semantic memory would rely on RAG before consolidation, with the relevant event retrieved into the context from the hippocampus, but here only the neocortical memory is shown. To simulate forgetting in the neocortical network, we tested the recall of the consolidated memories as the network was trained on new stories (representing the consolidation of new memories, in the absence of hippocampal reminders of the old memories). Over successive stages of new memory consolidation, the neocortex’s reconstruction moves away from the encoded story in embedding space (Fig. 3c). See Table 2 for an example of the same event’s neocortical reconstruction accuracy improving during consolidation and declining during forgetting. Putting these results together suggests an account of the inaccuracy of human episodic memory. To start with, the encoded version of an event is already highly compressed, relying on schemas from the neocortex. Then, as consolidation progresses, memory becomes less dependent on the hippocampal trace and more dependent on the neocortical reconstruction. We assume that the hippocampal trace is no longer maintained once consolidated into the neocortex, with consolidation continuing until the trace can be approximately reconstructed by the neocortex, rather than until it can be perfectly reconstructed. The consolidated version is therefore even less accurate than the encoded version. In addition, once fully consolidated into the neocortical model, forgetting occurs due to interference. Changes in memory content over time The HIPPOCORPUS dataset18 contains participants’ written descriptions of recalled autobiographical events, with a subset of the recalled events retold months later. We analysed it to track how concrete vs. abstract, how rich vs. poor in detail, and how specific vs. general memories were for more recent and more remote episodic memories. We then compared the original, encoded, and consolidated versions of stories in our simulations using the same metrics. This was achieved by using an LLM as an annotator (gpt-4o-mini via the OpenAI API). Figure 4d shows that the more remote memories are less concrete, rich in detail, and specific, in line with many similar findings. For comparison, we simulated the process of encoding and consolidation in our model with the ROC Stories dataset61 as described above, because the HIPPOCORPUS dataset does not contain ‘ground truth’ descriptions of the remembered events. Figure 4a shows that the encoded memories are significantly less concrete, detailed, and specific than the original stories, and the consolidated stories score even lower on these attributes. In Supplementary Tables S3 and S4, SI, we show that reduction in these attributes does not simply reflect less memory being recalled by controlling for length, confirming that there are still significant differences. In addition, Fig. 4e, g show that more commonly used words occur in the remembered stories. Furthermore, memory concreteness, richness, and specificity decline as forgetting proceeds (Fig. 4b). How memory content changes over time. a Memory concreteness, richness in detail, and specificity (as scored by gpt-4o-mini) for the original ROC Stories dataset61, the memories based on the compressed hippocampal encoding, and the memories reconstructed by the neocortex alone after consolidation. b Memory concreteness, richness, and specificity decline as forgetting proceeds. c As in (a) but for original and immediately recalled stories in the Naturalistic Free Recall Dataset (NFRD19). d As in (a) but experimental data for ‘recalled’ stories, and the same stories ‘retold’ at a later date from the HIPPOCORPUS dataset18. e The mean word frequency (across English language text) for words in the original, pre-consolidation, and post-consolidation stories. f Likewise, but experimental data for the NFRD texts. g The distributions of word frequencies for the categories in (e), with means shown as dotted lines and medians and interquartile ranges shown as bars, showing increased use of high frequency words after encoding and consolidation. h As in (g) but experimental data for the NFRD texts. Sample sizes: part a, n = 500 ROC Stories per stage; part b, n = 500 stories per forgetting stage; part c, n = 4 original NFRD stories and n = 458 recalled NFRD transcripts; part d, n = 2779 recalled, and n = 1319 retold HIPPOCORPUS memories; part e, n = 500 stories per stage; part f, n = 4 original NFRD stories and n = 458 recalled NFRD transcripts. Significance annotations denote two-sided tests: ns, p≥0.05; *, p< 0.05; **, p< 0.01; ***, p< 0.001. Part a uses paired t-tests by story item; parts c and f use one-sample t-tests on recall-minus-matched-original NFRD changes; parts d and e use Welch two-sample t-tests. Error bars throughout give the SEM. Source data are provided as a Source Data file. The Naturalistic Free Recall Dataset (NFRD19) consists of transcripts for four stories recalled by more than one hundred participants each, immediately after listening to the story. We repeated the analysis above for the original and recalled stories for comparison to the original and encoded versions in our simulation, observing a similar decrease in those attributes (Fig. 4c), and furthermore, the use of more common words in the recalled NFRD stories (Fig. 4f, h). (We note that schematisation is not the only explanation for vocabulary changes; even with a perfect memory for an event, one might paraphrase its content in more familiar language.) Modelling the interplay of semantic and episodic memory in consolidation We have sketched out how narrative memories change over time, but now we explore the effect of existing schemas on memory in more depth. Here, we investigate how the model distorts memories towards expectations, consistent with human data. Gist-based distortions and prior experience In the Bartlett experiment20, students heard a story called ‘The War of the Ghosts’ and were asked to recall it after different time intervals. The recalled story was distorted to be more consistent with the students’ background knowledge of the world, with distortion increasing over time64 (Fig. 5b). To simulate encoding, the story was compressed as described above (here with no poorly predicted details for greater interpretability). To simulate consolidation, Mistral 7B was then trained on its own reconstruction of the encoded story. Recall of the story was explored by giving the network its first few words (‘One night, two young men from Egulac’), and inspecting the predicted continuation. The consolidated story displayed even more distortion than the encoded version (Fig. 5f), suggesting that consolidation of a compressed memory is an ‘approximation of an approximation’. The effect of prior knowledge on narrative distortions. a The neocortical network (pre-trained Mistral 7B) is fine-tuned on Wikipedia content65 from each of five categories plus the Bartlett20 story (with the original rather than compressed version ‘replayed’ to isolate the effect of consolidation in parts a, c–e, and g). The embeddings of the training data plus the recalled stories for each model (100 sampled per topic, at epoch five and a temperature of 0.5) are obtained, using ‘all-MiniLM-L6-v2’118. These embeddings are projected into 2D with PCA, showing that recall moves each story towards that network’s training data. b Data reproduced from the Bergman and Roediger64 replication of Bartlett20 showing that the fraction of recalled propositions in the story with major distortions (defined as showing semantic influences) increases at greater delays in a repeated reproduction experiment. Error bars give the SEM in parts b-f. c The cosine distance (1 - cosine similarity) between the mean embedding for each category and the original story (red) and recalled stories (pink), at epoch five and a temperature of 0.5. Recalled stories become more similar to the background dataset. 100 recalled stories are sampled. d The effect of temperature on the number of new words in the recalled Bartlett stories (‘semantic intrusions’). Results are for the model at epoch ten. e The effect of epoch on the number of new words in the recalled Bartlett stories, where an epoch is one presentation of the story. Results use a temperature of 0.5. f The mean cosine distance between the recalled and original stories before and after consolidation (at a temperature of 0.5). Ten data points are overlaid per condition. g Word clouds showing ‘semantic intrusions’ at epoch 5 and temperature 1.0 (10,000 samples per topic), created with the ‘wordcloud’ Python library. h Event extension and contraction results, showing omission errors and extension errors after short (left) and long (right) retention intervals. Data were replotted from raw data provided by the authors of Raykov et al.21 with permission, with n = 56 short delay and n = 84 long delay participants per story condition. i The proportion of stories (100 per type) recalled with omission and extension errors, both before consolidation (short delay) and after (long delay). In parts (h) and (i), error bars give the 95% confidence intervals of the mean. Source data are provided as a Source Data file. To explore the effect of the model’s ‘priors’ (i.e., its expectations based on previous learning) on recall of narratives, the Bartlett story20 was consolidated after pre-training on ‘background data’. A dataset of Wikipedia content65 was used, with five categories of articles selected (‘Politics’, ‘Health’, ‘Universe’, ‘Sport’, and ‘Nature’). The training data for each model was made up of 1000 articles sampled from the relevant category (with the first 5000 characters of each article taken). The model was trained for 10 epochs on this dataset, where each epoch is one full presentation of the training set. Next, the Bartlett story was consolidated into the network in the same way (with the original story used for training to isolate the impact of consolidation rather than encoding). The temperature for sampling continuations from the model was also varied between 0 and 1.5 (see Section S1.1.2, SI). When the Bartlett story is ‘consolidated’ into the generative network, memory distortions are observed, as in the human data (see Table 3). Distortions in recalled stories reflect the ‘priors’ of the generative network. The word clouds in Fig. 5g show that new words added to the story (i.e., ‘semantic intrusions’) are representative of the background dataset used, and Fig. 5a and c show that the recalled stories move closer towards the background dataset in text embedding space. See also Table 3 for selected examples of semantic intrusions from the five models. Whilst distortion occurs even with ‘greedy decoding’, more ‘semantic intrusions’ are observed at higher temperatures, as Fig. 5d and Supplementary Table S5, SI, show. Table 3Examples of the effects of prior knowledge on narrative distortiona) Semantic intrusions in consolidated memory reflect priorsNatureA party of fifteen men had gone out in a canoe to hunt caribou, and a sudden storm had broken their canoe to pieces.They also gathered fruits and berries from the forest, and they fished in the river for fresh fish.[They] set up camp in a meadow halfway between the two lakes.HealthThe incident ended with the serious sickness and eventual death of the injured man.The doctor bandaged the man’s wound and gave him some medicine for the pain.The perpetrators... were eventually arrested and convicted of murder.SportThey were brave men, and when they were alive, they were good hunters and warriors.The party proceeded to the field, and the men played football. This time, the Egulac men won.The hunter took up his bow and arrow and crept out into the darkness.PoliticsThey sneak into the enemy village, and the man from Egulac kills the enemy chief.The men were therefore acquitted of the chargeThey travelled to the chief’s tribe and found him seated by the firesideUniverseThey took shelter under a hill, and as the lightning flashed, they saw canoes coming over the water.While they were there, it became foggy and calm as the moon rose.After the creation of the earth itself, the god brought forth the sun, the moon, the stars, and all animate and inanimate things.b) Event extension and contraction examples before consolidationTypeOriginalRecalledTypicalJohn worked at a law firm. They decided not to make him a partner. He decided to quit. He went to work for a new firm. The new firm made him a partner!John worked at a law firm. John was not made a partner at his old firm, so he quit and went to work for a new firm. The new firm made him a partner.IncompleteJake wanted to become an awesome guitar player. He bought a guitar and started watching instructional videos. After weeks of practice, he applied to audition for a band. He was nervous, but he played several songs for theJake wanted to become an awesome guitar player. Jake auditioned for a local band and got accepted. He continued to practice and improve his skills, and eventually became an awesome guitar player.UpdatedEmily had gotten a new dog. The dog ran out of the front yard. Emily was scared of losing the dog. She ran after it. Emily started letting the dog out on a leash. She felt a lot of peer pressure since they bought a lot ofEmily had gotten a new dog. Emily let the dog out of the front yard, and it ran away. She chased after it and eventually found it.a) Extracts from retrievals of the Bartlett20 story for networks trained on different categories of prior knowledge (temperature = 1.0), showing how semantic intrusions reflect the priors of the generative network.b) Event extension and contraction examples before consolidation, showing the effect of encoding with compression. The selected incomplete and updated stories (which are of the same length) display extension and omission errors, respectively. The model is prompted with the first 100 characters of each story. Examples of the effects of prior knowledge on narrative distortion a) Extracts from retrievals of the Bartlett20 story for networks trained on different categories of prior knowledge (temperature = 1.0), showing how semantic intrusions reflect the priors of the generative network. b) Event extension and contraction examples before consolidation, showing the effect of encoding with compression. The selected incomplete and updated stories (which are of the same length) display extension and omission errors, respectively. The model is prompted with the first 100 characters of each story. Event extension and contraction Distortions can apply to the structure as well as the content of narratives. Event extension is the tendency to extend certain events in memory, while event contraction is the tendency to curtail them. Raykov et al.21 demonstrate this with three types of video: complete videos end at a natural event boundary, incomplete videos are curtailed before this point, and updated videos are extended beyond it. They find that incomplete videos are often extended in memory (extension errors), while updated videos are often shortened (omission errors), and show that this is observed at long and short delays (Fig. 5h). We simulate the effect of consolidation in ref. 21, using simple stories in text form66. The majority are ‘typical’ stories, unmodified from the ROC Stories66 dataset, whereas an ‘incomplete’ story has characters removed, and an ‘updated’ story has characters added, so it continues beyond its natural ending. The data consisted of 100 typical stories, 100 incomplete stories, and 100 updated stories, where ‘updated’ and ‘incomplete’ stories are all the same length. First, the stories were encoded in compressed form as described above, and pre-consolidation recall was tested by giving the first 100 characters of each story as a query for retrieval-augmented generation. Then the reconstructions of these compressed memories were replayed to simulate consolidation; Mistral 7B29 was fine-tuned on the reconstructed stories for three epochs, and its recall tested. Both before and after consolidation, omission errors are more common in updated than incomplete stories, while extension errors are more common in incomplete than updated ones (Fig. 5i). By bringing the story to a more natural conclusion or by removing an incongruous ending, the stories are made more similar to the network’s expectations. See Table 3 for examples. This mirrors findings on boundary extension and contraction in memory for images67,68; in both cases, memories are distorted towards a prior encoded in the network by its previous ‘experience’, consistent with Bayesian views of memory27,47. Neocortical learning of sequential structure for problem solving Next, we investigate how consolidation develops generalisable knowledge which supports problem-solving. To enable simulations of training from scratch, we use a smaller LLM architecture, GPT-2 Medium23, and simplify the model by storing memories in full rather than compressed form. The ability to anticipate future observations improves with consolidation69–71, leading to downstream improvements on a diverse range of tasks even as perceptual details fade. We demonstrate how the neocortical network can extract sequential structure from memories through consolidation using the example of structural inference, which also the model’s application to non-linguistic stimuli. A spatial example of structural inference is the finding of shortcuts, as this relies on the common structure of space, and a non-spatial example is inferring that A is the grandfather of C from the knowledge that A is the father of B, and B is the father of C, which relies on the common structure of family trees. The relations in these tasks can be seen as edges in graphs, so that multiple tasks with a common transition structure can be simulated using a fixed graph with different nodes for each task22. We simulate how consolidation enables structural inference by the neocortical network. We consider inference in two types of graph: a spatial graph and a simple family tree graph (as in ref. 22). We trained GPT-2 models on random walks on these graphs, corresponding to sequences of observations encoded in the hippocampus, and then tested the models’ inference abilities on novel sequences with the same underlying structure. In the spatial graph, a three-by-three grid represents a simple 2D environment, where the nine nodes are locations and the edges between them (‘NORTH’, ‘EAST’, ‘SOUTH’ and ‘WEST’) are possible movements (Fig. 6a). Whilst each graph’s structure is the same, nodes are labelled with names (random pairs of letters) to represent arbitrary features at a particular location. Trajectories through the environment are walks on the resulting directed graph, which are represented as strings such as ‘ab EAST wd SOUTH ea WEST hn’. (See graph transformers for an alternative approach72.) Learning structural regularities. a, b The spatial graph task. a Spatial trajectories are modelled as walks on the graph, and represented as strings. b Structural inference is tested as the ability to complete sequences on unseen graphs based on structural regularities, e.g., inferring that the next location in the sequence shown is ‘jn’. c, d As above, but for the family tree graph task. e The training and validation losses (i.e., aggregated prediction errors) of GPT-2 trained for one epoch on sequences from 100,000 spatial graphs. f As above, but for the family tree graph task. g Inference, as shown in parts b and d, was tested using sequences from novel environments / family trees in which the final entity can be inferred from the sequence so far. Performance is grouped by the number of transitions. Mean accuracy across ten tasks (see overlaid points) is shown; the accuracy of each task is computed in 100 random graphs. Error bars in g, h and k give the SEM. h Generalisation to different grid sizes. The spatial inference model - trained on paths in a 3 x 3 grid of locations - was tested on a range of square grid sizes (from 2 x 2 to 6 x 6), and generalises beyond its training data. 400 tests are used per grid size. i The fraction of ‘imagined’ spatial trajectories (as generated by the spatial inference model when given a random location as an input) which are structurally valid, at a range of temperatures. A sequence is marked as invalid if different locations are found at the same inferred co-ordinates (e.g., ‘sz WEST eq EAST zr’). j RAG applied to an inference task. Relevant sequences are retrieved from the hippocampus and then provided as input to the generative network together with the new task. k Inference accuracy for the spatial and family tree inference tasks with RAG, in both a single-memory and multi-memory mode, and a hippocampus (HPC) only and neocortex (NC) only baseline for comparison. Each bar gives the mean across 200 tasks. Source data are provided as a Source Data file. The family tree graph has a simple structure for illustrative purposes, consisting of two children, their parents, and two sets of grandparents. See Fig. 6c. We model this as a directed graph with edges ‘PARENT_OF’, ‘CHILD_OF’, ‘SPOUSE_OF’, ‘SIBLING_OF’, ‘GRANDPARENT_OF’, and ‘GRANDCHILD_OF’. As in the spatial case, all graphs have the same structure, but each graph has different names assigned to its nodes. Walks on the graph are represented by strings such as ‘lk PARENT_OF nd SIBLING_OF re’. In each case, we created 100,000 graphs with the same structure but randomly chosen labels (pairs of letters) for the nodes. 20 random walks, of a number of transitions sampled from between 1 and 50, were gathered from each graph to create the training data, representing sequences of observations that might be experienced, encoded in the hippocampus, then replayed offline. GPT-2’s medium-sized architecture was then trained from scratch for one epoch (i.e., the full dataset was presented once). After training, we tested inference of the next node given the sequence so far, using greedy decoding of the output probabilities. For example, the next item after ‘uq NORTH sx EAST tp SOUTH ec WEST’ can be inferred to be ‘uq’, and the next item after ‘qk PARENT_OF xm PARENT_OF vw GRANDCHILD_OF zq SPOUSE_OF’ can be inferred to be ‘qk’. The sequences used for testing were sequences of relationships in new family trees or spatial grids. Figure 6e,f show the ‘loss’ (aggregated error on the training data) of the spatial and family tree models, respectively. In both cases, the loss gradually decreases, indicating improved ability to predict the next node on the set of graphs used for training, which corresponds to the consolidation of previous experience. Supplementary Tables S6 and S7, SI, and Fig. 6g show good performance on a range of novel structural inference tasks, involving sets of labels representing environments which have not been experienced before. Simpler inferences include inferring that going one step west then east takes one back to the original location, e.g., the correct continuation of ‘ab EAST cd WEST’ is ‘ab’. Similarly, a correct continuation of ‘ab PARENT_OF cd CHILD_OF’ is ‘ab’. But surprisingly complex inferences are also possible, e.g., that ‘kh CHILD_OF oi SPOUSE_OF tv CHILD_OF fh SPOUSE_OF xr GRANDPARENT_OF gq SIBLING_OF’ is followed by ‘kh’. Furthermore, given a single new item, the trained spatial model can ‘imagine’ new trajectories consistent with the learned graph structure (Fig. 6i). We also tested inference on larger grids than the model was trained on to show that inference cannot solely be attributed to ‘rote memory’ of sequences of transitions (Fig. 6h). Inference from memory as retrieval-augmented generation Episodic memories can be combined with general knowledge to support problem-solving. Next, we simulate how the generative network and hippocampal network could work together to achieve this via retrieval-augmented generation (RAG). In the simulations above, inference was tested as the continuation of a single new sequence in working memory. However, working memory is limited16,17, so the brain must selectively recall relevant experiences. In addition, problem solving can synthesise information from more than one episodic memory retrieved from the hippocampus; multiple fragments of information encoded at different times can be recombined with semantic knowledge, e.g., when inferring that A is the sibling of C from the observation on one day that A is the child of B, and on another that B is the parent of C. We create a ‘toy example’ of retrieval-augmented generation using the two models in the structural inference results above, and consider a task in which multiple memories, from recent tasks which have not yet been consolidated, are required to infer the right answer (Fig. 6j). For example, from the pair of memories ‘ab EAST cd SOUTH ef’ and ‘ef WEST gh’, it can be inferred that ‘gh NORTH ab’. 200 such tasks were created for each of the spatial and family tree experiments. To simulate RAG, the top two matches to the query (e.g., ‘gh NORTH?’) are first retrieved from the model hippocampus. Then the model neocortex predicts the solution to the query with these retrieved sequences in context. We compare this to baselines of RAG with a single memory, the hippocampus only solution (randomly selecting one of the locations / people in the retrieved sequences), and the neocortex only solution (conditioning the generative network on the task alone, without retrieving sequences from the hippocampus). The results show that RAG supports structural inference based on hippocampal memories, whereas relying on either the hippocampal network or generative network alone gives worse results, and that the model can solve problems using information from multiple episodes (Fig. 6k). Inspecting the hidden representations supporting inference To understand how the neocortical network generalises to a new set of labels, we inspected representations in its hidden layers. Activations (vectors of length 1024) for each token of a random walk were extracted from the innermost layer (i.e., the 12th of the 24 transformer blocks) of the model neocortex trained in the spatial task above, and from the same layer of the GPT-2 medium model pre-trained on language and released by ref. 23 as a baseline. Having obtained a vector for each location, we analysed the distances between these representations. We found that the nearer the points in the environment, the closer their hidden representations (Fig. 7f). This is not true of the baseline model (Fig. 7e). We then ran principal component analysis across representations extracted from random walks in many different environments. Figure 7b shows that our model has learned a shared structure across episodes, whereas this structure is not present in the baseline GPT-2 medium (Fig. 7a); the same is true of the family tree model, in which the three ‘generations’ in the tree are separated in our model (Fig. 7d) but not in the baseline model (Fig. 7c). Inference is supported by learned neocortical hidden representations. a, b Representations in the spatial inference model. Activations for each token of a random walk were extracted from the middle layer (12) to obtain a vector for each location. Principal component analysis was performed across representations extracted from random walks in many different environments (upper row), colouring each location by its co-ordinates in the environment, so that points with the same colour are different nodes which occupy the same position in the 3 x 3 grid (see example environments below, where dotted lines indicate possible transitions). GPT-2 medium pretrained on language is shown in (a), and our model - GPT-2 medium trained on structural inference - in (b). c, d As above, but in the family tree inference model. Here, three colours indicate the three generations in the family tree. e In the GPT-2 medium baseline, there is little relationship between Manhattan distance in the environment and Euclidean distance in token space. f As in (e), but for our model, showing that the token representations reflect distances in the environment. In (e, f), box plots show the median in orange, with boxes spanning the 25th-75th percentiles, and whiskers extending to 1.5 x IQR. g The Pearson correlation between Manhattan distance in the environment and Euclidean distance in each layer of our model (red) and GPT-2 medium (blue). h The Pearson correlation between Manhattan distance in the environment and Euclidean distance in layer 12 for prompt lengths differing in number of transitions, for our model (red) and pre-trained GPT-2 medium (blue). For both GPT-2 and our model, sample sizes for grid distances 1–4 are n = 10,552, 12,238, 6958 and 1698. Source data are provided as a Source Data file. These results mirror the finding that a shared abstract representation of different spatial tasks with the same structure develops in the cortex through consolidation, as has been shown using cross-map repetition suppression effects in fMRI73. Once this abstraction of the task has developed, it supports reasoning based on information in the prompt, or ‘in-context learning’. The correlation between distance in the grid and distance between the representations increases with the length of a random walk on a new graph, but is high from the outset, suggesting the model can rapidly align new stimuli to its learned map (Fig. 7h). Discussion We set out to address three questions: how episodic memory shapes semantic memory, how semantic memory shapes episodic memory, and how the two systems interact to provide complementary contributions to memory and problem solving. We hypothesise that semantic knowledge emerges from learning to reconstruct episodes; that episodic memory, which must use neocortical representations for input and output, is compressed using semantic knowledge and therefore shares its biases; and that specific episodic information is combined with general semantic knowledge in working memory for memory retrieval and problem solving. We propose that neocortical memory takes the form of a generative network, trained to reconstruct sequential events replayed by the hippocampus. This enables the network to acquire generalisable semantic knowledge through consolidation, to reconstruct episodes efficiently from highly compressed hippocampal traces, and eventually to capture specific events. During recall, the hippocampus retrieves episodic details which serve as context for the neocortical network to generate predictions based on semantic knowledge. Over time, consolidation allows the neocortex to reconstruct memories independently, reducing reliance on hippocampal traces. Likewise, episodic memories guide problem-solving by providing relevant context from the hippocampus for neocortical generation. In our simulations, the neocortical generative network interacts with stored sequences retrieved from the hippocampus as retrieval-augmented generation, but with the neocortex gradually trained on memories from the hippocampus through consolidation, and the hippocampus storing memories in a highly compressed form which can be decoded by the neocortex. Furthermore, the stored traces in the hippocampus can combine surprising details with gists in vector form, binding together information at different levels of abstraction in memory. This model suggests an explanation for several features of human cognition, such as the bidirectional influences between episodic and semantic memory; semantic memory is acquired by learning to reconstruct episodic memories, but also shapes the representation of these episodes, both in terms of their initial encoding, as reflected by the compressed representations observed in hippocampus38, and their gradual schematisation20. To support this, we demonstrated how distortions arise in narratives due to their encoding in compressed form, then increase with consolidation and subsequent forgetting, and compared changes in memory over time to human data (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’). We also showed how memory reflects priors in the generative model derived from previous experience (‘Modelling the interplay of semantic and episodic memory in consolidation’). Our model also explains how a number of capabilities improve through consolidation, in addition to the memorisation of ‘replayed’ sequences. In particular, the neocortical generative network supports problem solving, e.g., in relational inference tasks, by learning an implicit structure underlying observations (‘Neocortical learning of sequential structure for problem solving’). In recent years, neuroscience has seen a move from a modular view of many semi-independent networks learning particular tasks to a focus on the learning of multipurpose representations. Similarly, there has been a transition in machine learning from task-specific models to larger task-general ones trained through self-supervised learning, sometimes referred to as ‘foundation models’24. The use of a self-supervised task - simply the prediction of the next item in the sequence - to train such models is key to their scaling, as external supervision is not required. We can think of the brain as learning neural ‘foundation models’ too, and this paper suggests how memory consolidation could contribute to their development (see also ref. 27). This view aligns with the growing consensus that prediction error minimisation is key to biological intelligence (e.g., ref. 74), neuroimaging evidence of large task-general networks (e.g., ref. 75), and the relative lack of external supervision in learning. We do not argue that modern LLMs mirror the brain, but that some kind of neural ‘foundation model’ being trained to minimise prediction error on replayed sequences, and used in conjunction with specific memories, provides a good model for a wide range of data on memory, inference and planning. See Fig. 2f–i for a summary of the model’s predictions. Limitations and future work This work is primarily a model of psychological, rather than neural, data. More could be done to bridge the gap between these ideas and the growing understanding of sequence representations at a neural level. For example, the model allows consideration of navigation, which is associated with distinctive cell types such as place, grid and head-direction cells, so the connection between these different levels of explanation could be explored (see refs. 22,76–79). There is a trade-off between the complexity of the behaviour that can be simulated and our ability to understand the inner workings of the model. However, progress in ‘mechanistic interpretability’ (e.g., ref. 80.) enables the hidden layer activations of LLMs to be compared to neuroimaging data, allowing future comparison of representations at a population level. LLMs are trained on a simple next item prediction task23, mirroring how prediction error minimisation is thought to drive biological learning74. But even if the objective is shared, the implementations may be very different. In particular, error backpropagation - which calculates each parameter’s ‘responsibility’ for the network’s error - is thought to be implausible because of its non-local nature81. More biologically plausible approximations to error backpropagation do exist81, as well as other families of networks that mimic the brain more closely74,82,83, which could be explored in future work. We draw on a body of research connecting memory and schema formation8,12,84. The SLIMM framework8 suggests that whilst high prediction error memory elements may be preferentially encoded in the hippocampus, schema-congruent information may be remembered more easily by the cortex. Consistent with this account, our model could predict a U-shaped relationship between surprise and recall performance, in which moderately surprising elements - those not unpredictable enough to be encoded in detail, nor familiar enough to be captured by existing schema - are especially prone to forgetting (see ref. 85). Whilst we show that memories become more schema congruent overall in our simulations, and propose a mechanism for the storage of surprising details in hippocampus, we do not model how the retention of different aspects of a memory depends on their predictability, and how this changes over time. Future research could compare our model’s predicted pattern of memorability as a function of schema congruence with these data. Memories bind together neocortical representations, with anterior hippocampus connected to anterior neocortical regions capturing more conceptual content, and posterior hippocampus connected to posterior neocortical regions capturing more sensory content; consolidation is associated with growing dependence on the former39. If the gist is associated with the anterior neocortex, future work could test the prediction that the more schema-congruent an episodic memory, the greater the dependence on anterior as opposed to posterior hippocampus. A related issue that could be explored further is how the hippocampal encoding evolves over time. In our model, hippocampal traces are assumed to fade once the neocortical network can approximately reconstruct an event, but this is not simulated explicitly. A complexity of this process is that the stored gist representation must be ‘in sync’ with the neocortical model to be decompressed correctly, so as the neocortex changes, the gist would drift from its original meaning, which could be another factor leading to hippocampal forgetting. Long-term retention of hippocampal traces might therefore require some mechanism to preserve the correspondence between conceptual representations in hippocampus and in cortex86. A separate question is whether different elements of a hippocampal trace decay uniformly, or whether details might fade away once they become more predictable, with the gist representation the last element to be forgotten. Whilst experience is continuous, it is discretised into events in memory87,88, a process known as event segmentation. Event segmentation is thought to be based on prediction error89–91 (but see ref. 92 for an alternative account). Whilst the sequences used in our simulations are ‘pre-segmented’ for simplicity, this is potentially consistent with our model, as the generative network can provide an ongoing measure of ‘surprise’ during perception90,91,93,94, e.g., as quantified by perplexity23. Future work could explore the connections between these ideas. Human learning can display leaps of insight (e.g., ref. 95), with abrupt improvements thought to correspond to the reorganisation of mental representations96. Future work could explore the development of an LLM’s internal representations as a task is learned, comparing this to the corresponding ‘behavioural’ changes, to see if sudden increases in performance correspond to sudden representational changes. This could then be compared to the dynamics of learning in humans. Future research could also explore how retrieval, encoding, and replay processes are orchestrated. We simplify matters by using pre-segmented sequences, encoding every event with a fixed level of detail, and ‘hard-coding’ when the hippocampus is queried. However, one could train a ‘meta-controller’ to coordinate these processes with reinforcement learning97. This might learn to use different cognitive resources when it was optimal to do so, e.g., prioritising memories for replay to maximise future performance, and only encoding memories when it is worth the cognitive cost. This could build on previous studies like ref. 98, which prototypes these ideas with simpler models. Finally, catastrophic forgetting refers to the overwriting of old knowledge by new knowledge when a neural network learns multiple tasks or distributions consecutively99. Whilst gradual forgetting is expected in cognition, catastrophic forgetting is a more dramatic interference of new with old knowledge, which is not typically observed. The LLM’s gradient of forgetting in ‘Modelling consolidation and forgetting’ is steeper than would be expected of human memory, suggesting that the brain has mechanisms for maintaining neocortical memories over the course of a lifetime that LLMs do not. Future research could investigate these mechanisms, e.g., whether generative replay in the neocortex might facilitate continual learning86,100. Episodic and semantic memory In our model, episodic and semantic memory combine to reconstruct the past and predict the future, but how clear is this distinction? If a series of narratives representing ‘episodes’ are ‘consolidated’ into the neocortical LLM, the resulting LLM could support both memory for specific episodes28 and for semantic ‘facts’, with the latter learned as a side-effect of reconstructing the former. Not only are ‘beliefs’ influenced by ‘episodes’ the network was trained on, but the ‘episodes’ are reshaped by the ‘beliefs’, as we saw in a range of memory distortion simulations. This can be the case even for the initial hippocampal encoding of episodes in the full model, as demonstrated in ‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’. How do we distinguish remote memory from imagination, if both consist largely of schema-based predictions from a generative model? The generative model’s prediction error for sequences from the training data tends to be lower than for novel sequences101, so the generative network could provide a measure of recognition memory even if the original hippocampal trace has faded away. However, this might correspond to a ‘sense of familiarity’ rather than definitive knowledge that a stimulus was experienced. Thus, the presence of similar real memories56,57, or the rehearsal of imagined events102, can induce false memory, presumably from a ‘sense of familiarity’ in the generative network. This could align with hippocampal amnesics’ sometimes preserved ability to make relative judgements of familiarity103. On the other hand, episodic memories arguably need to include some details from a hippocampal trace to be recognised confidently103,104. Memories may retain some limited trace in the hippocampus proper for a long time, so the retrieval of event-unique details which have not yet been consolidated could help to distinguish real memories from imagination. The neocortical network may contain many semantic facts relating to one’s personal history, e.g., that one went on a holiday to a particular place as a child, and could thus construct events consistent with these facts. However, if episodic recollection requires retrieval of unique details encoded during the event, then even an accurate reconstruction might be considered a ‘semanticized’ memory rather than a truly episodic one (e.g., ref. 7). Conclusion We have proposed a model of the bidirectional interactions between episodic and semantic memory, in which semantic memory in the neocortex relies on a generative network, trained to reconstruct sequential events replayed by the hippocampus. Episodic memories are encoded in the hippocampus in compressed form optimised for efficient neocortical reconstruction, and recalled by ‘decompressing’ these traces with the generative network. In other words, the hippocampus and neocortex work together in encoding, recall, and problem-solving to provide episodic details and predictions from semantic knowledge based on these details, respectively. During consolidation, these reconstructions are then replayed to update the neocortical generative network to predict the next item in each sequence. We make use of recent advances in the large language model literature to simulate this hippocampo-neocortical interaction as ‘retrieval-augmented generation’ (RAG), with the addition of mechanisms to compress episodic memories and to consolidate them into the generative network. This model explains changes in the characteristics of autobiographical memories over time and accounts for gist or schema-based distortions. As well as capturing the gist of specific episodes, the neocortical network extracts statistical patterns that generalise to new situations. We also show how the hippocampal and neocortical networks could jointly contribute to problem solving through RAG, allowing memories retrieved from the hippocampus to provide the context for prediction using the ‘general knowledge’ of the neocortical network. We demonstrate that, by simply learning to predict the next item, the neocortical network can acquire abstracted knowledge of commonalities across experience, which can be used to support inference and planning. Methods Large language models Our simulations use large language models (LLMs), which are neural networks trained to predict the next item in a sequence based on preceding items23. Most commonly, the sequence is text, and the items are small chunks of characters (or ‘tokens’). These models operate autoregressively: given a partial sequence, they estimate a probability distribution over possible next items. Training consists of adjusting the network’s parameters so that it assigns high probability to the actual next item observed in the data, a procedure known as causal language modelling. Once trained, the model can be used to generate new sequences by repeatedly predicting and selecting the next item. The simplest approach is greedy decoding, in which the most probable next item is always selected, but sequences can also be generated by sampling from the model’s predicted probability distribution, with a temperature parameter controlling how variable the output is. Higher temperatures increase variability by flattening the distribution, while lower temperatures favour the most likely continuations. Most LLMs use a mechanism called attention105, which allows each element to be enriched by ‘attending to’ information earlier in the sequence. This matches the intuition that the meaning of a word is understood in the context of the text so far, not in isolation, as in earlier approaches to text processing with machine learning. See Section S1.1, SI, for further details. Modelling hippocampal-neocortical interaction as retrieval-augmented generation The compression and encoding of episodic memory In ‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’ and ‘Modelling the interplay of semantic and episodic memory in consolidation’, we use Mistral-7B-Instruct-v0.229 to explore the compression, encoding, retrieval-augmented generation, consolidation, and forgetting of narrative events. In addition to the LLM itself, pre-trained weights from ref. 33 are used to transform episodes into compressed xRAG vectors. In brief, these weights are trained so that the text reconstructed by the LLM is as close as possible to the original. A key advantage is that the underlying idea in ref. 33 is not specific to language but could, in principle, work for arbitrary types of sequence. Note that xRAG weights are only compatible with a particular LLM; Mistral-7B-Instruct-v0.2 is used because the xRAG weights are trained to transform embeddings into token representations of this specific model. But in principle, the context compression mechanism could work for any LLM. See Section S1.1.4, SI, for a discussion of alternative context compression approaches. To identify surprising details given the xRAG vector, we first segmented the original story into short clause-like phrases using the spaCy Python library, splitting at (i) sentence boundaries and (ii) commas or conjunctions. Note that this can leave a whole sentence intact when there is no comma or coordinating conjunction. To quantify how unpredictable each candidate phrase p is given the gist vector, we scored it by conditional perplexity. Specifically, we concatenated the gist vector with the candidate phrase and computed the model’s perplexity. Higher perplexity indicates that the phrase is less predictable from the gist. For a requested detail level n, we selected the top-n highest-perplexity phrases as the ‘unpredictable details’. (Since the perplexity of each phrase is evaluated as a whole, phrases may be high perplexity because some subset of tokens within them is surprising, even if other tokens are consistent with the gist.) The prompt used to recall a memory with retrieval-augmented generation is as follows, formatted as suggested in the xRAG33 code. Note that the ‘INST’ tag denotes the instruction for Mistral 7B, which is an instruction-tuned model: [INST] Refer to the background document and answer the question. [INST] Refer to the background document and answer the question. Background: {XRAG_TOKEN} Background: {XRAG_TOKEN} Question: {FIRST_LINE} What happened (in detail)? Question: {FIRST_LINE} What happened (in detail)? Other details to include: {SURPRISING_DETAILS}. [/INST] The answer is: Other details to include: {SURPRISING_DETAILS}. [/INST] The answer is: Here, ‘XRAG_TOKEN’ is the single token encoding the gist produced by the xRAG projection weights, and ‘SURPRISING_DETAILS’ is a comma-separated list of the n highest perplexity phrases. (The line beginning ‘Other details’ is omitted from the prompt when no details are provided.) Modelling the hippocampus Recall requires both pattern completion of the current item and prediction of the next one. For example, recalling a day out from a photo requires first retrieving the scene depicted in the photo, and then remembering the events following and preceding that scene. Modelling sequence memory, therefore, requires a mechanism that supports both autoassociative reconstruction of the present state from a partial input and heteroassociative transitions to future states106. Here, this is implemented with a pair of modern Hopfield networks (see Section S1.1.5, SI, for further details). A standard modern Hopfield network (MHN) retrieves items from their noisy versions31, while an asymmetric MHN107 retrieves the next item for a given item. This simple model demonstrates the idea of combining autoassociative and heteroassociative connectivity in a ‘biologically plausible’ way108, but many alternative implementations of sequence memory exist, e.g., ref. 109, and could be compatible with this proposal. In our model, the xRAG gist vector for an event can be retrieved autoassociatively from a cue, while each token in the sequence of details can be retrieved heteroassociatively from the previous token. (Note that we consider transitions going forward in time for simplicity, but a fuller model could involve heteroassociative transitions going backward in time too.) To encode text as sequences of characters, it is crucial to distinguish between different instances of the same character, i.e., the correct next character after ‘a’ depends on the letters that precede ‘a’. (Because the same words feature in different episodes, the history that determines the correct transition stretches back many characters.) The asymmetric MHN is by default a ‘Markov chain’ model of sequential memory, so it is necessary to use a state representation that captures the history. This is based on previous work involving the role of temporal context in memory110,111. Here, we simply add a decaying representation of the previous item to a given item’s representation, with the weighting of the previous item set by a decay constant. In summary, each element of a sequence is represented by a state vector, which consists of the current state plus a decaying trace of previous states. In the main text simulations, the first such state is the xRAG vector, and subsequent states are characters in the string of unpredictable details. (In ‘Neocortical learning of sequential structure for problem solving’, which omits the compression aspect for simplicity, the states are characters in the string.) To describe this in terms of the Krotov and Hopfield108 formulation, the values of each state vector are encoded in the weights to and from a given memory unit in an MHN, while the asymmetric MHN stores state n in incoming weights to a memory unit, and state n+1 in outgoing weights. Retrieval proceeds in two stages (described with reference to the main text simulations). First, given a query xRAG vector computed from the query text, the MHN computes similarities to stored states (Supplementary Fig. S2a, SI) and applies a softmax function with inverse temperature β. In the limit of high β, this results in autoassociative pattern completion of a single stored state, ideally that of the stored gist vector. Subsequent elements, corresponding to characters of the unpredictable details, are then retrieved heteroassociatively, with the output of one retrieval becoming the input to the next. A range of values of the decay constant described above can give good results (Supplementary Fig. S2b, SI). High β is also used here to retrieve a single next state prediction rather than a superposition (Supplementary Fig. S2c, SI). Note that we do not model forgetting in the hippocampus, or errors that result from retrieval failures rather than compressed encoding. The ability to retrieve stored sequences is near perfect for the parameters chosen, as shown in Supplementary Fig. S2, SI, but future work could explore how retrieval errors in the hippocampus affect the behaviour of the system. Consolidation of reconstructed episodes In simulations involving Mistral 7B, the model is fine-tuned in a different way from the ‘usual’ end-to-end method because of its size. Instead of updating the entire model, parameter-efficient fine-tuning (PEFT) methods update only a small subset of parameters. One such method is low-rank adaptation (LoRA63), which injects trainable low-rank matrices alongside frozen pre-trained weights. We implement LoRA with the ‘PEFT’ Python library112. We used a LoRA rank of r= 64, scaling parameter α= 16, and dropout probability 0.05. LoRA adaptors were applied to the linear weight matrices used to compute the query, key, value, and output projections in the attention mechanism, as well as to the gate, up, and down projection matrices of the feed-forward network. The training data is formatted as follows, in the form recommended for an instruction-tuned model such as Mistral 7B: [INST] {FIRST_LINE} What happened (in detail)? [/INST] {REST_OF_STORY} [INST] {FIRST_LINE} What happened (in detail)? [/INST] {REST_OF_STORY} The prompt used to test consolidated episodic memory, i.e., memory without any retrieval from the model hippocampus, is as follows: [INST] {FIRST_LINE} What happened (in detail)? [/INST] [INST] {FIRST_LINE} What happened (in detail)? [/INST] The prompt used to test consolidated semantic memory is as follows, where the prompt contains enough context to pick out a single relevant story, and the question has an answer in that story: [INST] Remember the story in which {FIRST_LINE}. {QUESTION} Answer with a word or short phrase. [/INST] [INST] Remember the story in which {FIRST_LINE}. {QUESTION} Answer with a word or short phrase. [/INST] Analysing the HIPPOCORPUS and NFRD datasets The HIPPOCORPUS dataset18 consists of 2779 recalled stories, 1319 retold ones, and 2756 imagined ones (not included in these analyses). The Naturalistic Free Recall Dataset (NFRD19;) consists of 116 recalls for the ‘pieman’ story, 116 for ‘eyespy’, 113 for ‘baseball’, and 113 for ‘oregontrail’. The following prompt was used to label the attributes of recalled and retold items in the HIPPOCORPUS dataset, and of original and recalled items in the NFRD dataset with gpt-4o-mini via the OpenAI API: Your task is to score text on three metrics: how concrete (vs abstract) it is, how rich in detail it is, and how specific (vs general) it is. Your task is to score text on three metrics: how concrete (vs abstract) it is, how rich in detail it is, and how specific (vs general) it is. Return ONLY a JSON dictionary with 3 keys, each a float 0-1: Return ONLY a JSON dictionary with 3 keys, each a float 0-1: {‘concrete_vs_abstract’: 0-1, ‘rich_vs_poor_details’: 0-1, ‘specific_vs_general’: 0-1} {‘concrete_vs_abstract’: 0-1, ‘rich_vs_poor_details’: 0-1, ‘specific_vs_general’: 0-1} A higher score corresponds to more concrete, richer in detail, or more specific text. A higher score corresponds to more concrete, richer in detail, or more specific text. This reflects standard practices for using LLMs for data annotation113. Word frequencies were computed with the ‘wordfreq’ Python library. Modelling the interplay of semantic and episodic memory in consolidation The simulations in ‘Modelling the interplay of semantic and episodic memory in consolidation’ investigate how the model distorts memories towards expectations, using stimuli from Bartlett20 and Raykov et al.21. In both cases, Mistral 7B29 is used as the model neocortex, and fine-tuned with LoRA63 on narrative stimuli. In ‘Gist-based distortions and prior experience’, this is preceded by training the model neocortex on one of five categories of Wikipedia data, to manipulate the priors from previous learning. Event extension and contraction are modelled with simple stories in text form, each only a few lines long66. Three types of narrative are used. Some are ‘typical’ stories, unmodified from the66 dataset, whereas an ‘incomplete’ story has characters removed, and an ‘updated’ story has characters added, so it continues beyond its natural ending. Note that this is a challenging task as some stories could be interpreted as either incomplete or updated. To disentangle the effect of story category from the effect of length, the incomplete and updated stories were all the same length. To achieve this, text from a different random story was added to stories shorter than average, extending them to the mean length, while longer than average stories were shortened to the mean length. After encoding and reconstructing the stories, the Mistral 7B model29 was fine-tuned on the reconstructions for three epochs, and then recall was tested by giving the model the first 100 characters and observing the output. Neocortical learning of sequential structure for inference and planning In ‘Neocortical learning of sequential structure for problem solving’, we train sequence models (GPT-223) on non-linguistic sequences, choosing a smaller model to run more extensive simulations of learning structural inference from scratch. The ‘medium’ sized GPT-2 model is used, which has 345 million parameters including 24 transformer blocks, and is trained from scratch with randomly initialised weights. See ‘Summary of models trained’ for a summary of the models trained. We simulated how consolidation enables structural inference by the neocortical network, considering the example of a spatial graph and a simple family tree graph (as in ref. 22). We trained GPT-2 medium from scratch on random walks on these graphs, corresponding to sequences of observations encoded in the hippocampus, and then tested the models’ inference abilities on novel sequences with the same underlying structure. Note that on some of the family tree inference problems there are multiple possible answers which are consistent with, but cannot be inferred from, the sequence so far (such as the imagined family member ‘ef’ in ‘ab CHILD_OF cd PARENT_OF ef’, whereas ‘ab’ is the expected answer), and these are counted as incorrect, making this quite a harsh performance metric. We simulate episodic contributions to problem solving as retrieval-augmented generation, using the two models in the structural inference results above (one trained on spatial graphs, and one on family tree graphs, such that each has learned the structural regularities of the stimuli). In each case, the hippocampus encodes sequences from new spatial or family tree graphs, corresponding to observations in a new spatial environment or of a new family’s relationships. 200 new graphs were constructed for each task, each missing a randomly chosen edge. Two short walks on each graph, which did not include the missing edge, were stored in the ‘hippocampus’ (simply a list of strings in this example). The two walks were chosen so that solving the task requires combining information from both, e.g., it requires both ‘ab EAST cd SOUTH ef’ and ‘ef WEST gh’ to infer that ‘gh NORTH ab’. For each missing edge, a query of the form ‘gh NORTH’ or ‘cd PARENT_OF’ was constructed for the spatial and family tree graphs, respectively. Testing involved two stages, retrieval followed by generation: first, the hippocampus is queried for relevant traces, simply by finding sequences from the same environment or family tree. Then the generative network produces an output conditioned on the retrieved sequences concatenated with the sequence for the task (Fig. 6j), i.e., with the retrieved sequence and the current task in the LLM’s prompt. Greedy decoding was used to generate predictions. Summary of models trained In summary, we train the following models: For the narratives simulation (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’), we further train Mistral-7B-Instruct-v0.2 plus xRAG weights using LoRA63. The training data are reconstructed English language text from the ROC Stories dataset61. For the narratives simulation (‘Modelling hippocampal-neocortical interaction as retrieval-augmented generation’), we further train Mistral-7B-Instruct-v0.2 plus xRAG weights using LoRA63. The training data are reconstructed English language text from the ROC Stories dataset61. For the gist-based distortions simulation (‘Gist-based distortions and prior experience’), we further train Mistral-7B-Instruct-v0.2 plus xRAG weights using LoRA63. The training data are reconstructed English language text from Bartlett20 and Wikipedia65. For the gist-based distortions simulation (‘Gist-based distortions and prior experience’), we further train Mistral-7B-Instruct-v0.2 plus xRAG weights using LoRA63. The training data are reconstructed English language text from Bartlett20 and Wikipedia65. For the event extension and contraction simulation (‘Event extension and contraction’), we further train Mistral-7B-Instruct-v0.2 plus xRAG weights using LoRA63. The training data are reconstructed English language text from the ROC Stories dataset61. For the event extension and contraction simulation (‘Event extension and contraction’), we further train Mistral-7B-Instruct-v0.2 plus xRAG weights using LoRA63. The training data are reconstructed English language text from the ROC Stories dataset61. For the structural inference simulations (‘Neocortical learning of sequential structure for problem solving’), we train GPT-2 medium from scratch for one epoch with a learning rate of 5×10−5. The training data are 20 random walks of length 50 transitions on each of 100,000 graphs, with format ‘mv SOUTH sz WEST li EAST sz...’ for spatial graphs and ‘yu GRANDPARENT OF mi SIBLING OF vb...’ for family trees. For the structural inference simulations (‘Neocortical learning of sequential structure for problem solving’), we train GPT-2 medium from scratch for one epoch with a learning rate of 5×10−5. The training data are 20 random walks of length 50 transitions on each of 100,000 graphs, with format ‘mv SOUTH sz WEST li EAST sz...’ for spatial graphs and ‘yu GRANDPARENT OF mi SIBLING OF vb...’ for family trees. Here, ‘reconstructed’ means the version of the original data reconstructed by the LLM from its compressed encoding. Reporting summary Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article. Supplementary information Supplementary Information Reporting Summary Transparent Peer Review file Source data Source data 1 Source data 2 Acknowledgements We thank Oliver Vikbladh, Misun Kim, Steve Fleming, Kris Jensen, and Mathias Sablé-Meyer for their insightful comments on the manuscript. Author contributions Conceptualisation: E.S. and N.B. Methodology: E.S. and N.B. Investigation: E.S. Visualisation: E.S. Supervision: N.B. Writing: E.S. and N.B. Peer review Peer review information Nature Communications thanks the anonymous reviewers for their contribution to the peer review of this work. A peer review file is available. Funding Funding support for this work was received from a Wellcome Principal Research Fellowship ‘Neural mechanisms of memory and prediction: Finding structure in experience’ (222457/Z/21/Z) (N.B.) and a Wellcome Collaborative Award ‘Organising knowledge for flexible behaviour in the prefrontal-hippocampal circuitry’ (214314/Z/18/Z) (N.B.). Data availability All datasets used are publicly available, and the codebase for the project loads the data required for each simulation automatically. Source data for all figures are provided with this paper. Source data are provided in this paper. Code availability Code for all simulations can be found at https://github.com/ellie-as/hippocampal-neocortical-RAG. This repository is archived on Zenodo114. Competing interests There are no competing interests to declare. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Contributor Information Eleanor Spens, Email: ellie.spens@ndcn.ox.ac.uk. Neil Burgess, Email: n.burgess@ucl.ac.uk. Supplementary information The online version contains supplementary material available at 10.1038/s41467-026-74357-6. References Associated Data Supplementary Materials Supplementary Information Reporting Summary Transparent Peer Review file Source data 1 Source data 2 Data Availability Statement All datasets used are publicly available, and the codebase for the project loads the data required for each simulation automatically. Source data for all figures are provided with this paper. Source data are provided in this paper. Code for all simulations can be found at https://github.com/ellie-as/hippocampal-neocortical-RAG. This repository is archived on Zenodo114.