Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

Jul 16, 2026

Language Models Need Sleep proposes a continual-learning paradigm for large language models that addresses their inherent static knowledge after deployment. The approach introduces a “Sleep” stage that consolidates short-term, fragile in-context memories into stable long-term parameters. Sleep combines Knowledge Seeding via upward distillation into a larger network and a “Dreaming” phase that uses RL to create synthetic-data curricula for rehearsal and improvement without human supervision. Experiments on long-horizon continual learning support the value of Sleep for knowledge incorporation and few-shot generalization.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages26

Reading Time1 h 6 min

Introduction

Memory Staleness in Static LLMs

Continual Learning Challenges

In-Context Learning as Continual Learning

Sleep Paradigm: Knowledge Seeding and Dreaming

arXiv :2606 .03979v2 [ cs .LG] 10 Jul 2026 Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories Ali Behrouz†, Farnoosh Hashemi‡, Adel Javanmard †, and Vahab Mirrokni † † ‡ Abstract The past few decades have witnessed significant advances in the design of machine learning algorithms–from early studies on task-specific shallow models to more general deep Large Language Models (LLMs) . Despite showing promising results in tasks that require instant prediction or in-context learning, existing models lack the ability to continually learn and effectively transfer their temporal in-context knowledge to their long-term parameters. Inspired by human learning process, we introduce a “Sleep” paradigm that allows the models to continually learn, distill their short-term fragile memories into stable long-term knowledge with replay, and recursively improve themselves with “Dreaming” process. In more detail, sleep consists of two stages: (1) Memory Consolidation: an upward distillation process, called Knowledge Seeding, where the memories of a smaller-self are distilled into a larger network to provide more capacity while preserving the knowledge. As a proof of concept, we present a new Generalized Distillation process for Knowledge Seeding (i.e., the combination of on-policy distillation with Reinforcement Learning (RL)-based imitation learning); (2) Dreaming: a self-improvement phase, where the model uses RL to generate a curriculum of synthetic data to rehearse new knowledge and refine existing capabilities without human supervision. Our experiments on long-horizon, continual learning, knowledge incorporation, and few-shot generalization tasks support the importance of the sleep stage. 1 Introduction The development of Large Language Models (LLMs) marks a pivotal milestone in machine learning research: a paradigm shift from task-specific models to more general-purpose systems with various emergent capabilities (Brown et al. 2020; Schaeffer et al. 2023) . Despite LLMs’ remarkable capabilities in diverse sets of tasks (Nijkamp et al. 2023; Wang et al. 2023; Comanici et al. 2025), they are largely static after their initial deployment, meaning that they successfully perform tasks learned during pre-or post-training, but are unable to continually acquire new capabilities beyond their immediate context. This inherent static nature creates a crucial vulnerability: The model’s knowledge and skills become progressively stale, operating with a fixed "knowledge cutoff" date beyond which it is unaware of new facts, events, and evolving information (Cheng et al. 2024) . Efforts to overcome this limitation have primarily focused on: (1) re-pretraining on an expanded dataset, which despite its effectiveness, is computationally expensive and impractical for frequent updates (Ibrahim et al. 2024); (2) using expensive continual parameter updates or other lightweight alternatives, such as fine-tuning or low-rank adaption (Hu et al. 2022; Akyürek et al. 2024a), which with iterative updates often results in Catastrophic Forgetting (CF) (Kemker et al. 2018; Shi et al. 2024)–a well-known phenomenon where the model’s proficiency on original tasks degrades catastrophically as it learns new ones. This dilemma—between knowledge obsolescence on one hand and catastrophic forgetting as well as the prohibitive cost or destructive nature of updates on the other—underscores a critical, unresolved challenge: enabling LLMs to learn incrementally and efficiently throughout their lifecycle. In recent years, In-Context Learning (ICL) (Brown et al. 2020) has gained attention as a highly efficient and successful form of continual learning (Akyürek et al. 2022, 2024b; Dong et al. 2024; Li et al. 2025) . Initially, ICL was known as an emergent ability of LLMs that is trained on large scale data, enabling them to adapt fast to the context and so perform zero-or few-shot tasks (Brown et al. 2020) . Later, more studies revealed and formali

FAQs

Why do existing language models struggle with continual learning after deployment?

They are largely static after initial deployment, so knowledge becomes stale beyond a fixed knowledge cutoff and the model cannot continually acquire new capabilities outside the immediate context window.

What is the “Sleep” paradigm proposed in the document?

Sleep is a two-stage process that helps models continually learn by consolidating short-term fragile memories into stable long-term knowledge, combining Memory Consolidation (Knowledge Seeding) and a self-improvement “Dreaming” phase.

How do Knowledge Seeding and Dreaming work together?

Knowledge Seeding performs upward distillation from a smaller model into a larger network to preserve knowledge while adding capacity. Dreaming then uses reinforcement learning to generate synthetic-data curricula for rehearsal, refining existing capabilities without human supervision.

Jul 16, 2026

Upload by Olivia Brown