[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81663-en":3,"doc-seo-81663-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},81663,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","SHARP Sleep-based Hierarchical Accelerated Replay for Long Range Non-stationary Temporal Pattern Recognition","Learning long-range non-stationary temporal patterns is a central challenge for modern sequence models, especially in strict streaming settings where data must be processed in one pass. SHARP (Sleep-based Hierarchical Accelerated Replay) separates temporal learning into a structured memory module and a pattern-recognition module, avoiding long backpropagation through time for long-range credit assignment. Inspired by accelerated replay during rodent slow-wave sleep, SHARP uses offline sleep phases to replay temporally structured memory traces. Experiments and ablations on TEXT8 and PG-19 show improved next-token prediction on past data while learning from the current stream and generalizing to future unseen sequences. The hierarchical design yields exponentially increasing effective temporal context with linear-time compute.","arXiv :2606 .00732v4 [ cs .AI] 10 Jul 2026  \nSHARP: SLEEP-BASED HIERARCHICAL ACCELERATED REPLAY FOR LONG RANGE NON-STATIONARY TEMPORAL  \nPATTERN RECOGNITION  \nJayanta Dey 1 Shikhar Srivastava2 Itamar Lerner 1 Christopher Kanan2 Dhireesha Kudithipudi 1  \n1University of Texas at San Antonio, USA  \n2 University of Rochester, USA  \nABSTRACT  \nLearning long-range non-stationary temporal patterns remains a core challenge for modern sequence models, particularly in strict streaming settings. In these settings, data arrive sequentially and must be processed in a single pass without simultaneously revisiting past observations. Standard architectures, including recurrent neural networks and transformers, are constrained by either truncated backpropagation through time horizon or explicit input window length for long range credit assignment. To address these limitations, we propose SHARP (Sleep-based Hierarchical Accelerated Replay), a framework that decomposes temporal learning into two complementary components: a memory module that accumulates a structured history of past inputs, and a pattern-recognition module that operates over this memory. This separation enables resource-and compute-efficient adaptation to non-stationary dynamics by eliminating the need for backpropagation through time across many steps for long-range credit assignment. Inspired by the accelerated replay observed in rodents during slow-wave sleep, SHARP incorporates offline (sleep) phases in which temporally structured memory traces are replayed in an accelerated form and integrated into higher-level memory representations, improving long-range context retention. Through controlled simulations and ablation studies, we characterize the key properties of the proposed framework. In benchmark datasets such as TEXT8 and PG-19, we demonstrate that SHARP improves over recurrent baselines by retaining next-token predictive performance on previously seen data while continuing to learn from the current stream and generalizing to future unseen data. These gains are enabled by its hierarchical structure, which yields an exponentially increasing effective temporal context with only linear-time computational cost.  \n1 INTRODUCTION  \nIn many real-world settings, observations arrive sequentially without the possibility of revisiting past data. Learning algorithms must therefore continually integrate new information while preserving the structure of prior experience (Harun et al.) . This imposes a strict constraint: learning must proceed online, with limited opportunity for long-horizon credit assignment. The challenge is further exacerbated under distribution shift, where the underlying data-generating process evolves over time.  \nFrom a modeling perspective, continual learning under streaming constraints can be naturally formulated as a sequential learning problem. To generalize under these constraints, a system must retain information about past inputs even after they are no longer directly accessible. Classical sequence models such as recurrent neural networks (RNNS) and long short-term memory networks (LSTMS) attempt to encode memory within recurrent dynamics. However, their effective memory is governed by backpropagation through time (BPTT), which limits credit assignment to a finite temporal horizon and introduces numerical instabilities such as vanishing and exploding gradients. Although recurrent models have a theoretically unbounded context memory, in practice their memory is lossy: information dissipates, interferes, or becomes entangled over time, restricting the reliable capture of long-range temporal structure (Bengio et al., 1994 ; Hochreiter & Schmidhuber, 1997) . A common consequence of the limited temporal horizon is the  \nImplementation of SHARP is available at [https://github.com/jdey4/sharp](https://github.com/jdey4/sharp. Email:)[. Email:](https://github.com/jdey4/sharp. Email:) [deyjayanta76@gmail.com](deyjayanta76@gmail.com)  \nstored salient experience  \nEn","cbCaieIERiDXPsuO","https://ap.wps.com/l/cbCaieIERiDXPsuO","pdf",969772,4,1,19,"English","en",105,"# Introduction\n## Motivation: streaming constraints and long-range credit assignment\n## Limitations of recurrent models and catastrophic forgetting\n## Related work: regularization-based and replay-based continual learning\n# SHARP Overview\n## Conceptual framework and sleep-based accelerated replay","[{\"question\":\"What problem does SHARP address in temporal sequence modeling?\",\"answer\":\"SHARP targets learning long-range non-stationary temporal patterns under strict streaming constraints, where past observations cannot be revisited and long-horizon credit assignment is limited.\"},{\"question\":\"How does SHARP reduce the need for long backpropagation through time?\",\"answer\":\"SHARP decomposes learning into a structured memory module and a pattern-recognition module, enabling long-range credit assignment without performing backpropagation through time across many steps.\"},{\"question\":\"What is the role of the sleep phase in SHARP?\",\"answer\":\"In the sleep phase, temporally structured memory traces are replayed in an accelerated form and integrated into higher-level memory representations, improving long-range context retention and consolidation.\"}]",1784175271,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"sharp-sleep-based-hierarchical-accelerated-replay-for-long-range-non-stationary-temporal-pattern-recognition","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/sharp-sleep-based-hierarchical-accelerated-replay-for-long-range-non-stationary-temporal-pattern-recognition/81663/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-23","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does SHARP address in temporal sequence modeling?","Question",{"text":75,"@type":76},"SHARP targets learning long-range non-stationary temporal patterns under strict streaming constraints, where past observations cannot be revisited and long-horizon credit assignment is limited.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does SHARP reduce the need for long backpropagation through time?",{"text":80,"@type":76},"SHARP decomposes learning into a structured memory module and a pattern-recognition module, enabling long-range credit assignment without performing backpropagation through time across many steps.",{"name":82,"@type":73,"acceptedAnswer":83},"What is the role of the sleep phase in SHARP?",{"text":84,"@type":76},"In the sleep phase, temporally structured memory traces are replayed in an accelerated form and integrated into higher-level memory representations, improving long-range context retention and consolidation.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]