[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85190-en":3,"doc-seo-85190-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85190,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","DSSMs: State Space Models with Explicit Memory via Delay Differential Equations","State Space Models (SSMs) enable efficient long-sequence learning but must compress an unbounded history into a fixed-size state, making precise retrieval over long ranges inherently difficult. Delay State Space Models (DSSMs) extend diagonal SSM recurrences with explicit delayed-state feedback inspired by delay differential equations. The approach introduces new stability parameterization, history management, and discretization, while deriving frequency-domain transfer functions to enable accurate FFT training. Experiments show strong gains on delayed-retrieval tasks and competitive results on standard sequence metrics.","arXiv :2607 . 10244v 1 [ cs .LG] 11 Jul 2026  \nDSSMs: State Space Models with Explicit Memory via Delay Differential Equations  \nYixiao Qian 1 Song Chen2 ,∗ Jiaxu Liu3 Shengze Cai 1 Chao Xu 1 ,∗  \n1 College of Control Science and Engineering, Zhejiang University  \n2Department of Mathematics, National University of Singapore  \n3 School of Mathematical Sciences, Zhejiang University  \n[song.chen@nus.edu.sg](song.chen@nus.edu.sg) , [cxu@zju.edu.cn](cxu@zju.edu.cn)  \n∗ Corresponding authors.  \nAbstract  \nState Space Models (SSMs) have emerged as a powerful paradigm for efficient long-sequence modeling, offering parallel training and fast linear-time recurrent inference. However, like other recurrent architectures, SSMs must compress an unbounded history into a fixed-size state, which limits context retention and makes precise retrieval over long-range context inherently difficult. To overcome this limitation, we propose Delay State Space Models (DSSMs), a delay differential equation (DDE)-inspired extension of diagonal SSMs that augments discrete SSM recurrences with explicit delayed-state feedback. Making explicit delayed feedback practical requires new stability parameterization, history management, and FFTtraining tools. We address these challenges with a practical discretization and parameterization grounded in a simple delay-independent stability condition. To bypass direct time-domain kernel construction, we derive the DSSM transfer function and compute kernels in the frequency domain, using a kernel contour shift to suppress aliasing and recover accurate FFT training. Empirically, DSSMs substantially improve targeted delayed-retrieval tasks while outperforming S4D on most standard sequence metrics and remaining close on the others.  \n1 Introduction  \nSequence modeling is a core problem in modern deep learning. Recently, State Space Models (SSMs) have drawn increasing attention as an efficient family of sequence models because they enable parallel training and fast linear-time inference while showing strong performance on long-sequence tasks [Guet al., 2021, 2022, Gu and Dao, 2024] . However, like other recurrent architectures, SSMs must compress past information into a finite recurrent state, where it is represented through exponentially decaying modes [Wang and Xue, 2023, Wang and Li, 2023], making exact copying and precise retrieval harder than in attention-based models [Jelassi et al., 2024] . This limitation has motivated several complementary directions, including recall-oriented SSM layers such as H3 [Fu et al., 2022], more expressive state-space designs [Lahoti et al., 2026], and hybrid architectures that combine attention with SSMs, such as Jamba, Zamba, and Samba [Lieber et al., 2024, Glorioso et al., 2024, Ren et al., 2024] . This leaves open how to strengthen precise long-range memory while retaining efficient parallel training and fast inference: Transformers offer strong long-range memory and parallel training but incur expensive inference [Shazeer, 2019, Pope et al., 2023], whereas standard SSMs achieve parallel training and efficient inference yet still struggle with precise long-range retrieval.  \nTo overcome this bottleneck, we revisit the underlying dynamics and introduce time-delays into the state evolution. Unlike standard ordinary differential equations (ODEs), delay differential equations  \nPreprint.  \n\n|  | \u003Cbr> |\n| --- | --- |\n\nFigure 1: Overview of the discrete DSSM update. The state combines the input term˜, the main-branch  \nterm from ht−1, and the delayed-branch term from the buffered delayed state ht−τ . The formal discrete recurrence is developed in Sec. 3.2.  \n(DDEs) explicitly incorporate historical states through delayed terms, thereby providing a natural mechanism for improving precise long-range memory. Related work in scientific machine learning likewise suggests that introducing delay-aware continuous dynamics can improve expressive power and long-horizon prediction [Zhu et al., 2021, Gup","cbCaitdvrnm1PcVs","https://ap.wps.com/l/cbCaitdvrnm1PcVs","pdf",1396548,3,1,23,"English","en",105,"# Introduction\n# Background\n## Continuous State Space Models\n## Discretization and Convolutional View","[{\"question\":\"Why do standard state space models struggle with long-range precise retrieval?\",\"answer\":\"They must compress an unbounded history into a fixed-size state represented by exponentially decaying modes, which makes exact copying and precise long-range retrieval harder than attention-based models.\"},{\"question\":\"What is a Delay State Space Model (DSSM)?\",\"answer\":\"A DSSM is a delay differential equation-inspired extension of diagonal SSMs that augments discrete SSM recurrences with explicit delayed-state feedback to improve long-range memory.\"},{\"question\":\"How does DSSM enable efficient FFT training despite delayed feedback?\",\"answer\":\"It derives the DSSM transfer function and computes kernels in the frequency domain, using a kernel contour shift to suppress time-domain aliasing and recover accurate FFT training.\"}]",1784201641,58,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"dssms-state-space-models-with-explicit-memory-via-delay-differential-equations","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/dssms-state-space-models-with-explicit-memory-via-delay-differential-equations/85190/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do standard state space models struggle with long-range precise retrieval?","Question",{"text":75,"@type":76},"They must compress an unbounded history into a fixed-size state represented by exponentially decaying modes, which makes exact copying and precise long-range retrieval harder than attention-based models.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is a Delay State Space Model (DSSM)?",{"text":80,"@type":76},"A DSSM is a delay differential equation-inspired extension of diagonal SSMs that augments discrete SSM recurrences with explicit delayed-state feedback to improve long-range memory.",{"name":82,"@type":73,"acceptedAnswer":83},"How does DSSM enable efficient FFT training despite delayed feedback?",{"text":84,"@type":76},"It derives the DSSM transfer function and computes kernels in the frequency domain, using a kernel contour shift to suppress time-domain aliasing and recover accurate FFT training.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]