[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85818-en":3,"doc-seo-85818-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85818,8796095461564,"Liam","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Looped State-Space Language Models with Adaptive Exit-State Selection","Recent research on looped language models shows that increased computational depth can benefit reasoning more than adding independent parameters. This work tests whether the same principle holds for state-space language models by studying Looped Mamba and Looped Hybrid Mamba–Transformer architectures that reuse shared Mamba (or hybrid) blocks in explicit finite-depth loops. On Mano and p-hop induction, Looped Mamba outperforms non-looped parameter-matched baselines and matches equal effective-depth models in several settings. Under matched iso-parameter and iso-FLOPs pre-training protocols, looped models stay competitive with far fewer distinct parameters, though deeper non-looped models can win on validation perplexity. An adapted Ouro-style two-stage exit gate enables threshold-controlled adaptive exit-state selection, improving downstream performance at intermediate depths while preserving per-step execution, so wall-clock savings require extra mechanisms.","Looped State-Space Language Models with Adaptive Exit-State Selection  \nZhenxuan Yu* Rikkyo University Yutaka Matsuo†  \nThe University of Tokyo  \nTakeshi Kojima† The University of Tokyo  \nYusuke Iwasawa† The University of Tokyo  \narXiv :2607 . 10 1 10v 1 [ cs .AI] 11 Jul 2026  \nAbstract  \nRecent work on looped language models suggests that many reasoning problems benefit from greater computational depth rather than from additional independent parameters. Existing studies, however, focus almost exclusively on Transformer backbones, leaving open whether this principle also applies to state-space language models. We investigate Looped Mamba and Looped Hybrid Mamba– Transformer architectures, which repeatedly apply a shared Mamba (or hybrid) block to introduce explicit finite-depth recurrent computation. On two controlled reasoning tasks—Mano (modular-arithmetic manipulation) and p-hop induction—Looped Mamba consistently outperforms parameter-matched non-looped baselines and, in several settings, matches or exceeds non-looped models of equal effective depth. We then extend the study to language model pre-training under matched isoparameter and iso-FLOPs protocols, which jointly disentangle the effects of parameter sharing and effective depth: looped models remain competitive on downstream benchmarks with substantially fewer distinct parameters, although deeper non-looped models retain an advantage in validation perplexity under strict isoFLOPs comparisons. Finally, we adapt Ouro’s two-stage exit gate to Looped Mamba for threshold-controlled selection among recurrentstep outputs. Since all recurrent steps are still executed, the selected exit step represents prediction depth rather than reduced wall-clock computation. At the scales studied, adaptive exit-state selection improves downstream performance at intermediate depths, while actual inference-time savings require additional statehandling mechanisms.1  \n* E-mail: [24wr006m@rikkyo.ac.jp](24wr006m@rikkyo.ac.jp)  \n†Email: {t.kojima, matsuo,  \n[iwasawa}@weblab.t.u-tokyo.ac.jp](iwasawa}@weblab.t.u-tokyo.ac.jp)[ ](iwasawa}@weblab.t.u-tokyo.ac.jp)1Code will be released publicly.  \n1 Introduction  \nLarge Language Models (LLMs) have achieved remarkable progress in language understanding, text generation, and complex reasoning (DeepSeek-AI et al., 2025 ; Yang et al., 2025) . However, these advances have largely been driven by scaling model size, training data, and compute-intensive training and test-time inference (Hoffmann et al., 2022 ; Snell et al., 2024) . This scaling paradigm introduces substantial challenges for practical deployment: model weights and intermediate activations increase memory consumption, while Transformerbased self-attention requires maintaining large keyvalue caches during long-context inference, leading to rapidly growing inference costs as context length increases (Beltagy et al., 2020) . Improving model capability under fixed parameter, FLOP, or memory budgets has therefore become a central problem in the design of efficient LLM architectures.  \nState Space Models (SSMs), particularly Mamba-2 (Gu and Dao, 2024 ; Dao and Gu, 2024), provide an efficient alternative to attention with favorable long-context scaling, and Hybrid MambaTransformer designs (Lieber et al., 2024 ; Waleffeet al., 2024 ; Blakeman et al., 2025) have further emerged as practical backbones for modern LLMs. Orthogonally, Looped Transformers reuse a shared block across depth to gain effective computational depth without extra parameters, and recent work shows that this benefits reasoning (Dehghani et al., 2019 ; Saunshi et al., 2025) .  \nOur core idea is to share the parameters of Mamba or hybrid blocks across layers and apply them repeatedly through explicit finite-depth loops, constructing a deeper effective computational path without increasing the number of distinct trainable parameters. This design allows us to disentangle the effects of parameter count, compute budget, and effective depth","cbCaie3o0pxnkLas","https://ap.wps.com/l/cbCaie3o0pxnkLas","pdf",484280,1,19,"English","en",105,"# Introduction\n# Related Work","[{\"question\":\"What problem does this paper address for efficient language model architectures?\",\"answer\":\"It addresses how to improve capability under fixed parameter, FLOP, or memory budgets, where scaling LLMs increases memory use and inference costs, especially for long-context attention.\"},{\"question\":\"How do Looped Mamba and Looped Hybrid Mamba–Transformer differ from standard models?\",\"answer\":\"They repeatedly apply a shared Mamba (or hybrid) block in explicit finite-depth loops, creating greater effective computational depth without increasing the number of distinct trainable parameters.\"},{\"question\":\"What is “adaptive exit-state selection” in this work and what does it achieve?\",\"answer\":\"It adapts Ouro’s two-stage exit gate to Looped Mamba to select a prediction depth via a threshold-controlled exit mechanism; all recurrent steps still run, so savings require additional state-handling, but downstream performance improves at intermediate depths.\"}]",1784206441,48,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"looped-state-space-language-models-with-adaptive-exit-state-selection","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/looped-state-space-language-models-with-adaptive-exit-state-selection/85818/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does this paper address for efficient language model architectures?","Question",{"text":75,"@type":76},"It addresses how to improve capability under fixed parameter, FLOP, or memory budgets, where scaling LLMs increases memory use and inference costs, especially for long-context attention.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do Looped Mamba and Looped Hybrid Mamba–Transformer differ from standard models?",{"text":80,"@type":76},"They repeatedly apply a shared Mamba (or hybrid) block in explicit finite-depth loops, creating greater effective computational depth without increasing the number of distinct trainable parameters.",{"name":82,"@type":73,"acceptedAnswer":83},"What is “adaptive exit-state selection” in this work and what does it achieve?",{"text":84,"@type":76},"It adapts Ouro’s two-stage exit gate to Looped Mamba to select a prediction depth via a threshold-controlled exit mechanism; all recurrent steps still run, so savings require additional state-handling, but downstream performance improves at intermediate depths.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]