[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84523-en":3,"doc-seo-84523-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},84523,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers","Language models typically perform chain-of-thought (CoT) reasoning by generating explicit intermediate tokens sequentially, which is computationally expensive. Latent CoT instead executes multi-step reasoning in hidden states, avoiding decoded intermediate tokens and improving efficiency. Existing latent methods, however, lag behind explicit CoT beyond 1B parameters, with the gap increasing at larger scales. This work introduces LOTUS, a looped padded Transformer using parallel supervision on gold CoT steps to bridge that gap at the 3B scale while reducing thought-phase latency.","arXiv :2606 .3 1779v2 [ cs .LG] 13 Jul 2026  \nBridging the Gap Between Latent and Explicit Reasoning with Looped Transformers  \nYing Fan 1˚ Anej Svete2 Kangwook Lee3 ,4  \n1Microsoft Research 2ETH Zürich 3 KRAFTON 4Ludo Robotics  \nAbstract  \nLanguage models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token. Latent CoT offers an alternative: it performs multi-step reasoning in the model’s hidden states, replacing decoded tokens with continuous representations for greater efficiency. However, existing latent CoT methods underperform explicit CoT beyond 1B parameters, and the gap widens with scale. Looped, or recurrent-depth, Transformers, which reuse their weights to increase computation depth without adding parameters, are a natural fit for latent reasoning. We therefore ask whether looped Transformers can bridge this gap.  \nWe answer affirmatively with a simple recipe: a looped padded Transformer that processes K latent blocks in parallel for R iterations, with a cross-entropy losson each latent position’s gold CoT-step token, similar to explicit CoT supervision.  \nWe instantiate it as LOTUS (Looped Transformers with parallel supervision on latents) . LOTUS is, to our knowledge, the first latent-CoT method to bridge the gap to explicit CoT at the 3B scale, while cutting thought-phase latency by 2.5 ˆ– 6.9 ˆ from compact math expressions to natural language. Projecting LOTUS’s post-loop latents through the base LM head recovers the gold reasoning steps and even surfaces alternative valid intermediate steps, evidence that its latent space is interpretable and CoT-aligned. Ablations confirm that both the looped backbone and the parallel supervision on gold CoT tokens are essential. Code is available at [https://github.com/yingfan-bot/lotus](https://github.com/yingfan-bot/lotus).  \n1 Introduction  \nScaling inference compute, i.e., letting a model “think” before it answers, has become a dominant lever for increasing language model capabilities, with stronger performance now coming from longer reasoning chains rather than from model size alone [DeepSeek-AI, 2025, OpenAI, 2026] . Chain-ofthought (CoT) reasoning [Wei et al., 2022], where the model emits intermediate reasoning steps, drives this trend. However, since each token must be decoded sequentially, generating a CoT of length N takes N sequential model evaluations, making reasoning costly.  \nLatent reasoning aims to achieve the same at a fraction of the cost: it carries out the intermediate computation in continuous hidden states rather than decoded tokens, condensing many steps into fewer model evaluations. On small backbones such as GPT-2 [Radford et al.], latent methods [Hao et al., 2025, Shen et al., 2025, Wei et al., 2025] already match CoT accuracy. Yet at the scales where CoT matters most, the promise breaks down: beyond 1B parameters, no existing latent method matches explicit CoT on math reasoning, and the gap widens with model size [Wei et al., 2025] .  \nWe identify two common issues in latent methods. (P1) Sequential generation: methods like Coconut [Hao et al., 2025], CODI [Shen et al., 2025], and SIM-CoT [Wei et al., 2025] produce latent thoughts in a pure autoregressive way, so the number of sequential forward passes still scales linearly with the number of latent tokens, keeping CoT’s sequential generation bottleneck while giving up the  \n˚ Preprint. Work partially done at UW-Madison.  \nGSM8K test accuracy (%)  \n70  \n65  \n60  \n55  \n50  \n45  \n(a) Math-expression CoT  \nGPT-2 124M Llama-3 . 2-1B Llama-3 . 2-3B  \nModel size  \n70  \n65  \n60  \n55  \n50  \n45  \n(b) Natural-language CoT (3B)  \nExplicit LOTUS KaVa CODI PCCoT CoT (ours)  \nFigure 1: LOTUS bridges the latent–explicit CoT accuracy gap across scale on GSM8K test.(a) Math-expression CoT across model sizes: prior latent methods fall further below explicit CoT asthe backbone grows, whereas LOTUS tracks the explicit-CoT ceiling while cutting thought-phase latency by 2.5","cbCaieXGIWv4X1Oh","https://ap.wps.com/l/cbCaieXGIWv4X1Oh","pdf",614206,5,1,30,"English","en",105,"# Abstract\n# Introduction\n## Background: chain-of-thought and cost\n## Latent reasoning: promise and scaling gap\n## Identified issues in latent methods (sequential bottleneck, lack of grounding)\n## Proposed approach: looped transformers with direct gold CoT supervision\n## Model and contributions: LOTUS","[{\"question\":\"Why is explicit chain-of-thought reasoning costly for language models?\",\"answer\":\"Explicit CoT requires sequential token decoding, so a longer reasoning chain needs more sequential model evaluations, making reasoning expensive.\"},{\"question\":\"What causes latent CoT methods to underperform explicit CoT at larger parameter scales?\",\"answer\":\"The gap arises from issues such as sequential generation that retains a bottleneck in forward passes and insufficient CoT grounding that lets latent traces drift during training at scale.\"},{\"question\":\"How does LOTUS bridge the gap between latent and explicit CoT?\",\"answer\":\"LOTUS uses a looped, recurrent-depth (weight-reusing) Transformer with padded latent blocks processed in parallel over multiple iterations, supervised directly on gold CoT-step tokens with a cross-entropy loss.\"}]",1784196304,76,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"bridging-the-gap-between-latent-and-explicit-reasoning-with-looped-transformers","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/bridging-the-gap-between-latent-and-explicit-reasoning-with-looped-transformers/84523/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-21","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why is explicit chain-of-thought reasoning costly for language models?","Question",{"text":76,"@type":77},"Explicit CoT requires sequential token decoding, so a longer reasoning chain needs more sequential model evaluations, making reasoning expensive.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What causes latent CoT methods to underperform explicit CoT at larger parameter scales?",{"text":81,"@type":77},"The gap arises from issues such as sequential generation that retains a bottleneck in forward passes and insufficient CoT grounding that lets latent traces drift during training at scale.",{"name":83,"@type":74,"acceptedAnswer":84},"How does LOTUS bridge the gap between latent and explicit CoT?",{"text":85,"@type":77},"LOTUS uses a looped, recurrent-depth (weight-reusing) Transformer with padded latent blocks processed in parallel over multiple iterations, supervised directly on gold CoT-step tokens with a cross-entropy loss.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,115,120,122,127,130,134],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":22,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":20,"slug":137},19,"General","general"]