[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-81591-en":3,"doc-seo-81591-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},81591,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","Latent Thoughts Tuning Bridging Context and Reasoning with Fused Information in Latent Tokens","Explicit Chain-of-Thought (CoT) strengthens large language model reasoning but forces thoughts into a discrete token vocabulary, limiting flexibility. Recent continuous latent-space reasoning can avoid explicit text constraints, yet existing approaches often experience feature collapse and instability from distribution mismatch, or misalignment from reliance on assistant models. Latent Thoughts Tuning (LT-Tuning) is a post-training framework that rebuilds latent thought construction and deployment via Context-Prediction-Fusion and a three-stage curriculum, enabling dynamic switching between latent and explicit thinking modes.","Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens  \nWeihao Liu 1 Dehai Min 1 Lu Cheng 1  \narXiv :2602 . 10229v2 [ cs .CL] 10 Jul 2026  \nAbstract  \nWhile explicit Chain-of-Thought (CoT) equips Large Language Models (LLMs) with strong reasoning capabilities, it constrains the model’s thoughts to a discrete vocabulary space. Recently, reasoning in continuous latent space has emerged as a promising alternative, but current paradigms suffer from feature collapse and instability due to distribution mismatch when recurrently reusing hidden states, or alignment issues when relying on assistant models. To address this, we propose Latent Thoughts Tuning (LT-Tuning), a post-training framework that redefines how latent thoughts are constructed and deployed. Instead of relying solely on raw hidden states, our method introduces a Context-Prediction-Fusion mechanism that jointly leverages contextual hidden states and predictive semantic guidance from the vocabulary embedding space. Combined with a progressive three-stage curriculum learning pipeline, LT-Tuning also enables dynamic switching between latent and explicit thinking modes.  \nExperiments demonstrate that our method outperforms existing latent reasoning baselines, effectively mitigating feature collapse and achieving robust reasoning accuracy. 2  \n1. Introduction  \nThe capability of Large Language Models (LLMs) to perform multi-step reasoning has largely depended on generating explicit text steps, known as Chain-of-Thought (CoT)(Wei et al., 2022 ; Chen et al., 2023) . Although effective, this approach requires the model to perform reasoning ina discrete token sequence, which means the model can not  \n1Department of Computer Science, University of Illinois Chicago, Chicago, IL, USA. Correspondence to: Weihao Liu \u003C[wliu681@uic.edu](wliu681@uic.edu) >, Lu Cheng \u003C[lucheng@uic.edu](lucheng@uic.edu) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \n2[https://github.com/NeosKnight233/Latent-Thoughts-Tuning](https://github.com/NeosKnight233/Latent-Thoughts-Tuning)  \nFigure 1. Comparison of reasoning paradigms. Explicit CoT verbalizes all steps as text tokens. Coconut uses a fixed number of latent tokens from hidden states. Soft-Thinking constructs latent tokens via probability-weighted interpolation with entropy-based stopping. Assistant-based methods rely on external models. Our LT-Tuning dynamically interleaves text and latent tokens through confidence-driven insertion and Context-Prediction Fusion.  \n“think twice before acting”, or they demand extra cost on extremely long text output (Jaech et al., 2024 ; Min et al., 2026 ; Yeo et al., 2025 ; Guo et al., 2025 ; Seed et al., 2025) and self-reflection (Renze & Guven, 2024 ; Kang et al., 2025 ; Yu et al., 2025) .  \nMotivated by these limitations, recent work has explored reasoning in continuous latent spaces as an alternative (Zhu et al., 2025a ; Chen et al., 2025) . By allowing models to reason directly in high-dimensional hidden states rather than explicit tokens (Hao et al., 2024 ; Shen et al., 2025 ; Wei et al., 2025), this line of research aims to decouple internal reasoning from explicit text generation. While promising,  \nlatent-space reasoning methods face two fundamental challenges:  \n• Constructing well-aligned latent representations. Latent tokens must be semantically expressive while remaining compatible with the model’s internal embedding space. Methods relying on external assistant models (Xu et al., 2025 ; He et al., 2025) struggle with representational misalignment, whereas purely intrinsic approaches (Chen et al., 2025) risk distribution mismatch between input embeddings and output hidden states—particularly in models with untied input and output embeddings—which can lead to instability or feature collapse.  \n• Adapting reasoning cost dynamically. Most existing methods employ static","cbCaidotxLMhba2r","https://ap.wps.com/l/cbCaidotxLMhba2r","pdf",667030,1,21,"English","en",105,"# Abstract\n# Introduction\n## Limitations of explicit and existing latent reasoning\n## Latent Thoughts Tuning (LT-Tuning) overview and core innovations\n## Main contributions and empirical evaluation","[{\"question\":\"What problem does LT-Tuning aim to solve in latent-space reasoning?\",\"answer\":\"It targets instability and feature collapse caused by distribution mismatch when reusing hidden states, as well as misalignment issues that can arise when relying on assistant models.\"},{\"question\":\"How does Context-Prediction-Fusion construct latent thoughts?\",\"answer\":\"It fuses contextual hidden states with predictive semantic guidance derived from the vocabulary embedding space, bridging the output space and the input embedding manifold.\"},{\"question\":\"What training strategy does LT-Tuning use to improve stability?\",\"answer\":\"LT-Tuning applies a progressive three-stage curriculum that transitions from purely explicit CoT toward reasoning with latent thoughts, mitigating latent-space optimization issues.\"}]",1784174569,53,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"latent-thoughts-tuning-bridging-context-and-reasoning-with-fused-information-in-latent-tokens","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/latent-thoughts-tuning-bridging-context-and-reasoning-with-fused-information-in-latent-tokens/81591/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does LT-Tuning aim to solve in latent-space reasoning?","Question",{"text":75,"@type":76},"It targets instability and feature collapse caused by distribution mismatch when reusing hidden states, as well as misalignment issues that can arise when relying on assistant models.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Context-Prediction-Fusion construct latent thoughts?",{"text":80,"@type":76},"It fuses contextual hidden states with predictive semantic guidance derived from the vocabulary embedding space, bridging the output space and the input embedding manifold.",{"name":82,"@type":73,"acceptedAnswer":83},"What training strategy does LT-Tuning use to improve stability?",{"text":84,"@type":76},"LT-Tuning applies a progressive three-stage curriculum that transitions from purely explicit CoT toward reasoning with latent thoughts, mitigating latent-space optimization issues.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]