[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-343245-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-343245-en":131},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","lightthinker-thinking-step-by-step-compression","LightThinker: Thinking Step-by-Step Compression","","Large language models (LLMs) excel at complex reasoning, yet their efficiency is constrained by the high memory and computational costs of generating long token sequences. The paper introduces LightThinker, a method that dynamically compresses intermediate thoughts during reasoning. It trains the model to decide when and how to compress, maps hidden states to compact gist tokens, and uses specialized attention masks to discard original chains. A Dependency metric quantifies compression by measuring reliance on historical tokens.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/lightthinker-thinking-step-by-step-compression/343245/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/lightthinker-thinking-step-by-step-compression/343245.png","ImageObject",300,407,{"name":42,"@type":43},"Bulrr","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-11-14","2026-09-22",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",12,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does LightThinker address?","Question",{"text":63,"@type":64},"It addresses the efficiency limits of LLM reasoning caused by memory and computation costs from generating many intermediate tokens.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does LightThinker compress intermediate thoughts?",{"text":68,"@type":64},"It trains the model to determine when and how to compress, converts hidden states into compact gist tokens, and uses specialized attention masks to reduce stored context.",{"name":70,"@type":61,"acceptedAnswer":71},"How is the degree of compression measured?",{"text":72,"@type":64},"The paper introduces the Dependency (Dep) metric, which quantifies how much the generation relies on historical tokens.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},343245,1790380483,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":25,"category_name":129,"show_sort_weight":97,"slug":130},19,"General","general",{"code":4,"msg":82,"data":132},{"doc_id":79,"user_id":133,"nickname":42,"user_avatar":134,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":140,"language":141,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":12,"update_tm":145,"read_time":146,"preview_page_count":33,"preview_urls":147},5909892115043,"https://ap-avatar.wpscdn.com/davatar_6f874abed73319feea01a86fa6f0fab8","LightThinker: Thinking Step-by-Step Compression  \nJintian Zhang♠♡ * , Yuqi Zhu♠♡ * , Mengshu Sun♣♡ , Yujie Luo♠♡ , Shuofei Qiao♠♡ , Lun Du♣♡ , Da Zheng♣♡ , Huajun Chen♠♡ , Ningyu Zhang♠♡†  \n♠Zhejiang University ♣Ant Group  \n♡Zhejiang University-Ant Group Joint Laboratory of Knowledge Graph  \n{zhangjintian,[zhangningyu}@zju.edu.cn](zhangningyu}@zju.edu.cn)  \n [https://github.com/zjunlp/LightThinker](https://github.com/zjunlp/LightThinker)  \nAbstract  \nLarge language models (LLMs) have shown remarkable performance in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens. In this paper, we propose LightThinker, a novel method that enables LLMs to dynamically compress intermediate thoughts during reasoning.  \nInspired by human cognitive processes, LightThinker compresses verbose thought steps into compact representations and discards the original reasoning chains, thereby significantly reducing the number of tokens stored in the context window. This is achieved by training the model on when and how to perform compression through data construction, mapping hidden states to condensed gist tokens, and creating specialized attention masks. Additionally, we introduce the Dependency (Dep) metric to quantify the degree of compression by measuring the reliance on historical tokens during generation. Extensive experiments on four datasets and two models show that LightThinker reduces peak memory usage and inference time, while maintaining competitive accuracy. Our work provides a new direction for improving the efficiency of LLMs in complex reasoning tasks without sacrificing performance.  \n1 Introduction  \nRecent advancements in Large Language Models (LLMs) have demonstrated their remarkable capabilities in complex reasoning tasks (Zhao et al., 2023 ; Azaria et al., 2024) . As research in this domain progresses, the reasoning patterns of these models have gradually evolved from “fast thinking”to “slow thinking”. This transition is exemplified by methods such as Chain-of-Thought (CoT) (Wei et al., 2022) prompting, which enhances reasoning by breaking down complex problems into sequential sub-steps. Building on this, the o1-like  \n* Equal Contribution.† Corresponding Author.  \nInput:  \nMike has 12 apples. He gives away half of his apples to his friend and then buys 5 more apples from the market. How many apples does Mike have now?  \nOutput:  \n[Thought1] Frist, Mike starts with 12 apples and gives away half of them, which is 12 ÷ 2 = 6 apples, leaving him with 12 −  6 6 apples. [Thought2] Then, He buys 5 more apples from the market, bringing him total to 6 + 5 = 11 apples. [Answer] Therefore, Mike now has 11 apples.  \n(a) A CoT example requiring two-step reasoning.  \n(b) Diagram of the reasoning process for Vanilla and LightThinker  \nFigure 1: (a) A CoT case. Tokens highlighted in yellow represent critical reasoning tokens, while the remaining tokens primarily ensure fluency. Humans typically only write the yellow parts when solving this problem. (b) Comparison of reasoning between Vanilla and LightThinker.“C Ti” denotes the i-th compressed thought representation, and we illustrate the semantic information potentially expressed after compression.  \nthinking mode (Jaech et al., 2024 ; Qwen., 2024 ; DeepSeek-AI et al., 2025) introduces multiple reasoning abilities such as trial-and-error, backtracking, correction, and iteration, further improving the success rate of models in solving complex problems. However, this performance improvement comes at the cost of generating a large number of tokens (Wu et al., 2024b) . Given that current LLMs are predominantly based on the Transformer architecture (Vaswani et al., 2017), the computational complexity of the attention mechanism grows quadratically with the context length, while the storage overhead of the KV Cache increases linearly with the context length. For example, in the case of Qwen32B (Yang et al.","cbCairrir8e0Ccs7","https://ap.wps.com/l/cbCairrir8e0Ccs7","pdf",7518162,22,"English","# Introduction\n## Motivation and related approaches\n## Dynamic compression method\n## Dependency metric\n## Experiments and results","[{\"question\":\"What problem does LightThinker address?\",\"answer\":\"It addresses the efficiency limits of LLM reasoning caused by memory and computation costs from generating many intermediate tokens.\"},{\"question\":\"How does LightThinker compress intermediate thoughts?\",\"answer\":\"It trains the model to determine when and how to compress, converts hidden states into compact gist tokens, and uses specialized attention masks to reduce stored context.\"},{\"question\":\"How is the degree of compression measured?\",\"answer\":\"The paper introduces the Dependency (Dep) metric, which quantifies how much the generation relies on historical tokens.\"}]","LightThinker: Thinking Step-by-Step Compression | PDF",1790050411,55,[148,152,156,160],{"page_no":22,"url":149,"width":150,"height":151},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-16/77984e0f3b6621ed1cbd027019c672b2.webp",983,1390,{"page_no":26,"url":153,"width":154,"height":155},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-16/ffb0ccac0db93e3621abe9688b22e958.webp",982,1388,{"page_no":30,"url":157,"width":158,"height":159},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-16/b396a6511ff093a9fe1f0522bda666af.webp",981,1387,{"page_no":33,"url":161,"width":162,"height":163},"https://res-academy.cache.wpscdn.com/upload/kdocshare/2026-11-16/cd4c6ed91317e75aa7e7d644908a0e44.webp",980,1385]