[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85208-en":3,"doc-seo-85208-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85208,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","Structured Thoughts For Improved Reasoning And Context Pruning","Large language models can produce lengthy reasoning traces that are verbose and inefficient for memory and context. This work introduces Structured Thoughts, formatting reasoning as alternating \u003Ctry> and \u003Coutcome> blocks where each outcome summarizes the corresponding try. A dataset is built by segmenting traces and prompting an LLM to generate step summaries. Fine-tuning on this structured data yields up to 8.08% gains on reasoning benchmarks versus standard SFT. The explicit interface also supports context pruning by masking try tokens during training, enabling inference-time removal of scratch while retaining conclusions, achieving about 85% context savings with an 8.67% performance drop on mathematical tasks.","STRUCTURED THOUGHTS FOR IMPROVED REASONING AND CONTEXT PRUNING  \narXiv :2607 . 10386v 1 [ cs .CL] 11 Jul 2026  \nZain Sarwar  \nUniversity of Chicago  \n[zsarwar@uchicago.edu](zsarwar@uchicago.edu)  \nSupriyo Chakraborty Berkcan Kapusuzoglu  \nChia-Hsuan Lee Anirban Das  \nStephen Rawls Kartik Balasubramaniam  \nSambit Sahu  \nCapital One  \nABSTRACT  \nLarge language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating \u003Ctry> and \u003Coutcome> blocks: \u003Ctry> captures exploratory scratch work, while \u003Coutcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into \u003Ctry> blocks and prompting an LLM to summarize each step into its corresponding \u003Coutcome> . Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each \u003Ctry>/\u003Coutcome> pair, the \u003Ctry> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85% memory / context savings with an 8.67% performance drop across mathematical tasks.  \n1 INTRODUCTION  \nStandard LLMs often fail on tasks that require multi-step logical or mathematical reasoning. Reasoning models extend LLM capabilities on such complex tasks by explicitly generating intermediate steps before arriving at a final answer. This distinction between generic LLMs and reasoningoriented LLMs has proven to be critical for performing well on challenging domains such as competition-level mathematics, logical reasoning and programming (OpenAI, 2024; Comanici et al., 2025; Guo et al., 2025; Bercovich et al., 2025)  \nTransforming a pretrained model into a reasoning model requires post-training with reasoning traces. The most common first step is supervised fine-tuning (SFT), where the model is trained to imitate curated chains-of-thought (CoT) that spell out intermediate steps (Wei et al., 2022) . SFT is critical because it provides the model with an initial format and skill set for reasoning. Once the model has learned to reliably generate reasoning trajectories, reinforcement learning (RL) methods are applied to further refine them. RL teaches the model to prefer reasoning trajectories that lead to the correct answers. Recent large-scale efforts illustrate the power of this recipe. OpenAI’s o3 ando3-mini models, and DeepSeek’s R1 (OpenAI, 2025; Guo et al., 2025) and other models combine SFT and RL on large scale reasoning datasets, yielding state-of-the-art performance on mathematical and coding benchmarks. This indicates that reasoning-oriented post-training can substantially extend the capabilities of pretrained LLMs.  \nA major focus of current reasoning research is on improving the RL stage of post-training. These include introducing process supervision to reward intermediate steps (Lightman et al., 2023; Uesato et al., 2022; Shao et al., 2024) as well as refining the optimization objective/algorithm itself, as in recent analyses of R1-zero-style training (Liu et al.) . In contrast, our work does not modify the optimization procedure or reward design. We impose a syntax in which each problem-solving step is organized into two blocks: a \u003Ctry> block containing exploratory scratch work (mathematical derivations, verification, intermediate proofs) and an \u003Coutcome> block which contains the  \nconclusion of the step. This alternation enforces a clean interface between working and result, analogous to human reasoning where we deliberate before summarizing a finding. Empirically, we find that finetuning models on such type of structured traces improves benchmark perf","cbCaiawATwhTLskA","https://ap.wps.com/l/cbCaiawATwhTLskA","pdf",507203,3,1,13,"English","en",105,"# Introduction\n## Related Work","[{\"question\":\"What are Structured Thoughts and how do they organize reasoning?\",\"answer\":\"Structured Thoughts format reasoning into alternating \\u003ctry\\u003e and \\u003coutcome\\u003e blocks. The \\u003ctry\\u003e holds exploratory scratch work, while the \\u003coutcome\\u003e contains the distilled conclusion for that step.\"},{\"question\":\"How is the training data for Structured Thoughts constructed?\",\"answer\":\"Reasoning traces are segmented into \\u003ctry\\u003e blocks, and an LLM is prompted to summarize each step into its corresponding \\u003coutcome\\u003e. The resulting structured pairs are used for fine-tuning.\"},{\"question\":\"How does Structured Thoughts enable context pruning at inference time?\",\"answer\":\"During training, \\u003ctry\\u003e blocks are masked so the model learns to rely on outcomes. At inference, completed \\u003ctry\\u003e spans can be pruned while retaining the \\u003coutcome\\u003e blocks, reducing context length and memory usage.\"}]",1784201758,33,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"structured-thoughts-for-improved-reasoning-and-context-pruning","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/structured-thoughts-for-improved-reasoning-and-context-pruning/85208/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What are Structured Thoughts and how do they organize reasoning?","Question",{"text":75,"@type":76},"Structured Thoughts format reasoning into alternating \u003Ctry> and \u003Coutcome> blocks. The \u003Ctry> holds exploratory scratch work, while the \u003Coutcome> contains the distilled conclusion for that step.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is the training data for Structured Thoughts constructed?",{"text":80,"@type":76},"Reasoning traces are segmented into \u003Ctry> blocks, and an LLM is prompted to summarize each step into its corresponding \u003Coutcome>. The resulting structured pairs are used for fine-tuning.",{"name":82,"@type":73,"acceptedAnswer":83},"How does Structured Thoughts enable context pruning at inference time?",{"text":84,"@type":76},"During training, \u003Ctry> blocks are masked so the model learns to rely on outcomes. At inference, completed \u003Ctry> spans can be pruned while retaining the \u003Coutcome> blocks, reducing context length and memory usage.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]