[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85521-en":3,"doc-seo-85521-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85521,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","BRIDGE Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking","Chain-of-Thought (CoT) reasoning improves LLM math solving, yet distilling it into smaller students is hindered by capacity mismatch between verbose teachers and compact learners. Directly copying long reasoning chains overwhelms students, causing truncation, repetition, or shallow imitation. BRIDGE introduces a structure-aware curriculum: masked reconstruction for dependency awareness, GRPO-based reinforcement to balance accuracy and brevity, and teacher-guided rewriting for failures. On GSM8K, BRIDGE boosts Qwen2.5-3B accuracy by 11.29% while reducing tokens by 27.4%, with zero-shot gains on SVAMP and MATH-500.","BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking  \nBowen Yu 1 , Sheng Zhang 1 , Binhao Wang 1 , Yi Wen 1 , Jingtong Gao 1 ,  \nBowen Liu 1 , Zimo Zhao 1 , Shanshan Ye2 , Wanyu Wang 1 , Maolin Wang 1∗, Xiangyu Zhao 1∗ 1 City University of Hong Kong 2 Mohamed Bin Zayed University of Artificial Intelligence  \n{bowyu2-c, szhang844-c, binhawang2-c, wenyiwy2022, jt.g,  \nboweliu6-c, zmzhao6-c, [wanyuwang4-c}@my.cityu.edu.hk](wanyuwang4-c}@my.cityu.edu.hk), [Morin.wang@my.cityu.edu.hk](Morin.wang@my.cityu.edu.hk), [xianzhao@cityu.edu.hk](xianzhao@cityu.edu.hk), [shanshan.ye@mbzuai.ac.ae](shanshan.ye@mbzuai.ac.ae)  \narXiv :2602 . 17686v 5 [ cs .LG] 12 Jul 2026  \nAbstract. Chain-of-Thought (CoT) reasoning has significantly improved LLMs’ mathematical problemsolving capabilities, but distilling such capabilities into smaller models remains challenging due to the capacity mismatch between verbose teachers and compact students. Directly copying teachers’ lengthy reasoning chains causes capacity overload, resulting in truncated outputs or repetitive failure. Existing remedies each sacrifice a critical property of CoT: implicit reasoning methods (e.g., compressing reasoning into hidden states) trade away interpretability and verifiability, while heuristic compression strategies (e.g., random step pruning) destroy logical integrity. To address this, we propose BRIDGE, a curriculum framework that first establishes structural understanding via masked reconstruction, then uses GRPO-based reinforcement learning to guide students in self-discovering the optimal balance between accuracy and brevity, and finally internalizes complex reasoning through teacher-guided rewriting on failure cases. On GSM8K, BRIDGE enables Qwen2 .5-3B to achieve 11.29% accuracy improvement and 27.4% token reduction over the original model, outperforming instruction-tuned variants and distillation baselines. Zero-shot transfer experiments on SVAMP and MATH-500 further confirm the generalization of internalized reasoning. Our code and model checkpoints are publicly available at [https://github.com/](https://github.com/)[ ](https://github.com/)Applied-Machine-Learning-Lab/SDM2026 _BRIDGE and [https://huggingface.co/bowen0815/BRIDGE](https://huggingface.co/bowen0815/BRIDGE).  \n1 Introduction. Chain-of-Thought (CoT) reasoning has emerged as a transformative technique for eliciting complex problem-solving capabilities in large language models (LLMs) [33, 32, 26] . By prompting models to decompose tasks into explicit intermediate steps (e.g., via Chain-of-Thought exemplars), CoT enables models to tackle arithmetic and symbolic reason-  \ning challenges with remarkable success. For instance, prompting strategies have been shown to boost GSM8K accuracy from 17.9% to 58.1% in few-shot settings [29] and from 10.4% to 40.7% in zero-shot scenarios [10] . However, these gains are predominantly observed in massive models (e.g., models with tens of billions of parameters) . Deploying such capabilities in resourceconstrained environments requires distilling them into smaller models (e.g., 3B parameters), a process that remains computationally challenging.  \nA fundamental obstacle in this distillation process is the capacity mismatch between teacher and student [14] . Capable teachers (e.g., DeepSeek-R1-14B) often rely on lengthy reasoning chains to ensure correctness. When compact students (e.g., 3B models) attempt to reproduce these lengthy sequences via standard supervised fine-tuning, they lack the representational bandwidth to process or memorize such content effectively. This manifests as truncated outputs, repetition loops, or superficial mimicry without genuine understanding [19] .  \nSeveral approaches have been proposed to address this mismatch, yet none satisfies the need for explicit, verifiable reasoning. Implicit reasoning methods [3, 13, 24] bypass sequence length constraints by compressing reasoning into hidden states or continuous represe","cbCaiqQs06QgKt3j","https://ap.wps.com/l/cbCaiqQs06QgKt3j","pdf",2167464,1,13,"English","en",105,"# Introduction\n## Distillation and capacity mismatch\n## Prior approaches and their limitations\n## BRIDGE: structure-aware curriculum","[{\"question\":\"Why is distilling Chain-of-Thought reasoning into smaller models difficult?\",\"answer\":\"Because of capacity mismatch: verbose teacher reasoning chains are too long for compact student models, leading to truncated outputs, repetition loops, or superficial mimicry.\"},{\"question\":\"What are the three stages of BRIDGE?\",\"answer\":\"BRIDGE uses masked reconstruction to build structural understanding, then applies GRPO on masked completion to learn the accuracy–brevity trade-off, and finally performs teacher-guided rewriting on failure cases to internalize reasoning in concise form.\"},{\"question\":\"What improvements does BRIDGE report on GSM8K?\",\"answer\":\"BRIDGE enables Qwen2.5-3B to achieve an 11.29% accuracy improvement and reduce token usage by 27.4% compared with the original model, outperforming several instruction-tuned and distillation baselines.\"}]",1784204151,33,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"bridge-bridging-reasoning-in-distillation-gap-elimination-via-structure-aware-masking","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/bridge-bridging-reasoning-in-distillation-gap-elimination-via-structure-aware-masking/85521/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is distilling Chain-of-Thought reasoning into smaller models difficult?","Question",{"text":75,"@type":76},"Because of capacity mismatch: verbose teacher reasoning chains are too long for compact student models, leading to truncated outputs, repetition loops, or superficial mimicry.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the three stages of BRIDGE?",{"text":80,"@type":76},"BRIDGE uses masked reconstruction to build structural understanding, then applies GRPO on masked completion to learn the accuracy–brevity trade-off, and finally performs teacher-guided rewriting on failure cases to internalize reasoning in concise form.",{"name":82,"@type":73,"acceptedAnswer":83},"What improvements does BRIDGE report on GSM8K?",{"text":84,"@type":76},"BRIDGE enables Qwen2.5-3B to achieve an 11.29% accuracy improvement and reduce token usage by 27.4% compared with the original model, outperforming several instruction-tuned and distillation baselines.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]