[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"detail-sidebar-cat-0-en-105":3,"doc-seo-267425-105":59,"doc-detail-267425-en":131},{"code":4,"msg":5,"data":6},0,"success",[7,13,18,23,28,33,38,43,48,51,55],{"id":8,"doc_module":4,"doc_module_name":9,"category_name":10,"show_sort_weight":11,"slug":12},1,"Document","Story & Novel",90,"story-novel",{"id":14,"doc_module":4,"doc_module_name":9,"category_name":15,"show_sort_weight":16,"slug":17},2,"Literature",80,"literature",{"id":19,"doc_module":4,"doc_module_name":9,"category_name":20,"show_sort_weight":21,"slug":22},4,"Exam",70,"exam",{"id":24,"doc_module":4,"doc_module_name":9,"category_name":25,"show_sort_weight":26,"slug":27},5,"Comic",60,"comic",{"id":29,"doc_module":4,"doc_module_name":9,"category_name":30,"show_sort_weight":31,"slug":32},6,"Technology",50,"technology",{"id":34,"doc_module":4,"doc_module_name":9,"category_name":35,"show_sort_weight":36,"slug":37},7,"Healthcare",40,"healthcare",{"id":39,"doc_module":4,"doc_module_name":9,"category_name":40,"show_sort_weight":41,"slug":42},8,"Research & Report",30,"research-report",{"id":44,"doc_module":4,"doc_module_name":9,"category_name":45,"show_sort_weight":46,"slug":47},9,"Religion & Spirituality",20,"religion-spirituality",{"id":46,"doc_module":4,"doc_module_name":9,"category_name":49,"show_sort_weight":46,"slug":50},"World Cup","world-cup",{"id":52,"doc_module":4,"doc_module_name":9,"category_name":53,"show_sort_weight":52,"slug":54},10,"Lifestyle","lifestyle",{"id":56,"doc_module":4,"doc_module_name":9,"category_name":57,"show_sort_weight":24,"slug":58},19,"General","general",{"code":4,"msg":60,"data":61},"ok",{"site_id":62,"language":63,"slug":64,"title":65,"keywords":66,"description":67,"schema_data":68,"social_meta":124,"head_meta":126,"extra_data":128,"updated_unix":130},105,"en","guiding-through-complexity-what-makes-good-supervision-for-hard-reasoning-tasks","Guiding Through Complexity - What Makes Good Supervision for Hard Reasoning Tasks?","","Hard reasoning tasks strain both average human annotators and existing AI systems, making effective supervision essential for LLM alignment. This paper studies data-driven strategies that vary supervision quality across task complexity. It compares supervision from hard full tasks versus supervision from easier subtasks and finds that even noisy hard-task supervision can outperform perfectly correct subtask supervision on multiple hard math benchmarks. The key driver is step-wise error severity, not just outcome error rate.",{"@graph":69,"@context":123},[70,84,106],{"@type":71,"itemListElement":72},"BreadcrumbList",[73,77,79,82],{"item":74,"name":75,"@type":76,"position":8},"https://docshare.wps.com","Home","ListItem",{"item":78,"name":9,"@type":76,"position":14},"https://docshare.wps.com/document/",{"item":80,"name":40,"@type":76,"position":81},"https://docshare.wps.com/document/research-report/",3,{"item":83,"name":65,"@type":76,"position":19},"https://docshare.wps.com/document/guiding-through-complexity-what-makes-good-supervision-for-hard-reasoning-tasks/267425/",{"url":83,"name":65,"@type":85,"image":86,"author":91,"headline":65,"publisher":94,"fileFormat":97,"inLanguage":63,"description":67,"dateModified":98,"datePublished":99,"encodingFormat":97,"isAccessibleForFree":100,"interactionStatistic":101},"DigitalDocument",{"url":87,"@type":88,"width":89,"height":90},"https://docshare.wps.com/thumbnails/guiding-through-complexity-what-makes-good-supervision-for-hard-reasoning-tasks/267425.png","ImageObject",300,407,{"name":92,"@type":93},"Aria Callaghan","Person",{"url":74,"name":95,"@type":96},"DocShare","Organization","application/pdf","2026-10-23","2026-09-14",true,{"@type":102,"interactionType":103,"userInteractionCount":105},"InteractionCounter",{"@type":104},"ViewAction",13,{"@type":107,"mainEntity":108},"FAQPage",[109,115,119],{"name":110,"@type":111,"acceptedAnswer":112},"What two supervision strategies does the paper compare for hard reasoning tasks?","Question",{"text":113,"@type":114},"It compares supervising from hard full tasks directly (hard full task supervision) versus supervising from corresponding easier subtasks (subtask supervision). Both are controlled by different supervision quality levels.","Answer",{"name":116,"@type":111,"acceptedAnswer":117},"How can noisy supervision on hard tasks still help performance?",{"text":118,"@type":114},"Even with high outcome error rates (e.g., 90%), training on hard-task supervision can outperform perfectly correct subtask supervision on multiple hard math benchmarks.",{"name":120,"@type":111,"acceptedAnswer":121},"Which factor more strongly affects training performance: outcome error rate or step-wise error rate?",{"text":122,"@type":114},"Step-wise error rate matters more. Similar outcome error rates but different step-wise error rates can produce a large accuracy gap on the MATH benchmark.","https://schema.org",{"og:url":83,"og:type":125,"og:title":65,"og:site_name":95,"og:description":67},"article",{"robots":127,"canonical":83},"index,follow",{"doc_id":129,"site_id":62},267425,1789418457,{"code":4,"msg":5,"data":132},{"doc_id":129,"user_id":133,"nickname":92,"user_avatar":134,"doc_module":4,"category_id":39,"category_name":40,"doc_title":65,"doc_description":67,"doc_content":135,"file_id":136,"file_url":137,"file_type":138,"file_size":139,"view_count":105,"is_deleted":4,"is_public":8,"is_downloadable":8,"audit_status":8,"page_count":140,"language":141,"language_code":63,"site_id":62,"html_lang":63,"table_of_contents":142,"faqs":143,"seo_title":144,"seo_description":67,"update_tm":130,"read_time":145,"preview_page_count":4},962084926284,"https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0","Guiding Through Complexity: What Makes Good Supervision for Hard  \nReasoning Tasks?  \nXuan He*♣♣  \nDa Yin∗♠ Nanyun Peng♠ Tsinghua University  \n♠ University of California, Los Angeles  \n[hexuan21@mails.tsinghua.edu.cn](hexuan21@mails.tsinghua.edu.cn) , {da.yin,[violetpeng}@cs.ucla.edu](violetpeng}@cs.ucla.edu)  \nAbstract  \nHow can “weak teacher models” (Bowmanet al., 2022) such as average human annotators or existing AI systems, effectively supervise LLMs to improve performance on hard reasoning tasks, especially those that challenge and requires expertise or daily practice from the teacher models? In this paper, we seek for empirical answers to this question by investigating various data-driven strategies that offer supervision data at different quality levels upon tasks of varying complexity. Two intuitive strategies emerge for teacher models to provide supervision during alignment training: 1) using lower-quality supervision from hard full tasks that match the difficulty of the target reasoning tasks, and 2) leveraging higher-quality supervision from easier subtasks that are less challenging. Interestingly, we find that even when the outcome error rate for hard task supervision is high (e.g., 90%), training on such data can outperform perfectly correct supervision on easier subtasks on multiple hard math benchmarks. We further identify a more critical factor influencing training performance:  \nstep-wise error rates, which indicate the severity of errors in solutions. Specifically, training on hard task supervision with the same outcome error rates but disparate step-wise error rates can lead to a 30% accuracy gap on MATH benchmark. Our results also reveal that supplementing hard task supervision with the corresponding subtask supervision can yield notable performance improvements than simply combining rephrased hard full task supervision, suggesting new avenues for data augmentation. Data and code will be released at [https:](https:)//[github.com/hexuan21/Weak-to-Strong](github.com/hexuan21/Weak-to-Strong).  \n1 Introduction  \nThe rapid advancements in Large Language Models (LLMs) (OpenAI, 2024b ; Anthropic, 2024 ;  \n* These two authors contributed equally. The order of authorship decided by the flip of a coin.  \nReid et al., 2024 ; Dubey et al., 2024) indicate that AI systems will soon be capable of progressively solving hard reasoning tasks that challenge both humans and current LLMs. To enhance the performance of today’s AI systems (Bowman et al., 2022 ; Burns et al., 2023) on such hard reasoning tasks, especially those that exceed the reasoning abilities of the average human or current LLMs, what strategies should we adopt to consistently and effectively supervise the AI systems?  \nWe are concerned what supervision data those“weak teacher models”, humans and current LLMs, should annotate or synthesize for better hard downstream task performance. We mainly consider two possible data-driven supervision strategies – Strategy 1: Sampling or annotating solutions directly on the hard tasks that both human annotators and AI models typically struggle with; Strategy 2: Sampling or annotating solutionson hard tasks’ corresponding subtasks, which are more manageable and where humans or AI models are more likely to succeed (Gülçehre and Bengio, 2016 ; Glasmachers, 2017 ; Wies et al., 2023 ; Jothimurugan et al., 2023) . For Strategy 1, while the tasks used for supervision match the reasoning difficulty of our studied hard downstream tasks, the quality of supervision may be compromised due to the difficulty that challenges the human and AI teacher models. In contrast, Strategy 2 involves easier tasks that do not match the difficulty level of the downstream tasks, but the supervision quality tends to be higher because these tasks are simpler for annotators or models to handle. Should we obtain low-quality supervision for hard tasks, or employ high-quality supervision for decomposed subtasks for fine-tuning?  \nTo fairly compare the two s","cbCaikCLiG7kaCgU","https://ap.wps.com/l/cbCaikCLiG7kaCgU","pdf",2107485,41,"English","# Abstract\n# Introduction\n## Supervision strategies and simulated setting\n## Experimental findings across hard math tasks\n# Data, evaluation, and key factors","[{\"question\":\"What two supervision strategies does the paper compare for hard reasoning tasks?\",\"answer\":\"It compares supervising from hard full tasks directly (hard full task supervision) versus supervising from corresponding easier subtasks (subtask supervision). Both are controlled by different supervision quality levels.\"},{\"question\":\"How can noisy supervision on hard tasks still help performance?\",\"answer\":\"Even with high outcome error rates (e.g., 90%), training on hard-task supervision can outperform perfectly correct subtask supervision on multiple hard math benchmarks.\"},{\"question\":\"Which factor more strongly affects training performance: outcome error rate or step-wise error rate?\",\"answer\":\"Step-wise error rate matters more. Similar outcome error rates but different step-wise error rates can produce a large accuracy gap on the MATH benchmark.\"}]","Guiding Through Complexity - What Makes Good Supervision for Hard Reasoning Tasks? | PDF",103]