[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83919-en":3,"doc-seo-83919-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83919,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Reason Reward Refine Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models","Physics reasoning fails structurally in small language models: a mistake at any step propagates forward and corrupts subsequent inferences. Limited domain knowledge, hallucination during multi-step derivation, and distributional sensitivity compound this issue. The framework introduces a step-level reward that locates the first reasoning error, produces targeted structured feedback, and trains revision via policy gradient with KL regularization while avoiding ground-truth solutions as generation targets. Across five physics benchmarks, it improves accuracy by 17–20% versus CoT prompting and 10–16% versus the strongest baseline, also sharply reducing calculation, miscomprehension, and conceptual misapplication errors.","Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models  \nRaj Jaiswal 1∗, Dhruv Jain2∗, Rishabh Dhawan 1∗, Sree Krishna Uppalapati 1∗,  \nShin’ichi Satoh4 , Tanuja Ganu5 , Rajiv Ratn Shah3  \n1IIIT Delhi 2IIT BHU 3IIT Kanpur 4NII Tokyo 5Microsoft Research India  \n{jaiswalp, rishabh23002, [sree23533}@iiitd.ac.in](sree23533}@iiitd.ac.in)  \n[dhruv.jain.ece21@itbhu.ac.in](dhruv.jain.ece21@itbhu.ac.in) , [satoh@nii.ac.jp](satoh@nii.ac.jp) , [rajivratn@iitk.ac.in](rajivratn@iitk.ac.in) , [tanuja.ganu@microsoft.com](tanuja.ganu@microsoft.com)  \n∗Equal contribution  \narXiv :2607 .05 199v 1 [ cs .AI] 6 Jul 2026  \nAbstract  \nPhysics reasoning fails structurally in small language models: an error at any step propagates forward, corrupting every inference that follows. Limited domain knowledge, hallucination under multi-step derivation, and distributional sensitivity compound this failure. We propose a step-level reward framework that identifies the first reasoning error, generates targeted structured feedback, and trains the model to revise its solution via policy gradient with KL regularization, without exposing it to ground truth solutions as generation targets. Unlike annotation-dependent step-level methods, no preference data construction is required and the external verifier operates exclusively at training time. Across five physics benchmarks, our framework delivers accuracy gains of 17–20% over CoT prompting and 10–16% over the strongest baseline, reduces calculation errors from 56.9% to 23.5%, and reduces miscomprehension errors from 22.3% to 12.0% in the best observed cases. Conceptual errors reduce from 89.7% to 68.7%, yet persist as the hardest failure mode across all conditions.  \nCode, prompts, and experimental details are provided in the Appendix section (§9) .  \n1 Introduction  \nPhysics reasoning is inherently sequential: each solution step depends on the correctness of all preceding steps, and an error at any point invalidates every inference that follows (Jaiswal et al., 2024 ; Anand et al., 2024 ; Ding et al., 2023) . Language models have made substantial progress on reasoning tasks (Wei et al., 2022 ; Fu et al., 2023 ; Zhang et al., 2022), yet physics reasoning remains an open challenge for small language models (SLMs) (Srivastava et al., 2025 ; Boye and Myrberg, 2025) . Larger models benefit from knowledge compression across broad training distributions; smaller models do not, and the reasoning failures that  \nTable 1: Our Framework Achieves the Highest Average Accuracy Across All Conditions. Per-model average (%) across five benchmarks; Cell shading indicates per-row rank across conditions. Discussed in §5 .  \n\n| Model | CoT | RAG | SFT | DPO | Ours |\n| --- | --- | --- | --- | --- | --- |\n| Qwen 2.5 1.5B | \u003Cbr>46.1 | \u003Cbr>53.3 | 41.0 | 40.7 | \u003Cbr>58.5 |\n| LLaMA 3.2 1B | 32.5 | 41.1 | 37.5 | 37.5 | 53.9 |\n| LLaMA 3.2 3B | 44.8 | 51.2 | 44.0 | 44.1 | 64.4 |\n| Phi 3.5 Mini 3.8B | 50.5 | 54.3 | 51.0 | 52.1 | 61.5 |\n\nemerge are structurally distinct from those observed at scale (Zhang et al., 2024 ; Srivastava et al., 2025) .  \nPrior work has progressively improved physics reasoning in language models, with each approach addressing a distinct limitation (Ding et al., 2023 ; Pang et al., 2024 ; Anand et al., 2024) . Chain-ofThought prompting (Wei et al., 2022) improves reasoning transparency by externalizing intermediate steps, though step-level error propagation across dependent steps remains unaddressed. RetrievalAugmented Generation (Lewis et al., 2020) provides access to relevant domain knowledge at inference time, though correct retrieval does not guarantee correct application of retrieved content (Anand et al., 2024) . Supervised fine-tuning on expert reasoning trajectories (Ho et al., 2022 ; Luo et al., 2023) improves structural response quality, though models trained under token prediction objectives have been shown to reproduce surface reasoning pa","cbCaivhnVvj0I33a","https://ap.wps.com/l/cbCaivhnVvj0I33a","pdf",3598784,1,40,"English","en",105,"# Introduction\n## Step-Level Reward Mechanism\n## Error-Type Reduction via Structured Feedback\n## Comparison with Prior Work\n## Contributions and Results","[{\"question\":\"Why do errors in small language models fail to recover in physics reasoning?\",\"answer\":\"Physics solutions are sequential: an incorrect step invalidates all following inferences. In small models, this propagation is amplified by limited domain knowledge and multi-step hallucination.\"},{\"question\":\"What is the key idea of the proposed step-level reward framework?\",\"answer\":\"It identifies the first reasoning error and assigns a step-level reward, enabling the model to correct at the precise failure point rather than only optimizing complete responses.\"},{\"question\":\"How does structured feedback affect different error types during training?\",\"answer\":\"Feedback conditioned on error type is generated during training, reducing calculation errors, problem miscomprehension, and conceptual misapplication, with the largest observed improvements reported for conceptual misapplication.\"}]",1784191446,101,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"reason-reward-refine-step-level-errors-corrections-with-structured-feedback-for-physics-reasoning-in-small-language-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/reason-reward-refine-step-level-errors-corrections-with-structured-feedback-for-physics-reasoning-in-small-language-models/83919/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do errors in small language models fail to recover in physics reasoning?","Question",{"text":75,"@type":76},"Physics solutions are sequential: an incorrect step invalidates all following inferences. In small models, this propagation is amplified by limited domain knowledge and multi-step hallucination.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the key idea of the proposed step-level reward framework?",{"text":80,"@type":76},"It identifies the first reasoning error and assigns a step-level reward, enabling the model to correct at the precise failure point rather than only optimizing complete responses.",{"name":82,"@type":73,"acceptedAnswer":83},"How does structured feedback affect different error types during training?",{"text":84,"@type":76},"Feedback conditioned on error type is generated during training, reducing calculation errors, problem miscomprehension, and conceptual misapplication, with the largest observed improvements reported for conceptual misapplication.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":21,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]