[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83179-en":3,"doc-seo-83179-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83179,2336464648746,"Skyler","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation","Agentic language models need to learn when to call tools, when to use tool responses, and when to answer directly. Multi-teacher on-policy distillation can train specialization by letting one teacher focus on tool calls, another on direct responses, and the student learn from both during its own rollouts. The study finds a behavior shift that aggregate losses miss: vanilla GKD can raise over-calling even while improving tool-call recall. It attributes this to behavior leverage imbalance at token-level mode-entry and structural positions, then proposes Soft Clamp to calibrate extreme token divergences. On APIGen-MT and BFCL diagnostics, Soft Clamp reduces tool-call loops without harming decision accuracy.","arXiv :2607 .07050v 1 [ cs .CL] 8 Jul 2026  \nBehavior Leverage Imbalance in Multi-Teacher On-Policy  \nDistillation  \nJiabin Shen 1 , Guang Chen2 , and Chengjun Mao3  \n[1](1 tjusjb@gmail.com)[ tjusjb@gmail.com](1 tjusjb@gmail.com)  \n[2](2 cg234573@antgroup.com)[ cg234573@antgroup.com](2 cg234573@antgroup.com)  \n[3](3 chengjun.mcj@antgroup.com)[ chengjun.mcj@antgroup.com](3 chengjun.mcj@antgroup.com)  \nAbstract  \nAgentic language models must learn when to call tools, when to consume tool responses, and when to answer directly. This makes multi-teacher on-policy distillation a natural training strategy: one teacher can specialize in tool calls, another in direct responses, and the student can learn from both on its own generated distribution. We show that this strategy can induce a behavior shift that is invisible from aggregate losses alone. In a two-teacher tool-use setting, vanilla generalized knowledge distillation improves tool-call recall but also moves the model toward over-calling, where it calls tools on examples that should be answered directly. Aggregate explanations are insufficient: tool-call samples do not receive more token exposure, and full-sequence per-token divergence is not larger for the tool-call teacher. We instead analyze behavior leverage imbalance: local token-level signals at mode-entry and structural positions, such as \u003Ctool   call> and function names, can have disproportionate control over the global generation mode. We propose Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero gradients. On APIGen-MT, Soft Clamp reduces over-calling from 13.7% to 9.0% relative to vanilla GKD while matching its decision accuracy. In a BFCL multi-turn diagnostic, it also lowers tool-call loops and repeated calls among GKD variants. These results suggest that multi-teacher OPD should monitor where teacher signals act, not only how large they are in aggregate.  \n1 Introduction  \nAgentic language models often act through interleaved dialogue and tool-use trajectories, such as  \nSystem → User → ToolCall 1 → ToolResponse1 → ToolCall2 → ToolResponse2 → Response1 → User → · · · .  \nAlong such trajectories, the model must decide not only how to use a tool, but also whether to use one and when to stop using tools. This decision is brittle in multi-turn systems. A model that overuses tools increases latency and cost, and it can enter repeated tool-call loops after receiving observations. A model that underuses tools fails tasks that require external information or actions.  \nMulti-teacher on-policy distillation (MOPD) appears well matched to this problem [11] . A tool-call teacher can specialize in structured function calls; a response teacher can specialize in direct natural-language answers. The student then learns from teacher distributions on its own rollouts, as in generalized knowledge distillation (GKD) [1] . This setup offers specialization without manually merging heterogeneous behaviors.  \nWe find a failure mode that is not captured by loss curves or aggregate teacher contributions alone. In our APIGen-MT tool-use setting, vanilla GKD improves decision accuracy and tool-call recall, but it also raises over-calling. The model becomes better at calling tools when tools are needed, yet it becomes more likely to call tools when a direct answer is appropriate.  \nSeveral aggregate explanations are not supported by the training logs. Tool-call samples do not dominate token exposure. Full-sequence per-token Jensen-Shannon divergence (JSD) and a gradient proxy are not larger for the tool-call teacher. In several runs, response tokens receive more exposure and larger aggregate divergence. The final behavior therefore cannot be explained by sample count, token count, or total divergence alone.  \nWe argue that the missing variable is behavior leverage imbalance. A token-level learning signal matters not only by its magnitude, but ","cbCaib3d8iS8tCXy","https://ap.wps.com/l/cbCaib3d8iS8tCXy","pdf",344933,1,17,"English","en",105,"# Abstract\n# Introduction\n## Problem Setting: Tool-Use Trajectories\n## Failure Mode Not Captured by Losses\n## Behavior Leverage Imbalance\n## Soft Clamp Mitigation\n## Contributions and Evaluation","[{\"question\":\"What problem does multi-teacher on-policy distillation address for agentic language models?\",\"answer\":\"It trains models to decide when to call tools, when to consume tool responses, and when to answer directly during interleaved tool-use dialogues.\"},{\"question\":\"Why can vanilla generalized knowledge distillation increase over-calling even if accuracy improves?\",\"answer\":\"Because aggregate explanations do not capture a behavior leverage imbalance: small local token-level signals at mode-entry and structural positions can disproportionately control the global generation mode.\"},{\"question\":\"How does Soft Clamp mitigate the over-calling issue?\",\"answer\":\"Soft Clamp performs per-token divergence calibration by dynamically capping extreme token-level Jensen-Shannon divergence via a detached scaling factor, preserving nonzero gradients while compressing harmful extremes.\"}]",1784185781,43,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/behavior-leverage-imbalance-in-multi-teacher-on-policy-distillation/83179/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does multi-teacher on-policy distillation address for agentic language models?","Question",{"text":74,"@type":75},"It trains models to decide when to call tools, when to consume tool responses, and when to answer directly during interleaved tool-use dialogues.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"Why can vanilla generalized knowledge distillation increase over-calling even if accuracy improves?",{"text":79,"@type":75},"Because aggregate explanations do not capture a behavior leverage imbalance: small local token-level signals at mode-entry and structural positions can disproportionately control the global generation mode.",{"name":81,"@type":72,"acceptedAnswer":82},"How does Soft Clamp mitigate the over-calling issue?",{"text":83,"@type":75},"Soft Clamp performs per-token divergence calibration by dynamically capping extreme token-level Jensen-Shannon divergence via a detached scaling factor, preserving nonzero gradients while compressing harmful extremes.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":105,"slug":137},19,"General","general"]