[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85515-en":3,"doc-seo-85515-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85515,34359740700684,"Finn","https://ap-avatar.wpscdn.com/avatar/1f400023980c374ae676?_k=1777273430885731487",8,"Research & Report","Disentangling Intrinsic Importance from Emergent Structure in Multi-Expert Orchestration","Multi-expert systems that coordinate multiple Large Language Models (LLMs) for complex reasoning are increasingly used, yet orchestration policies remain difficult to interpret. The paper proposes INFORM, an interpretability analysis that formalizes orchestration as explicit computation, separating interaction structure, execution order, and functional attribution. Experiments on GSM8K, HumanEval, and MMLU use instruction-tuned expert sets from LLaMA-3.1, Qwen3, and DeepSeek-R1. Results show routing dominance poorly predicts necessity, while gradient sensitivity reveals intrinsic importance and hub-like experts.","arXiv :2602 .0429 1v 3 [ cs .LG] 12 Jul 2026  \n lcs2 , iit delhi June 2026  \nDisentangling Intrinsic Importance from Emergent Structure in  \nMulti-Expert Orchestration  \nSudipto Ghosh 1* Sujoy Nath 2* Sunny Manchanda 3 Tanmoy Chakraborty 1,2  \n1 Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi, India  \n2 Department of Electrical Engineering, Indian Institute of Technology Delhi, India  \n3 DYSL-AI, Defence Research and Development Organisation, India  \n* Equal Contribution  \n[sudipto.ghosh@scai.iitd.ac.in](sudipto.ghosh@scai.iitd.ac.in) , [sujoynathofficial@gmail.com](sujoynathofficial@gmail.com) , [sunny.dysl-ai@gov.in](sunny.dysl-ai@gov.in) , [tanchak@iitd.ac.in](tanchak@iitd.ac.in)  \n[a](a github.com/parmanu-lcs2/inform)[ github.com/parmanu-lcs2/inform](a github.com/parmanu-lcs2/inform) î [openreview.net/forum?id=4W7sgat04A](openreview.net/forum?id=4W7sgat04A)  \nAbstract Multi-expert systems, where multiple Large Language Models (LLMs) collaborate to solve complex tasks, are increasingly adopted for high-performance reasoning and generation. However, the orchestration policies governing expert interaction and sequencing remain largely opaque. We introduce INFORM, an interpretability analysis that treats orchestration as an explicit, analyzable computation, enabling the decoupling of expert interaction structure, execution order, and functional attribution. We use INFORM to evaluate an orchestrator on GSM8K, HumanEval, and MMLU using a homogeneous consortium of ten instruction-tuned experts drawn from LLaMA-3.1 8B, Qwen3 8B, and DeepSeek-R1 8B, with controlled decoding-temperature variation, and a secondary heterogeneous consortium spanning 1B–7B parameter models. Across tasks, routing dominance is a poor proxy for functional necessity. We reveal a divergence between relational importance, captured by routing mass and interaction topology, and intrinsic importance, measured via gradient sensitivity: frequently selected experts often act as interaction hubs with limited influence, while sparsely routed experts can be structurally critical. Orchestration behaviors emerge asynchronously, with expert centralization preceding stable routing confidence and expert ordering remaining non-deterministic. Targeted ablations show that masking intrinsically important experts induces disproportionate collapse in interaction structure compared to masking frequent peers, confirming that INFORM exposes functional and structural dependencies beyond accuracy metrics alone.  \n 1 Introduction and Prior Art  \nLarge Language Models (LLMs) are increasingly deployed not as standalone solvers but as components within multiexpert and multi-agent systems, where multiple models interact to solve complex reasoning tasks [1–3] . Rather than relying on a single monolithic model, these systems coordinate experts through an orchestration mechanism [4] that determines which expert is invoked, in what order, and under what context. This paradigm has enabled strong empirical gains across reasoning, coding, and decision-making benchmarks [5, 6] . Existing approaches to orchestration span several design philosophies. Some systems rely on externally controlled execution graphs that explicitly manage state and control flow [1, 7], while others encode coordination through role-based prompting or social simulation [3, 8] . More recent work introduces learned or routing mechanisms that dynamically select experts during inference [9–11] . Despite their architectural differences, these methods share a common assumption: orchestration policies are optimized for performance but are rarely examined as objects of analysis.  \nRelated Work. Conditional computation has long been studied to improve efficiency and specialization, exemplified by the Mixture-of-Experts (MoE) paradigm, which routes inputs to subsets of parameters [12, 13] or selects among multiple frozen language models [9, 10] . While systems like LLM-Blender [14] focus on aggregating ou","cbCaimEf1rrXIKnc","https://ap.wps.com/l/cbCaimEf1rrXIKnc","pdf",1333227,1,34,"English","en",105,"# Introduction and Prior Art\n## Related Work\n## Motivation and Crisis","[{\"question\":\"What problem does INFORM address in multi-expert LLM orchestration?\",\"answer\":\"It addresses the lack of interpretability in orchestration policies, clarifying how expert interaction structure, execution order, and functional attribution relate to task performance.\"},{\"question\":\"How does the paper distinguish intrinsic importance from routing frequency?\",\"answer\":\"It finds that experts chosen more often can act as interaction hubs with limited influence, while sparsely routed experts can be intrinsically critical, measured via gradient sensitivity.\"},{\"question\":\"What do the targeted ablations show about expert influence?\",\"answer\":\"Masking intrinsically important experts causes a disproportionate collapse in interaction structure compared with masking frequent peers, indicating dependencies beyond accuracy metrics.\"}]",1784204117,86,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"disentangling-intrinsic-importance-from-emergent-structure-in-multi-expert-orchestration","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/disentangling-intrinsic-importance-from-emergent-structure-in-multi-expert-orchestration/85515/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does INFORM address in multi-expert LLM orchestration?","Question",{"text":75,"@type":76},"It addresses the lack of interpretability in orchestration policies, clarifying how expert interaction structure, execution order, and functional attribution relate to task performance.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper distinguish intrinsic importance from routing frequency?",{"text":80,"@type":76},"It finds that experts chosen more often can act as interaction hubs with limited influence, while sparsely routed experts can be intrinsically critical, measured via gradient sensitivity.",{"name":82,"@type":73,"acceptedAnswer":83},"What do the targeted ablations show about expert influence?",{"text":84,"@type":76},"Masking intrinsically important experts causes a disproportionate collapse in interaction structure compared with masking frequent peers, indicating dependencies beyond accuracy metrics.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]