[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84390-en":3,"doc-seo-84390-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84390,7971461741311,"Ophelia","https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826",8,"Research & Report","It Takes a MAESTRO To Prune Bad Experts","Sparsely-activated Mixture-of-Experts (MoE) language models improve inference efficiency by activating only a small subset of experts per token, yet they still require all experts to be stored in memory, creating a major deployment bottleneck. Existing structured pruning approaches largely rely on locally computed heuristics tailored for dense transformers and miss MoE routing interdependencies. MAESTRO introduces an MoE-specific global importance heuristic using ergodic Markov chains over routing trajectories, delivering up to 10.61% better performance retention under 50% compression and reduced cross-task variance across Safety, Bias, and Ethics domains.","It Takes a MAESTRO To Prune Bad Experts  \nPalaash Goel  \nIndian Institute of Technology Delhi, India [goelpalaash@scai.iitd.ac.in](goelpalaash@scai.iitd.ac.in)  \nAyush Maheshwari  \nNVIDIA, India [aymaheshwari@nvidia.com](aymaheshwari@nvidia.com)  \nTanmoy Chakraborty  \nIndian Institute of Technology Delhi, India [tanchak@iitd.ac.in](tanchak@iitd.ac.in)  \narXiv :2607 .0860 1v 1 [ cs .CL] 9 Jul 2026  \nAbstract  \nSparsely-activated Mixture-of-Experts (MoE) language models achieve remarkable inference efficiency by activating only a small fraction of parameters per token, yet their full expert banks reside in memory at all times, creating a prohibitive deployment bottleneck. Existing structured pruning methods, largely designed for dense transformers, assess expert importance using locally derived heuristics that are blind to the interdependent nature of MoE routing. We introduce MAESTRO (Markovchain Aproximated Expert Sparsification via Transition-based ROuting), a structured pruning framework designed for MoE architectures that models autoregressive expert activation trajectories as Ergodic Markov chains whose stationary distributions encode cross-layer dependencies, yielding a globally aware importance heuristic. Evaluated across five diverse domains including Safety, Bias, and Ethics, MAESTRO outperforms state-of-the-art baselines by up to 10.61% in average performance retention under a strict 50% compression regime, while exhibiting substantially lower cross-task variance, indicating that global, routing-congruent pruning produces models that generalize more consistently across heterogeneous tasks.  \n1 Introduction  \nSparsely-activated Mixture-of-Experts (MoE) architectures have emerged as the dominant recipe for scaling large language models without a commensurate increase in inference compute (Jiang et al., 2024 ; DeepSeek-AI, 2025 ; Yang et al., 2025 ; OpenAI et al., 2025 ; NVIDIA et al., 2025) . By replacing the dense feed-forward block of each transformer layer with a bank of E expert sub-networksand a learned router that dispatches each token to only k ≪ E of them, MoE models decouple parameter count from per-token FLOPs. Such models routinely instantiate tens to hundreds of billions of parameters while activating only a few billion per  \ntoken, yielding inference throughput comparable to far smaller dense models.  \nThis same design choice, however, is the source of a sharply contradictory deployment profile. While MoE models are computationally efficient at inference time, they remain spatially prohibitive: the full bank of experts must reside in memory, regardless of how few are selected for any given token. This results in a deployment bottleneck wherein state-of-the-art MoE models do not fit on commodity accelerators, are expensive to host even on data-centre hardware, and are effectively excluded from on-device, edge, and memoryconstrained settings where their per-token compute profile would otherwise be ideal. Closing the gap between activated and total parameters is therefore a central practical problem for the deployment of modern MoE LLMs.  \nA natural response is structured pruning: permanently removing parameters from the model so that both compute and memory drop simultaneously. The pruning literature on large language models (LLMs), however, has overwhelmingly targeted dense transformers (Ma et al., 2023 ; Ashkbooset al., 2024 ; Men et al., 2025 ; Guo et al., 2025 ; Shopkhoev et al., 2026 ; Song et al., 2024 ; Sengupta et al., 2025) . Methods such as magnitude-(Han et al., 2016), activation- (Sun et al., 2024), and gradient-based pruning (Ma et al., 2023 ; Frantar et al., 2022), layer dropping (Shopkhoev et al., 2026 ; Song et al., 2024 ; Men et al., 2025), and width reduction (Ma et al., 2023 ; Ashkboos et al., 2024 ; Sengupta et al., 2025) are designed around the assumption that every parameter participates in every forward pass and can therefore be scored by global, token-averaged signals. These assumption","cbCaiau6bFw6kelQ","https://ap.wps.com/l/cbCaiau6bFw6kelQ","pdf",783109,1,16,"English","en",105,"# Abstract\n# Introduction\n## Deployment bottleneck in MoE inference\n## Structured pruning limitations for MoE\n## MAESTRO: Markovchain Approximated Expert Sparsification\n## Markov modeling of routing trajectories","[{\"question\":\"What problem does the document address in deploying sparsely-activated MoE models?\",\"answer\":\"MoE models activate only a few experts per token, but they must keep the full expert bank in memory, making them difficult to host on memory-constrained hardware and creating an inference deployment bottleneck.\"},{\"question\":\"Why do existing structured pruning methods not transfer well to MoE architectures?\",\"answer\":\"Prior pruning methods often use token-averaged or locally isolated importance signals designed for dense transformers, which are blind to MoE routing behavior and cross-layer inter-expert dependencies.\"},{\"question\":\"How does MAESTRO estimate expert importance for pruning?\",\"answer\":\"MAESTRO models token routing as an autoregressive ergodic Markov chain over (layer, expert) slots, uses empirically estimated transitions on a calibration corpus, and ranks experts by their stationary distribution mass to capture global routing dependencies.\"}]",1784195262,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"it-takes-a-maestro-to-prune-bad-experts","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/it-takes-a-maestro-to-prune-bad-experts/84390/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the document address in deploying sparsely-activated MoE models?","Question",{"text":74,"@type":75},"MoE models activate only a few experts per token, but they must keep the full expert bank in memory, making them difficult to host on memory-constrained hardware and creating an inference deployment bottleneck.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"Why do existing structured pruning methods not transfer well to MoE architectures?",{"text":79,"@type":75},"Prior pruning methods often use token-averaged or locally isolated importance signals designed for dense transformers, which are blind to MoE routing behavior and cross-layer inter-expert dependencies.",{"name":81,"@type":72,"acceptedAnswer":82},"How does MAESTRO estimate expert importance for pruning?",{"text":83,"@type":75},"MAESTRO models token routing as an autoregressive ergodic Markov chain over (layer, expert) slots, uses empirically estimated transitions on a calibration corpus, and ranks experts by their stationary distribution mass to capture global routing dependencies.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,118,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":28,"slug":117},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]