On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain

Sep 21, 2026

Mixture-of-Experts (MoE) models accelerate inference through selective activation but keep the full network loaded, creating high memory costs. Structured expert pruning reduces deployment expenses in resource-limited environments, yet prior work mainly measures benchmark utility and rarely examines how pruning affects factual reliability, especially in high-stakes biomedicine. This study evaluates four MoE models, six pruning methods, and multiple pruning ratios across generation and classification tasks in-domain and cross-domain.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages24

Reading Time1 h 0 min

AccessFree to download

Abstract

Introduction

Related Work

On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain Atsuki Yamaguchi 1,2 Szymon Palucha2 Léo Bijar2 Aline Villavicencio 1,3,4 Nikolaos Aletras 1 1University of Sheffield, United Kingdom 2AstraZeneca 3University of Exeter, United Kingdom 4Federal University of Rio Grande do Norte, Brazil {ayamaguchi1,a.villavicencio, [n.aletras}@sheffield.ac.uk](n.aletras}@sheffield.ac.uk) arXiv :2607 .0 1444v 1 [ cs .LG] 1 Jul 2026 Abstract Mixture-of-Experts (MoE) models offer inference speedups via selective activation but impose substantial memory requirements because the whole network must remain loaded. Structured expert pruning is a practical approach for reducing deployment costs in resource-constrained settings. However, prior studies primarily evaluate benchmark utility, leaving the effect of pruning on factual reliability underexplored, particularly in high-stakes domains such as biomedicine. In this paper, we investigate how domainspecific expert pruning affects both utility and reliability. We assess four MoE models, six pruning methods, and multiple pruning ratios across generation and classification tasks under in-domain (biomedical) and cross-domain settings. Results reveal that moderate pruning preserves in-domain utility without immediate reliability decline, although hallucination risks increase at extreme pruning ratios. When shifting to the general domain, both utility and reliability degrade rapidly. These findings indicate that safe compression depends heavily on the task and domain. Evaluating pruned MoE models solely on utility is inadequate for high-stakes deployment without reliability assessment.1 1 Introduction Modern large language models (LLMs) such as Qwen3.6 (Qwen Team, 2026), GPT-OSS (OpenAI et al., 2025), and Nemotron3 (NVIDIA et al., 2025) follow a Mixture-of-Experts (MoE) architecture. During the forward pass, MoE models activate only a subset of the network by routing each token through specialized subnetworks called experts (Du et al., 2022 ; Cai et al., 2025) . They achieve strong performance while offering substantial inference speedups. However, this efficiency 1Our code is available at [https://github.com/](https://github.com/)[ ](https://github.com/)gucci-j/moe-pruning-reliability. Eutrophic electrotherapy was more e4ective than a uniform 10 Hz electrotherapy ... uniform 10 Hz electrotherapy ignored the spike train information, which was the signal inducing rapid... adaptation. Figure 1: An example of hallucination in biomedical text summarization, where the text highlighted in red represents a term that inverts the original meaning of the source while it is still plausible. does not eliminate the memory overhead during deployment. At inference, all experts must remain loaded in memory, which results in substantially larger memory footprints than those of dense models with comparable active parameter counts. A popular approach to mitigating this constraint is expert pruning, which eliminates redundant experts based on an informativeness (i.e., saliency) criterion estimated from a small calibration dataset (Chen et al., 2022 ; Muzio et al., 2024 ; Lu et al., 2024, inter alia) . Among these approaches, domain-specific expert pruning has been proposed to tailor the compressed model for a particular field (Dong et al., 2025), such as biomedicine. However, previous studies primarily evaluate pruning methods on downstream benchmark performance, or utility. Consequently, they overlook the reliability of the resulting compressed models. This includes factual consistency, faithfulness, and hallucination prevention (Ji et al., 2023 ; Pal et al., 2023) . This oversight poses serious risks particularly in high-stakes domains such as biomedicine, where factual errors can lead to critical real-world failures (Weidinger et al., 2022 ; Moor et al., 2023) . Furthermore, benchmark utility does not always align with factual reliability. Ch

FAQs

Why can expert pruning affect factual reliability in MoE models?

Expert pruning removes entire experts, which changes the model architecture and can force reliance on fallback experts. These alternatives may stay fluent while introducing subtle factual inaccuracies.

What does the paper find about moderate pruning in the biomedical domain?

Moderate pruning preserves in-domain utility without an immediate decline in reliability. However, hallucination risks rise when pruning becomes extreme.

How do utility and reliability change under domain shift?

When moving from the biomedical domain to a general domain, both utility and reliability degrade rapidly.

Sep 21, 2026

Uploaded by Olivia Brown