MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration

Jul 16, 2026

Upload by Skyler

Vision-language models such as CLIP support zero-shot classification by matching image and text in a shared embedding space, relying on global comparability of logits across candidate classes. Fine-tuning with LoRA often improves in-domain accuracy but harms out-of-domain performance, causing fragmented ecosystems of specialized experts. Multi-Expert-Domain (MED) classification merges independently trained domain adapters, yet yields cross-domain logit miscalibration and interference. MED-DSLC restores global logit comparability using domain supervision and domain-wise logit scaling, improving mean accuracy and robustness under scalable multi-domain zero-shot recognition.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages26

Reading Time1 h 6 min

Introduction

Problem: fragmented multi-domain experts and logit miscalibration

Approach: MED-DSLC mixture-of-experts with domain supervision and logit scaling

arXiv :2607 . 10985v1 [ cs .CV] 13 Jul 2026 MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration Zheng Zeng∗1, Deepak Sridhar∗1, and Nuno Vasconcelos 1 University of California, San Diego, USA {zhz396,[desridha}@ucsd.edu](desridha}@ucsd.edu) * Equal contribution Abstract. Vision-language models (VLMs) such as CLIP enable zeroshot classification by comparing image features with text prompts in a shared embedding space. A fundamental property underlying this capability is the global comparability of logits across arbitrary candidate classes. However, VLMs are often adapted to fine-grained domains using techniques such as LoRA. While this improves in-domain accuracy, outof-domain accuracy degrades. This leads to a higly fragmented model ecosystem, with thousands of specialized models. Multi-Expert-Domain (MED) classification seeks to address this problem, by merging LoRAstrained independently on specialized domains. However, due to the independent training, the various domain experts no longer produce globally calibrated logits. As a result, when evaluating over the union of multiple domain-specific class sets, heterogeneous logit scales induce cross-domain interference and artificially high confidence for out-of-domain classes, inducing prediction errors. In this work, we identify domain supervision and cross-domain logit miscalibration as the key issue to scalable multi-domain zero-shot recognition. We propose a mixture-of-experts MED architecture, MED-DSLC, combining domain supervised training and domain-wise logit scaling, to explicitly restore global logit comparability. MED-DSLC is a lightweight solution for MED classification, which is shown to preserve within-domain discrimination while reducing cross-domain logit interference with minimal data. Extensive experiments across diverse fine-grained benchmarks demonstrate that it substantially improves mean accuracy (+15%), cross-domain robustness, and scalability in the size of MED classification problem. Our results show that restoring outputlevel calibration is essential under highly data imbalanced settings for achieving a truly zero-shot VLM under multi-domain specialization. Code is publicly available at MED-DSLC. Keywords: Few-shot Open-set Recognition · Adapter Merging · Mixture of Experts · Generalization 1 Introduction Vision-language models (VLMs), such as CLIP [16], have transformed visual recognition by enabling zero-shot classification with open class vocabularies. By 2 Z. Zeng, D. Sridhar et al. Fig. 1: Logits for classification of an image of the class ‘FA-18’. Only largest logit value is shaded.(a) CLIP logits. (b) Logits of LoRA adapted domain expert trained on the image domain. (c) Logits of an expert trained on another domain.(d) Logit miscalibration across domains of an existing MED model (MoLE) . (e) Logits of the proposed MED-DSLC classifier. (a) MoLE (b) MED-LCDS (Ours) Fig. 2: Logit histograms for images of the EuroSAT dataset and classes of a MED classifier with 10 domain experts (identified by color) . (a) Logits produced by MoLE exhibit overlap between domains (EuroSAT and DTD) . (b) The overlap is significantly reduced by MED-DSLC. aligning images and text in a shared embedding space, these models can classify images with respect to arbitrary class names without retraining. Given feature vectors for image and class labels, a similarity score (logit) is computed per label and a softmax function used to output class probabilities. A central property underlying the zero shot capability is that the logits of all classes are globally comparable, allowing softmax normalization over any set of labels. Despite this strong zero-shot generalization, performance often degrades on fine-grained or domain-specific datasets involving image classes not commonly found in the public domain. This is illustrated in Fig. 1(a), which visualizes the logits (only largest one shaded) of the CLIP model for

FAQs

为什么多专家域（MED）分类会出现跨域干扰和错误的置信度？

由于各个域专家在独立训练后不再产生全局可比的logit尺度，评测多个域的类别集合时会产生跨域logit失准与干扰，导致对域外类别产生异常高置信度并引发预测错误。

MED-DSLC的核心思路是什么？

MED-DSLC提出混合专家架构MED-DSLC，结合域监督训练与按域的logit缩放，显式恢复全局logit可比性，从而缓解跨域失准问题。

MED-DSLC的效果如何衡量？

通过覆盖多种细粒度基准的实验评估，MED-DSLC在平均准确率（+15%）、跨域鲁棒性与MED分类规模可扩展性方面显著提升，并强调在高度数据不均衡下恢复输出级校准对真正的零样本多域专用至关重要。

Jul 16, 2026

Upload by Skyler