Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps

Jul 16, 2026

Advanced foundation models such as ModernBERT deliver stronger dense retrieval than earlier architectures, yet they underperform established BERT-base baselines in learned sparse retrieval (LSR). The paper attributes this gap to a Vocabulary Gap: modern tokenizers use raw, case-sensitive vocabularies that map single semantics to redundant surface forms, wasting capacity on morphological noise and reducing lexical matching quality. It introduces a theoretical framework and Vocabulary Transfer (VT) to normalize vocabularies while preserving semantic integrity and aligning manifolds with sparsity constraints.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages12

Reading Time30 min

Introduction

Dense vs. learned sparse retrieval

Performance anomaly under sparse retrieval

Root cause: Vocabulary Gap

Vocabulary Transfer (VT) approach

Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps arXiv :2607 .00004v1 [ cs .IR] 20 Apr 2026 Zhichao Geng Amazon Web Service Shanghai, China [zhichaog@amazon.com](zhichaog@amazon.com) Abstract While advanced foundation models like ModernBERT significantly outperform older architectures in dense retrieval, they surprisingly lag behind the aging BERT-base baseline in learned sparse retrieval (LSR) . We identify the root cause as the Vocabulary Gap: modern tokenizers utilize raw, case-sensitive vocabularies designed for lossless reconstruction, which map single semantic units to redundant surface forms, wasting model capacity on morphological noise and hindering lexical matching. We formalize this intuition through a theoretical framework, demonstrating that appropriate vocabulary coarse-graining can tighten the generalization bounds by reducing complexity of the hypothesis class, provided that semantic integrity is preserved. To resolve this, we propose Vocabulary Transfer (VT), a model-agnostic framework that migrates advanced encoders to sparse-friendly, normalized vocabularies with minimal computational cost. VT utilizes a novel Semantic Initialization via spatial topology to preserve geometric structure and an Activation Potential Calibration (APC) mechanism to align pre-trained manifolds with sparsity constraints, preventing the dead neuron and dense collapse observed in standard fine-tuning. Empirically, VT is universally effective: it enables ModernBERT to achieve state-ofthe-art performance on the BEIR benchmark (52.4 nDCG, a +4.7 improvement), resuscitates failing models like RoBERTa-large, and generalizes seamlessly to inference-free architectures and specialized domains. These results confirm that the performance lag isnot an architectural deficiency but a solvable vocabulary mismatch. We’ve released our code and models.1 CCS Concepts • Information systems → Retrieval models and ranking. Keywords SPLADE, learned sparse representations, passage retrieval ACM Reference Format: Zhichao Geng and Yang Yang. 2026. Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia. ACM, New York, NY, USA, 12 pages. [https://doi.org/10.1145/](https://doi.org/10.1145/)[ ](https://doi.org/10.1145/)3805712.3809724 1[https://anonymous.4open.science/r/vocab-transfer/. All details](https://anonymous.4open.science/r/vocab-transfer/. All details) included. This work is licensed under a Creative Commons Attribution 4 .0 International License. SIGIR ’26, Melbourne, VIC, Australia © 2026 Copyright held by the owner/author(s) . ACM ISBN 979-8-4007-2599-9/2026/07 [https://doi.org/10.1145/3805712.3809724](https://doi.org/10.1145/3805712.3809724) Yang Yang Amazon Web Service Shanghai, China [yych@amazon.com](yych@amazon.com) Performance Gap with BERT-uncased (nDCG@10) +2 +0 -2 -4 -10 -20 -40 Dense Retrieval Sparse Retrieval Sparse Retrieval (Input Lowercased) Figure 1: The Vocabulary Gap anomaly. While advanced encoders like ModernBERT significantly outperform BERT in dense retrieval, they lag behind in sparse retrieval under standard fine-tuning. 1 Introduction The landscape of neural information retrieval has bifurcated into two dominant paradigms: dense retrieval, which encodes queries and documents into continuous low-dimensional embeddings [21, 56], and learned sparse retrieval (LSR), which projects text into highdimensional, weighted lexical vectors [15, 33] . While dense retrievers excel at capturing semantic nuances, sparse retrievers—exemplified by models like SPLADE [15]—retain the interpretability and efficiency of inverted indices while mitigating the lexical mismatch problem of traditional BM25 [34, 47] . In

FAQs

Why do advanced encoders lag in learned sparse retrieval even when they excel in dense retrieval?

The paper links the degradation to the Vocabulary Gap: modern tokenization yields raw, redundant surface forms that hinder lexical matching and waste model capacity under sparse objectives.

What is the Vocabulary Gap, according to the document?

It is the mismatch between modern tokenizers’ non-normalized, case-sensitive vocabularies (aimed at lossless reconstruction) and the lexical matching needs of learned sparse retrieval.

How does the proposed Vocabulary Transfer (VT) framework address the issue?

VT migrates advanced encoders to sparse-friendly normalized vocabularies with minimal compute cost, using Semantic Initialization via spatial topology and Activation Potential Calibration to prevent dead neurons and dense collapse.

Jul 16, 2026

Upload by Emma Wilson