The Geometry of Saturation: Effective Rank Predicts When Labels Stop Helping in Few-Shot Classification

Q: What problem does the paper address in few-shot label acquisition?

It addresses the lack of a label-free signal that tells when adding more labels stops improving classification accuracy in few-shot settings.

Q: How is the spectral saturation index S(K) defined and interpreted?

S(K)=erank(W(K))/K, where erank is exponential spectral entropy of pooled within-class covariance and K is per-class support size. A drop of S(K) below a threshold indicates saturation of the explored spectral subspace.

Q: What threshold and decision rule does the paper recommend?

Using τ=0.02, the method turns into a stop/continue rule. For PCA-50 it uses a hard halt when S(K)<0.02, while for CLIP/DINOv2 it monitors S(K) decreasing from about 0.3 to 0.05 as a diminishing-returns signal.

Jul 16, 2026

Upload by Eliana

Few-shot label acquisition lacks a label-free signal for identifying when additional labels no longer improve accuracy. The paper proposes the spectral saturation index S(K)=erank(W(K))/K, derived from exponential spectral entropy of pooled within-class covariance and per-class support size K. When the explored spectral subspace saturates, S(K) falls below τ=0.02 and marginal accuracy gains vanish. Across 49 tasks and three frozen backbones, S(K) strongly predicts gains from doubling support and supports practical stop/continue decisions.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages46

Reading Time1 h 56 min

Introduction

Motivation and problem setting

Limitations of existing stopping criteria

Related Work

Effective rank, spectral entropy, and random matrix theory

Method and Core Idea

Spectral saturation index S(K)

Theory

Convergence and saturation point Ksat

Experiments

Benchmarks across tasks and backbones

Ablations and robustness

Practical Guidance

Two-regime stopping rule

arXiv :2606 .24903v2 [ cs .LG] 11 Jul 2026 The Geometry of Saturation: Effective Rank Predicts When Labels Stop Helping in Few-Shot Classification Arnav Gupta [arnav.gupta.ai@outlook.com](arnav.gupta.ai@outlook.com) Independent Researcher, Nepal Abstract Few-shot label acquisition lacks a label-free signal for when additional labels cease to improve accuracy. Existing stopping criteria either require a held-out validation set (violating the few-shot premise) or rely on heuristic proxies with no theoretical grounding. We introduce the spectral saturation index S (K) = erank(W(K))/K, where erank is the exponential spectral entropy of the pooled within-class covariance and K is the per-class support size. S (K) measures the exploration rate per label; when the explored spectral subspace saturates, S (K) drops below a fixed threshold τ = 0 .02 and marginal accuracy gains vanish. Across 49 real tasks (binary, 5-way, 10-way) and three frozen backbones (PCA-50, CLIP ViT-B/32, DINOv2 ViT-S/14), S (K) correlates strongly with the marginal gain on doubling the support set (ρpool = 0 .6366 , p = 2 .9×10−57, cluster-bootstrap 95% CI [0 .551 , 0.720]) . A fixed τ = 0 .02 classifies stop/continue decisions with cluster-bootstrap AUC = 0 .787 [95% CI: 0.713 , 0.860] and achieves high recall on meaningful gains (∆A > 1%) . A partial correlation controlling for log K yields ρpartial = 0 .324 (p = 1 .65 × 10−13), confirming S(K) carries spectral information beyond the shared K-dependence. Theory predicts this from first principles: the population effective rank sets the saturation scale Ksat ≈ erank(ΣW )/τ ; τ = 0 .02 sits at the boundary between the first and second descent (Nakkiran et al., 2021); and the O(1/K) bias in sample effective rank explains the small-K hump in S (K) . For practitioners using unregularized linear probes (C = ∞ ): halt when S(K) < 0.02 (PCA-50, hard stop); monitor S(K) dropping from ∼ 0.3 → 0.05 (foundation models, diminishing-returns signal) . Computation is ∼ 1 ms at d = 50 . Keywords: few-shot learning, effective rank, spectral entropy, label acquisition, stopping rule, covariance estimation 1 Introduction Adapting a frozen CLIP or DINOv2 backbone to a new 5-way task with 16 labels per class costs minutes of annotation; pushing to 256 labels costs hours but often yields negligible gain—the saturation point lies beyond practical budgets, and no label-free signal tells you when to stop. Current stopping criteria either demand a held-out validation set (breaking the few-shot premise) or use heuristic proxies such as activation sparsity or weight variance that lack theoretical grounding (Koyuncu et al. 2022; Stutz et al. 2021) . Methods requiring the population spectrum (Yao et al. 2007; Raskutti et al. 2014) are unusable in practice. Without a per-instance, label-free diagnostic, practitioners under-collect (losing accuracy) or over-collect (wasting budget), especially for foundation models where saturation exceeds feasible K. 1 LLMs were used by the authors for text polishing, literature search, and code assistance throughout the preparation of this manuscript. All LLM-generated content was reviewed by the authors for accuracy and correctness. We introduce the spectral saturation index S (K) = eranW(K)􀀁~~ ~~ , (1) wSithehadrennropoW(Keben) tloisrowthpyτpf0oitos0le2n, dotwrmheitachlioinzevadcilaseiganseccneovvaspaluecretiansprumceecftrhromumas saKStuexKatpealanesdsuprefuesrr cththelaerselsxaapbelnoldrseatyiiranoneldkradisteimthpineeriselhixpabngoenlr:eenwtuthriaennsl Thesis: the spectral saturation index S (K) is a label-free, theoretically grounded stopping rule that predicts marginal accuracy gains from the within-class covariance spectrum. 1. Sfempaoectudetrelsrat(l saranCLIstfPo,urramDItatNioioOnnvs2n;rdeeqforxuirPeC(Ses oA-cn54tth)he: eSsbua(Kpsips)estraetonn, katnh(W(Kissu)fp)lKlyortilsasibetnvel-afrriaeenftotrofrinveozenrtibfoulenldianteaiorn P 2. Theoretical grounding (Sec.

FAQs

What problem does the paper address in few-shot label acquisition?

It addresses the lack of a label-free signal that tells when adding more labels stops improving classification accuracy in few-shot settings.

How is the spectral saturation index S(K) defined and interpreted?

S(K)=erank(W(K))/K, where erank is exponential spectral entropy of pooled within-class covariance and K is per-class support size. A drop of S(K) below a threshold indicates saturation of the explored spectral subspace.

What threshold and decision rule does the paper recommend?

Using τ=0.02, the method turns into a stop/continue rule. For PCA-50 it uses a hard halt when S(K)<0.02, while for CLIP/DINOv2 it monitors S(K) decreasing from about 0.3 to 0.05 as a diminishing-returns signal.

Jul 16, 2026

Upload by Eliana