A Mechanism-Driven Theory of Phase Transitions in Active Learning - Abstract

Sep 21, 2026

Uploaded by Jiven

Active learning performance depends on the labeling budget, but existing stage definitions rely on heuristic label counts that do not generalize across datasets or architectures. The paper reframes budget regimes as shifts in the dominant generalization mechanism by interpreting PAC-style risk components as interacting dynamic terms. Dominance shifts are proven structurally unavoidable, forming a moving bottleneck for generalization. A measurable tripartite taxonomy is identified via segmented regression.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages45

Reading Time1 h 53 min

AccessFree to download

Introduction

Budget-dependent dominance of AL strategies

Mechanism-driven view of AL regimes

Measurable proxies and three-phase taxonomy

Empirical results on benchmarks

arXiv :2607 .00144v1 [ cs .CV] 30 Jun 2026 A Mechanism-Driven Theory of Phase Transitions in Active Learning Julia Machnio , Mads Nielsen , and Mostafa Mehdipour Ghazi Pioneer Centre for AI, University of Copenhagen, Copenhagen, Denmark {juma,madsn,[ghazi}@di.ku.dk](ghazi}@di.ku.dk) Abstract. Active learning (AL) performance is known to be budgetdependent, yet regimes are typically defined by heuristic label counts that fail to generalize across datasets or architectures. We characterize AL dynamics by reframing budget regimes as shifts in the dominant generalization mechanism. By reinterpreting PAC-style risk components as dynamic interacting terms, we prove that dominance shifts are structurally unavoidable, creating a moving bottleneck for generalization. We operationalize this using measurable proxies and a segmented regression procedure to identify a tripartite taxonomy: datadriven, transition, and model-driven phases. Our framework explains the long-standing observation that representativeness, coverage, and uncertainty strategies excel at different stages. Experiments across natural and medical imaging show that AL efficiency depends on the alignment between the strategy’s inductive bias and the active bottleneck. Moreover, self-supervised representation shift transitions earlier along the labeling trajectory, highlighting the role of representation quality in shaping AL dynamics. Overall, this work provides a unified framework for the next generation of transition-aware AL algorithms. The code is available at: [https://github.com/juliamachnio/PALM](https://github.com/juliamachnio/PALM). Keywords: Active Learning · PAC · Generalization · Phase Transition 1 Introduction Active learning (AL) aims to reduce annotation cost by iteratively selecting informative samples from a large unlabeled pool [25] . Empirical studies consistently show that the effectiveness of different AL strategies depends strongly on the labeling budget [12, 31] . This behavior is commonly described through early- , mid-, and late-budget stages in which different families of methods dominate: representativeness-based strategies perform best early [1, 12 , 28 , 30], coverageoriented approaches become effective at intermediate budgets [24, 31], and uncertainty-driven methods dominate later [2, 9 , 16 , 18 , 23] . However, these regimes are typically defined by heuristic label counts that vary across datasets, architectures, and representations, limiting their structural interpretability and provide limited guidance for the design of acquisition strategies. We hypothesize a different perspective: AL regimes are not fixed by budget, but emerge from shifts in the dominant generalization mechanism during 2 J. Machnio et al. Fig. 1: Active learning exhibits dynamic bottleneck transitions. Top: Smoothed, normalized upper-bound proxies across labeling budgets. Distinct dominance regimes emerge: representativeness dominates early, coverage and complexity peak mid-stage, and empirical risk with statistical confidence dominate late. Bottom: Ranking of the methods. Transition points (τ) denote detected shifts in proxy dominance. In low-complexity datasets like CIFAR-10, these transitions occur early, while in more complex dataset shifts them further along the trajectory. Pretrained models (e.g., SimCLR) accelerate these transitions, shortening the initial data-driven phase. sampling. Building on true risk decompositions under sampling bias and covariate shift [8, 21 , 27], we reinterpret four structural components: empirical risk, distributional discrepancy, hypothesis-space complexity, and confidence term [21], as interacting terms whose relative influence evolves along labeling trajectory. Within this perspective, it is important to distinguish between distributional representativeness [12, 28 , 30] and geometric coverage [24, 31] . Representativeness measures how well the labeled subset reflects the underlying data distribution, contr

FAQs

How does the paper reinterpret labeling-budget stages in active learning?

It treats budget regimes as shifts in the dominant generalization mechanism rather than as fixed stages determined by heuristic label counts.

What is meant by the “moving bottleneck” for generalization?

When dominance shifts structurally during the labeling trajectory, different risk components become limiting factors, creating a bottleneck that moves across stages.

How are the paper’s three phases identified empirically?

Using measurable proxies tied to PAC-style risk components and a segmented regression procedure along the labeling-budget axis to detect transition points.

Sep 21, 2026

Uploaded by Jiven

A Mechanism-Driven Theory of Phase Transitions in Active Learning - Abstract_Page 1

Open in WPS to view all 45 pages