Critical Damping as a Momentum Schedule: Multi-Seed Validation, a Hybrid Recipe, and an Exhaustive Negative Result on Surgical Layer Selection

Jul 16, 2026

Upload by Himbo

Critical damping applied to the damped harmonic oscillator model of SGD with momentum yields a tuning-free momentum schedule, fixing µ(t)=1−2√α(t) (with a fixed, never-tuned stability clamp). The schedule is validated on ResNet-18/CIFAR-10 with a 200-epoch cosine schedule across five seeds, reaching 90% test accuracy 2.34× faster than µ=0.9, but with a measurable final-accuracy deficit of 0.46 pp. A Hybrid-90 strategy removes the deficit while keeping the speedup. An exhaustive study finds gradient-attribution-based error signals do not select optimal surgical layers. The full layer-combination landscape is released.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages23

Reading Time58 min

Abstract

Introduction

Momentum–learning-rate coupling and critical damping

Speed–accuracy validation and trade-off

Hybrid-90 schedule to remove deficit

Negative result on surgical layer selection

Contributions

arXiv :2603 .2892 1v 3 [ cs .LG] 12 Jul 2026 Critical Damping as a Momentum Schedule: Multi-Seed Validation, a Hybrid Recipe, and an Exhaustive Negative Result on Surgical Layer Selection Ivan Pasichnyk We Label Data Inc. [ivan@welabeldata. com](ivan@welabeldata. com) [https://orcid. org/0009-0004-8154-3345](https://orcid. org/0009-0004-8154-3345) Abstract The critical damping condition of the damped harmonic oscillator model of SGD with momentum (Qian, 1999) yields a momentum schedule with no tuned hyperparameters: µ(t ) = 1 −2 pα (t )(a fixed, never-tuned stability clamp aside) . We validate this schedule across five random seeds on ResNet-18/CIFAR-10 (200-epoch cosine learning-rate schedule): it reaches 90% test accuracy 2.34 × faster than constant µ = 0.9 (range 1 .71–2.86 ×, 5/5 seeds, one-sided paired t-test p = 4 × 10−4), at the cost of a real final-accuracy deficit of 0 .46 pp (5/5 seeds, p = 0.009) . A shortschedule control shows the speedup is not a schedule-length artifact: baselines with the cosine schedule compressed to 60/100 epochs either pay 0 .5–0.9 pp of final accuracy or remain slower to 90% than the physics schedule at equal accuracy—physics Pareto-dominates the accuracy-matched compressed baseline. A hybrid recipe—critical-damping momentum until 90% accuracy, then constant µ = 0.9—removes the deficit while keeping the speedup: 95 .45 ± 0.05% final accuracy (indistinguishable from baseline) at 2 .4 × faster progress to 90%(n = 5) . The speedup generalizes across architectures—VGG-16 without skip connections reaches 90% test accuracy 1 .72 × faster (n = 3)—while on CIFAR-100 the effect is smaller: early-training gains persist (2–4 × to mid-training thresholds) but the pure schedule pays a larger final-accuracy cost (−1 .7 pp), compressing the accuracymatched gain to 1 .14 × and delimiting the schedule’s scope. We also report an exhaustive negative result on surgical layer selection. Version 2 of this paper claimed that gradient attribution computed on misclassified images selects which layers to retrain. Running the identical surgical-correction protocol on all 35 combinations of 3-of-7 layer groups shows the diagnostically selected triple ranks 11th of 35 (exact p = 0.31)—no better than random selection. What survives is weaker but real: combinations containing the top-ranked layer outperform those without it (+6 .2 vs −1 .4 mean net error reduction), and the bottom of the gradient-norm ranking reliably predicts the most harmful interventions (down to −20 net errors) . Gradient attribution on errors is a harm-avoidance signal, not an optimizer of layer selection. We release the full 35-combination landscape as a baseline we believe should accompany any layer-selection claim. Keywords: momentum scheduling, critical damping, tuning-free momentum schedule, surgical finetuning, negative results, layer attribution 1 Introduction Momentum and learning rate are usually tuned as independent hyperparameters. The damped harmonic oscillator model of SGD with momentum [15] says they should not be: for any learning-rate schedule α (t ), the critical damping condition fixes the momentum trajectory completely, µ (t ) = 1−2 pα (t ), with no tuned hyperparameters (the implementation applies a fixed, never-tuned stability clamp; Section 4) . This paper asks two questions about that formula, and answers a third one we did not plan to ask. Does it work? Yes, with a quantified trade-off (Section 4) . Across five random seeds on ResNet- 18/CIFAR-10 under a fixed 200-epoch cosine schedule, critical-damping momentum reaches 90% test accuracy 2 .34 × faster than the standard constant µ = 0.9 (range 1 .71–2.86 ×, 5/5 seeds, one-sided paired t-test p = 4 × 10−4) . The price is real, not seed noise: a final-accuracy deficit of 0 .46 pp (5/5 seeds, p = 0.009) . A short-schedule control (Section 4.2) rules out the schedule-length explanation: physics is faster to 90% than any compressed baseline of equal final

FAQs

What does the critical damping condition imply for SGD with momentum?

It fixes the momentum trajectory from the learning-rate schedule as µ(t)=1−2√α(t) without tuned hyperparameters, aside from a fixed stability clamp.

How does the tuning-free critical-damping momentum schedule perform versus constant momentum?

On ResNet-18/CIFAR-10 it reaches 90% test accuracy 2.34× faster than µ=0.9, but incurs a final-accuracy deficit of 0.46 pp.

Does gradient attribution on misclassified images correctly identify layers to retrain?

No. Across all 35 combinations of 3-of-7 layer groups, the diagnostically selected triple ranks 11th of 35 (p=0.31), indicating no better-than-random selection.

Jul 16, 2026

Upload by Himbo