Does Continual Learning Equally Forget All Parameters? - Forgetting Prioritized Finetuning（FPF）与k-FPF方法

Aug 4, 2026

Distribution shift in continual learning (CL) typically causes catastrophic forgetting as models overwrite previously learned weights. While experience replay can alleviate this via buffered data, every-step replay is costly. This study analyzes training dynamics to identify which neural modules are most prone to forgetting, finding forgetting concentrated in a small set of task-sensitive modules. Finetuning only those modules with a small final buffer yields efficient gains via Forgetting Prioritized Finetuning (FPF). A lazy variant, k-FPF, removes frequent replay and triggers FPF only k times, achieving comparable accuracy with much lower overhead and cost across class- and domain-incremental benchmarks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages24

Reading Time1 h 0 min

AccessFree to download

Abstract

Introduction

Continual learning and catastrophic forgetting

Experience replay and its limitations

Task-specific adapters and plasticity-stability trade-off

Research questions and proposed approach

Does Continual Learning Equally Forget All Parameters? Haiyan Zhao 1 Tianyi Zhou 2 Guodong Long 1 Jing Jiang 1 Chengqi Zhang 1 Abstract Distribution shift (e.g., task or domain shift) in continual learning (CL) usually results in catastrophic forgetting of previously learned knowledge. Although it can be alleviated by repeatedly replaying buffered data, the every-step replay is time-consuming. In this paper, we study which modules in neural networks are more prone to forgetting by investigating their training dynamics during CL. Our proposed metrics show that only a few modules are more task-specific and sensitive to task change, while others can be shared across tasks as common knowledge. Hence, we attribute forgetting mainly to the former and find that finetuning them only on a small buffer at the end of any CL method can bring non-trivial improvement. Due to the small number of finetuned parameters, such “Forgetting Prioritized Finetuning (FPF)” is efficient in computation. We further propose a more efficient and simpler method that entirely removes the every-step replay and replaces them by only k-times of FPF periodically triggered during CL. Surprisingly, this “kFPF” performs comparably to FPF and outperforms the SOTA CL methods but significantly reduces their computational overhead and cost. In experiments on several benchmarks of classand domain-incremental CL, FPF consistently improves existing CL methods by a large margin, and k-FPF further excels in efficiency without degrading the accuracy. We also empirically studied the impact of buffer size, epochs per task, and finetuning modules on the cost and accuracy of our methods. 1University of Technology Sydney 2University of Maryland. Correspondence to: Haiyan Zhao , Tianyi Zhou <[tianyi@umd.edu](tianyi@umd.edu) >, Guodong Long, Jing Jiang, Chengqi Zhang <{guodong.long, jing.jiang, [chengqi.zhang](chengqi.zhang}@uts.edu.au)[}](chengqi.zhang}@uts.edu.au)[@uts.edu.au](chengqi.zhang}@uts.edu.au)>. Proceedings of the 40 th International Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023 . Copyright 2023 by the author(s) . 1. Introduction Empowered by advancing deep learning techniques and neural networks, machine learning has achieved unprecedented promising performance on challenging tasks in different fields, mostly under the i.i.d. offline setting. However, its reliability and performance degenerate drastically in continual learning (CL) where the data distribution or task in training changes over time, as the model quickly adapts toa new task and overwrites the previously learned weights. This leads to a severe bias toward more recent tasks and“catastrophic forgetting” of previously learned knowledge, which is detrimental to a variety of practical applications. A widely studied strategy to mitigate forgetting is experience replay (ER) (Ratcliff, 1990 ; Robins, 1995) and its variants (Riemer et al., 2018 ; Buzzega et al., 2020 ; Boschini et al., 2022), which store a few data from previous tasks in the limited memory and train the model using both the current and buffered data. However, they only bring marginal improvements when the memory is too small to store sufficient data for recovering previously learned knowledge, which is common due to the complicated distributions of previous tasks. In contrast, multi-task learning (Caruana, 1997) usually adopts a model architecture composed of atask-agnostic backbone network and multiple task-specific adapters on top of it. While the backbone needs to be pre-trained on large-scale data, the adapters are usually lightweight and can be achieved using a few data. In CL, however, we cannot explicitly pre-define and separate the task-agnostic parts and task-specific parts. Although previous methods (Schwarz et al., 2018 ; Zenke et al., 2017) have studied to restrict the change of parameters critical to previous tasks, such an extra constra

FAQs

Why does continual learning suffer catastrophic forgetting under distribution shift?

As tasks or data distributions change over time, models quickly adapt to new tasks and overwrite weights learned for earlier tasks, creating a strong bias toward recent tasks.

How does this work identify which parameters cause forgetting?

It investigates training dynamics by measuring how model parameters change over time during continual learning, revealing that only a few modules change more drastically between tasks.

What are FPF and k-FPF, and how do they reduce computational cost?

FPF finetunes task-specific parameters using buffered data only at the end of continual learning methods. k-FPF further removes every-step replay by triggering FPF periodically only k times, reducing overhead while maintaining comparable accuracy.

Aug 4, 2026

Upload by Theodore