Large language models (LLMs) face catastrophic forgetting and plasticity loss when updating parameters for downstream tasks. This work introduces a fast-slow learning framework for LLMs, utilizing model parameters as "slow" weights and optimized context as "fast" weights to adapt efficiently without compromising general reasoning.
This content introduces novel metrics for Continual Learning, broadening evaluation beyond just preventing catastrophic forgetting. It proposes a more comprehensive view for measuring AI model performance in sequential learning scenarios.
This research analyzes Supervised Fine-Tuning (SFT), revealing that instruction-following capabilities emerge distinctly across layers: middle layers are stable while final layers are highly sensitive. Leveraging this, the authors propose Mid-Block Efficient Tuning, which updates critical intermediate layers, outperforming standard LoRA with reduced parameter overhead.
This paper investigates the mechanistic origins of catastrophic forgetting in Large Language Models (LLMs), comparing Reinforcement Learning (RL) with Supervised Fine-Tuning (SFT). It reveals that RL preserves internal computational circuits more effectively, mitigating the forgetting of prior capabilities, unlike SFT which causes greater circuit disruption.
Sparse Memory Finetuning (SMF) addresses catastrophic forgetting in pretrained language models by updating only a small subset of memory rows. Experiments show SMF improves performance on a medical exam task while substantially mitigating forgetting compared to LoRA and full finetuning.