RESEARCHarXiv CS.CL·5/6/2026
Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
Sparse Memory Finetuning (SMF) addresses catastrophic forgetting in pretrained language models by updating only a small subset of memory rows. Experiments show SMF improves performance on a medical exam task while substantially mitigating forgetting compared to LoRA and full finetuning.
27