Polynomial Expansion

Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions

Polynomial Expansion Rank Adaptation (PERA) is a novel method to enhance low-rank adaptation (LoRA) for fine-tuning large language models. It introduces structured polynomial expansion into the low-rank factor space to model richer nonlinear high-order interactions, overcoming LoRA's linear limitations without increasing rank or inference cost.