Parameter-efficient fine-tuning

Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures

Aletheia introduces a gradient-guided layer selection method for LoRA fine-tuning, identifying the most task-relevant layers and applying adapters selectively with asymmetric rank. This approach achieves a significant 15-28% training speedup across diverse large language models and architectures while broadly matching downstream behavior.