Learnability-Informed Fine-Tuning of Diffusion Language Models

This research introduces LIFT, a learnability-informed fine-tuning algorithm designed to enhance the reasoning capabilities of diffusion language models. LIFT addresses the shortcomings of standard SFT by adaptively learning tokens based on their difficulty and available context during different diffusion time steps, showing improved performance over existing baselines.