ReAD: Reinforcement-Guided Capability Distillation for Large Language Models

ReAD proposes a Reinforcement-guided Capability Distillation framework for Large Language Models, aiming to compress LLMs while preserving essential abilities for downstream tasks. It explicitly accounts for the interdependence of capabilities, optimizing token budget usage and mitigating degradation of useful abilities.