Distilling Genomic Models for Efficient mRNA Representation Learning via Embedding Matching

This paper introduces a distillation framework to make large genomic foundation models for mRNA representation learning more efficient, reducing model size by 200-fold. By using embedding-level distillation, the smaller model achieves state-of-the-art performance on mRNA-related tasks, demonstrating an effective strategy for scalable biological AI.