When Your LLM Reaches End-of-Life: A Framework for Confident Model Migration in Production Systems

This research introduces a framework for migrating production LLM systems when their underlying models reach end-of-life or need replacement. It employs a Bayesian statistical approach to calibrate automated evaluation metrics against human judgments, ensuring confident model comparison with limited manual data.