We benchmarked TranslateGemma against 5 other LLMs on subtitle translation across 6 languages. At first glance the numbers told a clean story, but then human QA added a chapter. [D]

This content presents a benchmark study evaluating six Large Language Models (LLMs), including TranslateGemma-12b, on English subtitle translation into six languages. The models were ranked using reference-free Quality Evaluation (QE) metrics and a custom combined metric called TQI, where TranslateGemma-12b emerged as the top-performing model overall.