Automated Evaluation

Should We be Pedantic About Reasoning Errors in Machine Translation?

This research identifies and quantifies reasoning errors in machine translation across multiple language pairs, using an automated annotation protocol. Experiments with interventions on reasoning traces suggest that strong corrections improve error resolution, though with mixed gains in overall translation quality.