We benchmarked 18 LLMs on OCR (7k+ calls) — cheaper/old models oftentimes win. Full dataset + framework open-sourced. [R]

Researchers benchmarked 18 LLMs for OCR, finding that cheaper and older models often match or exceed the accuracy of flagship models at a fraction of the cost. They open-sourced their dataset and benchmarking framework.