We Ran 52 AI Coding Benchmarks. Here's Every Uncomfortable Thing We Found.

This study ran 52 AI coding benchmarks, finding that the biggest variable in AI-assisted development is the initial brief, not the model or tool. A structured brief (CONTRACT.md) reduces costs by 54% and boosts quality from 5/10 to 9/10, while agent teams and retry loops proved costly or detrimental.