I Let a Small Model Train on Its Own Mistakes. It Reached 80% on HumanEval and Beat GPT-3.5 on Math

An experiment showed that a small AI model can train itself to code by inventing problems, solving them, and fine-tuning on its own corrections. The model achieved 80% on HumanEval and outperformed GPT-3.5 on math, using only a Python interpreter as the judge.