The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models

This research study reveals that small instruction-tuned language models (LMs) using Chain-of-Thought (CoT) for arithmetic often employ a positional shortcut, copying whichever number occupies the trailing position before the answer delimiter. This shortcut dominates, even when intermediate reasoning is correct, significantly impacting answer accuracy.