Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR

This paper examines the impact of systematic verification errors on Reinforcement Learning with Verifiable Rewards (RLVR), a method used to enhance the reasoning capabilities of large language models. Unlike prior analyses that treated errors as random, this work shows that systematic errors can lead models to learn unwanted behaviors. Experiments on arithmetic tasks reveal that systematic false negatives have similar effects to random noise, while systematic false positives can have more complex impacts.