Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution

This paper introduces Stepwise Confidence Attribution (SCA), a framework for closed-source LLMs that diagnoses multi-step reasoning failures by assigning step-level confidence. SCA applies the Information Bottleneck principle, flagging deviations from consensus structures as potential errors, and proposes two complementary methods: NIBS and GIBS.