The Scaling Properties of Implicit Deductive Reasoning in Transformers

This paper investigates the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. Deep models with a bidirectional prefix mask approach explicit CoT performance, though CoT remains necessary for depth extrapolation.