Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models

This paper rigorously analyzes how numerical instability from finite precision leads to unpredictability in LLMs, a critical reliability issue in agentic workflows. It details rounding error propagation, identifying a chaotic "avalanche effect" in early layers and universal, scale-dependent chaotic behaviors.