KL-drift

Gemma 4 has a systemic attention failure. Here's the proof.

The author developed a diagnostic method for LLMs, revealing a systemic attention failure in Gemma 4 26B A4B. It identified 29 tensors with significant distribution drift, 21 of which are in attention layers, indicating a compromised attention mechanism.