On the Invariants of Softmax Attention

This research defines the "energy field" in softmax attention, revealing essential invariant properties. It distinguishes between mechanism-level invariants, derived from algebraic structure, and model-level regularities observed in autoregressive language models.