softmax

2 items

ARTICLEDEV.to AI·4/11/2026

Output Layer Explained — Logits, Softmax, Cross-Entropy, and Why They Work Together

This article explains how neural networks generate meaningful predictions through probabilities, detailing the role of logits, softmax, and cross-entropy. It discusses how these components transform raw outputs into interpretable probability distributions and how the loss function measures accuracy, ensuring numerical stability.

neural networks Logits softmax Cross-Entropy

RESEARCHarXiv CS.LG·5/6/2026

On the Invariants of Softmax Attention

This research defines the "energy field" in softmax attention, revealing essential invariant properties. It distinguishes between mechanism-level invariants, derived from algebraic structure, and model-level regularities observed in autoregressive language models.

neural networks softmax machine learning NLP