RESEARCHarXiv CS.LG·1d ago
WAV: Multi-Resolution Block Residual Routing for Deep Decoder-Only Transformers
The paper introduces WAV v1, a lightweight multi-resolution residual routing method for decoder-only Transformers. It improves upon standard residual connections by augmenting each block with directional detail bases that contrast attention and MLP updates, and early-vs-late sublayer dynamics.
60