LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs

LayerBoost proposes an optimization for LLMs by selectively modifying the attention mechanism based on the sensitivity of individual transformer layers. This aims to reduce the quadratic complexity of softmax attention, a major bottleneck for efficient inference, without significant model quality degradation.