Rotary Position Embedding

Efficient Matrix Implementation for Rotary Position Embedding

This research proposes RoME, a novel and computationally efficient reformulation of Rotary Position Embedding (RoPE), a core component in modern Transformer architectures. By replacing vector-level operations with unified matrix transformations, RoME significantly reduces computational overhead and improves hardware utilization.