Positional Encoding

LaneRoPE: Positional Encoding for Collaborative Parallel Reasoning and Generation

LaneRoPE is a novel technique designed to enhance parallel Large Language Model (LLM) generation by enabling coordination and collaboration among multiple sequences at test time. It achieves this through an inter-sequence attention mask and a RoPE extension that injects positional information, demonstrating promising results on mathematical reasoning tasks.