SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decoding

This paper proposes SENSE (Semantic Embedding Navigation with Soft-gated Evaluation) to enhance Retrieval-based Speculative Decoding (RSD) for LLMs. SENSE addresses RSD's rigid lexical dependencies by using robust semantic alignment and a soft-gated evaluation module to validate semantic equivalence.