A Comparative Theoretical Analysis of Entropy Control Methods in Reinforcement Learning

This paper provides a comparative theoretical analysis of entropy control strategies in Reinforcement Learning, focusing on traditional regularization versus a novel covariance-based mechanism for LLM training. It establishes a unified framework, showing that covariance-based methods achieve asymptotic unbiasedness by selectively regularizing high-covariance tokens, unlike traditional methods that introduce persistent bias.