O artigo apresenta a Memória Associativa de Fase (PAM), um modelo de sequência recorrente que utiliza representações de valores complexos e opera em um espaço de Hilbert complexo. O PAM alcança uma perplexidade de validação de 30.0 no WikiText-103, um desempenho competitivo com transformadores, e aborda a degradação da capacidade em modelos de estado vetorial.
This paper proposes Belief-State RWKV, a stronger RL formulation where the recurrent state is explicitly interpreted as a belief state. The method maintains a compact uncertainty-aware state, allowing policies to depend on both memory and confidence in partially observed settings.
This research systematically investigates LoRA placement in hybrid language models, which combine attention and recurrent components. It finds that adapting the attention pathway consistently outperforms full-model adaptation with significantly fewer parameters, while the effect of adapting the recurrent backbone varies drastically depending on the hybrid architecture (sequential vs. parallel).
This paper investigates how action information can be incorporated into the state update function of a recurrent cell within recurrent neural networks (RNNs) for reinforcement learning (RL). The authors discuss several choices and empirically evaluate the resulting architectures on illustrative domains.