MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs

MemQ integrates TD($\lambda$) eligibility traces with memory Q-values, propagating credit backward through a provenance DAG to account for memory dependencies. This approach significantly improves LLM agents' ability to accumulate and retrieve experience, achieving high success rates across various benchmarks.