essay · 2026-09-27

How to tune forgetting rates in LLM long-term memory systems

Recent research on long-term memory mechanisms demonstrates that storing external knowledge and historical interactions is crucial for improving decision quality in AI systems, yet unmanaged data accumulation can increase reasoning costs significantly. Studies indicate that selective forgetting strategies are viable because they reduce overall memory storage by approximately 10% without causing a detectable drop in token F1 scores during knowledge-update tasks. However, the performance gap between graph-based retrieval and flat retrieval methods on such questions is substantial, with F1 scores of only 0.456 versus 0.511 respectively for specific query types. This suggests that relying solely on extraction mechanisms fails to resolve conflicts effectively, necessitating explicit conflict-resolution policies like last-write-wins within a hybrid memory architecture. Implementing these structured retention policies allows agents to persist context and preferences across sessions while preventing the accumulation of stale or irrelevant information.

Sources

• tandfonline.com/doi/full/10.1080/10926771.2021.2013375
• Vector Storage Based Long-term Memory Research on LLM
• LLM Long-Term Memory: How to Make AI Agents... - HydraDB
• GitHub - revectores/awesome-llm-memory: Curated papers on...
• Fine-Tuning Large Language Models (LLMs) Without Catastrophic Forgetting | by Sachchida Nand Singh | Towards AI
• Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agents · Pith
• Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning | OpenReview