From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

TL;DR

HippoRAG 2 is a framework that improves LLM long-term memory through RAG-based non-parametric continual learning.

Problem

LLMs struggle with continuously acquiring new knowledge, and existing RAG relies on vector retrieval, failing to mimic the dynamic and interconnected nature of human memory. Recent structural approaches (RAG+KG) degrade performance on factual memory tasks.

Approach

HippoRAG 2 builds on HippoRAG's Personalized PageRank algorithm, adding deeper passage integration and more effective online use of an LLM, improving factual, associative, and sense-making memory tasks.

Results & Contribution

Achieves 7% improvement in associative memory tasks over state-of-the-art embedding models, with superior factual and sense-making memory. Paves the way for non-parametric continual learning; code and data are publicly available.