A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

TL;DR

A survey paper systematically reviewing the integration of Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs).

Problem

LLMs suffer from hallucinations and outdated internal knowledge. RAG technology, which leverages external knowledge, has gained attention to address these issues. However, there is a lack of comprehensive understanding of the integration methods, optimization, and limitations of RAG and LLMs.

Approach

This survey reviews RA-LLM (Retrieval-Augmented Large Language Models) research by categorizing it into three main technical perspectives: (1) integration methods of retriever and generator, (2) strategies for utilizing retrieval results, and (3) optimization techniques for training and inference. It also discusses current limitations and future research directions.

Results & Contribution

Provides a comprehensive taxonomy of the RAG-LLM field, analyzes the strengths and weaknesses of each approach, and offers direction for researchers. Additionally, it contributes to the community by providing a continuously updated website with related resources.