Harnessing Multiple Large Language Models: A Survey on LLM Ensemble

TL;DR

A pioneering survey that categorizes LLM ensemble methods by inference timing and synthesizes related research.

Problem

Individual LLMs have varying strengths and weaknesses; a single model cannot optimally answer all types of queries. There is a lack of systematic understanding of ensemble methodologies that effectively combine multiple LLMs.

Approach

The survey classifies LLM ensemble into 'ensemble-before-inference', 'ensemble-during-inference', and 'ensemble-after-inference', reviewing specific methods under each category. It also introduces related benchmarks and applications and proposes future research directions.

Results & Contribution

As the first systematic survey in the LLM ensemble field, it provides a taxonomy of methods, synthesizes existing research, and lays a foundation for future work. A curated list of related papers is also made available via a GitHub repository.