Large language models and emergence: a complex systems perspective.

TL;DR

A review paper analyzing the capabilities and intelligence of Large Language Models through the lens of emergence concepts from complexity science.

Problem

There is a lack of a theoretical framework for defining and measuring the novel capabilities observed in Large Language Models (LLMs). Furthermore, there are no clear criteria for determining whether these capabilities constitute true 'intelligence' beyond mere scaling.

Approach

The paper introduces the concept of 'emergence' from complexity science, specifically the principle of 'More is Different,' to analyze LLM capabilities. It reviews various approaches to quantifying emergence proposed in the literature and evaluates whether LLM capabilities satisfy the characteristics of intelligence captured by the idea 'Less is More.'

Results & Contribution

The paper systematically reviews how LLM capabilities manifest similarly to emergent phenomena in complex systems. It raises theoretical questions about whether LLMs possess true emergent intelligence and suggests directions for future research.