TL;DR
LLMs are a degenerate special case of world models with token sequences as state space and token appending as the sole action. A natural continuum exists from Next-Token Prediction (NTP) to JEPA, with intermediate steps already populated by current research.
Problem
The AI community often frames LLMs (token prediction) and world models (reality simulation) as a dichotomy. This paper argues this framing is unnecessary and that LLMs can be naturally situated within a more general framework of world models.
Approach
Redefining LLMs as World Models: Explains LLMs' state space (all token sequences), action (append one token), and objective (predict next token) within the category of world models.
Proposing a Continuum: Positions intermediate steps like multi-token prediction, future-summary prediction, and next-latent prediction—already populated by current research—on a spectrum from NTP to JEPA.
Analyzing Challenges: Examines how the two practical advantages of LLMs (internet-scale self-supervised data, transformer architecture co-designed for discrete token prediction) are progressively surrendered along this spectrum, focusing on the data cliff (from self-supervised text to instrumented environments) and the architecture question (transformer generalization to continuous states).
Results & Contribution
Theoretical Insight: Reconceptualizes the relationship between LLMs and world models from opposition to inclusion, guiding future research direction.
Research Roadmap: Visualizes the progression from NTP to JEPA as a continuous spectrum and clearly defines key research questions (data and architecture) to be solved along the way.
Practical Considerations: Analyzes how theoretical advances connect to practical trainability (data, architecture), encouraging research to consider real-world constraints.