Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen et al.
This is the first survey paper that defines the overthinking phenomenon that hinders reasoning efficiency in LLMs and systematically categorizes various approaches to address it.
Recent Large Reasoning Models (LRMs) use long Chain-of-Thought reasoning to solve complex problems, but this leads to generating unnecessarily many tokens and a surge in computational cost, known as the 'overthinking phenomenon.' The efficiency degradation relative to accuracy improvement has become a serious issue.
The authors categorize existing research into three main directions: (1) Model-based: optimizing full-length reasoning models into more concise ones or directly training efficient reasoning models; (2) Output-based: dynamically reducing reasoning steps and length during inference; (3) Input prompt-based: enhancing efficiency based on input prompt properties such as difficulty or length control. They also cover training reasoning models with efficient data, reasoning capabilities of small language models, and evaluation methods and benchmarks.
This paper provides the first systematic survey on the overthinking problem, helping researchers comprehensively understand the field and set future research directions. It also plans to continuously update related research through the project website.