Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaoshen Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu et al.
Vision-R1 leverages reinforcement learning to enhance reasoning in multimodal large language models, using a high-quality CoT dataset and progressive thinking suppression training to significantly improve performance on math reasoning benchmarks.
Existing multimodal large language models (MLLMs) struggle to activate reasoning capabilities through reinforcement learning (RL). In particular, there is a lack of high-quality multimodal reasoning data needed to learn complex reasoning processes such as questioning and reflection.
1) Construct a high-quality multimodal CoT dataset (Vision-R1-cold) of 200K samples by leveraging an existing MLLM and DeepSeek-R1 through modality bridging and data filtering. 2) Propose a Progressive Thinking Suppression Training (PTST) strategy to mitigate optimization challenges caused by overthinking after cold start. 3) Use GRPO algorithm with a hard formatting result reward function to gradually fine-tune the model on 10K multimodal math data.
Achieves an average improvement of ~6% across various multimodal math reasoning benchmarks. Vision-R1-7B achieves 73.5% accuracy on MathVista, only 0.4% lower than OpenAI O1. Vision-R1-32B and Vision-R1-72B achieve 76.4% and 78.2% on MathVista, respectively. Datasets and code will be released.