Xiaoyang Han, Jianhua Li, Kewang Deng, Zukai Chen, Xuanke Shi, Sihan Wang, Boxuan Li, Linyan Wang et al.
This work reformulates computer vision as a unified multimodal generation task, enabling a single model to solve diverse vision problems without task-specific architectures.
Computer vision often requires separate architectures and prediction heads for each task, making it difficult to integrate into general-purpose foundation models. This research addresses this by redefining vision within a unified framework of text and image generation.
SenseNova-Vision uses natural language instructions and optional visual prompts to specify tasks, target regions, and decoding conventions. It generates text for symbolic outputs and images for dense spatial predictions. A large-scale training corpus, the SenseNova-Vision Corpus, was created by converting diverse vision annotations into instruction-response examples. An off-the-shelf multimodal model was primarily trained on this corpus.
Experiments show that a single unified model can match leading task-specialized systems across structured visual understanding, dense geometric prediction, segmentation, and multi-view visual geometry. The results suggest unified multimodal generation as a scalable route for integrating computer vision capabilities into general-purpose foundation models. The model and corpus are publicly available.