Peraza, J. A., Kent, J. D., Nichols, T. E., Poline, J.-B., de la Vega, A., Laird, A. R.
NiCLIP is a contrastive language-image pretrained model that aligns brain activation images with text to predict cognitive tasks, advancing functional decoding in neuroimaging.
Predicting cognitive processes from brain activation maps has been challenging. Existing meta-analytic methods rely on limited metrics that fail to capture semantic context from publications.
Trained a CLIP model on over 23,000 full-text neuroscientific articles to contrastively align text with brain activation patterns. Applied a curated cognitive ontology and fine-tuned LLMs like BrainGPT.
Accurately predicted cognitive tasks (emotion, language, motor) from group-level activation maps from the Human Connectome Project and characterized functional roles of specific brain regions. Showed limitations with noisy subject-level maps, but represents a significant advancement in quantitative functional decoding.