- Published on
A New Neuroimaging Foundation Model Trained With Vol-JEPA on Uncurated Clinical MRI and CT Volumes
- Authors
- Name
- DeepHorus
Researchers at the University of Michigan have developed NeuroVFM, a groundbreaking foundation model for neuroimaging that learns directly from uncurated clinical MRI and CT scans. Published in Nature Medicine, the model achieves state-of-the-art performance on 156 diagnostic tasks while training efficiently on real-world hospital data. This advance addresses a critical gap where general AI models underperform on brain imaging due to the scarcity of clinical neuroimaging data on the internet.
Clinical neuroimaging rarely appears in public datasets because MRI and CT scans contain identifiable facial features, limiting model development. The Michigan team overcame this through 'health system learning,' training NeuroVFM on 5.24 million clinical volumes from 566,915 studies spanning two decades at Michigan Medicine. Unlike previous approaches, the model requires no paired radiology reports or disease-specific curation, instead learning directly from routine clinical data.
The core innovation is Vol-JEPA, a self-supervised algorithm that predicts representations in a learned latent space rather than reconstructing pixels. Applied to 3D volumes, Vol-JEPA uses foreground-focused masking and a student-teacher framework to model shared neuroanatomy. Training consumed fewer than 1,000 GPU hours and was seven times faster than a 3DINO baseline, while fitting 16 times larger batches. The base encoder has 85.8 million parameters.
NeuroVFM achieved a macro-averaged AUROC of 92.68 on CT and 92.49 on MRI across 74 MRI and 82 CT diagnoses, outperforming all baselines including PRIMA, HLIP, and NeuroMAE. The researchers attribute this to latent prediction outperforming report supervision and voxel reconstruction. Beyond classification, the model supports report generation, triage, grounded predictions, and cross-modal transfer with minimal performance drop.
'Training on uncurated health system data allows the model to learn robust representations that generalize across diverse clinical scenarios,' the authors stated. In a one-week prospective study of 1,155 scans, NeuroVFM achieved 92.6% balanced triage accuracy compared to 71.2% for GPT-5, though it missed 21 of 155 critical findings, yielding 86.5% sensitivity.
The team plans to expand NeuroVFM's capabilities and address limitations in sensitivity for critical findings. By demonstrating that health system learning can produce powerful foundation models without curated datasets, this work opens the door to broader adoption of AI in clinical neuroimaging, potentially improving diagnostic support across healthcare systems.
