Vision Foundation Models for Medical Imaging

We evaluate and adapt vision foundation models (VFMs) for medical imaging tasks, establishing benchmarks and demonstrating their potential for out-of-the-box medical image registration and segmentation.

Key Contributions

• Foundation model evaluation: Conducted rapid, large-scale evaluation of SAM2 across 21 diverse medical imaging datasets within 72 hours of its release
• Registration benchmarks: Established benchmarks for foundation model-based registration in breast MRI, demonstrating their readiness for out-of-the-box medical image registration (Best Paper Award at MICCAI Workshop 2025)
• 2D and 3D segmentation: Extended Segment Anything Model (SAM2) to handle both 2D and 3D medical imaging tasks, showing significant improvements in segmentation flexibility
• Multi-modal adaptation: Participated in developing multiple medical segmentation models through foundation model fine-tuning strategies

Impact

• Best Paper Award: “Are Vision Foundation Models Ready for Out-of-the-Box Medical Image Registration?” won Best Paper Award at Deep-Breath @ MICCAI 2025
• Early contribution: Established an early and impactful contribution to the field by rapidly evaluating SAM2 across diverse medical datasets
• Clinical translation: Demonstrated the feasibility of using pre-trained foundation models for medical imaging tasks, reducing the need for extensive domain-specific training

Applications

• Medical image registration using pre-trained vision foundation models
• Zero-shot and few-shot segmentation in medical imaging
• Rapid deployment of AI models for new medical imaging tasks
• Foundation model fine-tuning strategies for medical applications