UniMed‑CLIP: Towards a Unified Image‑Text Pre‑training Paradigm for Diverse Medical Imaging Modalities

Dataset Overview

Dataset Name

UniMed‑CLIP

Dataset Description

UniMed‑CLIP is a unified image‑text pre‑training dataset for multiple medical imaging modalities. It comprises over 5.3 million image‑text pairs covering six modalities: X‑ray, CT, MRI, Ultrasound, Pathology, and Fundus.

Dataset Features

1. Multimodal Coverage: Includes six distinct medical imaging modalities, providing rich multimodal data.
2. Large Scale: Over 5.3 million image‑text pairs furnish a solid foundation for training general medical VLMs.
3. Open‑Source: Accompanied by detailed code and annotation files for dataset preparation, fostering open research in medical VLMs.

Dataset Applications

UniMed‑CLIP primarily supports training and evaluating medical visual‑language models (VLMs), especially excelling in zero‑shot evaluation.

Dataset Contributions

1. UniMed Dataset: An open‑source large‑scale multimodal medical dataset with over 5.3 million samples across six modalities.
2. UniMed‑CLIP VLMs: Contrastive learning VLMs trained on UniMed, outperforming existing general VLMs across multiple medical modalities.
3. Extensive Evaluation: Provides ablation studies on design choices and releases training code, dataset, and model checkpoints to advance medical VLM research.

Dataset Performance

Dataset Updates

• 13 December 2024: Released annotation and code scripts for preparing the UniMed pre‑training dataset, along with training and inference code and pretrained checkpoints for UniMed‑CLIP.

Dataset Preparation

Detailed instructions and annotation files for dataset preparation are available in the UniMed‑DATA.md document.

Pre‑trained Models

Three UniMed‑CLIP model weights are provided:

Citation

If you use this dataset, please cite the following paper:

@inproceedings{khattakuniemed,
  title={UniMed‑CLIP: Towards a Unified Image‑Text Pre‑training Paradigm for Diverse Medical Imaging Modalities},
  author={Khattak, Muhammad Uzair and Kunhimon, Shahina and Muzzamal, Naseer and Khan, Salman and Khan, Fahad Shahbaz},
  journal={arXiv:2412.10372},
  year={2024}
}