classification

EVA-X: X-ray Image Classification

Installation

First, clone the repo and install required packages:

conda create --name evax python=3.8 -y
conda activate evax

git clone [email protected]:hustvl/EVA-X.git
cd EVA-X
pip install -r requirements.txt

Then, install Apex and xFormer following the official instruction.

Core packages:

• Pytorch version 1.12.1
• torchvision version 0.13.0
• timm version 0.5.4
• DeepSpeed version 0.6.5 (fp16 training and ZeRO optimizer), fine-tuning with bfloat16 requires version 0.8.1
• Apex (fused layer norm)
• xFormer (fast and memory efficient MHSA)

Data Preparation

We have done the most of the work before. You could use the datasets with simple steps.

1. Download Chest X-Ray14 / CheXpert / CovidX-CXR-3 / CovidX-CXR-4 from the given links. (Note: for CovidX dataset, we use the version-5 and version-9.)

2. Prepare them by running the following codes. It will prepare and check the datasets automatically.

   python datasets/prepare_dataset.py
   

If there is no error, it's done!

Weights Preparation

Run EVA-X Only

If you just want to use EVA-X for medical image classification, you only need to download the weights for EVA-X.

EVA-X Series	Architecture	#Params	Checkpoint	MIM epochs
EVA-X-Ti	ViT-Ti/16	6M	🤗download	900
EVA-X-S	ViT-S/16	22M	🤗download	600
EVA-X-B	ViT-B/16	86M	🤗download	600

Reproduce all methods

In our experiments, in addition to EVA-X, we used 15 different visual representations for comparison. You can choose to download the weights you need.

Architecture	#Params	Methods / Downloads
ResNet50	24M	ImageNet; MoCov2; MedKLIP MGCA; BioViL
DenseNet121	8M	ImageNet; MoCov2; Medical MAE
ViT-S/16	22M	DEiT; MAE; EVA-02; Medical MAE

Some weights should be processed for training. We provide codes for them:

# process ResNet50s
python tools/preprocess_r50.py

# process EVA02
python tools/interpolate14to16.py

Run

Run EVA-X Only

We use 4 RTX 3090 GPUs for finetuning. We have fixed most of the randomness. So the results are reproducible. We have released our weights, you can also eval with them directly (how to eval? ) . Run the following codes to finetune EVA-X in classification datasets:

Chest X-Ray14

The reported mAUC is evaluated on the official test set.

sh train_files/eva_x/cxr14/vit_ti.sh
sh train_files/eva_x/cxr14/vit_s.sh
sh train_files/eva_x/cxr14/vit_s.sh

EVA-X Series	Architecture	#Params	MIM epochs	FT configs	FT epochs	Res	mAUC	Checkpoint
EVA-X-Ti	ViT-Ti/16	6M	900	config	45	224x224	82.4	🤗download
EVA-X-S	ViT-S/16	22M	600	config	30	224x224	83.3	🤗download
EVA-X-B	ViT-B/16	86M	600	config	10	224x224	83.5	🤗download

CheXpert

The reported mAUC is evaluated on the official val set.

sh train_files/eva_x/chexpert/vit_ti.sh
sh train_files/eva_x/chexpert/vit_s.sh

EVA-X Series	Architecture	#Params	MIM epochs	FT configs	FT epochs	Res	mAUC	Checkpoint
EVA-X-Ti	ViT-Ti/16	6M	900	config	5	224x224	89.6	🤗download
EVA-X-S	ViT-S/16	22M	600	config	5	224x224	90.1	🤗download

CovidX-CXR-3&4

We train EVA-X with these two dataset with 1%, 10% and 100% X-ray images of the training set. You could change the data percentage in the provided training files.

The reported mAcc is evaluated on the official test/val set.

sh train_files/eva_x/covidx3/vit_s.sh
sh train_files/eva_x/covidx4/vit_s.sh

EVA-X Series	Architecture	#Params	Dataset	FT configs	FT epochs	Res	mAcc	Checkpoint
EVA-X-S	ViT-S/16	22M	CovidX-CXR-3	config	10	480x480	97.5	🤗download
EVA-X-S	ViT-S/16	22M	CovidX-CXR-4	config	10	480x480	96.0	🤗download

Note: We do not claim that any of the above models can be used directly for disease diagnosis. Please seek professional medical help when needed.

Run with other methods

We compare EVA-X with 15 different visual representations. We compare them on three datasets: Chest X-Ray14, CovidX-CXR-3, CovidX-CXR-4. We have uploaded part of their finetuned weights for analysis. More results please refer to our paper. We appreciate all of their excellent research work!

Chest X-Ray14

For example, run:

# choose the file you want to reproduce
sh train_files/resnet/cxr14/r50_biovil.sh

PT Methods	Architecture	#Params	FT configs	FT epochs	Res	mAUC	Checkpoint
MoCov2	DN121	7M	config	75	224x224	80.8	🤗download
Medical MAE	DN121	7M	config	75	224x224	81.3	🤗download
BioViL	ResNet50	24M	config	75	224x224	80.8	🤗download
MGCA	ResNet50	24M	config	75	224x224	81.3	🤗download
MedKLIP	ResNet50	24M	config	75	224x224	81.2	🤗download
Medical MAE	ViT-S	22M	config	75	224x224	82.3	official download

Evaluate

You could eval with our finetuned models by slightly changing the train files.

1. Download our pre-trained CKPT above.
2. Change the train file:

   (1) change weights
   CKPT_PATH='/path/to/mim/weights' --> '/path/to/finetuned/weights'
   (2) add '--eval' to the end of args
   

Acknowledgements

The codes are built upon EVA, Medical MAE Thanks for their great work!