CSG: Contrastive Syn-to-Real Generalization

Paper

Contrastive Syn-to-Real Generalization.
Wuyang Chen, Zhiding Yu, Shalini De Mello, Sifei Liu, Jose M. Alvarez, Zhangyang Wang, Anima Anandkumar.
In ICLR 2021.

• Visda-17 to COCO

• train resnet101 with CSG
• evaluation

• GTA5 to Cityscapes

• train deeplabv2 (resnet50/resnet101) with CSG
• evaluation

Usage

Visda-17

• Download Visda-17 Dataset

Evaluation

• Download pretrained ResNet101 on Visda17
• Put the checkpoint under ./CSG/pretrained/
• Put the code below in train.sh

python train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--rand_seed 0 \
--csg 0.1 \
--apool \
--augment \
--csg-stages 3.4 \ 
--factor 0.1 \
--resume pretrained/csg_res101_vista17_best.pth.tar \
--evaluate

• Run CUDA_VISIBLE_DEVICES=0 bash train.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with CSG

• Put the code below in train.sh

python train.py \
--epochs 30 \
--batch-size 32 \
--lr 1e-4 \
--rand_seed 0 \
--csg 0.1 \
--apool \
--augment \
--csg-stages 3.4 \ 
--factor 0.1 \

• Run CUDA_VISIBLE_DEVICES=0 bash train.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

GTA5 → Cityscapes

• Download GTA5 dataset.
• Download the leftImg8bit_trainvaltest.zip and gtFine_trainvaltest.zip from the Cityscapes.
• Prepare the annotations by using the createTrainIdLabelImgs.py.
• Put the file of image list into where you save the dataset.
• Remember to properly set the C.dataset_path in the config_seg.py to the path where datasets reside.

Evaluation

• Download pretrained DeepLabV2-ResNet50 and DeepLabV2-ResNet101 on GTA5
• Put the checkpoint under ./CSG/pretrained/
• Put the code below in train_seg.sh

python train_seg.py \
--epochs 50 \
--switch-model deeplab50 \
--batch-size 6 \
--lr 1e-3 \ 
--num-class 19 \
--gpus 0 \
--factor 0.1 \
--csg 75 \
--apool \
--csg-stages 3.4 \
--chunks 8 \
--augment \
--evaluate \
--resume pretrained/csg_res101_segmentation_best.pth.tar \

• Change --switch-model (deeplab50 or deeplab101) and --resume (path to pretrained checkpoints) accordingly.
• Run CUDA_VISIBLE_DEVICES=0 bash train_seg.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Train with CSG

• Put the code below in train_seg.sh

python train_seg.py \
--epochs 50 \
--switch-model deeplab50 \
--batch-size 6 \
--lr 1e-3 \ 
--num-class 19 \
--gpus 0 \
--factor 0.1 \
--csg 75 \
--apool \
--csg-stages 3.4 \
--chunks 8 \
--augment

• Change --switch-model (deeplab50 or deeplab101) accordingly.
• Run CUDA_VISIBLE_DEVICES=0 bash train_seg.sh
  • Please update the GPU index via CUDA_VISIBLE_DEVICES based on your need.

Citation

If you use this code for your research, please cite:

@article{chen2021contrastive,
  title={Contrastive syn-to-real generalization},
  author={Chen, Wuyang and Yu, Zhiding and Mello, SD and Liu, Sifei and Alvarez, Jose M and Wang, Zhangyang and Anandkumar, Anima},
  year={2021},
  publisher={ICLR}
}