Skip to content

Latest commit

 

History

History
124 lines (96 loc) · 6.87 KB

README.md

File metadata and controls

124 lines (96 loc) · 6.87 KB
Raw

High Quality Segmentation for Ultra High-resolution Images

Tiancheng Shen, Yuechen Zhang, Lu Qi, Jason Kuen, Xingyu Xie, Jianlong Wu, Zhe Lin, Jiaya Jia

This project provides an implementation for the paper "High Quality Segmentation for Ultra High-resolution Images".


To segment 4K or 6K ultra high-resolution images needs extra computation consideration in image segmentation. Common strategies, such as down-sampling, patch cropping, and cascade model, cannot address well the balance issue between accuracy and computation cost. Motivated by the fact that humans distinguish among objects continuously from coarse to precise levels, we propose the Continuous Refinement Model~(CRM) for the ultra high-resolution segmentation refinement task. CRM continuously aligns the feature map with the refinement target and aggregates features to reconstruct these image details. Besides, our CRM shows its significant generalization ability to fill the resolution gap between low-resolution training images and ultra high-resolution testing ones. We present quantitative performance evaluation and visualization to show that our proposed method is fast and effective on image segmentation refinement.


Installation

This project is based on CascadePSP.

Dependencies

CRM can be trained and tested on PyTorch 1.7.1 or higher version. Other dependencies are needed to be installed by:

pip install progressbar2
pip install opencv-python

Download the dataset

Use the script in CRM/scripts/ to download the training dataset. The training dataset merges the following datasets: MSRA-10K, DUT-OMRON, ECSSD, and FSS-1000.

cd ./scripts/
python download_training_dataset.py

For the evaluation dataset BIG. Please download it follow the CascadePSP's instruction.

Training

To train model with 2 GPUs, run:

cd CRM/
python train.py Exp_ID -i 45000 -b 12 --steps 22500 37500 --lr 2.25e-4 --ce_weight 1.0 --l1_weight 0.5 --l2_weight 0.5 --grad_weight 2.0

Evaluation and Visualization

To evaluate a pre-trained model on BIG dataset, run:

python test.py \
    --dir /PathTO/BIG_PSPNet_SS \
    --model /PathTO/weights/Exp_ID/model_45705 \
    --output /PathTO/output/Exp_ID \
    --clear

python eval_post0.125.py --dir /PathTO/output/Exp_ID

python eval_post0.25.py --dir /PathTO/output/Exp_ID

python eval_post0.5.py --dir /PathTO/output/Exp_ID

python eval_post1.0.py --dir /PathTO/output/Exp_ID

Pretrained weights of CRM

Checkpoint Downloads File Size
This is the model that we used to generate all of our results in the paper. OneDrive 93MB

Segmentation Results

For convenience, we provide segmentation results from other models for evaluation and Visualization.

Dataset Coarse Mask Source Output Link
BIG (Test) DeeplabV3+ Link
RefineNet Link
PSPNet Link
FCN-8s Link

Citing Ours

Consider to cite High Quality Segmentation for Ultra High-resolution Images if it helps your research.

@article{shen2021high,
  title={High Quality Segmentation for Ultra High-resolution Images},
  author={Tiancheng Shen, Yuechen Zhang, Lu Qi, Jason Kuen, Xingyu Xie, Jianlong Wu, Zhe Lin, Jiaya Jia},
  journal={CVPR},
  year={2022}
}