README.md

Painter Experiments @Visual In-Context Learning

Brilliant-B

This repo is the modification and experiments of Painter.
for installation & data preparation & pretrained preparation, please refer to Official docs.

Models

The modified Painter model is in the directory $Painter_ROOT/models/

• Painter Variant 2: painter_variant_2.py
  add cr_banks and some other modifications
  
  • variant_3.py: attempt to delete encoder layers after xcr_depth, in order to examine the LLM-transferability.
    
• Painter Variant 1: painter_variant_1.py
  add controls of num_contexts/cr_depth/xcr_depth
• Painter Variant 0: painter_variant_0.py
  change some of the code structure
• Original Painter: models_painter.py
  

Training

The model will be trained, mostly finetuned, based on pretrained checkpoints under multiple hyper-parameters.
For new model training experiments, check the directory self_experiments/finetune
You can modify and run train_bash.sh

• Multi-datasets Training PORTAL: multi_finetune_portal.py
  
  • Modify it in the script. What's more, you can choose to use joint-dataset or seperate-dataset training.
    
• Hyper-parameter Testing: Finetune for ADE-20K semantic segmentation: finetune_ade20k_semseg.py
  

(more will be issued later)

Evaluation

For new model evaluation experiments, check the directory $Painter_ROOT/self_experiments/eval
You can modify and run eval_bash.sh

• Multi-datasets Evaluation PORTAL: multi_test_portal.py
  
  • Modify it in the script, where datasets are evaluated one-by-one and overall metrics will be generated
    
• Hyper-parameter Testing: Evaluation for ADE-20K semantic segmentation: test_ade20k_semseg.py
  

(more will be issued later)