python 3.6
Pytorch 0.4.1
In addition, please add the project folder to PYTHONPATH and pip install the following packages:
python-dateutileasydictpandastorchfilenltkscikit-imagespacyPyYAMLcffitorchtextdillCython
====================== DATASET AND PRE-MODELS are consistent with Obj-GAN ======================
Data
- Download our preprocessed metadata for coco and merge them to
data/coco - Download coco dataset, extract the images to
data/coco/images, and extract the annotations todata/coco/insanns
Training
- Train box generator:
cd box_generationpython sample.py --is_training 1
- Train shape generator:
cd shape_generation./make.shpython main.py --gpu '0,1' --FLAG
- Train image generator:
cd image_generation./make.shpython main.py --gpu '0,1' --FLAG
Pretrained Model
Download and save them to data/coco/pretrained/
Note that we have made some modifications (changing the obj attention estimation from "dot product between Glove embeddings" to "cosine similarity between Glove embeddings") based on the code for CVPR submission, and trained 120 epochs using batch size 16. Compared to the results in the paper, the updated results are better on FID and R-prsn scores, and worse on Inception score (because we do not get a chance to train the model using larger batch size).
| Methods | Box generator | Shape generator | Inception | FID |
|---|---|---|---|---|
| Obj-GAN | YES | YES | 32.26 | 18.25 |
| Obj-GAN_1 | YSE | NO | 31.41 | 19.21 |
| Obj-GAN_2 | NO | YES | 32.54 | 19.87 |
Tips for optimizing the Inception score (though it is boring):
- Increase the batch size as large as possible via distributed training
- Increase the weight for the DAMSM loss
Sampling
- Run box generator:
cd box_generationpython sample.py --is_training 0 --load_checkpoint [replace with your ckpt path]
- Run shape generator:
cd shape_generationpython main.py --gpu '0,1' --NET_G [replace with your ckpt path]
- Run image generator:
cd image_generationpython main.py --gpu '0,1' --NET_G [replace with your ckpt path]
More Results
