Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A. Efros (2017)
- Learning image-to-image translation without need for paired examples
- Capture special characteristics of one image collection and figure out how these could be translated into the other collection
- Cycle-consistent: translation of translation should be same as original --> ensure input-output is paired in a meaningful way!
- 2 generative adversarial networks (GANs), 2 consistency losses
- Only 1 GAN and no cycle-consistency: mode collapse (all input --> same output) or irrelevant mappings
- GAN: discriminator "helps" generator through backprop (gives info on how to improve)
- Endpoint of training: discriminator no better than random guessing (0.5)
- Only works for color/texture changes, not for geometric ones
- Other variations, not present in training data: e.g. network thinks man on horse is part of the horse
- Weak semantic supervision may be required to close the gap with paired translators