Deep Visual Analogy-Making

Tensorflow implementation of Deep Visual Analogy-Making. The matlab code of the paper can be found here.

This implementation contains a deep network trained end-to-end to perform visual analogy making with

1. Fully connected encoder & decoder networks
2. Analogy transformations by vector addition and deep networks (vector multiplication is not implemented)
3. Regularizer for manifold traversal transformations

This implementation conatins:

1. Analogy transformations of shape dataset
   • with objective for vector-addition-based analogies (L_add)
   • with objective for multiple fully connected layers (L_deep)
   • with manifold traversal transformations

Prerequisites

• Python 2.7 or Python 3.3+
• Tensorflow
• SciPy

Usage

First, you need to download the dataset with:

$ ./download.sh

To train a model with shape dataset:

$ python main.py --dataset shape --is_train True

To test a model with shape dataset:

$ python main.py --dataset shape 

Results

Result of analogy transformations of shape dataset with fully connected layers (L_deep) after 1 day of training.

From top to bottom for each : Reference, output, query, target, prediction, manifold prediction after 2 steps, and manifold prediction after 3 steps.

• Change on angle

• Change on scale

• Change on x position

• Change on y position

(in progress)

Training details

Reference

• NIPS 2015 slide

Author

Taehoon Kim / @carpedm20