Deep Visual Analogy-Making

Tensorflow implementation of Deep Visual Analogy-Making. The matlab code of the paper can be found here.

This implementation contains a deep network trained end-to-end to perform visual analogy making with

1. Fully connected encoder & decoder networks
2. Analogy transformations by vector addition and deep networks (vector multiplication is not implemented)
3. Regularizer for manifold traversal transformations

This implementation conatins:

1. Analogy transformations of  shape  dataset
   • with objective for vector-addition-based analogies (L_add)
   • with objective for multiple fully connected layers (L_deep)
   • with manifold traversal transformations

Prerequisites

• Python 2.7 or Python 3.3+
• Tensorflow
• SciPy

Usage

First, you need to download the dataset with:

normal$ ./download.sh
normal

To train a model with  shape  dataset:

normal$ python main.py --dataset shape --is_train True
normal

To test a model with  shape  dataset:

normal$ python main.py --dataset shape 
normal

Results

Result of analogy transformations of  shape  dataset with fully connected layers (L_deep) after 1 day of training.

From top to bottom for each : Reference, output, query, target, prediction, manifold prediction after 2 steps, and manifold prediction after 3 steps.

• Change on angle

• Change on scale

• Change on x position

• Change on y position

(in progress)

Training details

Reference

• NIPS 2015 slide

Author

Taehoon Kim / @carpedm20