Repository files navigation

Chen-Hsuan LinandSimon Lucey
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017 (oral presentation)

Project page:https://chenhsuanlin.bitbucket.io/inverse-compositional-STN
Paper:https://chenhsuanlin.bitbucket.io/inverse-compositional-STN/paper.pdf
Poster:https://chenhsuanlin.bitbucket.io/inverse-compositional-STN/poster.pdf
arXiv preprint:https://arxiv.org/abs/1612.03897

We provide TensorFlow code for the following experiments:

• MNIST classification
• traffic sign classification

[NEW!] The PyTorch implementation of the MNIST experiment is now up!

This code is developed with Python3 (python3) but it is also compatible with Python2.7 (python). TensorFlow r1.0+ is required. The dependencies can install by running

pip3 install --upgrade numpy scipy termcolor matplotlib tensorflow-gpu

If you're using Python2.7, usepip2 instead; if you don't have sudo access, add the--user flag.

The training code can be executed via the command

python3 train.py <netType> [(options)]

<netType> should be one of the following:

1. CNN - standard convolutional neural network
2. STN - Spatial Transformer Network (STN)
3. IC-STN - Inverse Compositional Spatial Transformer Network (IC-STN)

The list of optional arguments can be found by executingpython3 train.py --help.
The default training settings in this released code is slightly different from that in the paper; it is stabler and optimizes the networks better.

When the code is run for the first time, the datasets will be automatically downloaded and preprocessed.
The checkpoints are saved in the automatically created directorymodel_GROUP; summaries are saved insummary_GROUP.

We've included code to visualize the training over TensorBoard. To execute, run

tensorboard --logdir=summary_GROUP --port=6006

We provide three types of data visualization:

1. SCALARS: training/test error over iterations
2. IMAGES: alignment results and mean/variance appearances
3. GRAPH: network architecture

The PyTorch version of the code is stil under active development. The training speed is currently slower than the TensorFlow version. Suggestions on improvements are welcome! :)

This code is developed with Python3 (python3). It has not been tested with Python2.7 yet. PyTorch 0.2.0+ is required. Please seehttp://pytorch.org/ for installation instructions.
Visdom is also required; it can be installed by running

pip3 install --upgrade visdom

If you don't have sudo access, add the--user flag.

First, start a Visdom server by running

python3 -m visdom.server -port=7000

The training code can be executed via the command (using the same port number)

python3 train.py <netType> --port=7000 [(options)]

<netType> should be one of the following:

1. CNN - standard convolutional neural network
2. STN - Spatial Transformer Network (STN)
3. IC-STN - Inverse Compositional Spatial Transformer Network (IC-STN)

The list of optional arguments can be found by executingpython3 train.py --help.
The default training settings in this released code is slightly different from that in the paper; it is stabler and optimizes the networks better.

When the code is run for the first time, the datasets will be automatically downloaded and preprocessed.
The checkpoints are saved in the automatically created directorymodel_GROUP; summaries are saved insummary_GROUP.

We provide three types of data visualization on Visdom:

1. Training/test error over iterations
2. Alignment results and mean/variance appearances

If you find our code useful for your research, please cite

@inproceedings{lin2017inverse,  title={Inverse Compositional Spatial Transformer Networks},  author={Lin, Chen-Hsuan and Lucey, Simon},  booktitle={IEEE Conference on Computer Vision and Pattern Recognition ({CVPR})},  year={2017}}

Please contact me (chlin@cmu.edu) if you have any questions!