Repository files navigation

The official Faster R-CNN code (written in MATLAB) is availablehere.If your goal is to reproduce the results in our NIPS 2015 paper, please use theofficial code.

This repository contains a Pythonreimplementation of the MATLAB code.This Python implementation is built on a fork ofFast R-CNN.There are slight differences between the two implementations.In particular, this Python port

• is ~10% slower at test-time, because some operations execute on the CPU in Python layers (e.g., 220ms / image vs. 200ms / image for VGG16)
• gives similar, but not exactly the same, mAP as the MATLAB version
• isnot compatible with models trained using the MATLAB code due to the minor implementation differences
• includes approximate joint training that is 1.5x faster than alternating optimization (for VGG16) -- see theseslides for more information

By Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (Microsoft Research)

This Python implementation contains contributions from Sean Bell (Cornell) written during an MSR internship.

Please see the officialREADME.md for more details.

Faster R-CNN was initially described in anarXiv tech report and was subsequently published in NIPS 2015.

Faster R-CNN is released under the MIT License (refer to the LICENSE file for details).

If you find Faster R-CNN useful in your research, please consider citing:

@inproceedings{renNIPS15fasterrcnn,    Author = {Shaoqing Ren and Kaiming He and Ross Girshick and Jian Sun},    Title = {Faster {R-CNN}: Towards Real-Time Object Detection             with Region Proposal Networks},    Booktitle = {Advances in Neural Information Processing Systems ({NIPS})},    Year = {2015}}

1. Requirements: software
2. Requirements: hardware
3. Basic installation
4. Demo
5. Beyond the demo: training and testing
6. Usage

NOTE If you are having issues compiling and you are using a recent version of CUDA/cuDNN, please consultthis issue for a workaround

1. Requirements forCaffe andpycaffe (see:Caffe installation instructions)

Note: Caffemust be built with support for Python layers!

# In your Makefile.config, make sure to have this line uncommentedWITH_PYTHON_LAYER := 1# Unrelatedly, it's also recommended that you use CUDNNUSE_CUDNN := 1

You can download myMakefile.config for reference.2. Python packages you might not have:cython,python-opencv,easydict3. [Optional] MATLAB is required forofficial PASCAL VOC evaluation only. The code now includes unofficial Python evaluation code.

1. For training smaller networks (ZF, VGG_CNN_M_1024) a good GPU (e.g., Titan, K20, K40, ...) with at least 3G of memory suffices
2. For training Fast R-CNN with VGG16, you'll need a K40 (~11G of memory)
3. For training the end-to-end version of Faster R-CNN with VGG16, 3G of GPU memory is sufficient (using CUDNN)

1. Clone the Faster R-CNN repository

# Make sure to clone with --recursivegit clone --recursive https://github.com/rbgirshick/py-faster-rcnn.git

2. We'll call the directory that you cloned Faster R-CNN intoFRCN_ROOT

   Ignore notes 1 and 2 if you followed step 1 above.

   Note 1: If you didn't clone Faster R-CNN with the--recursive flag, then you'll need to manually clone thecaffe-fast-rcnn submodule:

   git submodule update --init --recursive

   Note 2: Thecaffe-fast-rcnn submodule needs to be on thefaster-rcnn branch (or equivalent detached state). This will happen automaticallyif you followed step 1 instructions.

3. Build the Cython modules

   cd$FRCN_ROOT/libmake

4. Build Caffe and pycaffe

   cd$FRCN_ROOT/caffe-fast-rcnn# Now follow the Caffe installation instructions here:#   http://caffe.berkeleyvision.org/installation.html# If you're experienced with Caffe and have all of the requirements installed# and your Makefile.config in place, then simply do:make -j8&& make pycaffe

5. Download pre-computed Faster R-CNN detectors

   cd$FRCN_ROOT./data/scripts/fetch_faster_rcnn_models.sh

   This will populate the$FRCN_ROOT/data folder withfaster_rcnn_models. Seedata/README.md for details.These models were trained on VOC 2007 trainval.

After successfully completingbasic installation, you'll be ready to run the demo.

To run the demo

cd$FRCN_ROOT./tools/demo.py

The demo performs detection using a VGG16 network trained for detection on PASCAL VOC 2007.

1. Download the training, validation, test data and VOCdevkit

   wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tarwget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tarwget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar

2. Extract all of these tars into one directory namedVOCdevkit

   tar xvf VOCtrainval_06-Nov-2007.tartar xvf VOCtest_06-Nov-2007.tartar xvf VOCdevkit_08-Jun-2007.tar

3. It should have this basic structure

   $VOCdevkit/# development kit$VOCdevkit/VOCcode/# VOC utility code$VOCdevkit/VOC2007# image sets, annotations, etc.# ... and several other directories ...

4. Create symlinks for the PASCAL VOC dataset

   cd$FRCN_ROOT/dataln -s$VOCdevkit VOCdevkit2007

   Using symlinks is a good idea because you will likely want to share the same PASCAL dataset installation between multiple projects.

5. [Optional] follow similar steps to get PASCAL VOC 2010 and 2012

6. [Optional] If you want to use COCO, please see some notes underdata/README.md

7. Follow the next sections to download pre-trained ImageNet models

Pre-trained ImageNet models can be downloaded for the three networks described in the paper: ZF and VGG16.

cd$FRCN_ROOT./data/scripts/fetch_imagenet_models.sh

VGG16 comes from theCaffe Model Zoo, but is provided here for your convenience.ZF was trained at MSRA.

To train and test a Faster R-CNN detector using thealternating optimization algorithm from our NIPS 2015 paper, useexperiments/scripts/faster_rcnn_alt_opt.sh.Output is written underneath$FRCN_ROOT/output.

cd$FRCN_ROOT./experiments/scripts/faster_rcnn_alt_opt.sh [GPU_ID] [NET] [--set ...]# GPU_ID is the GPU you want to train on# NET in {ZF, VGG_CNN_M_1024, VGG16} is the network arch to use# --set ... allows you to specify fast_rcnn.config options, e.g.#   --set EXP_DIR seed_rng1701 RNG_SEED 1701

("alt opt" refers to the alternating optimization training algorithm described in the NIPS paper.)

To train and test a Faster R-CNN detector using theapproximate joint training method, useexperiments/scripts/faster_rcnn_end2end.sh.Output is written underneath$FRCN_ROOT/output.

cd$FRCN_ROOT./experiments/scripts/faster_rcnn_end2end.sh [GPU_ID] [NET] [--set ...]# GPU_ID is the GPU you want to train on# NET in {ZF, VGG_CNN_M_1024, VGG16} is the network arch to use# --set ... allows you to specify fast_rcnn.config options, e.g.#   --set EXP_DIR seed_rng1701 RNG_SEED 1701

This method trains the RPN module jointly with the Fast R-CNN network, rather than alternating between training the two. It results in faster (~ 1.5x speedup) training times and similar detection accuracy. See theseslides for more details.

Artifacts generated by the scripts intools are written in this directory.

Trained Fast R-CNN networks are saved under:

output/<experiment directory>/<dataset name>/

Test outputs are saved under:

output/<experiment directory>/<dataset name>/<network snapshot name>/