Repository files navigation

This repository contains the source code of the algorithm described in aCVPR 2015 paperMid-level Deep Pattern Miningand also a technical reportMining Mid-level Visual Patterns with Deep CNN Activations. More details are provided on theproject page.This package has been tested using Matlab 2014a on a 64-bit Linux machine. This code is for research purposes only.

If you find MDPM useful in your research, please consider citing:

@inproceedings{LiLSH15CVPR,    author = {Yao Li and Lingqiao Liu and Chunhua Shen and Anton van den Hengel},    title = {Mid-level Deep Pattern Mining},    booktitle = {CVPR},    year = {2015}    pages = {971-980},}

0. Prerequisites
1. Caffe: install Caffe by following itsinstallation instructions.Do not forget to runmake matcaffe to compile Caffe's Matlab interface. You also need to download the ImageNet mean file (runget_ilsvrc_aux.sh fromdata/ilsvrc12).Note: As we only use Caffe CNN as a feature extractor, installing Caffe using the CPU mode is OK.
2. CNN models. We use consider two CNN models in the experiment. The first one is BVLC Reference CaffeNet (CaffeRef for short),this model can be downloaded by runningdownload_model_binary.py models/bvlc_reference_caffenet fromscripts.The second is VGG 19-layer Very Deep model (VGGVD for short), which can be downloaded fromhere.
3. Apriori algorithm: we usethis implementation. Click the link to download this package. You needto uncompress it and runmake to compile it in theapriori/apriori/src.Detailed usage of this package can be foundhere.
4. Liblinear: download liblinear and compile it by following its instructions.
5. KSVDS-Box v11: as we use theim2colstep function in this toolbox,you need to download and compile it (im2colstep is found inksvdsbox11/private).
6. Configuring MDPM
7. Download MDPM:git clone https://github.com/yaoliUoA/MDPM.
8. Download MIT Indoor dataset fromhere.
9. Openinit.m in the Matlab. Change values of sereval variables based on your configuration, includingconf.pathToLiblinear,conf.pathToCaffe,conf.dataset andconf.imgDir based on yourlocal configuration.
10. Copy the executable fileaprior under directoryapriori/apriori/src and paste it undermining directory.
11. Copy the mex fileim2colstep and paste it undercnn directory.
12. Running MDPM
13. Run thedemo.m. It should be working properly for MIT Indoor dataset if you have followed aforementioned instructions. Note that we have notreleased a demo for PASCAL VOC datasets as the dataset setting for VOC datasets is different.
14. Important: It may takes some time to get the final classification result, so it is suggested to run MDPM on a clusterwhere jobs can be run in parallel. The*.sh scripts are provided to submit jobs on a cluster.

We provide final image features generated by the proposed MDPM algorithm using different CNN models (CaffeRef or VGGVD).You should able to reproduce our result presented in the CVPR 2015 paper and technical report.

0. MIT Indoor dataset
1. feature_MITIndoor_CaffeRef andfeature_MITIndoor_VGGVD.After uncompressing the downloaded file, copy the.mat files todata/MIT67/feaFinal_128_32_150 directory (create by yourself), you should be able to runclassify.munderclassify to reproduce the classification accuracy presented in the technical report.
2. PASCAL VOC 2007 dataset
3. feature_VOC2007_CaffeRef andfeature_VOC2007_VGGVD.After uncompressing the downloaded file, copy the.mat files todata/VOC2007/feaFinal_128_32_150 directory (create by yourself), you should be able to runtrain_VOC.mand thentest_VOC.m underclassify to reproduce the mean average precision presented in the technical report.
4. PASCAL VOC 2012 dataset
5. feature_VOC2012_VGGVD.After uncompressing the downloaded file, copy the.mat files todata/VOC2012/feaFinal_128_32_150 directory (create by yourself), you should be able to runtrain_VOC.mand thentest_VOC_txt.m underclassify. The generated .txt files can be submitted to theevaluation server.

If you have any issues (question, feedback) or find bugs in the code, please contactyao.li01@adelaide.edu.au.