Repository files navigation

This repository contains the architectures, Models, logs, etc pertaining to the SimpleNet Paper(Lets keep it simple: Using simple architectures to outperform deeper architectures ) :https://arxiv.org/abs/1608.06037

SimpleNet-V1 outperforms deeper and heavier architectures such as AlexNet, VGGNet,ResNet,GoogleNet,etc in a series of benchmark datasets, such as CIFAR10/100, MNIST, SVHN.It also achievs a higher accuracy (currently72.03/90.32) in imagenet, more than VGGNet, ResNet, MobileNet, AlexNet, NIN, Squeezenet, etc with only 5.7M parameters. It also achieves74.23/91.748) with 9m version.
Slimer versions of the architecture work very decently against more complex architectures such as ResNet, WRN and MobileNet as well.

If you find SimpleNet useful in your research, please consider citing:

@article{hasanpour2016lets,  title={Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures},  author={Hasanpour, Seyyed Hossein and Rouhani, Mohammad and Fayyaz, Mohsen and Sabokrou, Mohammad},  journal={arXiv preprint arXiv:1608.06037},  year={2016}}

(Check the successor of this architecture atTowards Principled Design of Deep Convolutional Networks: Introducing SimpNet)
 

OfficialPytorch implementation
 
 

ImageNet result below was achieved using thePytorch implementation

• Achieved using Pytorch implementation

SimpleNet outperforms much deeper and larger architectures on the ImageNet dataset:

SimpleNet performs very decently, it outperforms VGGNet, variants of ResNet and MobileNets(1-3)
and is pretty fast as well! and its all using plain old CNN!.
To view the full benchmark results visit thebenchmark page.
To view more results checkout thethe Pytorch implementation page

*Note that the Fractional max pooling[13] uses deeper architectures and also uses extreme data augmentation.۞ means No zero-padding or normalization with dropout and ۩ means Standard data-augmentation- with dropout. To our knowledge, our architecture has the state of the art result, without aforementioned data-augmentations.

*Note that we didn’t intend on achieving the state of the artperformance here as we are using a single optimization policy withoutfine-tuning hyper parameters or data-augmentation for a specific task,and still we nearly achieved state-of-the-art on MNIST. **Resultsachieved using an ensemble or extreme data-augmentation

Table 6-Slimmed version Results on Different Datasets

*Since we presented their results in their respective sections, we avoided mentioning the results here again.

** Achieved using several data-augmentation tricks

Flops and Parameter Comparison:[tab:Flops_appndx]

Flops and Parameter Comparison of Models trained on ImageNet

*Inception v3, v4 did not have any Caffe model, so we reported theirsize related information from MXNet and Tensorflow respectively.Inception-ResNet-V2 would take 60 days of training with 2 Titan X toachieve the reported accuracy. Statistics are obtained usinghttp://dgschwend.github.io/netscope

1# Data-augmentation method used by stochastic depth paper:https://github.com/Pastromhaug/caffe-stochastic-depth.

2#https://github.com/dmlc/mxnet-model-gallery/blob/master/imagenet-1k-inception-v3.md

3#https://github.com/tensorflow/models/tree/master/slim

4#https://github.com/revilokeb/inception\_resnetv2\_caffe

This was based on my Master's thesis titled "Object classification using Deep Convolutional neural networks" back in 1394/2015.

If you find SimpleNet useful in your research, please consider citing:

@article{hasanpour2016lets,  title={Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures},  author={Hasanpour, Seyyed Hossein and Rouhani, Mohammad and Fayyaz, Mohsen and Sabokrou, Mohammad},  journal={arXiv preprint arXiv:1608.06037},  year={2016}}