Repository files navigation

https://deep-learning-geeks-slack.herokuapp.com/

This is a project to demo how to useCaffe2/OpenCV 2 to build an iOS application doing real time object classification.

• iOS (Swift/Objective-C/C++) with Caffe2

• Test build in models (tinyYolo, SqueezeNet) with your own photo

• Memory Consumption and Time Elapse Data

• Live (Real Time) detection

• Download your own model on the fly! And test it!

• Manage models locally on your iPhone

• Overall control on every layer (from beginger to expert)

• Warm community and welcome to contribute

• Star us if you like

If you are too lazy to build this repo, I also put this in App Store:

Check it out

• If it navigates you to a different country's app store, you just searchDeep Learning Pro.

Check ourWiki

Feel free to ask any questions from prepare environment to debug on Xcode and we are happy to help you.

We provide two stable versions in here with:

• Exper stable version for experts to explore more possibilities

Static Classifier	Real Time Classifier	Model Downloader

• Lite stable version for beginners to experience how the wrapper work and play with the demo

Static Classifier	Real Time Classifier

• Note that the number of FPS is subjective to the size you send to the device as well as type of the device. Those numbers were obtained with Height*Width = 227 * 227 on an iPhone 7 Plus.

You have to use a Mac with Xcode >= 8.0 (macOS Sierra) to keep going

iOS 10

Caffe2

OpenCV 2

brew install git-lfsgit lfs installgit lfs clone https://github.com/KleinYuan/Caffe2-iOS

• Notice, after thiscommit, I put all large files in git LFS and make sure when you clone uselfs clone

• Clone this repo into a folder, let's say~/Desktop/, then you will have~/Desktop/Caffe2-iOS when clone is done

• Navigate to~/Desktop/Caffe2-iOS/src folder and runbash ./setup.sh, which will automatically download and build iOS Caffe2 in a paralleled folder besides~/Desktop/Caffe2-iOS/src/caffe2-ios calledcaffe2 (it's important to make sure this step is done and it may take around 20-30 min to finish)

• When previous step is done, open~/DesktopCaffe2-iOS/src/caffe2-ios with Xcode (>8.0)

• DisableBitcode likethis

• (by default should be done) Adding$(inherited) -force_load caffe2-ios/libCaffe2_CPU.a toBuild Settings/Linking/Other Linker Flags . For thisissue

• Build with your iPhone plugged in

• Open the app and pressRun to check the result of a pre-loaded image (cute Panda!) and presslive to go to live mode

There are some potential issues that you will have (I will keep adding if I sense some in issues):

1-a.Error Message 1: When build project in Xcode you see this errorCannot find caffe2/proto/caffe2.pb.h

1-b. Error Message 2: When running setup.sh you see this in terminal${YOUR_PATH}/Caffe2-iOS/src/caffe2/third_party/protobuf/cmake: is a directory

2. Description: Those two are related and all because that you failed to build the caffe2 ios and check this folderarchitecture to validate your build (you should be able to see thecaffe2.pb.h)

3. Debug and how to fix it: Mostly, the root cause is that yourcmake is broken (not broken broken, more like configuration/path changed by other services/software) and you probably wanna runbrew install cmake to reinstall it

1. Error Message:Reading dangerously large protocol message. If the message turns out to be larger than 67108864 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.

2. Description: As you can see in the caffe2repo, that they reduced theprotobuf, which is the tool they use to hanlde the communication down to version 3.1.0 and only have64MB limit. Therefore, when you load a model larger than that, boooooomb, memory exploed and thread got killed.

3. Debug and how to fix it:

   • After youdownload and build the caffe2, hold on and modify something to increase the limit first

   • Find thisfile, which is the tool they use to hanlde the communication down to version 3.1.0 and only have64MB and change the limit to whatever you want (also change the warning limit)

   • Then build caffe2-ios and Tada

   • Alternative method seehere

The initial slope is for a static 4KB image, around 50 MB and Note that memory usage in live mode might not be the same as the one shown in Xcode (slightly different). And also, remember the memory data in the app is aggregated and therefore, if you are really interested in checking performance of a specific process, open Xcode :)

Checkhere

We have a clear scope for this repo below:

Scope

Caffe2-on-iOS-install

Caffe2-repo

License