- Notifications
You must be signed in to change notification settings - Fork748
Releases: ufoym/deepo
Deepo v2.0.0
Compare
Deepo2 is now a series ofDocker images that
- allows you to quickly set up your deep learning research environment
- supports almost all commonly used deep learning frameworks:theano,tensorflow,sonnet,pytorch,keras,lasagne,mxnet,cntk,chainer,caffe,torch
and their Dockerfile generator that
- allows you to customize your own environment with Lego-like modules
- automatically resolves the dependencies for you
Table of contents
Quick Start
Installation
Step 1. InstallDocker andnvidia-docker.
Step 2. Obtain the all-in-one image fromDocker Hub
docker pull ufoym/deepo
Usage
Now you can try this command:
nvidia-docker run --rm ufoym/deepo nvidia-smi
This should work and enables Deepo to use the GPU from inside a docker container.
If this does not work, searchthe issues section on the nvidia-docker GitHub -- many solutions are already documented. To get an interactive shell to a container that will not be automatically deleted after you exit do
nvidia-docker run -it ufoym/deepo bash
If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.
nvidia-docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo bash
This will make/host/data
from the host visible as/data
in the container, and/host/config
as/config
. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.
Please note that some frameworks (e.g. PyTorch) use shared memory to share data between processes, so if multiprocessing is used the default shared memory segment size that container runs with is not enough, and you should increase shared memory size either with--ipc=host
or--shm-size
command line options tonvidia-docker run
.
nvidia-docker run -it --ipc=host ufoym/deepo bash
You are now ready to begin your journey.
$ python
>>>importtensorflow>>>importsonnet>>>importtorch>>>importkeras>>>importmxnet>>>importcntk>>>importchainer>>>importtheano>>>importlasagne>>>importcaffe
$ caffe --version
caffe version 1.0.0
$ th
│ ______ __ | Torch7 │ /_ __/__ ________/ / | Scientific computing for Lua. │ / / / _ \/ __/ __/ _ \ | Type ? for help │ /_/ \___/_/ \__/_//_/ | https://github.com/torch │ | http://torch.ch │ │th>
Customization
Note thatdocker pull ufoym/deepo
mentioned inQuick Start will give you a standard image containing all available deep learning frameworks. You can customize your own environment as well.
I hate all-in-one solution
If you prefer a specific framework rather than an all-in-one image, just append a tag with the name of the framework.
Take tensorflow for example:
docker pull ufoym/deepo:tensorflow
Other python versions
Note that all python-related images usePython 3.6
by default. If you are unhappy withPython 3.6
, you can also specify other python versions:
docker pull ufoym/deepo:py27
docker pull ufoym/deepo:tensorflow-py27
Currently, we supportPython 2.7
andPython 3.6
.
Seehttps://hub.docker.com/r/ufoym/deepo/tags/ for a complete list of all available tags. These pre-built images are all built fromdocker/Dockerfile.*
andcircle.yml
. SeeHow to generatedocker/Dockerfile.*
andcircle.yml
if you are interested in how these files are generated.
Build your own customized image with Lego-like modules
Step 1. prepare generator
git clone https://github.com/ufoym/deepo.gitcd deepo/generatorpip install -r requirements.txt
Step 2. generate your customized Dockerfile
For example, if you likepytorch
andlasagne
, then
python generate.py Dockerfile pytorch lasagne
This should generate a Dockerfile that contains everything for buildingpytorch
andlasagne
. Note that the generator can handle automatic dependency processing and topologically sort the lists. So you don't need to worry about missing dependencies and the list order.
You can also specify the version of Python:
python generate.py Dockerfile pytorch lasagne python==3.6
Step 3. build your Dockerfile
docker build -t my/deepo.
This may take several minutes as it compiles a few libraries from scratch.
Comparison to alternatives
. | modern-deep-learning | dl-docker | jupyter-deeplearning | Deepo |
---|---|---|---|---|
ubuntu | 16.04 | 14.04 | 14.04 | 16.04 |
cuda | ❌ | 8.0 | 6.5-8.0 | 8.0 |
cudnn | ❌ | v5 | v2-5 | v6 |
theano | ❌ | ✔️ | ✔️ | ✔️ |
tensorflow | ✔️ | ✔️ | ✔️ | ✔️ |
sonnet | ❌ | ❌ | ❌ | ✔️ |
pytorch | ❌ | ❌ | ❌ | ✔️ |
keras | ✔️ | ✔️ | ✔️ | ✔️ |
lasagne | ❌ | ✔️ | ✔️ | ✔️ |
mxnet | ❌ | ❌ | ❌ | ✔️ |
cntk | ❌ | ❌ | ❌ | ✔️ |
chainer | ❌ | ❌ | ❌ | ✔️ |
caffe | ✔️ | ✔️ | ✔️ | ✔️ |
torch | ❌ | ✔️ | ✔️ | ✔️ |
Assets2
Deepo v1.0.0
Compare
Deepo is aDocker image with a full reproducible deep learning research environment. It contains most popular deep learning frameworks:theano,tensorflow,sonnet,pytorch,keras,
lasagne,mxnet,cntk,chainer,caffe,torch.
Quick Start
Installation
Step 1. InstallDocker andnvidia-docker.
Step 2. Obtain the Deepo image
You can either directly download the image from Docker Hub, or build the image yourself.
Option 1: Get the image from Docker Hub (recommended)
docker pull ufoym/deepo
Option 2: Build the Docker image locally
git clone https://github.com/ufoym/deepo.gitcd deepo&& docker build -t ufoym/deepo.
Note that this may take several hours as it compiles a few libraries from scratch.
Usage
Now you can try this command:
nvidia-docker run --rm ufoym/deepo nvidia-smi
This should work and enables Deepo to use the GPU from inside a docker container.
If this does not work, searchthe issues section on the nvidia-docker GitHub -- many solutions are already documented. To get an interactive shell to a container that will not be automatically deleted after you exit do
nvidia-docker run -it ufoym/deepo bash
If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.
nvidia-docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo bash
This will make/host/data
from the host visible as/data
in the container, and/host/config
as/config
. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.
You are now ready to begin your journey.
tensorflow
$ python
>>>importtensorflow>>>print(tensorflow.__name__,tensorflow.__version__)tensorflow1.3.0
sonnet
$ python
>>>importsonnet>>>print(sonnet.__name__,sonnet.__path__)sonnet ['/usr/local/lib/python3.5/dist-packages/sonnet']
pytorch
$ python
>>>importtorch>>>print(torch.__name__,torch.__version__)torch0.2.0_3
keras
$ python
>>>importkeras>>>print(keras.__name__,keras.__version__)keras2.0.8
mxnet
$ python
>>>importmxnet>>>print(mxnet.__name__,mxnet.__version__)mxnet0.11.0
cntk
$ python
>>>importcntk>>>print(cntk.__name__,cntk.__version__)cntk2.2
chainer
$ python
>>>importchainer>>>print(chainer.__name__,chainer.__version__)chainer3.0.0
theano
$ python
>>>importtheano>>>print(theano.__name__,theano.__version__)theano0.10.0beta4+14.gb6e3768
lasagne
$ python
>>>importlasagne>>>print(lasagne.__name__,lasagne.__version__)lasagne0.2.dev1
caffe
$ python
>>>importcaffe>>>print(caffe.__name__,caffe.__version__)caffe1.0.0
$ caffe --version
caffe version 1.0.0
torch
$ th
│ ______ __ | Torch7 │ /_ __/__ ________/ / | Scientific computing for Lua. │ / / / _ \/ __/ __/ _ \ | Type ? for help │ /_/ \___/_/ \__/_//_/ | https://github.com/torch │ | http://torch.ch │ │th>
Comparison to alternatives
. | modern-deep-learning | dl-docker | jupyter-deeplearning | Deepo |
---|---|---|---|---|
ubuntu | 16.04 | 14.04 | 14.04 | 16.04 |
cuda | ❌ | 8.0 | 6.5-8.0 | 8.0 |
cudnn | ❌ | v5 | v2-5 | v6 |
theano | ❌ | ✔️ | ✔️ | ✔️ |
tensorflow | ✔️ | ✔️ | ✔️ | ✔️ |
sonnet | ❌ | ❌ | ❌ | ✔️ |
pytorch | ❌ | ❌ | ❌ | ✔️ |
keras | ✔️ | ✔️ | ✔️ | ✔️ |
lasagne | ❌ | ✔️ | ✔️ | ✔️ |
mxnet | ❌ | ❌ | ❌ | ✔️ |
cntk | ❌ | ❌ | ❌ | ✔️ |
chainer | ❌ | ❌ | ❌ | ✔️ |
caffe | ✔️ | ✔️ | ✔️ | ✔️ |
torch | ❌ | ✔️ | ✔️ | ✔️ |