- Notifications
You must be signed in to change notification settings - Fork23
HowTo | TANGO Image Build and Run
If you have not installed the docker, please refer to following section.
The descriptions in this sections are based on follow test environments:
- Linux Ubuntu 20.04, 22.04 LTS
System Prerequisites
sudo apt-get updatesudo apt-get install ca-certificates curl gnupg lsb-releasecurl -fsSL https://download.docker.com/linux/ubuntu/gpg| sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpgecho"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu\$(lsb_release -cs) stable"| sudo tee /etc/apt/sources.list.d/docker.list> /dev/null
How to install docker engine
Refer to following reference for up-to-date information
sudo apt-get updatesudo apt-get install docker-ce docker-ce-cli containerd.io
Check the installeddocker version.
docker --version
How to install NVIDIA container toolkit
Refer to following reference for up-to-date information
TANGO can use GPU resources in some containers such as bms, autonn_nk, autonn_bb, etc.You would consider installing NVIDIA container toolkit.
- Make sure you have installed the NVIDIA driver and Docker 20.10 for your linux machine.
- refer to
- You do not need to install the CUDA toolkit on the host, butthe driver need to be installed.
- With the release of Docker 19.03, usage of nvidia-docker2 packages is deprecated since NVIDIA GPUs are now natively supported as devices in the Docker runtime.
$ distribution=$(. /etc/os-release;echo$ID$VERSION_ID)$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey| sudo apt-key add -$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list| sudo tee /etc/apt/sources.list.d/nvidia-docker.list$ sudo apt-get update&& sudo apt-get install -y nvidia-container-toolkit$ sudo systemctl restart docker
Once you have installed docker in your local host system, you can clone the GitHub TANGO repository into local host
If you have registered your public key with your github ID, you can use following command
git clone git@github.com:ML-TANGO/TANGO.git
Please refer toHow to Add SSH Keys to Your GitHub Account.
After cloning TANGO repository into your local host, change working directory into local TANGO repository.
cd TANGOBuild docker images and run the containers withdocker compose command.
docker compose up -d --build
Note
- run above command at directory where
compose.ymlordocker-compose.ymlfile is located.docker compose up -d --buildrequires a lot of times and sometimes it seems do nothing but it does something.Be patient!!
If you're in low bandwidth Internet environment or using problematic DNS server, from time to timedocker compose up -d --build command would be interrupted by following errors(Temporary failure in name resolution):
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to do request: Head "https://registry-1.docker.io/al tcp: lookup registry-1.docker.io: Temporary failure in name resolutionIf this is your case, you should repeatedly rundocker compose up -d --build before to get the following message(FINISHED):
[+] Building 1430.5s (114/114) FINISHED......Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them[+] Running 9/10...Once previous step completes successfully, following docker container images and containers can be found in your local host system.
Example list of Docker images
$ docker image ls| grep tangotango-code_gen latest 34cf7e11edcb 4 minutes ago 16.2GBtango-viz2code latest 7f6a3f83c99d 10 minutes ago 8.92GBtango-bms latest 72fb1b5e3fa0 12 minutes ago 7.27GBtango-autonn_yoloe latest 0264b9335389 13 minutes ago 16.6GBtango-project_manager latest 03a04bfb2bcb 20 minutes ago 1.95GBtango-kube_deploy latest e19d929c98de 20 minutes ago 1.6GBtango-cloud_deploy latest 1ea4c02c1f25 21 minutes ago 1.34GBtango-ondevice_deploy latest 9ecadad31d0f 21 minutes ago 978MBtango-labelling latest 32fa6ce9af2a 4 days ago 11.1GB- Note that the name of the docker images generated based on
compose.ymlordocker-compose.ymlis prefixed by its folder name (e.g,'tango-')
Example list of Docker containers
$ docker ps -a --format"table {{.Image}}\t{{.Names}}\t{{.Status}}\t{{.Command}}\t{{.Ports}}"IMAGE NAMES STATUS COMMAND PORTStango-project_manager tango-project_manager-1 Up 2 seconds"sh -c 'chmod 777 ./…" 0.0.0.0:8085->8085/tcp, :::8085->8085/tcptango-autonn_yoloe tango-autonn_yoloe-1 Up 2 seconds"/opt/nvidia/nvidia_…" 0.0.0.0:8090->8090/tcp, :::8090->8090/tcptango-labelling tango-labelling-1 Up 3 seconds"./start.sh" 0.0.0.0:8086->80/tcp, :::8086->80/tcp, 0.0.0.0:8095->10236/tcp, :::8095->10236/tcptango-code_gen tango-code_gen-1 Up 1 second"sh -c 'cd /app && p…" 0.0.0.0:8888->8888/tcp, :::8888->8888/tcptango-viz2code tango-viz2code-1 Exited (1) 2 seconds ago"sh -c 'cd ./visuali…"tango-bms tango-bms-1 Up 3 seconds"sh -c 'python manag…" 0.0.0.0:8081->8081/tcp, :::8081->8081/tcppostgres:latest tango-postgresql-1 Restarting (1) Less than a second ago"docker-entrypoint.s…"tango-ondevice_deploy tango-ondevice_deploy-1 Up 3 seconds"sh -c 'cd /app && p…" 0.0.0.0:8891->8891/tcp, :::8891->8891/tcptango-kube_deploy tango_k8s Exited (1) 2 seconds ago"bash -c 'cd /app &&…"registry registry Up 2 seconds"/entrypoint.sh /etc…" 0.0.0.0:8903->5000/tcp, :::8903->5000/tcptango-cloud_deploy tango-cloud_deploy-1 Up 3 seconds"/init" 0.0.0.0:7007->7007/tcp, :::7007->7007/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:8890->8890/tcp, :::8890->8890/tcp
- Note that the name of the docker containers genrated based on
compose.ymlordocker-compose.ymlis prefixed by its folder name (e.g,'tango-') and suffixed by the its instance ID (e.g,'-1').
TANGO in Web-browser
Now you can launch web browser and open URLhttp://localhost:8085 orhttp://aaa.bbb.ccc.ddd:8085.
aaa.bbb.ccc.dddis your host's DNS address or IP address.8085is published port fromTANGO_web_1container, which acts as front-end server of TANGO.
Then you can see the login page of TANGO as follows:
Once you can find the login page in the web browser, register new account and password and use the newly created account and password to login.
When you want remove all the images and containers previously built and run, you can use following commands;
# tear down all containers and remove all docker images created and volumes.$ docker compose down --rmi all --volumes#or tear down all containers and remove all docker images created except for volumes.$ docker compose down --rmi all# remove all images in the local docker host for preventing cached image layers side effect# when you are to build from the zero base.docker system prune -a# remove labelling dataset related folder if you want to start from the empty datasets$ sudo rm -rf ./labelling/data/$ sudo rm -rf ./labelling/datadb/$ sudo rm -rf ./labelling/dataset/
Note
- Run above command at project root directory (e.g
'TANGO') wherecompose.ymlordocker-compose.ymlfile exists.- After running of above commands, your account on project manager as well as datasets prepared with
labellingtool are removed, due to--volumesoption.- Hence, you recreate account for project manager and dataset from the scratch.
Currently we have following component containers;
- labelling : dataset labelling authoring tool:
- autonn: automatic neural network creation:
- target_image_build: target deployment image build
- target_deploy: image deployment to target:
For testing or debugging of the individual component container, you want to run container individually.
First of all, check your current working branch ismain.
$ git branch -a* main remotes/origin/HEAD -> origin/main remotes/origin/main remotes/origin/sub
labelling: container for labelling tool
Change current working directory intolabelling and image build withDockerfile
cd labellingdocker build -t labelling.
labelling container run
docker run -d --name labelling -p 8086:80 labelling:latest
autonn: container for automatic neural network creationn
You should build both of them at each directory respectively.
Forbackbone NAS, change current working directory intoautonn/backbone-nas and image build withDockerfile
cd ../autonn/backbone_nasdocker build -t autonn_bb.
autonn_bb container run
Be careful not to get the port number wrong,8087 is allocated for backbone-nas
docker run -d --name autonn_bb -p 8087:8087 -v autonn_bb:latest
If CUDA is available, you can use--gpus=all options
docker run -d --gpus=all --name autonn_bb -p 8087:8087 autonn_bb:latest
When you run into shared memory shortage, you should use--ipc=host options
docker run -d --gpus=all --ipc=host --name autonn_bb -p 8087:8087 autonn_bb:latest
Similary forneck NAS, change current working directory intoautonn/neck-nas and image build withDockerfile
cd ../autonn/neck_nasdocker build -t autonn_nk.
autonn_nk container run
Be careful not to get the port number wrong,8089 is allocated for neck-nas
docker run -d --name autonn_nk -p 8089:8089 autonn_nk:latest
If CUDA is available, you can use--gpus=all options
docker run -d --gpus=all --name autonn_nk -p 8089:8089 autonn_nk:latest
When you run into shared memory shortage, you should use--ipc=host options
docker run -d --gpus=all --ipc=host --name autonn_nk -p 8089:8089 autonn_nk:latest
target_image_build: container for target deployment image build
Change current working directory intotarget_image_build and image build withDockerfile
cd ../target_image_build/docker build -t target_image_build.
target_image_build container run
docker run -d --name target_image_build -p 8088:8088 target_image_build:latest
target_deploy: container for image deployment to target
Change current working directory intotarget_deploy and image build withDockerfile
cd ../target_deploy/docker build -t target_deploy.
target_deploy container run
docker run -d --name target_deploy -p 8089:8089 target_deploy:latest
👉Note
- This page might contain some wide rendered images. If you want to see the whole contents with wider page, please useWide GitHub extension of Chrome.
- If you are experiencing the error on rendered images due to the low-bandwith Internet delay, please use F5 function key for refreshing current page.