Repository files navigation

Python scripts performing on devive semantic segmentation (ADE20K 150 classes)using the TopFormer model in depthai.

Taken from:https://youtu.be/fzZEylhZTbI

• Check therequirements.txt file.
• Additionally,pafy andyoutube-dl are required for youtube video inference.

git clone https://github.com/ibaiGorordo/depthai-TopFormer-Semantic-Segmentation.gitcd depthai-TopFormer-Semantic-Segmentationpip install -r requirements.txt

pip install youtube_dlpip install git+https://github.com/zizo-pro/pafy@b8976f22c19e4ab5515cacbfae0a3970370c102b

The original Pytorch models were converted to MyriadX blob using the Google Colab notebook below. You can download the converted models fromGoogle Drive and save them into themodes folder.

• Conversion notebook: TopFormer_to_MyriadX_Conversion.ipynb
• The License of the models is Apache-2.0 License:https://github.com/hustvl/TopFormer/blob/main/LICENSE

The model can also run in ONNX using the scripts inthis repository.

The original Pytorch model can be found in this repository:https://github.com/hustvl/TopFormer

• Depthai Semantic Segmentation using the camera in the board (~20 fps for small model):

python main.py

• Video inference:https://youtu.be/x3UVNlPlFlc

python main_video.py

Original video:https://youtu.be/fzZEylhZTbI

• TopFormer model:https://github.com/hustvl/TopFormer
• ONNX TopForfmer:https://github.com/ibaiGorordo/ONNX-TopFormer-Semantic-Segmentation
• Luxonis Depthai:https://docs.luxonis.com/en/latest/
• PINTO0309's model zoo:https://github.com/PINTO0309/PINTO_model_zoo
• Original paper:https://arxiv.org/abs/2204.05525