hoya012/semantic-segmentation-tutorial-pytorchPublic

NotificationsYou must be signed in to change notification settings
Fork39
Star195

A simple PyTorch codebase for semantic segmentation using Cityscapes.

License

MIT license

195 stars 39 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 20 Commits
helpers		helpers
learning		learning
minicity		minicity
LICENSE		LICENSE
README.md		README.md
baseline.py		baseline.py
evaluate.py		evaluate.py
option.py		option.py
requirements.txt		requirements.txt
results.txt		results.txt

Repository files navigation

Semantic Segmentation Tutorial using PyTorch

Semantic Segmentation Tutorial using PyTorch. Based on2020 ECCV VIPriors Challange Start Code, implements semantic segmentation codebase and add some tricks.

Editer: Hoseong Lee (hoya012)

0. Experimental Setup

0-1. Prepare Library

pipinstall-rrequirements.txt

0-2. Download dataset (MiniCity from CityScapes)

We will use MiniCity Dataset from Cityscapes. This dataset is used for 2020 ECCV VIPriors Challenge.

workshop page:https://vipriors.github.io/challenges/
challenge link:https://competitions.codalab.org/competitions/23712
dataset download(google drive)
- move dataset intominicity folder.

0-3. Dataset Simple EDA (Exploratory Data Analysis) - Class Distribution, Sample Distribution

benchmark class

CityscapesClass('road',7,0,'flat',1,False,False, (128,64,128)),CityscapesClass('sidewalk',8,1,'flat',1,False,False, (244,35,232)),CityscapesClass('building',11,2,'construction',2,False,False, (70,70,70)),CityscapesClass('wall',12,3,'construction',2,False,False, (102,102,156)),CityscapesClass('fence',13,4,'construction',2,False,False, (190,153,153)),CityscapesClass('pole',17,5,'object',3,False,False, (153,153,153)),CityscapesClass('traffic light',19,6,'object',3,False,False, (250,170,30)),CityscapesClass('traffic sign',20,7,'object',3,False,False, (220,220,0)),CityscapesClass('vegetation',21,8,'nature',4,False,False, (107,142,35)),CityscapesClass('terrain',22,9,'nature',4,False,False, (152,251,152)),CityscapesClass('sky',23,10,'sky',5,False,False, (70,130,180)),CityscapesClass('person',24,11,'human',6,True,False, (220,20,60)),CityscapesClass('rider',25,12,'human',6,True,False, (255,0,0)),CityscapesClass('car',26,13,'vehicle',7,True,False, (0,0,142)),CityscapesClass('truck',27,14,'vehicle',7,True,False, (0,0,70)),CityscapesClass('bus',28,15,'vehicle',7,True,False, (0,60,100)),CityscapesClass('train',31,16,'vehicle',7,True,False, (0,80,100)),CityscapesClass('motorcycle',32,17,'vehicle',7,True,False, (0,0,230)),CityscapesClass('bicycle',33,18,'vehicle',7,True,False, (119,11,32)),

from 0 to 18 class, count labeled pixels

deeplab v3 baseline test set result

Dataset has severe Class-Imbalance problem.
- IoU of minor class is very low. (wall, fence, bus, train)

classesIoUnIoU--------------------------------road          :0.963nansidewalk      :0.762nanbuilding      :0.856nanwall          :0.120nanfence         :0.334nanpole          :0.488nantrafficlight :0.563nantrafficsign  :0.631nanvegetation    :0.884nanterrain       :0.538nansky           :0.901nanperson        :0.7320.529rider         :0.3740.296car           :0.8970.822truck         :0.4440.218bus           :0.2440.116train         :0.0330.006motorcycle    :0.4920.240bicycle       :0.6380.439--------------------------------ScoreAverage :0.5730.333--------------------------------

1. Training Baseline Model

I useDeepLabV3 from torchvision.
- ResNet-50 Backbone, ResNet-101 Backbone
I use 4 RTX 2080 Ti GPUs. (11GB x 4)
If you have just 1 GPU or small GPU Memory, please use smaller batch size (<= 8)

pythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50--crop_size5761152--batch_size8;

pythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet101--modelDeepLabv3_resnet101--train_size5121024--test_size5121024--crop_size384768--batch_size8;

1-1. Loss Functions

I tried 3 loss functions.
- Cross-Entropy Loss
- Class-Weighted Cross Entropy Loss
- Focal Loss
You can choose loss function using--loss argument.
- I recommend default (ce) or Class-Weighted CE loss. Focal loss didn'y work well in my codebase.

# Cross Entropy Losspythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50--crop_size5761152--batch_size8;

# Weighted Cross Entropy Losspythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_wce--crop_size5761152--batch_size8--lossweighted_ce;

# Focal Losspythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_focal--crop_size5761152--batch_size8--lossfocal--focal_gamma2.0;

1-2. Normalization Layer

I tried 4 normalization layer.
- Batch Normalization (BN)
- Instance Normalization (IN)
- Group Normalization (GN)
- Evolving Normalization (EvoNorm)
You can choose normalization layer using--norm argument.
- I recommend BN.

# Batch Normalizationpythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50--crop_size5761152--batch_size8;

# Instance Normalizationpythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_instancenorm--crop_size5761152--batch_size8--norminstance;

# Group Normalizationpythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_groupnorm--crop_size5761152--batch_size8--normgroup;

# Evolving Normalizationpythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_evonorm--crop_size5761152--batch_size8--normevo;

1-3. Additional Augmentation Tricks

Propose 2 data augmentation techniques (CutMix, copyblob)
CutMix Augmentation
- Based onOriginal CutMix, bring idea to Semantic Segmentation.
CopyBlob Augmentation
- To tackle Class-Imbalance, use CopyBlob augmentation with visual inductive prior.
  - Wall must be located on the sidewalk
  - Fence must be located on the sidewalk
  - Bus must be located on the Road
  - Train must be located on the Road

# CutMix Augmentationpythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_cutmix--crop_size5761152--batch_size8--cutmix;

# CopyBlob Augmentationpythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50_copyblob--crop_size5761152--batch_size8--copyblob;

2. Inference

After training, we can evaluate using trained models.
- I recommend same value fortrain_size andtest_size.

pythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50--batch_size4--predict;

2-1. Multi-Scale Infernece (Test Time Augmentation)

I use [0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0, 2.2] scales for Multi-Scale Inference. Additionaly, use H-Flip.
- Must use single batch (batch_size=1)

# Multi-Scale Inferencepythonbaseline.py--save_pathbaseline_run_deeplabv3_resnet50--batch_size1--predict--mst;

2-2. Calculate Metric using Validation Set

We can calculate metric and save results intoresults.txt.
- ex)My final validation set result

pythonevaluate.py--resultsbaseline_run_deeplabv3_resnet50/results_val--batch_size1--predict--mst;

3. Final Result

My final single model result is0.6069831962012341
- Achieve 5th place on the leaderboard.
- But, didn't submit short-paper, so my score is not official score.
If i use bigger model and bigger backbone, performance will be improved.. maybe..
If i use ensemble various models, performance will be improved!
Leader board can be found inCodalab Challenge Page

4. Reference

About

A simple PyTorch codebase for semantic segmentation using Cityscapes.

Releases

No releases published

Packages

No packages published

Movatterモバイル変換

License

hoya012/semantic-segmentation-tutorial-pytorch

Folders and files

Latest commit

History

Repository files navigation

Semantic Segmentation Tutorial using PyTorch

0. Experimental Setup

0-1. Prepare Library

0-2. Download dataset (MiniCity from CityScapes)

0-3. Dataset Simple EDA (Exploratory Data Analysis) - Class Distribution, Sample Distribution

benchmark class

from 0 to 18 class, count labeled pixels

deeplab v3 baseline test set result

1. Training Baseline Model

1-1. Loss Functions

1-2. Normalization Layer

1-3. Additional Augmentation Tricks

2. Inference

2-1. Multi-Scale Infernece (Test Time Augmentation)

2-2. Calculate Metric using Validation Set

3. Final Result

4. Reference

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages0

Uh oh!

Contributors2

Uh oh!

Languages

Packages