Repository files navigation

Official code for the paper "Pose2Seg: Detection Free Human Instance Segmentation"[ProjectPage][arXiv] @ CVPR2019.

TheOCHuman dataset proposed in our paper is releasedhere

Pipeline of our pose-based instance segmentation framework.

pip install cython matplotlib tqdm opencv-python scipy pyyaml numpypip install torchvision torchcd~/github-public/cocoapi/PythonAPI/python setup.py build_ext installcd -

• COCO 2017

  • COCO 2017 Train images [118K/18GB]
  • COCO 2017 Val images [5K/1GB]
  • COCOPersons Train Annotation (person_keypoints_train2017_pose2seg.json) [166MB]
  • COCOPersons Val Annotation (person_keypoints_val2017_pose2seg.json) [7MB]

• OCHuman

  • images [667MB] & annotations

Note:person_keypoints_(train/val)2017_pose2seg.json is a subset ofperson_keypoints_(train/val)2017.json (inCOCO2017 Train/Val annotations). We choose those instances with both keypoint and segmentation annotations for our experiments.

Thedata folder should be like this:

data  ├── coco2017│   ├── annotations  │   │   ├── person_keypoints_train2017_pose2seg.json │   │   ├── person_keypoints_val2017_pose2seg.json │   ├── train2017  │   │   ├── ####.jpg  │   ├── val2017  │   │   ├── ####.jpg  ├── OCHuman │   ├── annotations  │   │   ├── ochuman_coco_format_test_range_0.00_1.00.json   │   │   ├── ochuman_coco_format_val_range_0.00_1.00.json   │   ├── images  │   │   ├── ####.jpg

python train.py

Note: Currently we only support for single-gpu training.

This allows you to test the model on (1) COCOPersons val set and (2) OCHuman val & test set.

python test.py --weights last.pkl --coco --OCHuman

We retrained our model using this repo, and got similar results with our paper. The final weights can be downloadhere.

Pose templates clustered using K-means on COCO.

This repo already contains a template filemodeling/templates.json which was used in our paper. But you are free to explore different cluster parameters as discussed in our paper. Seevisualize_cluster.ipynb for an example.