Repository files navigation

*This is a pytorch realization ofResidual Steps Networkwhich won 2019 COCO Keypoint Challenge and ranks 1st place on both COCO test-dev and test-challenge datasets as shown inCOCO leaderboard.

• 2020.09 : Our RSN has been integrated into the greatMMPose framework. Thanks to their effort. Welcome to use theircodebase with theirpre-trained model zoo. ⭐
• 2020.07 : Our paper has been accepted as Spotlight by ECCV 2020 🚀
• 2019.09 : Our work won theFirst place andBest Paper Award in COCO 2019 Keypoint Challenge 🏆

Abstract:In this paper, we propose a novel method called Residual Steps Network (RSN). RSN aggregates features with the same spatialsize (Intra-level features) efficiently to obtain delicate local representations, which retain rich low-level spatial information and result in precise keypoint localization. In addition, we propose an efficient attention mechanism - Pose Refine Machine (PRM) to further refine the keypoint locations. Our approach won the 1st place of COCO Keypoint Challenge 2019 and achieves state-of-the-art results on both COCO and MPII benchmarks, without using extra training data and pretrained model. Our single model achieves 78.6 on COCO test-dev, 93.0 on MPII test dataset. Ensembled models achieve 79.2 on COCO test-dev, 77.1 on COCO test-challenge dataset.

• + means using ensemble models.
• All models are trained on 8 V100 GPUs
• We done all the experiments using our orginal DL-Platform, all results in our paper are reported on this DL-Platform. There are some differences between it and Pytorch.

This repo is organized as following:

$RSN_HOME|-- cvpack||-- dataset|   |-- COCO|   |   |-- det_json|   |   |-- gt_json|   |   |-- images|   |       |-- train2014|   |       |-- val2014|   ||   |-- MPII|       |-- det_json|       |-- gt_json|       |-- images|   |-- lib|   |-- models|   |-- utils||-- exps|   |-- exp1|   |-- exp2|   |-- ...||-- model_logs||-- README.md|-- requirements.txt

1. Install Pytorch referring toPytorch website.

2. Clone this repo, and configRSN_HOME in/etc/profile or~/.bashrc, e.g.

export RSN_HOME='/path/of/your/cloned/repo'export PYTHONPATH=$PYTHONPATH:$RSN_HOME

3. Install requirements:

pip3 install -r requirements.txt

4. Install COCOAPI referring tococoapi website, or:

git clone https://github.com/cocodataset/cocoapi.git $RSN_HOME/lib/COCOAPIcd $RSN_HOME/lib/COCOAPI/PythonAPImake install

1. Download images fromCOCO website, and put train2014/val2014 splits into$RSN_HOME/dataset/COCO/images/ respectively.

2. Download ground truth fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/COCO/gt_json/.

3. Download detection result fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/COCO/det_json/.

1. Download images fromMPII website, and put images into$RSN_HOME/dataset/MPII/images/.

2. Download ground truth fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/MPII/gt_json/.

3. Download detection result fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/MPII/det_json/.

Create a directory to save logs and models:

mkdir $RSN_HOME/model_logs

Go to specified experiment repository, e.g.

cd $RSN_HOME/exps/RSN50.coco

and run:

python config.py -logpython -m torch.distributed.launch --nproc_per_node=gpu_num train.py

thegpu_num is the number of gpus.

python -m torch.distributed.launch --nproc_per_node=gpu_num test.py -i iter_num

thegpu_num is the number of gpus, anditer_num is the iteration number you want to test.

Please considering citing our projects in your publications if they help your research.

@inproceedings{cai2020learning,  title={Learning Delicate Local Representations for Multi-Person Pose Estimation},  author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},  booktitle={ECCV},  year={2020}}@inproceedings{cai2019res,  title={Res-steps-net for multi-person pose estimation},  author={Cai, Yuanhao and Wang, Zhicheng and Yin, Binyi and Yin, Ruihao and Du, Angang and Luo, Zhengxiong and Li, Zeming and Zhou, Xinyu and Yu, Gang and Zhou, Erjin and others},  booktitle={Joint COCO and Mapillary Workshop at ICCV},  year={2019}}