Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

"Learning Delicate Local Representations for Multi-Person Pose Estimation" (ECCV 2020 Spotlight) & (COCO 2019 Human Keypoint Detection Challenge Winner) & (COCO 2019 Best Paper Award)

License

NotificationsYou must be signed in to change notification settings

caiyuanhao1998/RSN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

92 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PWC

PWCPWC

PWCPWC

Learning Delicate Local Representations for Multi-Person Pose Estimation (ECCV 2020 Spotlight)

winnerbpaarXivzhihu

*This is a pytorch realization ofResidual Steps Networkwhich won 2019 COCO Keypoint Challenge and ranks 1st place on both COCO test-dev and test-challenge datasets as shown inCOCO leaderboard.

News

  • 2020.09 : Our RSN has been integrated into the greatMMPose framework. Thanks to their effort. Welcome to use theircodebase with theirpre-trained model zoo. ⭐
  • 2020.07 : Our paper has been accepted as Spotlight by ECCV 2020 🚀
  • 2019.09 : Our work won theFirst place andBest Paper Award in COCO 2019 Keypoint Challenge 🏆

Abstract:In this paper, we propose a novel method called Residual Steps Network (RSN). RSN aggregates features with the same spatialsize (Intra-level features) efficiently to obtain delicate local representations, which retain rich low-level spatial information and result in precise keypoint localization. In addition, we propose an efficient attention mechanism - Pose Refine Machine (PRM) to further refine the keypoint locations. Our approach won the 1st place of COCO Keypoint Challenge 2019 and achieves state-of-the-art results on both COCO and MPII benchmarks, without using extra training data and pretrained model. Our single model achieves 78.6 on COCO test-dev, 93.0 on MPII test dataset. Ensembled models achieve 79.2 on COCO test-dev, 77.1 on COCO test-challenge dataset.


Pipieline of Residual Steps Network

Overview of RSN.

Architecture of Pose Refine Machine

Overview of RSN.

Some prediction resullts of our method on COCO and MPII valid datasets

Prediction Results of COCO-valid.

Prediction Results of MPII-valid.

Results(Original Version)

Results on COCO val dataset

ModelInput SizeGFLOPsAPAP50AP75APMAPLAR
Res-18256x1922.370.789.577.566.875.975.8
RSN-18256x1922.573.690.580.967.879.178.8
RSN-50256x1926.474.791.481.571.080.280.0
RSN-101256x19211.575.892.483.072.181.281.1
2×RSN-50256x19213.977.292.384.073.882.582.2
3×RSN-50256x19220.778.292.385.174.783.783.1
4×RSN-50256x19229.379.092.585.775.284.583.7
4×RSN-50384x28865.979.692.585.875.585.284.2

Results on COCO test-dev dataset

ModelInput SizeGFLOPsAPAP50AP75APMAPLAR
RSN-18256x1922.571.692.680.368.875.877.7
RSN-50256x1926.472.593.081.369.976.578.8
2×RSN-50256x19213.975.593.684.073.079.681.3
4×RSN-50256x19229.378.094.286.575.382.283.4
4×RSN-50384x28865.978.694.386.675.583.383.8
4×RSN-50+--79.294.487.176.183.884.1

Results on COCO test-challenge dataset

ModelInput SizeGFLOPsAPAP50AP75APMAPLAR
4×RSN-50+--77.193.383.672.283.682.6

Results on MPII dataset

ModelSplitInput SizeHeadShoulderElbowWristHipKneeAnkleMean
4×RSN-50val256x25696.796.792.388.290.389.085.391.6
4×RSN-50test256x25698.597.393.989.992.090.686.893.0

Results(Pytorch Version)

Results on COCO val dataset

ModelInput SizeGFLOPsAPAP50AP75APMAPLAR
Res-18256x1922.365.287.371.561.272.271.3
RSN-18256x1922.570.488.877.767.276.776.5

Note

  • + means using ensemble models.
  • All models are trained on 8 V100 GPUs
  • We done all the experiments using our orginal DL-Platform, all results in our paper are reported on this DL-Platform. There are some differences between it and Pytorch.

Repo Structure

This repo is organized as following:

$RSN_HOME|-- cvpack||-- dataset|   |-- COCO|   |   |-- det_json|   |   |-- gt_json|   |   |-- images|   |       |-- train2014|   |       |-- val2014|   ||   |-- MPII|       |-- det_json|       |-- gt_json|       |-- images|   |-- lib|   |-- models|   |-- utils||-- exps|   |-- exp1|   |-- exp2|   |-- ...||-- model_logs||-- README.md|-- requirements.txt

Quick Start

Installation

  1. Install Pytorch referring toPytorch website.

  2. Clone this repo, and configRSN_HOME in/etc/profile or~/.bashrc, e.g.

export RSN_HOME='/path/of/your/cloned/repo'export PYTHONPATH=$PYTHONPATH:$RSN_HOME
  1. Install requirements:
pip3 install -r requirements.txt
  1. Install COCOAPI referring tococoapi website, or:
git clone https://github.com/cocodataset/cocoapi.git $RSN_HOME/lib/COCOAPIcd $RSN_HOME/lib/COCOAPI/PythonAPImake install

Dataset

COCO

  1. Download images fromCOCO website, and put train2014/val2014 splits into$RSN_HOME/dataset/COCO/images/ respectively.

  2. Download ground truth fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/COCO/gt_json/.

  3. Download detection result fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/COCO/det_json/.

MPII

  1. Download images fromMPII website, and put images into$RSN_HOME/dataset/MPII/images/.

  2. Download ground truth fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/MPII/gt_json/.

  3. Download detection result fromGoogle Drive orBaidu Drive (code: fc51), and put it into$RSN_HOME/dataset/MPII/det_json/.

Log

Create a directory to save logs and models:

mkdir $RSN_HOME/model_logs

Train

Go to specified experiment repository, e.g.

cd $RSN_HOME/exps/RSN50.coco

and run:

python config.py -logpython -m torch.distributed.launch --nproc_per_node=gpu_num train.py

thegpu_num is the number of gpus.

Test

python -m torch.distributed.launch --nproc_per_node=gpu_num test.py -i iter_num

thegpu_num is the number of gpus, anditer_num is the iteration number you want to test.

Citation

Please considering citing our projects in your publications if they help your research.

@inproceedings{cai2020learning,  title={Learning Delicate Local Representations for Multi-Person Pose Estimation},  author={Yuanhao Cai and Zhicheng Wang and Zhengxiong Luo and Binyi Yin and Angang Du and Haoqian Wang and Xinyu Zhou and Erjin Zhou and Xiangyu Zhang and Jian Sun},  booktitle={ECCV},  year={2020}}@inproceedings{cai2019res,  title={Res-steps-net for multi-person pose estimation},  author={Cai, Yuanhao and Wang, Zhicheng and Yin, Binyi and Yin, Ruihao and Du, Angang and Luo, Zhengxiong and Li, Zeming and Zhou, Xinyu and Yu, Gang and Zhou, Erjin and others},  booktitle={Joint COCO and Mapillary Workshop at ICCV},  year={2019}}

About

"Learning Delicate Local Representations for Multi-Person Pose Estimation" (ECCV 2020 Spotlight) & (COCO 2019 Human Keypoint Detection Challenge Winner) & (COCO 2019 Best Paper Award)

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages


[8]ページ先頭

©2009-2025 Movatter.jp