Repository files navigation

Official Repository for the ICCV 2023 paperEMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild.

To receive access to the data, please fill out theapplication form. You will receive an e-mail with more information after your application has been approved.

For an overview of how EMDB is structured, please refer to thedataset overview.

We useaitviewer to visualize the data. The code was tested with Python 3.8 on Windows 10.

conda create -n emdb python=3.8pip install aitviewer tabulate

This does not automatically install a GPU-version of PyTorch. If your environment already contains it, you should be good to go, otherwise you may wish to install it manually, e.g. on Windows

pipinstalltorchtorchvisiontorchaudio--index-urlhttps://download.pytorch.org/whl/cu117

Please also download the SMPL model by following the instructions on theSMPL-X GitHub page.

1. Change theSMPLX_MODELS variable inconfiguration.py to where you downloaded the SMPL model. This folder should contain a subfoldersmpl with the respective model files.
2. Change theEMDB_ROOT variable inconfiguration.py to where you extracted the EMDB dataset. This folder should contain subfoldersP0,P1, etc.

Run the following command to visualize a sequence.SUBJECT_ID refers to the ID Of the participant, i.e.P0-P9 andSEQUENCE_ID is the 2-digit identifier that is prepended to each sequence's name:

pythonvisualize.py--subject {SUBJECT_ID}--sequence {SEQUENCE_ID}

By default, this opens the viewer in the 3D view. You can choose to show the reprojected poses instead by specifying--view_from_camera. If you specify--draw_2d the 2D keypoints and bounding boxes will be drawn on top of the image. If you pass--draw_trajectories the SMPL root and camera trajectories will be drawn in addition.

We provide a script to visualizeGLAMR results. An example result that reproduces Figure 9 of the main paper is provided inassets/GLAMR. To visualize it, run the following command:

pythonvisualize_GLAMR.py

We provide code to load and evaluate HybrIK results on the EMDB test set, which in the paper is referred to asEMDB 1. Based on this evaluation code, it should be straight-forward to extend the evaluation to other methods (see below).

To run the evaluation, use the following command:

pythonevaluate.py {RESULT_ROOT}

TheRESULT_ROOT is a folder that is expected to have the same general folder structure as EMDB, i.e.:

RESULT_ROOT├── PX    ├── sequence1        ├── hybrIK-out            ├── 000000.pkl            ├── 000001.pkl            ├── ...    ├── sequence2    ├── ...    ├── sequenceN

I.e.,EMDB_ROOT can function as aRESULT_ROOT, if the corresponding results are stored in a subfolderhybrIK-out for each sequence. The evaluation code computes the MPJPE, MPJAE, MVE, and jitter metrics as reported in the paper. It reports both the pelvis-aligned and Procrustes-aligned versions (*-PA), as well as standard deviations. Further, it prints the metrics for each sequence individually, as well as the average over all sequences.

In order to run the evaluations with your own results, follow these steps:

1. Inevaluation_loaders.py define a function to load your result. Follow the signature and return values of the existingload_hybrik function.
2. Inevaluation_engine.py register your method by giving it a name in a global variable, e.g.MYMETHOD = 'My Method'. Then using that name as a key, extend the following twodicts (follow the existing example with HybrIK for reference):
   • METHOD_TO_RESULT_FOLDER: This maps to the subfolder in{RESULT_ROOT}/{SUBJECT_ID}/{SEQUENCE_ID} where your methods result will be stored.
   • METHOD_TO_LOAD_FUNCTION: This maps to the loading function you defined in step 1.
3. In the functionEvaluationEngine.get_gender_for_baseline select the appropriate SMPL gender for your method.
4. Finally, inevaluate.py importMYMETHOD and add it to the list of methods that the evaluation engine should evaluate.

If you use this code or data, please cite the following paper:

@inproceedings{kaufmann2023emdb,author ={Kaufmann, Manuel and Song, Jie and Guo, Chen and Shen, Kaiyue and Jiang, Tianjian and Tang, Chengcheng and Z{\'a}rate, Juan Jos{\'e} and Hilliges, Otmar},title ={{EMDB}: The {E}lectromagnetic {D}atabase of {G}lobal 3{D} {H}uman {P}ose and {S}hape in the {W}ild},booktitle ={International Conference on Computer Vision (ICCV)},year ={2023}}

For any questions or problems, please open an issue or contactManuel Kaufmann.