Repository files navigation

MANO layer forPyTorch (tested with v0.4 and v1.x)

ManoLayer is a differentiable PyTorch layer that deterministically maps from pose and shape parameters to hand joints and vertices.It can be integrated into any architecture as a differentiable layer to predict hand meshes.

ManoLayer takesbatched hand pose and shape vectors and outputs corresponding hand joints and vertices.

The code is mostly a PyTorch port of the originalMANO model fromchumpy toPyTorch.It therefore builds directly upon the work of Javier Romero, Dimitrios Tzionas and Michael J. Black.

This layer was developped and used for the paperLearning joint reconstruction of hands and manipulated objects for CVPR19.Seeproject page anddemo+training code.

Itreusespart of the great code from thePytorch layer for the SMPL body model by Zhang Xiong (MandyMo) to compute the rotation utilities !

It also includes inmano/webuser partial content of files from the originalMANO code (posemapper.py,serialization.py,lbs.py,verts.py,smpl_handpca_wrapper_HAND_only.py).

If you find this code useful for your research, consider citing:

• the originalMANO publication:

@article{MANO:SIGGRAPHASIA:2017,  title = {Embodied Hands: Modeling and Capturing Hands and Bodies Together},  author = {Romero, Javier and Tzionas, Dimitrios and Black, Michael J.},  journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)},  publisher = {ACM},  month = nov,  year = {2017},  url = {http://doi.acm.org/10.1145/3130800.3130883},  month_numeric = {11}}

• the publication this PyTorch port was developped for:

@INPROCEEDINGS{hasson19_obman,  title     = {Learning joint reconstruction of hands and manipulated objects},  author    = {Hasson, Yana and Varol, G{\"u}l and Tzionas, Dimitris and Kalevatykh, Igor and Black, Michael J. and Laptev, Ivan and Schmid, Cordelia},  booktitle = {CVPR},  year      = {2019}}

The training code associated with this paper, compatible with manopth can be foundhere. The release includes a model trained on a variety of hand datasets.

• git clone https://github.com/hassony2/manopth
• cd manopth
• Install the dependencies listed inenvironment.yml
  • In an existing conda environment,conda env update -f environment.yml
  • In a new environment,conda env create -f environment.yml, will create a conda environment namedmanopth

• Go toMANO website
• Create an account by clickingSign Up and provide your information
• Download Models and Code (the downloaded file should have the formatmano_v*_*.zip). Note that all code and data from this download falls under theMANO license.
• unzip and copy themodels folder into themanopth/mano folder
• Your folder structure should look like this:

manopth/  mano/    models/      MANO_LEFT.pkl      MANO_RIGHT.pkl      ...  manopth/    __init__.py    ...

To check that everything is going well, runpython examples/manopth_mindemo.py, which should generate from a random hand using the MANO layer !

To be able to import and useManoLayer in another project, go to yourmanopth folder and runpip install .

cd /path/to/other/project

You can now usefrom manopth import ManoLayer in this other project!

Seeexamples/manopth_mindemo.py

Simple forward pass with random pose and shape parameters through MANO layer

importtorchfrommanopth.manolayerimportManoLayerfrommanopthimportdemobatch_size=10# Select number of principal components for pose spacencomps=6# Initialize MANO layermano_layer=ManoLayer(mano_root='mano/models',use_pca=True,ncomps=ncomps)# Generate random shape parametersrandom_shape=torch.rand(batch_size,10)# Generate random pose parameters, including 3 values for global axis-angle rotationrandom_pose=torch.rand(batch_size,ncomps+3)# Forward pass through MANO layerhand_verts,hand_joints=mano_layer(random_pose,random_shape)demo.display_hand({'verts':hand_verts,'joints':hand_joints},mano_faces=mano_layer.th_faces)

Result :

With more options, forward and backward pass, and a loop for quick profiling, look atexamples/manopth_demo.py.

You can run it locally with:

python examples/manopth_demo.py