- Notifications
You must be signed in to change notification settings - Fork14
[CVPR 2022] Pytorch implementation of "Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions" paper
nv-nguyen/template-pose
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Templates for 3D Object Pose Estimation Revisited:
Generalization to New objects and Robustness to Occlusions
git checkout 50a1087If our project is helpful for your research, please consider citing :
@inproceedings{nguyen2022template, title={Templatesfor 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions}, author={Nguyen, Van Nguyen and Hu, Yinlin and Xiao, Yang and Salzmann, Mathieu and Lepetit, Vincent}, booktitle={Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)}, year={2022}}You can also put a star ⭐, if the code is useful to you.
If you like this project, check out related works from our group:
- CNOS: A Strong Baseline for CAD-based Novel Object Segmentation (ICCV 2023 R6D)
- NOPE: Novel Object Pose Estimation from a Single Image (arXiv 2023)
- PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6DoF Tracking(3DV 2022)
- BOP visualization toolkit
We have introduced additional features and updates to the codebase:
- Adding wandb logger:training loggers,testing loggers
- Cropping in LINEMOD settings is done with input bounding boxes (there is also predicted in-plane rotation)
- Releasing synthetic templates with Pyrender for faster rendering
- Releasing ready-to-use universal model pretrained on different datasets of BOP challengeHomebrewedDB, HOPE, RU-APC, IC-BIN, IC-MI, TUD-L, T-LESS
- Adding code to generate poses (OpenCV coordinate) from icosahedron with Blender
- Parsing withhydra library, simplifying training_step, testing_step withpytorch lightning
- Path structure (of pretrained models, dataset) is defined as in our recent projectNOPE
Click to expand
$ROOT_DIR ├── datasets ├── linemod ├── models ├──test ├── tless ├── ruapc ├── ... ├── templates ├── pretrained ├── moco_v2_800ep_pretrain.pth ├── results ├── experiment1 ├── wandb ├── checkpoint ├── experiment2
This repository is running with the Weight and Bias logger. Ensure that you update thisuser's configuration before conducting any experiments.
Click to expand
conda env create -f environment.ymlconda activate template# require only for evaluation: pytorch3d 0.7.0git clone https://github.com/facebookresearch/pytorch3d.gitpython -m pip install -e .First, create template poses from icosahedron:
blenderproc run src/poses/create_poses.pyNext, download and process BOP datasets
./src/scripts/download_and_process_datasets.shThere are two options for the final step (rendering synthetic templates from CAD models):
python -m src.scripts.download_prerendered_templatesOptional: This pre-rendered template set can be manually downloaded fromhere (12GB).
Option 2: Rendering synthetic templates from scratch (this will take around 1 hour with Nvidia V100)
./src/scripts/render_pyrender_all.shClick to expand
It is important to verify that all the datasets are correctly downloaded and processed. For example, by counting the number of images of each folder:
for dir in $ROOT_DIR/datasets/* do echo ${dir} find ${dir} -name "*.png" | wc -l doneIf everything is fine, here are the number of images that you should get:
├──$ROOT_DIR/datasets ├── hb# 55080 ├── hope# 1968 ├── icbin# 19016 ├── icmi# 31512 ├── lm# 49822 ├── olm# 4856 ├── ruapc#143486 ├── tless# 309600 ├── tudl# 153152 ├── templates (12GB)# 84102
Click to expand
0. (Optional) We use pretrained weight from MoCo v2. You can download it fromhere or run:
python -m src.scripts.download_moco_weightsIf you don't want to use pretrained weights, you can remove the path inthis line.
python train.py name_exp=train_allThe parsing is done with Hydra library. You can override anything in the configuration by passing arguments. For example:
# experiment 1: change batch_size, using data augmentation, update name_exppython train.py machine.batch_size=2 use_augmentation=True name_exp=train_augmentation# experiment 2: change batch_size, using data augmentation, update name_exp, update_lrpython train.py machine.batch_size=2 use_augmentation=True model.lr=0.001 name_exp=train_augmentation_lr0.001Please check out thistraining loggers to see how the training loss looks like.
Please note that all testing objects are unseen during training!
Click to expand
0. You can download it fromthis link or run:
python -m src.scripts.download_checkpointTODO: This is not the final checkpoint. We will update it soon.
python test_lm.py name_exp=test_lm model.checkpoint_path=$CHECKPOINT_PATHpython test_tless.py name_exp=test_tless model.checkpoint_path=$CHECKPOINT_PATHPlease check out thistesting loggers to see how the retrieved results looks like.
The code is adapted fromNope,Temos,Unicorn,PoseContrast,CosyPose andBOP Toolkit.
The authors thank Martin Sundermeyer, Paul Wohlhart and Shreyas Hampali for their fast reply, feedback!
If you have any question, feel free to create an issue or contact the first author atvan-nguyen.nguyen@enpc.fr
About
[CVPR 2022] Pytorch implementation of "Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions" paper