Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

Official GitHub repo for VecKM. A very efficient and descriptive local geometry encoder / point tokenizer / patch embedder. ICML2024.

License

NotificationsYou must be signed in to change notification settings

dhyuan99/VecKM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

57 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Dehao Yuan , Cornelia Fermüller , Tahseen Rabbani , Furong Huang , Yiannis Aloimonos  

ICML2024     [Paper]    [Video Talk]

News and Updates

  • [Dec 8, 2024] We did an extension project of VecKM. We apply VecKM to event camera inputs and predict the motion field. Check outVecKM_flow.

Highlighted Features

API Usage

VecKM is a generic local point cloud encoder, a.k.a. point tokenizer, patch encoder, etc. The API is easily used by the following codes, after installing the package:

git clone https://github.com/dhyuan99/VecKM.gitcd VecKMconda create -n VecKM python=3.13conda activate VecKMpip install --upgrade pip setuptools wheelpython setup.py sdist bdist_wheelpip install .

Seedemo for the codes and data for running the demo:

cd demopython main.py

API Documentation

We provide two implementations of VecKM:ExactVecKM andFastVecKM, corresponding to Eqn. (3) and Eqn. (2) in the paper.

  • ExactVecKM computes an accurate local point cloud encoding, in a slower manner. Suitable for tasks where accurate geometry is needed, e.g. normal estimation.
  • FastVecKM computes a noisy local point cloud encoding with large point cloud inputs (e.g. size > 50000), in a faster manner. Suitable for tasks where only coarse geometry is needed, e.g. classification.

Both of them

  • receive inputs with shape (n, 3) and output (n, d), operations defined atHighlighted Features.
  • scalable to point cloud size > 50000 and neighborhood size > 500 on 16GB memory.

The API call is as simple as followed, as shown in./demo/main.py:

fromVecKM.encoderimportExactVecKM,FastVecKMpts=torch.tensor(pts).float()pts=pts.cuda()# pts has shape (n, 3).vkm=ExactVecKM(pt_dim=3,enc_dim=384,radius=0.1)vkm=vkm.cuda()G=vkm(pts)# G has shape (n, 384).vkm=FastVecKM(pt_dim=3,enc_dim=384,radius=0.1)vkm=vkm.cuda()G=vkm(pts)# G has shape (n, 384).

How to check the encoding quality?

VecKM.visualize.check_vkm_quality_3d provides a visual check of the encoding quality (Figure 5 in the paper). It will

  • visualize the local point cloud aroundpts[which_pt], with radius specified invkm.radius.
  • visualize the reconstructed point cloud distribution from the VecKM encoding.

After executing the script

fromVecKM.visualizeimportcheck_vkm_quality_3dcheck_vkm_quality_3d(vkm,pts,which_pt=0)

It will generate a gif showing the 3d visualization. If the distribution aligns with the point cloud, it means the encoding quality is good. Checkdemo for more details.

Watch the video

Downstream Tasks

Local point cloud encoding is usually followed by point-wise regression, classification, regression, etc., as shown below.

Watch the video

Citation

If you find it helpful, please consider citing our papers:

@InProceedings{pmlr-v235-yuan24b,  title =  {A Linear Time and Space Local Point Cloud Geometry Encoder via Vectorized Kernel Mixture ({V}ec{KM})},  author =       {Yuan, Dehao and Fermuller, Cornelia and Rabbani, Tahseen and Huang, Furong and Aloimonos, Yiannis},  booktitle =  {Proceedings of the 41st International Conference on Machine Learning},  pages =  {57871--57886},  year =  {2024},  editor =  {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix},  volume =  {235},  series =  {Proceedings of Machine Learning Research},  month =  {21--27 Jul},  publisher =    {PMLR},  pdf =  {https://raw.githubusercontent.com/mlresearch/v235/main/assets/yuan24b/yuan24b.pdf},  url =  {https://proceedings.mlr.press/v235/yuan24b.html},}

About

Official GitHub repo for VecKM. A very efficient and descriptive local geometry encoder / point tokenizer / patch embedder. ICML2024.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages


[8]ページ先頭

©2009-2025 Movatter.jp