- Notifications
You must be signed in to change notification settings - Fork0
MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors
License
min-hieu-netropy/MASt3R-SLAM
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Riku Murai* ·Eric Dexheimer* ·Andrew J. Davison
(* Equal Contribution)
conda create -n mast3r-slam python=3.11conda activate mast3r-slamCheck the system's CUDA version with nvcc
nvcc --versionInstall pytorch withmatching CUDA version following:
# CUDA 11.8conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=11.8 -c pytorch -c nvidia# CUDA 12.1conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia# CUDA 12.4conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidiaClone the repo and install the dependencies.
git clone https://github.com/rmurai0610/MASt3R-SLAM.git --recursivecd MAST3R-SLAM# if you've clone the repo without --recursive run# git submodule update --init --recursivepip install -e thirdparty/mast3rpip install -e thirdparty/in3dpip install --no-build-isolation -e .Setup the checkpoints for MASt3R and retrieval. The license for the checkpoints and more information on the datasets used is writtenhere.
mkdir -p checkpoints/wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth -P checkpoints/wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_trainingfree.pth -P checkpoints/wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_codebook.pkl -P checkpoints/bash ./scripts/download_tum.shpython main.py --dataset datasets/tum/rgbd_dataset_freiburg1_room/ --config config/calib.yamlConnect a realsense camera to the PC and run
python main.py --dataset realsense --config config/base.yamlOur system can process either MP4 videos or folders containing RGB images.
python main.py --dataset <path/to/video>.mp4 --config config/base.yamlpython main.py --dataset <path/to/folder> --config config/base.yamlbash ./scripts/download_tum.shbash ./scripts/download_7_scenes.shbash ./scripts/download_euroc.shbash ./scripts/download_eth3d.shAll evaluation script will run our system in a single-threaded, headless mode.We can run evaluations with/without calibration:
bash ./scripts/eval_tum.sh bash ./scripts/eval_tum.sh --no-calibbash ./scripts/eval_7_scenes.sh bash ./scripts/eval_7_scenes.sh --no-calibbash ./scripts/eval_euroc.sh bash ./scripts/eval_euroc.sh --no-calibbash ./scripts/eval_eth3d.shThere might be minor differences between the released version and the results in the paper after developing this multi-processing version.We run all our experiments on an RTX 4090, and the performance may differ when running with a different GPU.
We sincerely thank the developers and contributors of the many open-source projects that our code is built upon.
If you found this code/work to be useful in your own research, please considering citing the following:
@article{murai2024_mast3rslam,title={{MASt3R-SLAM}: Real-Time Dense {SLAM} with {3D} Reconstruction Priors},author={Murai, Riku and Dexheimer, Eric and Davison, Andrew J.},journal={arXiv preprint},year={2024},}