Repository files navigation

Project Page |arXiv

Zirui Wang,Wenjing Bian,Victor Adrian Prisacariu.

Active Vision Lab (AVL),University of Oxford.

• Environment
• Data
• Training
• Inferencing

We provide aenvironment.yaml file to set up aconda environment:

git clone https://github.com/ActiveVisionLab/CrossScore.gitcd CrossScoreconda env create -f environment.yamlconda activate CrossScore

TLDR: download thisfile (~3GB),put it indatadir:

mkdir datadircd datadirwget https://www.robots.ox.ac.uk/~ryan/CrossScore/MFR_subset_demo.tar.gztar -xzvf MFR_subset_demo.tar.gzrm MFR_subset_demo.tar.gzcd ..

To demonstrate a minimum working example for training and inferencing steps shown below,we provide a small pre-processed subset.The is a subset ofMap-Free Relocalisation (MFR)and is pre-processed using3D Gaussian Splatting (3DGS).This small demo dataset is available at thislink (~3GB).This is the file in TLDR.We only use this demo subset to present the expected dataloading structure.

In our actual training, our model is trained using MFR that pre-processed by three NVS methods:3DGS,TensoRF, andNeRFacto.Due to the preprocessed file size (~2TB), it is challenging to directly sharethis pre-processed data. One work around is to release a data pre-processing scriptfor MFR, which we are still tidying up.We aim to release the pre-processing script in Dec 2024.

We train our model with two NVIDIA A5000 (24GB) GPUs for about two days.However, the model should perform reasonably well after 12 hours of training.It is also possible to train with a single GPU.

python task/train.py trainer.devices='[0,1]'# 2 GPUs# python task/train.py trainer.devices='[0]'  # 1 GPU

We provide an example command to predict CrossScore for NVS rendered imagesby referencing real captured images.

git lfs install&& git lfs pull# get our ckpt using git LFSbash predict.sh

After running the script, our CrossScore score maps should be written topredict dir.The output should be similar to ourdemo videoon our project page.

• Create a HuggingFace demo page.
• Release ECCV quantitative results related scripts.
• Release data processing scripts
• Release PyPI and Conda package.

This research is supported by anARIAresearch gift grant from Meta Reality Lab. We gratefully thankShangzhe Wu,Tengda Han,Zihang Lai for insightful discussions, andMichael Hobley for proofreading.

@inproceedings{wang2024crossscore,title={CrossScore: Towards Multi-View Image Evaluation and Scoring},author={Zirui Wang and Wenjing Bian and Victor Adrian Prisacariu},booktitle={ECCV},year={2024}}