Repository files navigation

Work published at Robotics and Autonomous Systems [open-access paper,video]

If you use this code in an academic context, please cite our work:

@article{xu2025cuahn,title={CUAHN-VIO: Content-and-uncertainty-aware homography network for visual-inertial odometry},author={Xu, Yingfu and de Croon, Guido CHE},journal={Robotics and Autonomous Systems},volume={185},pages={104866},year={2025},publisher={Elsevier}}

The Python scripts in this folder were developed using Python==3.9.12, numpy==1.22.4, torch==1.7.1+cu101.

Runpython trace_model.py to generate the.pt model file from the Python.pth.tar model file.

A.pt model file is loaded bylibtorch in C++ environment and called in theHomographyNet package ofcuahn_ros to perform neural network inference.

cuahn_ros is a ROS 1 project built uponCommit 83ffb88 of OpenVINS. The required packages are the same as OpenVINS except forlibtorch. libtorch==1.7.1+cu101 is installed at$ENV{HOME}/libtorch of the developer's laptop computer with cuda 10.1. Please modify the CMAKE_PREFIX_PATH of libtorch according to your libtorch installation atcuahn_ros/cuahn/CMakeLists.txt (line 6) andcuahn_ros/homography_network/CMakeLists.txt (line 7).

The developer usescatkin build to build the cuahn_ros project. A known issue during building is related to libtorch. The developer uses a workaround as follows. After the first build attempt, error messages containing the following could appear

/home/ws/src/cuahn_ros/cuahn/src/state/StateOptions.h:26:10: fatal error: types/LandmarkRepresentation.h: No such file or directory/home/ws/src/cuahn_ros/cuahn/src/core/VioManagerOptions.h:30:10: fatal error: feat/FeatureInitializerOptions.h: No such file or directory

In this case, after the building finishes ([build] Summary: 5 of 6 packages succeeded.), comment outset(CMAKE_PREFIX_PATH $ENV{HOME}/libtorch) at line 6 ofcuahn_ros/cuahn/CMakeLists.txt, and thencatkin build again. Packagecuahn should be built successfully.

Modify line 58 ofcuahn_ros/cuahn/launch/uzhfpv.launch

<param name="network_model_path" type="string" value="$(find HomographyNet)/torch_script_models_laptop/traced_model_3_blocks_using_prior_showError.pt" />

to set the path to the.pt model file you want to use. If you want to run the full model without using EKF prior, setfalse to line 56 ofuzhfpv.launch. Otherwise, set it astrue and set the path to a 3-block model tonetwork_model_path (line 58).

Runroslaunch cuahn uzhfpv.launch to run CUAHN-VIO on a flight sequence of the UZH-FPV dataset. Set the name and path to the ROS bag of the sequence at lines 8 and 9 ofuzhfpv.launch.

If the.pt model file name containsshowError, a window displays the photometric error map of the two consecutive images aligned by the network's homography transformation prediction. Setfalse to line 57 ofuzhfpv.launch to disable video display.