Repository files navigation

(ECCV 2020) Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection

Gitee Mirror:https://gitee.com/p_lart/HDFNet

Author: Lart Pang(lartpang@163.com)

This is a complete, modular and easily modified code base based on PyTorch, which is suitable for the training and testing of significant target detection task model.

@inproceedings{HDFNet-ECCV2020,    author = {Youwei Pang and Lihe Zhang and Xiaoqi Zhao and Huchuan Lu},    title = {Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection},    booktitle = ECCV,    year = {2020}}

News:

• The proposed model (HDFNet) is an important baseline of the winning solution in NTIRE 2021 (Depth Guided Image Relighting Challenge) hosted in CVPR 2021 workshop (winner: AICSNTU-MBNet team (Asustek Computer Inc & National Taiwan University)). [PAPER] [COOD]

NOTE:

• In the link below, we provide the results of the two versions (with/without_STEREO).
• Specifically, in the file with_STEREO, two versions of the STEREO dataset are provided.
  • One of them contains 797 pictures, and the other contains 1000 pictures.
  • In our paper, the results evaluated on the latter are used, which is exactly what is provided in the file without_STEREO.

[Results & PretrainedParams (j9qu)]

• RGBD-DATASET
  • https://github.com/jiwei0921/RGBD-SOD-datasets
  • http://dpfan.net/d3netbenchmark/

• backbone: Store some code for backbone networks.
• loss: The code of the loss function.
• module: The code of important modules.
• network: The code of the network.
• output: It saves all results.
• utils: Some instrumental code.
  • data/*py: Some files about creating the dataloader.
  • transforms/*py: Some operations on data augmentation.
  • metric.py: max/mean/weighted F-measure, S-measure, E-measure and MAE. (NOTE: If you find a problem in this part of the code, please notify me in time, thank you.)
  • misc.py: Some useful utility functions.
  • tensor_ops.py: Some operations about tensors.
• config.py: Configuration file for model training and testing.
• train.py: I think you can understand.
• test.py andtest.sh: These files can evaluate the performance of the model on the specified dataset. And the filetest.sh is a simple example about how to configure and runtest.py.

I provided conda environment configuration file (hdfnet.yaml), you can refer to the package version information.

And you can tryconda env create -f hdfnet.yaml to create an environment to run our code.

• Add your own module into themodule.
• Add your own network into thenetwork and import your model in thenetwork/__init__.py.
• Modifyconfig.py:
  • change the dataset path:datasets_root
  • change items inarg_config
    • model corresponds to the name of the model innetwork
    • suffix: finally, the form of<model>_<suffix> is used to form the alias of the model of this experiment and all files related to this experiment will be saved to the folder<model>_<suffix> inoutput folder
    • resume: set it toFalse to train normally
    • data_mode: set it toRGBD orRGB for using RGBD SOD datasets or RGB SOD datasets to train mdoel.
    • other items, likelr,batch_size and so on...
• Run the script:python train.py

If the training process is interrupted, you can use the following strategy to resume the training process.

• Setresume toTrue.
• Run the scripttrain.py again.

There are two ways:

1. For models that have been trained, you can setresume toTrue and run the scripttrain.py again.
2. Use the scriptstest.sh andtest.py. The specific method of use can be obtained by executing this command:python test.py --help.

You can use the toolkit released by us:https://github.com/lartpang/Py-SOD-VOS-EvalToolkit.

• (ECCV 2020 Oral) Suppress and Balance: A Simple Gated Network for Salient Object Detection:https://github.com/Xiaoqi-Zhao-DLUT/GateNet-RGB-Saliency
• (ECCV 2020) A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection:https://github.com/Xiaoqi-Zhao-DLUT/DANet-RGBD-Saliency
• (CVPR 2020) Multi-scale Interactive Network for Salient Object Detection:https://github.com/lartpang/MINet