Repository files navigation

This repo is the official implementation of "GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector" (T-PAMI 2023).

Note that this work has more than 70% new staff (methods, discussion, etc.) and much better performance compared with the GCoNet (the conference version).

Authors:Peng Zheng,Huazhu Fu,Deng-Ping Fan,Qi Fan,Jie Qin,Yu-Wing Tai,Chi-Keung Tang, &Luc Van Gool.

[arXiv] [IEEE] [code] [stuff] [中文版]

Try our online demo for inference:

Try ourFPS benchmark of previous CoSOD methods:CoSOD_fps_collection.

In this paper, we present a novel end-to-end group collaborative learning network, termed GCoNet+, which can effectively and efficiently (250 fps) identify co-salient objects in natural scenes. The proposed GCoNet+ achieves the new state-of-the-art performance for co-salient object detection (CoSOD) through mining consensus representations based on the following two essential criteria: 1) intra-group compactness to better formulate the consistency among co-salient objects by capturing their inherent shared attributes using our novel group affinity module (GAM); 2) inter-group separability to effectively suppress the influence of noisy objects on the output by introducing our new group collaborating module (GCM) conditioning on the inconsistent consensus. To further improve the accuracy, we design a series of simple yet effective components as follows: i) a recurrent auxiliary classification module (RACM) promoting the model learning at the semantic level; ii) a confidence enhancement module (CEM) helping the model to improve the quality of the final predictions; and iii) a group-based symmetric triplet (GST) loss guiding the model to learn more discriminative features. Extensive experiments on three challenging benchmarks, i.e., CoCA, CoSOD3k, and CoSal2015, demonstrate that our GCoNet+ outperforms the existing 12 cutting-edge models. Code has been released athttps://github.com/ZhengPeng7/GCoNet_plus.

The figure of network architecture is drawn byInkscape (0.92.5) as a .svg file. You can download and modify it if you find it useful.

• Comparison with the previous state-of-the-art methods with different training sets:

• Ablation study:

To see the better performance of ourGCoNet+, we select the currently latest and top models (UFO-arXiv2022,DCFM-CVPR2022, andCADC-ICCV2021) for the qualitative comparison.

We not only show the selected extremely hard samples in the test sets but also simply put the unscreened samples (the first 10 samples in the first group in CoCA) for more objective and fair qualitative comparisons.

• The first^2 samples:

• The extremely hard cases:

1. Environment

   GPU: V100 x 1Install Python 3.7, PyTorch 1.8.2pip install requirements.txt

2. Datasets preparation

   Download necessary datasets:
   from my google-drive:DUTS_class,COCO-9k,COCO-SEG, andCoSOD_testsets, or
   from my BaiduDisk:DUTS_class,COCO-9k,COCO-SEG, andCoSOD_testsets.
   TheCoSOD_testsets contains CoCA, CoSOD3k and CoSal2015.

   The file directory structure on my machine is as follows:

   +-- datasets|   +-- sod|       +-- images|           +-- DUTS_class|           +-- COCO-9k|           ...|           +-- CoSal2015|       +-- gts|           +-- DUTS_class|           +-- COCO-9k|           ...|           +-- CoSal2015|   ......+-- codes|   +-- sod|       +-- GCoNet_plus|       ...|   ......

3. Update the paths

   Replace all/root/datasets/sod/GCoNet_plus and/root/codes/sod/GCoNet_plus in this project to/YOUR_PATH/datasets/sod/GCoNet_plus and/YOUR_PATH/codes/sod/GCoNet_plus, respectively.

4. Training + Test + Evaluate + Select the Best

   ./gco.sh

   If you can apply more GPUs on the DGX cluster, you can./sub_by_id.sh to submit multiple times for more stable results.

   If you have the OOM problem, plz decreasebatch_size inconfig.py.

5. Adapt the settings of modules in config.py

   You can change the weights of losses, try variousbackbones or use differentdata augmentation strategies. There are also some modules coded but not used in this work, likeadversarial training, therefiner inBASNet, weightedmultiple output and supervision used inGCoNet andGICD, etc.

   

​Findwell-trained models +predicted saliency maps and all other stuff on mygoogle-drive folder for this work:

We appreciate the codebases ofGICD,GCoNet. Thanks for the CoSOD evaluation toolbox provided ineval-co-sod. Thanks for the drawing codes of figure 1 fromDGNet.

@article{zheng2022gconet+,  author={Zheng, Peng and Fu, Huazhu and Fan, Deng-Ping and Fan, Qi and Qin, Jie and Tai, Yu-Wing and Tang, Chi-Keung and Van Gool, Luc},  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},   title={GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector},   year={2023},  volume={45},  number={9},  pages={10929-10946},  doi={10.1109/TPAMI.2023.3264571}}@inproceedings{fan2021gconet,  title = {Group Collaborative Learning for Co-Salient Object Detection},  author = {Fan, Qi and Fan, Deng-Ping and Fu, Huazhu and Tang, Chi-Keung and Shao, Ling and Tai, Yu-Wing},  booktitle = CVPR,  year = {2021}}

Any question, discussion or even complaint, feel free to leave issues here or send me e-mails (zhengpeng0108@gmail.com).