Repository files navigation

This repository is the official implementation of the paper:

Semantically Coherent Out-of-Distribution Detection
Jingkang Yang, Haoqi Wang, Litong Feng, Xiaopeng Yan, Huabin Zheng, Wayne Zhang, Ziwei Liu
Proceedings of the IEEE International Conference on Computer Vision (ICCV 2021)

We useconda to manage our dependencies, and CUDA 10.1 to run our experiments.

You can specify the appropriatecudatoolkit version to install on your machine in theenvironment.yml file, and then run the following to create theconda environment:

conda env create -f environment.ymlconda activate scood

The SC-OOD dataset introduced in the paper can be downloaded here.

Our codebase accesses the dataset from the root directory in a folder nameddata/ by default, i.e.

├── ...├── data│   ├── images│   └── imglist├── scood├── test.py├── train.py├── ...

The entry point for training is thetrain.py script. The hyperparameters for each experiment is specified by a.yml configuration file (examples given inconfigs/train/).

All experiment artifacts are saved in the specifiedargs.output_dir directory.

python train.py \    --config configs/train/cifar10_udg.yml \    --data_dir data \    --output_dir output/cifar10_udg

Evaluation for a trained model is performed by thetest.py script, with its hyperparameters also specified by a.yml configuration file (examples given inconfigs/test/)

Within the configuration file, you can also specify which post-processing OOD method to use (e.g. ODIN or Energy-based OOD detector (EBO)).

The evaluation results are saved in a.csv file as specified.

python test.py \    --config configs/test/cifar10.yml \    --checkpoint output/cifar10_udg/best.ckpt \    --data_dir data \    --csv_path output/cifar10_udg/results.csv

We report the mean ± std results from the current codebase as follows, which match the performance reported in our original paper.

You can run the following script (specifying the data and output directories) which perform training + testing for our main experimental results:

CIFAR-10, UDG

bash scripts/cifar10_udg.sh data_dir output_dir

CIFAR-10 (+ Tiny-ImageNet), ResNet18

CIFAR-10 (+ Tiny-ImageNet), DenseNet

CIFAR-10 (+ Tiny-ImageNet), WideResNet

You can run the following script (specifying the data and output directories) which perform training + testing for our main experimental results:

CIFAR-100, UDG

bash scripts/cifar100_udg.sh data_dir output_dir

CIFAR-100 (+ Tiny-ImageNet), ResNet18

CIFAR-100 (+ Tiny-ImageNet), DenseNet

CIFAR-100 (+ Tiny-ImageNet), WideResNet

Note:The work was originally built on the company's own deep learning framework, based on which we report all the results in the paper.We extracted all related code and built this standalone version for release, and checked that most of the results can be reproduced.We noticed that CIFAR-10 can easily match the paper results, but CIFAR-100 benchmark might have a few differences, perhaps due to some minor difference in framework modules and some randomness.We are currently enhancing our codebase and exploring udg on large-scale datasets.

This project is open-sourced under the MIT license.

The codebase is refactored by Ang Yi Zhe, and maintained by Jingkang Yang and Ang Yi Zhe.

If you find our repository useful for your research, please consider citing our paper:

@InProceedings{yang2021scood,author ={Yang, Jingkang and Wang, Haoqi and Feng, Litong and Yan, Xiaopeng and Zheng, Huabin and Zhang, Wayne and Liu, Ziwei},title ={Semantically Coherent Out-of-Distribution Detection},booktitle ={Proceedings of the IEEE International Conference on Computer Vision},year ={2021}}