Repository files navigation

New: We add more implementation code and models in this repo:

• Un-Mix w/SwAV on CIFAR-10 and CIFAR-100, please check our results and code out at./UnMix_SwAV/for_CIFAR (93.0% accuracy on CIFAR-10 and69.9% accuracy on CIFAR-100).

• Un-Mix w/SwAV on ImageNet-1K, please check it out at./UnMix_SwAV/for_ImageNet.

(Insights of Un-Mix on clustering-based methods like SwAV can be foundhere.)

• Un-Mix w/ MoCoV1,V2 on ImageNet-1K, please check it out at./UnMix_MoCo/for_ImageNet.

(Un-Mix w/MoCo on CIFAR-10 and CIFAR-100 has already been included in this repo.)

Notes: As Un-Mix implementation on CIFAR with SwAV was done in 2020, while the SwAV authors simplified their cluster assignment implementation inApril, 2021. Thus, our CIFAR code follows their old implementation before simplification, but the ImageNet code is based on their updated version. Nevertheless, the performance of them will not be affected.

• Un-Mix has been accepted in AAAI 2022!! Please check out our camera-ready paper onarXiv.

• We provide a demo of Un-Mix on CIFAR-10 and 100 adapted fromColab notebook of MoCo.

  To train an Un-Mix model with MoCo and symmetric loss, rununmix_c10.py orunmix_c100.py:

   # CIFAR-10 CUDA_VISIBLE_DEVICES=0 python unmix_c10.py   # CIFAR-100 CUDA_VISIBLE_DEVICES=1 python unmix_c100.py

  Results on CIFAR-10:

  Model	epochs	acc. (Top-1)	weights (last)	logs	args
  MoCo w/ asymmetric	1000	89.11%	link	link	link
  +Un-Mix w/ asymmetric	1000	91.34%	link	link	link
  MoCo w/ symmetric	1000	90.49%	link	link	link
  +Un-Mix w/ symmetric	1000	92.25%	link	link	link

  Results on CIFAR-100:

  Model	epochs	acc. (Top-1)	weights (last)	logs	args
  MoCo w/ asymmetric	1000	63.64%	link	link	link
  +Un-Mix w/ asymmetric	1000	67.33%	link	link	link
  MoCo w/ symmetric	1000	65.49%	link	link	link
  +Un-Mix w/ symmetric	1000	68.83%	link	link	link

• Our pre-trained ResNet-50 model on ImageNet (MoCo V2 based) is available at:Download. The training code is available at:code.

We update ourmanuscript with a more comprehensive study usingimage mixtures method on unsupervised learning. The core codes of our method can be summarized as follows:

# P: probability of global or local level mixtures # beta: hyperparameter for Beta distribution# lam: mixture ratio in global-level mixture or bounding box location in region-level mixtureargs.beta = 1.0for x in loader: # load a minibatch x with N samples    # Probability of choosing global or local level mixtures    prob = np.random.rand(1)    lam = np.random.beta(args.beta, args.beta)     images_reverse = torch.flip(x[0], (0,))    if prob < args.P:# global-level mixturesmixed_images = lam*x[0]+(1-lam)* images_reversemixed_images_flip = torch.flip(mixed_images, (0,))    else:# region-level mixturesmixed_images = x[0].clone()bbx1, bby1, bbx2, bby2 = utils.rand_bbox(x[0].size(), lam)mixed_images[:, :, bbx1:bbx2, bby1:bby2] = images_reverse[:, :, bbx1:bbx2, bby1:bby2] mixed_images_flip = torch.flip(mixed_images,(0,))lam = 1 - ((bbx2 - bbx1) * (bby2 - bby1) / (x[0].size()[-1] * x[0].size()[-2]))    # original loss term    loss_ori = model(x)    # In following two losses, we found using ''x[0]'' may perform better on some particular datasets    # loss for the normal order of mixtures    loss_m1 = model([x[1], mixed_images])    # loss for the reverse order of mixtures    loss_m2 = model([x[1], mixed_images_flip])    # final loss function (our core code)    loss = loss_ori + lam * loss_m1 + (1-lam) * loss_m2    # update gradients    optimizer.zero_grad()     loss.backward()     optimizer.step()    ...

This repo contains the implementation forUn-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning, which perturbs input image space to soften the output prediction space indirectly, meanwhile, assigning new label values in the unsupervised frameworks accordingly. So that the proposed method can smooth decision boundaries and prevent the learner from becoming over-confident.

We run our method withSimCLR,BYOL,MoCo andMoCo V2, the results are as follows:

If you find this repo useful for your research, please consider citing the paper

@inproceedings{shen2022unmix,  title={Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning},  author={Shen, Zhiqiang and Liu, Zechun and Liu, Zhuang and Savvides, Marios and Darrell, Trevor and Xing, Eric},  journal={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},  year={2022}}

For any questions and comments, please contact Zhiqiang Shen (zhiqiangshen0214 at gmail.com).

MoCo V1&V2 (https://github.com/facebookresearch/moco)

Whitening (https://github.com/htdt/self-supervised)

PyTorch Image Classification (https://github.com/hysts/pytorch_image_classification)