Repository files navigation

• CIFAR and ILSVRC training code withjit compiling anddistributed learning on the multi-GPU system.
• I highly recommend using Jit compiling because most of the algorithm is static and can be compiled, which gives memory usage reduction and training speed improvement.
• This repository is built bycustom layers andcustom training loop for my project, but if you only want to check how to use jit compiling with distributed learning, check 'train.py' and 'op_util.py'.

• Tensorflow >= 2.5
• Pillow

• ILSVRC

python train.py --compile --gpu_id {} --dataset ILSVRC --data_path /path/to/your/ILSVRC/home --train_path /path/to/log

• CIFAR{10,100}

python train.py --compile --gpu_id {} --dataset CIFAR{10,100} --train_path /path/to/log

• I used four 1080ti.
• Jit compiling gives a 40% speedup for training time.