learning-rate-scheduling

A guide that integrates Pytorch DistributedDataParallel, Apex, warmup, learning rate scheduler, also mentions the set-up of early-stopping and random seed.

pytorch distributed apex warmup early-stopping learning-rate-scheduling pytorch-distributeddataparallel random-seeds

UpdatedMay 22, 2022
Python

cmpark0126 /pytorch-polynomial-lr-decay

Star65

Polynomial Learning Rate Decay Scheduler for PyTorch

deep-learning pytorch learning-rate-decay pytorch-extension learning-rate-scheduling

UpdatedDec 25, 2021
Python

abhuse /cyclic-cosine-decay

Star49

Pytorch cyclic cosine decay learning rate scheduler

pytorch learning-rate-decay learning-rate-scheduling

UpdatedMay 31, 2021
Jupyter Notebook

YuchenJin /autolrs

Star46

Automatic learning-rate scheduler

machine-learning deep-learning hyperparameter-tuning automl learning-rate-scheduling

UpdatedApr 12, 2021
Python

lehduong /torch-warmup-lr

Star41

Warmup learning rate wrapper for Pytorch Scheduler

deep-learning pytorch learning-rate-scheduling learning-rate-warmup

UpdatedApr 12, 2020
Python

TheQuantScientist /LiteFormer

Star24

[EAAI] The official repo for the paper: "A Lightweight Multi-Head Attention Transformer for Stock Price Forecasting".

learning-rate-scheduling stock-prices-prediction dynamic-masking lightweight-transformer

UpdatedJul 12, 2025
Python

git-disl /LRBench

Star20

A learning rate recommending and benchmarking tool.

deep-learning learning-rate-scheduling learning-rate-benchmarking

UpdatedMay 19, 2023
Python

JonnoFTW /keras_find_lr_on_plateau

Star16

Keras Callback to Automatically Adjust the learning rate when it stops improving

keras learning-rate-scheduling

UpdatedJul 23, 2019
Python

ahundt /sharpDARTS

Star15

sharpDARTS: Faster and More Accurate Differentiable Architecture Search

computer-vision deep-learning pytorch neural-networks hyperparameter-optimization imagenet nas hyperparameter-tuning learning-rate cifar10 sota neural-architecture-search hyperparameter-search state-of-the-art architecture-search cosine-annealing learning-rate-scheduling differentiable-hyperparamter-search cosine-power-annealing power-cosine-annealing

UpdatedFeb 7, 2021
Python

timesler /lr-momentum-scheduler

Star12

Pytorch implementation of arbitrary learning rate and momentum schedules, including the One Cycle Policy

deep-learning pytorch learning-rate learning-rate-finder onecyclepolicy learning-rate-scheduling

UpdatedJul 15, 2020
Python

developer0hye /Learning-Rate-WarmUp

Star11

learning-rate-decay learning-rate-scheduling learning-rate-warmup

UpdatedApr 11, 2022

ZIB-IOL /BIMP

Star9

Code to reproduce the experiments of ICLR2023-paper: How I Learned to Stop Worrying and Love Retraining

sparsity deep-learning neural-network optimization pytorch pruning learning-rate-scheduling

UpdatedFeb 23, 2023
Python

cybertronai /pytorch-fd

Star8

Implementation of fluctuation dissipation relations for automatic learning rate annealing.

pytorch learning-rate-scheduling

UpdatedJul 24, 2019
Python

kardasbart /MultiLR

Star6

A method for assigning separate learning rate schedulers to different parameters group in a model.

deep-learning scheduler optimizer torch pytorch learning-rate lr lr-scheduling learning-rate-scheduling

UpdatedJul 20, 2023
Python

Comprehensive image classification for training multilayer perceptron (MLP), LeNet, LeNet5, conv2, conv4, conv6, VGG11, VGG13, VGG16, VGG19 with batch normalization, ResNet18, ResNet34, ResNet50, MobilNetV2 on MNIST, CIFAR10, CIFAR100, and ImageNet1K.