optimizers

Star

Here are 104 public repositories matching this topic...

Language:All

Filter by language

All104 Python59 Jupyter Notebook28 C++2 HTML2 Lua2 C1 C#1 Elixir1 Java1 Julia1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

labmlai /annotated_deep_learning_paper_implementations

Star64.8k

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

machine-learning reinforcement-learning deep-learning transformers pytorch transformer gan neural-networks literate-programming attention lora deep-learning-tutorial optimizers

UpdatedNov 11, 2025
Python

lucidrains /lion-pytorch

Star2.2k

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

deep-learning artificial-intelligence optimizers evolutionary-search

UpdatedNov 27, 2024
Python

elixir-nx /axon

Star1.7k

Nx-powered Neural Networks

elixir deep-learning neural-networks nx optimizers

UpdatedNov 14, 2025
Elixir

gugarosa /opytimizer

Sponsor

Star630

🐦 Opytimizer is a Python library consisting of meta-heuristic optimization algorithms.

python optimization artificial-intelligence meta-heuristic optimizers bioinspired meta-heuristic-optimization

UpdatedSep 22, 2025
Python

Yonghongwei /Gradient-Centralization

Star540

A New Optimization Technique for Deep Neural Networks

deep-neural-networks optimizers optimization-technique gradient-centralization

UpdatedJan 13, 2022
Python

CyberZHG /keras-radam

Star325

RAdam implemented in Keras & TensorFlow

tensorflow keras adam optimizers radam rectified-adam

UpdatedJan 22, 2022
Python

OverLordGoldDragon /keras-adamw

Star169

Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers

tensorflow keras sgd adamw adamwr nadam optimizers learning-rate-multipliers warm-restarts

UpdatedJan 6, 2022
Python

lucidrains /adam-atan2-pytorch

Star135

Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch

deep-learning artificial-intelligence stability adam optimizers

UpdatedOct 15, 2025
Python

decile-team /submodlib

Star120

Summarize Massive Datasets using Submodular Optimization

greedy-algorithms combinatorial-optimization submodular-optimization data-summarization optimizers submodular-functions submodular submodular-information-measures

UpdatedMay 14, 2025
Jupyter Notebook

warner-benjamin /optimi

Star118

Fast, Modern, and Low Precision PyTorch Optimizers

deep-learning pytorch optimizers

UpdatedSep 6, 2025
Python

usef-kh /fer

Star111

Code for the paper "Facial Emotion Recognition: State of the Art Performance on FER2013"

architecture tuning vggnet loss fer schedulers fer2013 saliency-maps optimizers facial-emotions

UpdatedMay 7, 2023
Jupyter Notebook

palle-k /DL4S

Star110

Accelerated tensor operations and dynamic neural networks based on reverse mode automatic differentiation for every device that can run Swift - from watchOS to Linux

swift machine-learning deep-neural-networks deep-learning automatic-differentiation autograd recurrent-neural-networks recurrent-networks neural-networks derivatives convolutional-neural-networks tensor gradient-descent swift-machine-learning optimizers