A collection of various gradient descent algorithms implemented in Python from scratch

• UpdatedFeb 28, 2023
• Python

[ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.

• UpdatedAug 30, 2022
• Python

NAG-GS: Nesterov Accelerated Gradients with Gauss-Siedel splitting

• UpdatedDec 3, 2022
• Python

Intelligent Detection for RIS-Assisted MIMO Systems: A First-and-Second Momentum Approach

• UpdatedFeb 19, 2025
• Jupyter Notebook

Overshoot: Taking advantage of future gradients in momentum-based stochastic optimization

• UpdatedJun 16, 2025
• Python

In this project it is used a Machine Learning model based on a method called Extreme Learning, with the employment of L2-regularization. In particular, a comparison was carried out between: (A1) which is a variant of incremental extreme learning machine that is QRIELM and (A2) which is a standard momentum descent approach, applied to the ELM.

• UpdatedJul 14, 2023
• MATLAB

Using Matrix Factorization/Probabilistic Matrix Factorization to solve Recommendation。矩阵分解进行推荐系统算法。

• UpdatedJan 8, 2020
• R

Python code for Gradient Descent, Momentum, and Adam optimization methods. Train neural networks efficiently.

• UpdatedAug 3, 2023
• Jupyter Notebook

Generic L-layer 'straight in Python' fully connected Neural Network implementation using numpy.

• UpdatedMay 29, 2021
• Python

Lightweight neural network library written in ANSI-C supporting prediction and backpropagation for Convolutional- and Fully Connected neural networks

• UpdatedFeb 24, 2025
• C

Simple Document Classification using Multi Class Logistic Regression & SVM Soft Margin from scratch

• UpdatedJul 30, 2022
• Jupyter Notebook

This is an implementation of different optimization algorithms such as: - Gradient Descent (stochastic - mini-batch - batch) - Momentum - NAG - Adagrad - RMS-prop - BFGS - Adam Also, most of them are implemented in vectorized form for multi-variate problems

• UpdatedApr 3, 2023
• Jupyter Notebook

Numerical Optimization for Machine Learning & Data Science

• UpdatedApr 29, 2022
• Jupyter Notebook

• UpdatedJun 7, 2022
• Jupyter Notebook

EE456 2022 mini project implementation of two-moons problem using multi-layer-perceptron with back-propagation with analyzing performance of initializing methods and momentum rule

• UpdatedDec 12, 2023
• MATLAB

A fully vectorized Deep Neural Network (DNN) implementation built from scratch using only NumPy - no deep learning frameworks involved. Covers forward/backward propagation, activation functions, modular architecture, and training with different optimizers - a hands-on deep dive into the fundamentals of deep learning.

• UpdatedJun 30, 2025
• Jupyter Notebook

Machine Learning, Deep Learning Implementations

• UpdatedOct 7, 2021
• Jupyter Notebook

This repository provides implementations of numerical optimization algorithms for machine learning and deep learning. It includes clear explanations, mathematical formulas, Python code, and visualizations to help understand the behavior of each optimizer.

• UpdatedJun 20, 2025
• Jupyter Notebook

This repository contains a python implementation of Feed Forward Neural Network with Backpropagation, along with the example scripts for training the network to classify images from mnist and fashion_mnist datasets from keras.

• UpdatedMar 18, 2021
• Jupyter Notebook

Comparsion of Machine Learning optimization algorithms with MNIST dataset

• UpdatedMay 26, 2022
• Jupyter Notebook