Repository files navigation

This repository contains an implementation ofDeep Deterministic Policy Gradient (DDPG), a Reinforcement Learning algorithm designed for environments with continuous action spaces. It featuresActor-Critic architecture,Experience Replay,Target Networks, andExploration Strategies, and is tested on the likeMountainCarContinuous-v0 environment.Batch Normalisation is also included to stabilise training. For a deeper explanation of DDPG, the theory behind it, how it works and the structure of the code, check out myMedium blog.

• Actor-Critic Architecture: DDPG uses two networks—one for policy (actor) and one for value estimation (critic).
• Replay Buffer: Stores past experiences to ensure more stable training.
• Target Networks: Helps to stabilise learning by slowly updating the target networks.
• Exploration Noise: ImplementsOrnstein-Uhlenbeck noise to facilitate smooth exploration in continuous action spaces.
• Batch Normalisation: Used to stabilise training by normalising activations and reducing internal covariate shifts.
• Testing & Visualisation: Includes functionality to test the agent’s performance and visualise its actions in the MountainCar environment.

To be able to run this DDPG implementation and experiment with it, please follow the steps described below by copying and pasting the relevant lines onto your command prompt.

git clone https://github.com/sirine-b/DDPG.gitcd DDPG

pip install -r requirements.txt

jupyter notebook DDPG_model.ipynb

You can now run the code yourself and experiment with the training and testing of the DDPG agent by trying out different hyperparameters, environments ... etc