Repository files navigation

This repository contains material from thethird Udacity DRL procjet and the coding exerciceDDPG-pendulum.

In this project, we train a MADDPG multi-agent to solve two types of environment.

FirstTennis :

In this environment, two agents control rackets to bounce a ball over a net. If an agent hits the ball over the net, it receives a reward of$+0.1$. If an agent lets a ball hit the ground or hits the ball out of bounds, it receives a reward of$-0.01$. Thus, the goal of each agent is to keep the ball in play.

The observation space consists of$24$ variables corresponding to the position and velocity of the ball and racket. Each agent receives its own, local observation. Two continuous actions are available, corresponding to movement toward (or away from) the net, and jumping.

Second,Soccer :

In this discrete control environment, four agents compete in a$2$ strikers vs$2$ goalies in soccer game. The goal for a Striker is to get the ball into the opponent's goal and for Goalie is to keep the ball out of the goal.A striker/goalie receive a reward of$\pm 1$ when ball enters goal and$\mp 10^{-3}$ for existential.

An environment is considered solved, when an average score of +0.5 over 100 consecutive episodes, and for each agent is obtained.

To set up your python environment to run the code in this repository, follow the instructions below.

1. Create (and activate) a new environment with Python 3.9.

   • Linux orMac:

   conda create --name drlndsource activate drlnd

   • Windows:

   conda create --name drlnd activate drlnd

2. Follow the instructions inPytorch web page to install pytorch and its dependencies (PIL, numpy,...). For Windows and cuda 11.6

   conda install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge

3. Follow the instructions inthis repository to perform a minimal install of OpenAI gym.

   • Install thebox2d environment group by following the instructionshere.

   pip install gym[box2d]

4. Follow the instructions inthird Udacity DRL procjet to get the environment.

5. Clone the repository, and navigate to thepython/ folder. Then, install several dependencies.

git clone https://github.com/eljandoubi/MADDPG-for-Collaboration-and-Competition.gitcd MADDPG-for-Collaboration-and-Competition/pythonpip install.

6. Create anIPython kernel for thedrlnd environment.

python -m ipykernel install --user --name drlnd --display-name"drlnd"

7. Before running code in a notebook, change the kernel to match thedrlnd environment by using the drop-downKernel menu.

You can train and/or inference Tennis environment:

Run the training and/or inference cell ofTennis.ipynb.

The pre-trained models with the highest score are stored inTennis_checkpoint.

Same for Soccer but the current checkpoint isn't the best.

The implementation and resultats are discussed in the report.