Repository files navigation

This repository contains PyTorch implementations of deep reinforcement learning algorithms.The repository will soon be updated including the PyBullet environments!

1. Deep Q-Network (DQN) _{^{(V. Mnih et al. 2015)}}
2. Double DQN (DDQN) _{^{(H. Van Hasselt et al. 2015)}}
3. Advantage Actor Critic (A2C)
4. Vanilla Policy Gradient (VPG)
5. Natural Policy Gradient (NPG) _{^{(S. Kakade et al. 2002)}}
6. Trust Region Policy Optimization (TRPO) _{^{(J. Schulman et al. 2015)}}
7. Proximal Policy Optimization (PPO) _{^{(J. Schulman et al. 2017)}}
8. Deep Deterministic Policy Gradient (DDPG) _{^{(T. Lillicrap et al. 2015)}}
9. Twin Delayed DDPG (TD3) _{^{(S. Fujimoto et al. 2018)}}
10. Soft Actor-Critic (SAC) _{^{(T. Haarnoja et al. 2018)}}
11. SAC with automatic entropy adjustment (SAC-AEA) _{^{(T. Haarnoja et al. 2018)}}

1. Classic control environments (CartPole-v1, Pendulum-v0, etc.) _{^{(as described inhere)}}
2. MuJoCo environments (Hopper-v2, HalfCheetah-v2, Ant-v2, Humanoid-v2, etc.) _{^{(as described inhere)}}
3. PyBullet environments (HopperBulletEnv-v0, HalfCheetahBulletEnv-v0, AntBulletEnv-v0, HumanoidDeepMimicWalkBulletEnv-v1 etc.) _{^{(as described inhere)}}

• Observation space: 8
• Action space: 3

• Observation space: 17
• Action space: 6

• Observation space: 111
• Action space: 8

• Observation space: 376
• Action space: 17

• Observation space: 15
• Action space: 3

• Observation space: 26
• Action space: 6

• Observation space: 28
• Action space: 8

• Observation space: 197
• Action space: 36

• PyTorch
• TensorBoard
• gym
• mujoco-py
• PyBullet

The repository's high-level structure is:

├── agents                        └── common ├── results      ├── data     └── graphs        └── save_model

To train all the different agents on PyBullet environments, follow these steps:

git clone https://github.com/dongminlee94/deep_rl.gitcd deep_rlpython run_bullet.py

For other environments, change the last line torun_cartpole.py,run_pendulum.py,run_mujoco.py.

If you want to change configurations of the agents, follow this step:

python run_bullet.py \    --env=HumanoidDeepMimicWalkBulletEnv-v1 \    --algo=sac-aea \    --phase=train \    --render=False \    --load=None \    --seed=0 \    --iterations=200 \    --steps_per_iter=5000 \    --max_step=1000 \    --tensorboard=True \    --gpu_index=0

To watch all the learned agents on PyBullet environments, follow these steps:

python run_bullet.py \    --env=HumanoidDeepMimicWalkBulletEnv-v1 \    --algo=sac-aea \    --phase=test \    --render=True \    --load=envname_algoname_... \    --seed=0 \    --iterations=200 \    --steps_per_iter=5000 \    --max_step=1000 \    --tensorboard=False \    --gpu_index=0

You should copy the saved model name insave_model/envname_algoname_... and paste the copied name inenvname_algoname_.... So the saved model will be load.