Repository files navigation

keras-rl implements some state-of-the art deep reinforcement learning algorithms in Python and seamlessly integrates with the deep learning libraryKeras.

Furthermore,keras-rl works withOpenAI Gym out of the box. This means that evaluating and playing around with different algorithms is easy.

Of course you can extendkeras-rl according to your own needs. You can use built-in Keras callbacks and metrics or define your own.Even more so, it is easy to implement your own environments and even algorithms by simply extending some simple abstract classes. Documentation is availableonline.

As of today, the following algorithms have been implemented:

• Deep Q Learning (DQN)[1],[2]
• Double DQN[3]
• Deep Deterministic Policy Gradient (DDPG)[4]
• Continuous DQN (CDQN or NAF)[6]
• Cross-Entropy Method (CEM)[7],[8]
• Dueling network DQN (Dueling DQN)[9]
• Deep SARSA[10]
• Asynchronous Advantage Actor-Critic (A3C)[5]
• Proximal Policy Optimization Algorithms (PPO)[11]

You can find more information on each agent in thedoc.

• Install Keras-RL from Pypi (recommended):

pip install keras-rl

• Install from Github source:

git clone https://github.com/keras-rl/keras-rl.gitcd keras-rlpython setup.py install

If you want to run the examples, you'll also have to install:

• gym by OpenAI:Installation instruction
• h5py: simply runpip install h5py

For atari example you will also need:

• Pillow:pip install Pillow
• gym[atari]: Atari module for gym. Usepip install gym[atari]

Once you have installed everything, you can try out a simple example:

python examples/dqn_cartpole.py

This is a very simple example and it should converge relatively quickly, so it's a great way to get started!It also visualizes the game during training, so you can watch it learn. How cool is that?

Some sample weights are available onkeras-rl-weights.

If you have questions or problems, please file an issue or, even better, fix the problem yourself and submit a pull request!

• Starcraft II Learning Environment

You're using Keras-RL on a project? Open a PR and share it!

To see graphs of your training progress and compare across runs, runpip install wandb and add the WandbLogger callback to your agent'sfit() call:

fromrl.callbacksimportWandbLogger...agent.fit(env,nb_steps=50000,callbacks=[WandbLogger()])

For more info and options, see theW&B docs.

If you usekeras-rl in your research, you can cite it as follows:

@misc{plappert2016kerasrl,author ={Matthias Plappert},title ={keras-rl},year ={2016},publisher ={GitHub},journal ={GitHub repository},howpublished ={\url{https://github.com/keras-rl/keras-rl}},}

1. Playing Atari with Deep Reinforcement Learning, Mnih et al., 2013
2. Human-level control through deep reinforcement learning, Mnih et al., 2015
3. Deep Reinforcement Learning with Double Q-learning, van Hasselt et al., 2015
4. Continuous control with deep reinforcement learning, Lillicrap et al., 2015
5. Asynchronous Methods for Deep Reinforcement Learning, Mnih et al., 2016
6. Continuous Deep Q-Learning with Model-based Acceleration, Gu et al., 2016
7. Learning Tetris Using the Noisy Cross-Entropy Method, Szita et al., 2006
8. Deep Reinforcement Learning (MLSS lecture notes), Schulman, 2016
9. Dueling Network Architectures for Deep Reinforcement Learning, Wang et al., 2016
10. Reinforcement learning: An introduction, Sutton and Barto, 2011
11. Proximal Policy Optimization Algorithms, Schulman et al., 2017