Repository files navigation

Getting Started |Docs |Baseline Results |Changelist

Dopamine is a research framework for fast prototyping of reinforcement learningalgorithms. It aims to fill the need for a small, easily grokked codebase inwhich users can freely experiment with wild ideas (speculative research).

Our design principles are:

• Easy experimentation: Make it easy for new users to run benchmarkexperiments.
• Flexible development: Make it easy for new users to try out research ideas.
• Compact and reliable: Provide implementations for a few, battle-testedalgorithms.
• Reproducible: Facilitate reproducibility in results. In particular, oursetup follows the recommendations given byMachado et al. (2018).

Dopamine supports the following agents, implemented with jax:

• DQN (Mnih et al., 2015)
• C51 (Bellemare et al., 2017)
• Rainbow (Hessel et al., 2018)
• IQN (Dabney et al., 2018)
• SAC (Haarnoja et al., 2018)
• PPO (Schulman et al., 2017)

For more information on the available agents, see thedocs.

Many of these agents also have a tensorflow (legacy) implementation, thoughnewly added agents are likely to be jax-only.

This is not an official Google product.

We provide docker containers for using Dopamine.Instructions can be foundhere.

Alternatively, Dopamine can be installed from source (preferred) or installedwith pip. For either of these methods, continue reading at prerequisites.

Dopamine supports Atari environments and Mujoco environments. Install theenvironments you intend to use before you install Dopamine:

Atari

1. These should now come packaged withale_py.
2. You may need to manually run some steps to properly installbaselines, seeinstructions.

Mujoco

1. Install Mujoco and get a licensehere.
2. Runpip install mujoco-py (we recommend using avirtual environment).

The most common way to use Dopamine is to install it from source and modifythe source code directly:

git clone https://github.com/google/dopamine

After cloning, install dependencies:

pip install -r dopamine/requirements.txt

Dopamine supports tensorflow (legacy) and jax (actively maintained) agents.View theTensorflow documentation formore information on installing tensorflow.

Note: We recommend using avirtual environment when working with Dopamine.

Note: We strongly recommend installing from source for most users.

Installing with pip is simple, but Dopamine is designed to be modifieddirectly. We recommend installing from source for writing your own experiments.

pip install dopamine-rl

You can test whether the installation was successful by running the followingfrom the dopamine root directory.

export PYTHONPATH=$PYTHONPATH:$PWDpython -m tests.dopamine.atari_init_test

View thedocs for more information on training agents.

We supplybaselines for each Dopamine agent.

We also provide a set ofColaboratory notebookswhich demonstrate how to use Dopamine.

Bellemare et al.,The Arcade Learning Environment: An evaluation platform forgeneral agents. Journal of Artificial Intelligence Research, 2013.

Machado et al.,Revisiting the Arcade Learning Environment: EvaluationProtocols and Open Problems for General Agents, Journal of ArtificialIntelligence Research, 2018.

Hessel et al.,Rainbow: Combining Improvements in Deep Reinforcement Learning.Proceedings of the AAAI Conference on Artificial Intelligence, 2018.

Mnih et al.,Human-level Control through Deep Reinforcement Learning. Nature,2015.

Schaul et al.,Prioritized Experience Replay. Proceedings of the InternationalConference on Learning Representations, 2016.

Haarnoja et al.,Soft Actor-Critic Algorithms and Applications,arXiv preprint arXiv:1812.05905, 2018.

Schulman et al.,Proximal Policy Optimization Algorithms.

If you use Dopamine in your work, we ask that you cite ourwhite paper. Here is an example BibTeX entry:

@article{castro18dopamine,  author    = {Pablo Samuel Castro and               Subhodeep Moitra and               Carles Gelada and               Saurabh Kumar and               Marc G. Bellemare},  title     = {Dopamine: {A} {R}esearch {F}ramework for {D}eep {R}einforcement {L}earning},  year      = {2018},  url       = {http://arxiv.org/abs/1812.06110},  archivePrefix = {arXiv}}