Repository files navigation

Acme is a library of reinforcement learning (RL) building blocks that strives toexpose simple, efficient, and readable agents. These agents first and foremostserve both as reference implementations as well as providing strong baselinesfor algorithm performance. However, the baseline agents exposed by Acme shouldalso provide enough flexibility and simplicity that they can be used as astarting block for novel research. Finally, the building blocks of Acme aredesigned in such a way that the agents can be run at multiple scales (e.g.single-stream vs. distributed agents).

The quickest way to get started is to take a look at the detailed working codeexamples found in theexamples subdirectory. These show how to instantiate anumber of different agents and run them within a variety of environments. Seethequickstart notebook for an even quicker dive into using asingle agent. Even more detail on the internal construction of an agent can befound inside ourtutorial notebook. Finally, a full description Acmeand its underlying components can be found by referring to thedocumentation.More background information and details behind the design decisions can be foundin ourtechnical report.

NOTE: Acme is first and foremost a framework for RL research written byresearchers, for researchers. We use it for our own work on a daily basis. Sowith that in mind, while we will make every attempt to keep everything in goodworking order, things may break occasionally. But if so we will make our besteffort to fix them as quickly as possible!

To get up and running quickly just follow the steps below:

1. While you can install Acme in your standard python environment, westrongly recommend using aPython virtual environmentto manage your dependencies. This should help to avoid version conflicts andjust generally make the installation process easier.

   python3 -m venv acmesource acme/bin/activatepip install --upgrade pip setuptools wheel

2. While the coredm-acme library can be pip installed directly, the set ofdependencies included for installation is minimal. In particular, to run anyof the included agents you will also need eitherJAX orTensorFlowdepending on the agent. As a result we recommend installing these componentsas well, i.e.

   pip install dm-acme[jax,tf]

3. Finally, to install a few example environments (includinggym,dm_control, andbsuite):

   pip install dm-acme[envs]

4. Installing from github: if you're interested in running thebleeding-edge version of Acme, you can do so by cloning the Acme GitHubrepository and then executing following command from the main directory(wheresetup.py is located):

   pip install .[jax,tf,testing,envs]

If you use Acme in your work, please cite the updated accompanyingtechnical report:

@article{hoffman2020acme,title={Acme: A Research Framework for Distributed Reinforcement Learning},author={        Matthew W. Hoffman and Bobak Shahriari and John Aslanides and        Gabriel Barth-Maron and Nikola Momchev and Danila Sinopalnikov and        Piotr Sta\'nczyk and Sabela Ramos and Anton Raichuk and        Damien Vincent and L\'eonard Hussenot and Robert Dadashi and        Gabriel Dulac-Arnold and Manu Orsini and Alexis Jacq and        Johan Ferret and Nino Vieillard and Seyed Kamyar Seyed Ghasemipour and        Sertan Girgin and Olivier Pietquin and Feryal Behbahani and        Tamara Norman and Abbas Abdolmaleki and Albin Cassirer and        Fan Yang and Kate Baumli and Sarah Henderson and Abe Friesen and        Ruba Haroun and Alex Novikov and Sergio G\'omez Colmenarejo and        Serkan Cabi and Caglar Gulcehre and Tom Le Paine and        Srivatsan Srinivasan and Andrew Cowie and Ziyu Wang and Bilal Piot and        Nando de Freitas},year={2020},journal={arXiv preprint arXiv:2006.00979},url={https://arxiv.org/abs/2006.00979},}