Repository files navigation

This repository contains software required for participation in the NeurIPS 2019 Challenge: Learn to Move - Walk Around. See more details about the challengehere. See full documentation of our reinforcement learning environmenthere. In this document we will give very basic steps to get you set up for the challenge!

Your task is to develop a controller for a physiologically plausible 3D human model to walk or run following velocity commands with minimum effort. You are provided with a human musculoskeletal model and a physics-based simulation environment, OpenSim. There will be three tracks:

1. Best performance
2. Novel ML solution
3. Novel biomechanical solution, where all the winners of each track will be awarded.

To model physics and biomechanics we useOpenSim - a biomechanical physics environment for musculoskeletal simulations.

We took into account comments from the last challenge and there are several changes:

• You can use experimental data (to greatly speed up learning process)
• We released the 3rd dimensions (the model can fall sideways)
• We added a prosthetic leg -- the goal is to solve a medical challenge on modeling how walking will change after getting a prosthesis. Your work can speed up design, prototying, or tuning prosthetics!

You haven't heard of NIPS 2017: Learning to run?Watch this video!

Anaconda is required to run our simulations. Anaconda will create a virtual environment with all the necessary libraries, to avoid conflicts with libraries in your operating system. You can get anaconda from herehttps://docs.anaconda.com/anaconda/install/. In the following instructions we assume that Anaconda is successfully installed.

For the challenge we preparedOpenSim binaries as a conda environment to make the installation straightforward

We support Windows, Linux, and Mac OSX (all in 64-bit). To install our simulator, you first need to create a conda environment with the OpenSim package.

OnWindows, open a command prompt and type:

conda create -n opensim-rl -c kidzik -c conda-forge opensim python=3.6.1activate opensim-rlpip install osim-rl

OnLinux/OSX, run:

conda create -n opensim-rl -c kidzik -c conda-forge opensim python=3.6.1source activate opensim-rlpip install osim-rl

These commands will create a virtual environment on your computer with the necessary simulation libraries installed. If the commandpython -c "import opensim" runs smoothly, you are done! Otherwise, please refer to ourFAQ section.

Note thatsource activate opensim-rl activates the anaconda virtual environment. You need to type it every time you open a new terminal.

To execute 200 iterations of the simulation enter thepython interpreter and run the following:

fromosim.envimportL2M2019Envenv=L2M2019Env(visualize=True)observation=env.reset()foriinrange(200):observation,reward,done,info=env.step(env.action_space.sample())

The functionenv.action_space.sample() returns a random vector for muscle activations, so, in this example, muscles are activated randomly (red indicates an active muscle and blue an inactive muscle). Clearly with this technique we won't go too far.

Your goal is to construct a controller, i.e. a function from the state space (current positions, velocities and accelerations of joints) to action space (muscle excitations), that will enable to model to travel as far as possible in a fixed amount of time. Suppose you trained a neural network mapping observations (the current state of the model) to actions (muscle excitations), i.e. you have a functionaction = my_controller(observation), then

# ...total_reward=0.0foriinrange(200):# make a step given by the controller and record the state and the rewardobservation,reward,done,info=env.step(my_controller(observation))total_reward+=rewardifdone:break# Your reward isprint("Total reward %f"%total_reward)

You can find details about theobservation object here.

• Option 1:submit solution in docker container
• Option 2: run controller on server environment:./examples/submission.py

In order to make a submission to AIcrowd, please refer tothis page

Organizers reserve the right to modify challenge rules as required.

• Osim-rl interface
• How to train a model?
• More on training models
• Experimental data
• Physics underlying the model
• Frequently Asked Questions
• Citing and credits
• OpenSim documentation

• Understanding the Challenge - Great materials from@seungjaeryanlee on how to start