Repository files navigation

An introductory tutorial about leveragingRaycore features for distributed patterns.

These examples have been tested in the following environments:

• Ubuntu 18.04 LTS
• macOS 11.6, Big Sur

Using:

• Ray versions 1.7+
• Python versions: 3.6, 3.7, 3.8

See theslides.pdf file for the presentation slide deck thataccompanies this tutorial.

To get started usegit to clone this public repository:

git clone https://github.com/DerwenAI/ray_tutorial.gitcd ray_tutorial

Set up a localvirtual environmentand activate it:

python3 -m venv venvsource venv/bin/activate

Then usepip to install the required dependencies:

python3 -m pip install -U pippython3 -m pip install -r requirements.txtpython3 -m ipykernel install

Alternatively, if you useconda for installing Python packages:

conda create -n ray_tutorial python=3.7conda activate ray_tutorialpython3 -m pip install -r requirements.txtconda install ipykernel --name Python3

Note: if you run into any problems on Python 3.8 with "wheels"during apip installation, you may need to use thecondaapproach instead.

For some of the visualizations inpi.ipynb you also need toinstall graphviz

Then launch theJupyterLabenvironment to run examples in this repo:

jupyter-lab

Browse tohttp://localhost:8888/lab to continue.

First, installdockeranddocker-compose,then:

docker-compose up -d

Docker compose will start a JupyterLab service without requiring useof a security token.

Browse tohttp://localhost:8888/lab to continue.

To stop this container:

docker-compose stop

A Guided Tour of Ray Core covers an introductory, hands-on codingtour through the core features of Ray, which provide powerful yeteasy-to-use design patterns for implementing distributed systems inPython. This training includes a brief talk to provide overview ofconcepts, then coding for remote functions, tasks, object references andresolutions, actors, and so on.

Then we'll follow with Q&A. All code is available in notebooks in the GitHub repo.

• Python developers who want to learn how to parallelize their application code

Note: this material is not intended as an introduction to the higherlevel components in Ray, such as RLlib and Ray Tune.

• Some prior experience developing code in Python
• Basic understanding of distributed systems

• What are the Ray core features and how to use them?
• In which contexts are the different approaches indicated?
• Profiling methods, to decide when to make trade-offs (compute cost, memory, I/O, etc.) ?

1. Introduction to Ray core features as apattern language for distributed systems
2. Overview of the main Ray core features and their intended usage
3. Background, primary sources, and closely related resources about distributed systems
4. Code samples:
   • Remote Functions:ex_01_remo_func.ipynb
   • Remote Functions:ex_02_remo_func.ipynb
   • Remote Objects:ex_02_remo_objs.ipynb
   • Remote Methods:ex_03_remo_meth.ipynb
   • Multiprocessing Pool:ex_04_mult_pool.ipynb
   • JobLib:ex_05_job_lib.ipynb
5. Profiling: comparing trade-offs and overhead
   • Estimate Pi:pi.ipynb
6. Ray Summit, Anyscale Connect, developer forums, and other resources
7. Q&A

• Ray Design Patterns
• UC Berkeley EECS"Our Pattern Language"
• "Futures and Promises"
• "Ray Distributed Library Patterns"
• python-patterns
• Python Data Science
• "Profiling and Optimizing Jupyter Notebooks"
• Anyscale Academy
• Ray Tutorial

@dmatrix,@penolove,@deanwampler,@ceteri.