Repository files navigation

Warning:
This project is aprototype and isunstable during training. Use at your own risk and expect breaking changes.

Platform Support:RelayRL runs onMacOS, Linux, and Windows (x86_64). Some dependencies may require additional setup on Windows.

RelayRL is a high-performance, distributed reinforcement learning (RL) framework designed for research and experimentation. By using proven transport-layer communication protocols, RelayRL integrates a client-server architecture. It provides a robust Rust backend with Python bindings, enabling seamless integration with modern ML workflows and high-speed system-level orchestration.

RelayRL isprimarily designed for single-agent RL.
While it is theoretically possible to launch multiple agent and training server processes for multi-agent scenarios, the framework doesnot natively support scalable multi-agent orchestration at this time.

• Distributed RL Orchestration:Run agents and training servers as separate processes, communicating via gRPC or ZeroMQ.
• Python & Rust Interoperability:Natively supports built-in or custom Python PyTorch for RL algorithms and Rust for orchestration and performance-critical components.
• Extensible Algorithm/Environment Support:Plug in your own RL algorithms/environments in Python (seeexamples/ for Jupyter notebooks).
• Benchmarking & Profiling:Tools and scripts for measuring network and runtime performance.
• Configurable Experiments:JSON-based configuration for easy experiment management.

Install RelayRL Framework:

pip install relayrl_framework

Supported Platforms: MacOS, Linux, Windows (x86_64)

1. Clone the repository:

   git clone https://github.com/jrcalgo/RelayRL-prototype.gitcd RelayRL-prototype

2. Build the Rust framework:

   cd relayrl_frameworkcargo build --release

3. Install Python bindings (from therelayrl_framework directory):

   pip install torch==2.5.1pip install maturinmaturin develop --release

4. Run an example (seeexamples/):

   • Jupyter notebooks demonstrate how to use the Python API for training and evaluation.

See the examples' README!

• relayrl_framework/ — Core Rust framework and Python bindings
• examples/ — Example scripts and Jupyter notebooks (Python API usage)
• README.md — This file (project overview)
• CONTRIBUTING.md — OSS Contribution instructions for relayrl_framework crate
• relayrl_framework/README.md — Framework-level and development instructions
• examples/README.md — Python integration and API utilization instructions

• Improved training stability and error handling for gRPC and ZeroMQ implementations
• Cleaned up user-level APIs (RelayRLAgent, TrainingServer, ConfigLoader, etc.)
• Better documentation and API reference
• Benchmarking against known frameworks (RayRLLib, Stable-Baselines3, CleanRL, etc.)
• Preservation of most critical components/logic

• Improved multi-agent support and orchestration by enhancing component modularity
• Enhanced stability, scalability and error handling during training through comprehensive network error handling, backpressure, configurable components, etc.
• Real-time configuration updates and shutdown/init signals through lifecycle control
• Exposing scalable components for containerized automation
• Easing execution and external application integration through compilation into CLI executables
• More built-in RL algorithms available

This project is aprototype and isunstable during training.As of now, although both transport loops are functional, ZMQ is provably more stable than gRPC in training tasks.APIs and features are subject to change.
Contributions and feedback are welcome!

See contribution instructions!

Apache License 2.0