[](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest)

[](https://github.com/Unity-Technologies/ml-agents/blob/release_22/LICENSE.md)

[](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/LICENSE.md)

([latest release](https://github.com/Unity-Technologies/ml-agents/releases/tag/latest_release)) ([all releases](https://github.com/Unity-Technologies/ml-agents/releases))

**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents. We provide implementations (based on PyTorch) of state-of-the-art algorithms to enable game developers and hobbyists to easily train intelligent agents for 2D, 3D and VR/AR games. Researchers can also use the provided simple-to-use Python API to train Agents using reinforcement learning, imitation learning, neuroevolution, or any other methods. These trained agents can be used for multiple purposes, including controlling NPC behavior (in a variety of settings such as multi-agent and adversarial), automated testing of game builds and evaluating different game design decisions pre-release. The ML-Agents Toolkit is mutually beneficial for both game developers and AI researchers as it provides a central platform where advances in AI can be evaluated on Unity’s rich environments and then made accessible to the wider research and game developer communities.

- 17+[example Unity environments](Learning-Environment-Examples.md)

- 17+[example Unity environments](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Examples.html)

- Support for multiple environment configurations and training scenarios

- Flexible Unity SDK that can be integrated into your game or custom Unity scene

- Support for training single-agent, multi-agent cooperative, and multi-agent competitive scenarios via several Deep Reinforcement Learning algorithms (PPO, SAC, MA-POCA, self-play).

- Support for learning from demonstrations through two Imitation Learning algorithms (BC and GAIL).

- Quickly and easily add your own[custom training algorithm](Python-Custom-Trainer-Plugin.md) and/or components.

- Quickly and easily add your own[custom training algorithm](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-Custom-Trainer-Plugin.html) and/or components.

- Easily definable Curriculum Learning scenarios for complex tasks

- Train robust agents using environment randomization

- Flexible agent control with On Demand Decision Making

- Train using multiple concurrent Unity environment instances

- Utilizes the[Inference Engine](Inference-Engine.md) to provide native cross-platform support

- Unity environment[control from Python](Python-LLAPI.md)

- Wrap Unity learning environments as a[gym](Python-Gym-API.md) environment

- Wrap Unity learning environments as a[PettingZoo](Python-PettingZoo-API.md) environment

- Utilizes the[Inference Engine](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Inference-Engine.html) to provide native cross-platform support

- Unity environment[control from Python](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html)

- Wrap Unity learning environments as a[gym](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-Gym-API.html) environment

- Wrap Unity learning environments as a[PettingZoo](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-PettingZoo-API.html) environment

The table below shows our latest release, including our`develop` branch which is under active development and may be unstable. A few helpful guidelines:

- The[Versioning page](Versioning.md) overviews how we manage our GitHub releases and the versioning process for each of the ML-Agents components.

- The[Versioning page](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Versioning.html) overviews how we manage our GitHub releases and the versioning process for each of the ML-Agents components.

- The[Releases page](https://github.com/Unity-Technologies/ml-agents/releases) contains details of the changes between releases.

- The[Migration page](Migrating.md) contains details on how to upgrade from earlier releases of the ML-Agents Toolkit.

- The[Migration page](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Migrating.html) contains details on how to upgrade from earlier releases of the ML-Agents Toolkit.

- The`com.unity.ml-agents` package is[verified](https://docs.unity3d.com/2020.1/Documentation/Manual/pack-safe.html) for Unity 2020.1 and later. Verified packages releases are numbered 1.0.x.

|**Version**|**Release Date**|**Source**|**Documentation**|**Download**|**Python Package**|**Unity Package**|

|:-----------:|:---------------:|:----------:|:-----------------:|:------------:|:------------------:|:-----------------:|

|**Release22**|**October 5, 2024**|**[source](https://github.com/Unity-Technologies/ml-agents/tree/release_22)**|**[docs](https://unity-technologies.github.io/ml-agents/)**|**[download](https://github.com/Unity-Technologies/ml-agents/archive/release_22.zip)**|**[1.1.0](https://pypi.org/project/mlagents/1.1.0/)**|**[3.0.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@3.0/manual/index.html)**|

|**develop (unstable)**| --|[source](https://github.com/Unity-Technologies/ml-agents/tree/develop)|[docs](https://github.com/Unity-Technologies/ml-agents/tree/develop/com.unity.ml-agents/Documentation~/index.md)|[download](https://github.com/Unity-Technologies/ml-agents/archive/develop.zip)|--| --|

|**Version**|**Release Date**|**Source**|**Documentation**|**Download**|**Python Package**|**Unity Package**|

|:----------------------:|:-------------------:|:-----------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------:|:-----------------------------------------------------:|:-------------------------------------------------------------------------------------:|

|**Release23**|**August 15, 2025**|**[source](https://github.com/Unity-Technologies/ml-agents/tree/release_23)**|**[docs](https://docs.unity3d.com/Packages/com.unity.ml-agents@4.0/manual/index.html)**|**[download](https://github.com/Unity-Technologies/ml-agents/archive/release_23.zip)**|**[1.1.0](https://pypi.org/project/mlagents/1.1.0/)**|**4.0.0**|

|**develop (unstable)**|--|[source](https://github.com/Unity-Technologies/ml-agents/tree/develop)|[docs](https://github.com/Unity-Technologies/ml-agents/tree/develop/com.unity.ml-agents/Documentation~/index.md)|[download](https://github.com/Unity-Technologies/ml-agents/archive/develop.zip)|--| --|

*[Introduction to ML-Agents by Huggingface](https://huggingface.co/learn/deep-rl-course/en/unit5/introduction)

*[Community created ML-Agents projects](https://discussions.unity.com/t/post-your-ml-agents-project/816756)

*[ML-Agents models on Huggingface](https://huggingface.co/models?library=ml-agents)

*[Blog posts](Blog-posts.md)

*[Blog posts](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Blog-posts.html)

*[Discord](https://discord.com/channels/489222168727519232/1202574086115557446)

The ML-Agents Toolkit is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our[contribution guidelines](CONTRIBUTING.md) and[code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release_22/CODE_OF_CONDUCT.md).

The ML-Agents Toolkit is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our[contribution guidelines](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/CONTRIBUTING.html) and[code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/CODE_OF_CONDUCT.md).

For problems with the installation and setup of the ML-Agents Toolkit, or discussions about how to best setup or train your agents, please create a new thread on the[Unity ML-Agents discussion forum](https://discussions.unity.com/tag/ml-agents). Be sure to include as many details as possible to help others assist you effectively. If you run into any other problems using the ML-Agents Toolkit or have a specific feature request, please[submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues).

},

"source": [

"# ML-Agents Open a UnityEnvironment\n",

]

},

{

},

"source": [

"# ML-Agents Q-Learning with GridWorld\n",

]

},

{

"id":"pZhVRfdoyPmv"

},

"source": [

"The [GridWorld](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Examples.md#gridworld) Environment is a simple Unity visual environment. The Agent is a blue square in a 3x3 grid that is trying to reach a green __`+`__ while avoiding a red __`x`__.\n",

"The [GridWorld](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Examples.html#gridworld) Environment is a simple Unity visual environment. The Agent is a blue square in a 3x3 grid that is trying to reach a green __`+`__ while avoiding a red __`x`__.\n",

"\n",

"The observation is an image obtained by a camera on top of the grid.\n",

"\n",

},

"source": [

"# ML-Agents Use SideChannels\n",

]

},

{

"## Side Channel\n",

"\n",

"SideChannels are objects that can be passed to the constructor of a UnityEnvironment or the `make()` method of a registry entry to send non Reinforcement Learning related data.\n",

"More information available [here](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#communicating-additional-information-with-the-environment)\n",

"More information available [here](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#communicating-additional-information-with-the-environment)\n",

"\n",

"\n",

},

"source": [

"### Engine Configuration SideChannel\n",

"The [Engine Configuration Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n",

"The [Engine Configuration Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n",

]

},

},

"source": [

"### Environment Parameters Channel\n",

"The [Environment Parameters Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#environmentparameters) is used to modify environment parameters during the simulation.\n",

"The [Environment Parameters Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#environmentparameters) is used to modify environment parameters during the simulation.\n",

]

},

},

"source": [

"### Creating your own Side Channels\n",

]

},

{

},

"source": [

"# ML-Agents run with Stable Baselines 3\n",

]

},

{

and this project adheres to

[Semantic Versioning](http://semver.org/spec/v2.0.0.html).

- Upgraded to Inference Engine 2.2.1 (#6212)

Inference Engine supports[all Unity runtime platforms](https://docs.unity3d.com/Manual/PlatformSpecific.html).

Scripting Backends : Inference Engine is generally faster with**IL2CPP** than with**Mono** for Standalone builds. In the Editor, It is not possible to use Inference Engine with GPU device selected when Editor Graphics Emulation is set to**OpenGL(ES) 3.0 or 2.0 emulation**. Also there might be non-fatal build time errors when target platform includes Graphics API that does not support**Unity Compute Shaders**.

Scripting Backends : Inference Engine is generally faster with**IL2CPP** than with**Mono** for Standalone builds. In the Editor, It is not possible to use Inference Engine with GPU device selected when Editor Graphics Emulation is set to**OpenGL(ES) 3.0 or 2.0 emulation**. Also, there might be non-fatal build time errors when target platform includes Graphics API that does not support**Unity Compute Shaders**.

In cases when it is not possible to use compute shaders on the target platform, inference can be performed using**CPU** or**GPUPixel** Inference Engine backends.

The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in[TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and[SentisModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs).

The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in[TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and[SentisModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs).

If you wish to run inference on an externally trained model, you should use Inference Engine directly, instead of trying to run it through ML-Agents.