Repository files navigation

This project emphasizes fast, open-source toolchains for NPU devices including LLVM-based code generation. IRON contains a close-to-metal toolkit that empowers performance engineers to create fast and efficient designs for Ryzen™ AI NPUs powered by AI Engines. It provides Python APIs that enable developers to harness the unique architectural capabilities of AMD’s NPUs. However, this project is not intended to represent an end-to-end compilation flow for all application designs---it is designed to complement, not replace, mainstream NPU tooling for inference like theAMD Ryzen™ AI Software Platform. Targeting researchers and enthusiasts, IRON is designed to unlock the full potential of NPUs for a wide range of workloads, from machine learning to digital signal processing and beyond. This repository includes programming guides and examples demonstrating the APIs. Additionally, thePeano component extends the LLVM framework by adding support for the AI Engine processor as a target architecture, enabling integration with popular compiler frontends such asclang. Developers can leverage theAIE API header library to implement efficient vectorized AIE core code in C++ that can be compiled by Peano.

This repository contains anMLIR-based toolchain for AI Engine-enabled devices, such asAMD Ryzen™ AI andVersal™. This repository can be used to generate low-level configurations for the AI Engine portion of these devices. AI Engines are organized as a spatial array of tiles, where each tile contains AI Engine cores and/or memories. The spatial array is connected by stream switches that can be configured to route data between AI Engine tiles scheduled by their programmable Data Movement Accelerators (DMAs). This repository contains MLIR representations, with multiple levels of abstraction, to target AI Engine devices. This enables compilers and developers to program AI Engine cores, as well as describe data movements and array connectivity.

The IRON Python API for Ryzen™ AI NPUs is described in the following paper:

E. Hunhoff, J. Melber, K. Denolf, A. Bisca, S. Bayliss, S. Neuendorffer, J. Fifield, J. Lo, P. Vasireddy, P. James-Roxby, E. Keller. "Efficiency, Expressivity, and Extensibility in a Close-to-Metal NPU Programming Interface". In 33rd IEEE International Symposium On Field-Programmable Custom Computing Machines, May 2025.

These instructions will guide you through everything required for building and executing a program on the Ryzen™ AI NPU, starting from a fresh bare-bonesUbuntu 24.04 orUbuntu 24.10 install.

Be sure you have the latest BIOS on your laptop or mini-PC that enables the NPU. Seehere.

If starting fromUbuntu 24.04 you may need to update the Linux kernel to 6.11+ by installing the Hardware Enablement (HWE) stack:

sudo apt updatesudo apt install --install-recommends linux-generic-hwe-24.04sudo reboot

Turn off SecureBoot (Allows for unsigned drivers to be installed):BIOS → Security → Secure boot → Disable

1. Execute the scripted build process:

   This script will install package dependencies, build the xdna-driver and xrt packages, and install them.These steps requiresudo access.

   bash ./utils/build_drivers.sh

2. Reboot as directed after the script exits.

   sudo reboot

3. Check that the NPU is working if the device appears with xrt-smi:

   source /opt/xilinx/xrt/setup.shxrt-smi examine

   At the bottom of the output you should see:

   Devices presentBDF             :  Name------------------------------------[0000:66:00.1]  :  NPU Strix

   IRON requires thatxrt-smi be in your path.

1. Install the following packages needed for MLIR-AIE:

   # Python versions 3.10, 3.12 and 3.13 are currently supported by our wheelssudo apt install \build-essential clang clang-14 lld lld-14 cmake ninja-build python3-venv python3-pip

2. (Optional) Install opencv which is needed for vision programming examples:

   sudo apt install libopencv-dev python3-opencv

1. Clonethe mlir-aie repository:

   git clone https://github.com/Xilinx/mlir-aie.gitcd mlir-aie

2. Setup a virtual environment:

   python3 -m venv ironenvsource ironenv/bin/activatepython3 -m pip install --upgrade pip

3. Install IRON library, mlir-aie and llvm-aie compilers from wheels and dependencies:

   You can install a specific version ofmlir-aie from the release wheels. To see available versions, check out therelease page.

   # Install IRON library and mlir-aie from a specific release,# e.g., <verison> in the following command could be replaced with v1.1.3python3 -m pip install mlir_aie -f https://github.com/Xilinx/mlir-aie/releases/expanded_assets/<version>

   Alternatively, you can install the latest released version ofmlir-aie.

   # Get the latest release versionlatest_tag_with_v=$(curl -s"https://api.github.com/repos/Xilinx/mlir-aie/releases/latest"| jq -r'.tag_name')latest_tag="${latest_tag_with_v#v}"# Install IRON library and mlir-aie from the latest stable releasepython3 -m pip install mlir_aie==${latest_tag} -f https://github.com/Xilinx/mlir-aie/releases/expanded_assets/${latest_tag_with_v}

   For the latest wheels (not necessarily a release):

   # Install IRON library and mlir-aie from the latest wheelpython3 -m pip install mlir_aie -f https://github.com/Xilinx/mlir-aie/releases/expanded_assets/latest-wheels-2

   You will also need to install thellvm-aie wheel for the Peano compiler and themlir-aie python extras.

   # Install Peano from llvm-aie wheelpython3 -m pip install llvm-aie -f https://github.com/Xilinx/llvm-aie/releases/expanded_assets/nightly# Install MLIR Python ExtrasHOST_MLIR_PYTHON_PACKAGE_PREFIX=aie python3 -m pip install -r python/requirements_extras.txt

4. (Optional) Install Python packages required for development and testing:

   # Install Python requirements for development and testingpython3 -m pip install -r python/requirements_dev.txt# This installs the pre-commit hooks defined in .pre-commit-config.yamlpre-commit install

5. Setup environment

   source utils/env_setup.sh

6. (Optional) Install ML Python packages for ml programming examples:

   # Install Torch for ML examplespython3 -m pip install -r python/requirements_ml.txt

7. (Optional) Install Jupyter Notebook Python packages:

   # Install Jupyter Notebookpython3 -m pip install -r python/requirements_notebook.txt# This creates an ipykernel (for use in notebooks) using the ironenv venvpython3 -m ipykernel install --user --name ironenv# Only for Release v1.0 and non wheel-based installs:# The install generally captures in the $PYTHONPATH by the `env_setup.sh` script.# However, jupyter notebooks don't always get access to the PYTHONPATH (e.g., if they are run with# vscode) so we save the ${MLIR_AIE_INSTALL_DIR}/python in a .pth file in the site packages dir of the# ironenv venv; this allows the iron ipykernel to find the install dir regardless of if PYTHONPATH is# available or not.MLIR_AIE_INSTALL=`$(pip show mlir_aie| grep ^Location:| awk'{print $2}')/mlir_aie` \venv_site_packages=`python3 -c'import sysconfig; print(sysconfig.get_paths()["purelib"])'` \echo${MLIR_AIE_INSTALL}/python>$venv_site_packages/mlir-aie.pth

For your design of interest, for instance fromprogramming_examples, 2 steps are needed: (i) build the AIE design and then (ii) build the host code.

1. Goto the design of interest and run:

   make

2. Build host code and execute the design:

   make run

1. Continue to theIRON AIE Application Programming Guide

2. Additional MLIR-AIE documentation is available on thewebsite

3. AIE API header library documentation for single-core AIE programming in C++ is avaiablehere

4. If you are a university researcher or student and interested in trying these tools on our Ryzen™ AI AUP Cloud systems, please contact theAMD University Program

You may skip the Vitis™ installation step if you intend to only target AMD XDNA™/AIE-ML (AIE2) and AMD XDNA™ 2 (AIE2P) using our open-source single-core compilerPeano. Compiling withxchesscc is not supported without installing AMD Vitis™ AIE Essentials.

1. Install Vitis™ AIE Essentials fromRyzen AI Software 1.3 Early Access. We will assume you use the installation directory,/tools/ryzen_ai-1.3.0/vitis_aie_essentials.

   This is an early access lounge, you must register and be granted access at this time.

   1. Download VAIML Installer for Linux based compilation:ryzen_ai-1.3.0ea1.tgz

   2. Extract the required tools:

         tar -xzvf ryzen_ai-1.3.0ea1.tgzcd ryzen_ai-1.3.0   mkdir vitis_aie_essentials   mv vitis_aie_essentials*.whl vitis_aie_essentialscd vitis_aie_essentials   unzip vitis_aie_essentials*.whl

2. Set up an AI Engine license.

   1. Get a local license for AI Engine tools fromhttps://www.xilinx.com/getlicense.

   2. Copy your license file (Xilinx.lic) to your preferred location, e.g./opt/Xilinx.lic:

3. Setup your environment using the following script for Vitis™ for AIETools:

   #!/bin/bash################################################################################## Setup Vitis AIE Essentials#################################################################################export AIETOOLS_ROOT=/tools/ryzen_ai-1.3.0/vitis_aie_essentialsexport PATH=$PATH:${AIETOOLS_ROOT}/binexport LM_LICENSE_FILE=/opt/Xilinx.lic

Be sure you have the latest BIOS for your laptop or mini PC, this will ensure the NPU (sometimes referred to as IPU) is enabled in the system. You may need to manually enable the NPU:Advanced → CPU Configuration → IPU

NOTE: Some manufacturers only provide Windows executables to update the BIOS, please do this before installing Ubuntu.

IRON AIE Application Programming Guide

Device Descriptions

Building mlir-aie tools from source

MLIR Dialect and Compiler Documentation

Interested in contributing MLIR-AIE?Information for developers

Copyright© 2019-2024 Advanced Micro Devices, Inc