Build the generate() API from source

• Pre-requisites
• Clone the onnxruntime-genai repo
• Build the generate() API
  • Build Python API
    • Windows CPU build
    • Windows DirectML build
    • Windows NvTensorRtRtx build
    • Linux build
    • Linux CUDA build
    • Mac build
  • Build Java API
  • Build for Android
• Install the library into your application
  • Install Python wheel
  • Install NuGet
  • Install JAR
  • Install AAR
  • Install C/C++ header file and library
    • Windows
    • Linux

Pre-requisites

• cmake
• .NET6 (if building C#)

Clone the onnxruntime-genai repo

git clone https://github.com/microsoft/onnxruntime-genaicdonnxruntime-genai

Build the generate() API

This step assumes that you are in the root of the onnxruntime-genai repo.

All of the build commands below have a--config argument, which takes the following options:

• Release builds release binaries
• Debug build binaries with debug symbols
• RelWithDebInfo builds release binaries with debug info

Build Python API

Windows CPU build

python build.py--config Release

Windows DirectML build

python build.py--use_dml--config Release

Windows NvTensorRtRtx build

python build.py--use_trt_rtx--config Release--cuda_home <cuda_path>

Linux build

python build.py--config Release

Linux CUDA build

python build.py--use_cuda--config Release

Mac build

python build.py--config Release

Build Java API

python build.py--build_java--config Release

Build for Android

If building on Windows, installninja.

pipinstallninja

Run the build script.

python build.py--build_java--android--android_home <path to your Android SDK>--android_ndk_path <path to your NDK installation>--android_abi[armeabi-v7a|arm64-v8a|x86|x86_64]--config Release

Install the library into your application

Install Python wheel

# Change dir to the folder containing the onnxruntime-genai wheel# Example for Linux: cd build/Linux/Release/wheel/pipinstall*.whl

Install NuGet

Coming soon

Install JAR

Copybuild/Windows/Release/src/java/build/libs/*.jar into your application.

Install AAR

Copybuild/Android/Release/src/java/build/android/outputs/aar/onnxruntime-genai-release.aar into your application.

Install C/C++ header file and library

Windows

Use the header insrc\ort_genai.h and the libraries inbuild\Windows\Release

Linux

Use the header insrc/ort_genai.h and the libraries inbuild/Linux/Release