Repository files navigation

NVIDIA NIM Operator is a Kubernetes Operator that is designed to facilitate the deployment, management, and scaling of NVIDIA NIM microservices on Kubernetes clusters.

NVIDIA NIM microservices deliver AI foundation models as accelerated inference microservices that are portable across data center, workstation, and cloud, accelerating flexible generative AI development, deployment and time to value.

To use the Operator in your cluster, refer todocsfor installation and configuration information.

• Kubernetes v1.28 and higher.
• NVIDIA GPUs that are supported by the NIM microservices to deploy.

Build and push your image to the location specified byIMAGE_NAME:VERSION:

make build IMAGE_NAME=<your-private-registry>/k8s-nim-operator VERSION=<tag> -f deployments/container/Makefile

Publish the image to a personal registry.You must be able to pull the image from the working environment.Make sure you have the proper permission to the registry if the preceding commands result in an error.

Install the CRDs into the cluster:

make install

Deploy the manager to the cluster with the image specified byIMG:

make deploy IMG=<your-private-registry>/k8s-nim-operator:<tag>

If you encounter RBAC errors, you might need to grant yourself cluster-adminprivileges or be logged in as admin.Ensure that the samples have default values.

Follow the guides in thedocs to deploy sample CR instances.

Delete the instances (CRs) from the cluster:

kubectl delete -k config/samples/

Delete the APIs (CRDs) from the cluster:

make uninstall

Undeploy the controller from the cluster:

make undeploy

Perform the following steps to build the installation manifests and distribute this project to users.

1. Build the manifests for the image built and published in the registry:

   make build-installer IMG=<your-private-registry>/k8s-nim-operator:<tag>

   The preceding Makefile target generates adist/install.yaml file.This file is built with Kustomize and contains the manifests for the CRDs and resources that are necessary to install this project withoutits dependencies.

2. Run the installer:

   kubectl apply -f https://raw.githubusercontent.com/<org>/k8s-nim-operator/<tag or branch>/dist/install.yaml

NVIDIA can work with partners to add platform support for the NIM Operator.The NIM Operator is open-source and permissivelylicensed with only minimal requirements for source codecontributions.

To get started with building the NIM Operator, follow these steps:

git clone git@github.com:NVIDIA/k8s-nim-operator.gitcd k8s-nim-operatormake build IMAGE_NAME=<your-private-registry>/k8s-nim-operator VERSION=<tag> -f deployments/container/Makefile

Runmake help for more information about additionalmake targets.

More information can be found in theKubebuilder Documentation.