Dynamically allocate devices to workloads with DRA

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

You can flexibly request devices for your Google Kubernetes Engine (GKE) workloadsby usingdynamic resource allocation (DRA).This document shows you how to create a ResourceClaimTemplate to requestdevices, and then create a workload to observe how Kubernetes flexibly allocatesthe devices to your Pods.

This document is intended forApplication operatorsandData engineerswho run workloads like AI/ML or high performance computing (HPC).

About requesting devices with DRA

When you set up your GKE infrastructure for DRA, the DRA driverson your nodes createDeviceClass objects in the cluster.A DeviceClass defines a category of devices, such as GPUs, that are available to request forworkloads.A platform administrator can optionally deploy additional DeviceClasses thatlimit which devices you can request in specific workloads.

To request devices within a DeviceClass, you create one of the following objects:

ResourceClaim: A ResourceClaim lets a Pod or a user request hardware resources by filtering for certain parameterswithin a DeviceClass.
ResourceClaimTemplate: A ResourceClaimTemplate defines a template that Pods can use to automatically create new per-PodResourceClaims.

For more information about ResourceClaims and ResourceClaimTemplates,seeWhen to use ResourceClaims and ResourceClaimTemplates.

The examples on this page use a basic ResourceClaimTemplate to request thespecified device configuration. For more information about all of the fieldsthat you can specify, see theResourceClaimTemplate API reference.

Limitations

Node auto-provisioning isn't supported.
Autopilot clusters don't support DRA.
You can't use the following GPU sharing features:
- Time-sharing GPUs
- Multi-instance GPUs
- Multi-process Service (MPS)

Requirements

To use DRA, your GKE version must be version 1.34or later.

You should also be familiar with the following requirements and limitations:

Before you begin

Before you start, make sure that you have performed the following tasks:

Enable the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

If you want to use the Google Cloud CLI for this task,install and theninitialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running thegcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.Note: For existing gcloud CLI installations, make sure to set thecompute/regionproperty. If you use primarily zonal clusters, set thecompute/zone instead. By setting a default location, you can avoid errors in the gcloud CLI like the following:One of [--zone, --region] must be supplied: Please specify location. You might need to specify the location in certain commands if the location of your cluster differs from the default that you set.

Ensure that yourGKE clusters are configured for DRA workloads.

Use DRA to deploy workloads

To request per-Pod device allocation, you create a ResourceClaimTemplate thathas your requested device configuration, such as GPUs of a specific type. Whenyou deploy a workload that references the ResourceClaimTemplate, Kubernetescreates ResourceClaims for each Pod in the workload based on theResourceClaimTemplate. Kubernetes allocates the requested resources andschedules the Pods on corresponding nodes.

To request devices in a workload with DRA, select one of the following options:

GPU

Save the following manifest asclaim-template.yaml:

apiVersion:resource.k8s.io/v1kind:ResourceClaimTemplatemetadata:name:gpu-claim-templatespec:spec:devices:requests:-name:single-gpuexactly:deviceClassName:gpu.nvidia.comallocationMode:ExactCountcount:1

Create the ResourceClaimTemplate:
```
kubectlcreate-fclaim-template.yaml
```

To create a workload that references the ResourceClaimTemplate, savethe following manifest asdra-gpu-example.yaml:

apiVersion:apps/v1kind:Deploymentmetadata:name:dra-gpu-examplespec:replicas:1selector:matchLabels:app:dra-gpu-exampletemplate:metadata:labels:app:dra-gpu-examplespec:containers:-name:ctrimage:ubuntu:22.04command:["bash","-c"]args:["echo$(nvidia-smi-L||echoWaiting...)"]resources:claims:-name:single-gpuresourceClaims:-name:single-gpuresourceClaimTemplateName:gpu-claim-templatetolerations:-key:"nvidia.com/gpu"operator:"Exists"effect:"NoSchedule"

Deploy the workload:
```
kubectlcreate-fdra-gpu-example.yaml
```

TPU

Save the following manifest asclaim-template.yaml:

apiVersion:resource.k8s.io/v1kind:ResourceClaimTemplatemetadata:name:tpu-claim-templatespec:spec:devices:requests:-name:all-tpusexactly:deviceClassName:tpu.google.comallocationMode:All

This ResourceClaimTemplate requests all TPUs, so all TPUs on a node areallocated to each resulting ResourceClaim.

Create the ResourceClaimTemplate:
```
kubectlcreate-fclaim-template.yaml
```

To create a workload that references the ResourceClaimTemplate, savethe following manifest asdra-tpu-example.yaml:

apiVersion:apps/v1kind:Deploymentmetadata:name:dra-tpu-examplespec:replicas:1selector:matchLabels:app:dra-tpu-exampletemplate:metadata:labels:app:dra-tpu-examplespec:containers:-name:ctrimage:ubuntu:22.04command:-/bin/sh--c-|echo "Environment Variables:"envecho "Sleeping indefinitely..."sleep infinityresources:claims:-name:all-tpusresourceClaims:-name:all-tpusresourceClaimTemplateName:tpu-claim-templatetolerations:-key:"google.com/tpu"operator:"Exists"effect:"NoSchedule"

Deploy the workload:
```
kubectlcreate-fdra-tpu-example.yaml
```

Verify the hardware allocation

You can verify that your workloads have been allocated hardware by checkingthe ResourceClaim or by looking at the logs for your Pod. To verify theallocation for GPUs or TPUs, select one of the following options: