Create GPU VMs in bulk
To learn more about bulk creation, seeAbout bulk creation of VMs.To learn more about creating VMs with attached GPUs, seeOverview of creating an instance with attached GPUs.
Before you begin
- To review limitations and additional prerequisite steps for creatinginstances with attached GPUs, such as selecting an OS image and checkingGPU quota, seeOverview of creating an instance with attached GPUs.
- To review limitations for bulk creation, seeAbout bulk creation of VMs.
- If you haven't already, set upauthentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
gcloud
Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:
gcloudinit
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.- Set a default region and zone.
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:
gcloudinit
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.
Required roles
To get the permissions that you need to create VMs, ask your administrator to grant you theCompute Instance Admin (v1) (roles/compute.instanceAdmin.v1) IAM role on the project. For more information about granting roles, seeManage access to projects, folders, and organizations.
This predefined role contains the permissions required to create VMs. To see the exact permissions that are required, expand theRequired permissions section:
Required permissions
The following permissions are required to create VMs:
compute.instances.createon the project- To use a custom image to create the VM:
compute.images.useReadOnlyon the image - To use a snapshot to create the VM:
compute.snapshots.useReadOnlyon the snapshot - To use an instance template to create the VM:
compute.instanceTemplates.useReadOnlyon the instance template - To specify a subnet for your VM:
compute.subnetworks.useon the project or on the chosen subnet - To specify a static IP address for the VM:
compute.addresses.useon the project - To assign an external IP address to the VM when using a VPC network:
compute.subnetworks.useExternalIpon the project or on the chosen subnet - To assign alegacy network to the VM:
compute.networks.useon the project - To assign an external IP address to the VM when using a legacy network:
compute.networks.useExternalIpon the project - To set VM instance metadata for the VM:
compute.instances.setMetadataon the project - To set tags for the VM:
compute.instances.setTagson the VM - To set labels for the VM:
compute.instances.setLabelson the VM - To set a service account for the VM to use:
compute.instances.setServiceAccounton the VM - To create a new disk for the VM:
compute.disks.createon the project - To attach an existing disk in read-only or read-write mode:
compute.disks.useon the disk - To attach an existing disk in read-only mode:
compute.disks.useReadOnlyon the disk
You might also be able to get these permissions withcustom roles or otherpredefined roles.
Overview
When creating VMs with attached GPUs using the bulk creation method, you can chooseto create VMs in a region (such asus-central1) or in a specific zone such as(us-central1-a).
If you choose to specify a region, Compute Engine places the VMs inany zone within the region that supports GPUs.
Machine types
The accelerator-optimized machine family contains multiple machine types.
Each accelerator-optimized machine type has a specific model of NVIDIA GPUsattached to support the recommended workload type.
| AI and ML workloads | Graphics and visualization |
|---|---|
| Accelerator-optimized A series machine types are designed for high performance computing (HPC), artificial intelligence (AI), and machine learning (ML) workloads. For these machine types, the GPU model is automatically attached to the instance. | Accelerator-optimized G series machine types are designed for workloads such as NVIDIA Omniverse simulation workloads, graphics-intensive applications, video transcoding, and virtual desktops. These machine types supportNVIDIA RTX Virtual Workstations (vWS). For these machine types, the GPU model is automatically attached to the instance. |
|
Create groups of A4X, A4, and A3 Ultra
To create instances in bulk for the A4X, A4, and A3 Ultra machine series, seetheDeployment options overviewin the AI Hypercomputer documentation.
Create groups of A3, A2, G4, and G2 VMs
This section explains you can create instances in bulk for the A3 High, A3 Mega,A3 Edge, A2 Standard, A2 Ultra, G4, and G2 machine series by usingGoogle Cloud CLI, orREST.
gcloud
To create a group of VMs, use thegcloud compute instances bulk createcommand. For moreinformation about the parameters and how to use this command, seeCreate VMs in bulk.
Example
This example creates two VMs that have attached GPUs by using the followingspecifications:
- VM names:
my-test-vm-1,my-test-vm-2 - Each VM has two GPUs attached, specified by usingthe appropriateaccelerator-optimized machine type
gcloud compute instances bulk create \ --name-pattern="my-test-vm-#" \ --region=REGION \ --count=2 \ --machine-type=MACHINE_TYPE \ --boot-disk-size=200 \ --image=IMAGE \ --image-project=IMAGE_PROJECT \ --on-host-maintenance=TERMINATE
Replace the following:
REGION: the region for the VMs. This region mustsupportyour selected accelerator-optimized machine type.MACHINE_TYPE: the machine type that you selected.Choose from one of the following:- AnA3 High machine type
- AnA3 Mega machine type
- AnA3 Edge machine type
- AnA2 Standard machine type
- AnA2 Ultra machine type
- AG4 machine type
- AG2 machine typeG2 machine types also support custom memory. Memory must be a multipleof 1024 MB and within the supported memory range. For example, tocreate a VM with 4 vCPUs and 19 GB of memory specify
--machine-type=g2-custom-4-19456.
IMAGE: an operating system imagethat supports GPUs.If you want to use the latest image in animage family, replace the
--imageflag with the--image-familyflag and setits value to an image family that supports GPUs. For example:--image-family=rocky-linux-8-optimized-gcp.You can also specify a custom image orDeep Learning VM Images.
IMAGE_PROJECT: theCompute Engineimage projectthat the OS image belongs to. If using a custom image orDeep Learning VM Images, specify the project that those images belong to.VWS_ACCELERATOR_COUNT: the number of virtual GPUsthat you need.
If successful, the output is similar to the following:
NAME ZONEmy-test-vm-1 us-central1-bmy-test-vm-2 us-central1-bBulk create request finished with status message: [VM instances created: 2, failed: 0.]
Optional flags
To further configure your instance to meet your workload or operating system needs, include oneor more of the following flags when you run thegcloud compute instances bulk create command.
| Feature | Description |
|---|---|
| Provisioning model | Sets the provisioning model for the instance. Specify eitherSPOT orFLEX_START.FLEX_START isn't supported for G4 instances. If you don't specify a model, then the standard model is used. For more information, see Compute Engine instances provisioning models.--provisioning-model=PROVISIONING_MODEL |
| Virtual workstation | Specifies anNVIDIA RTX Virtual Workstations (vWS) for graphics workloads. This feature is supported only for G4 and G2 instances.--accelerator=type=VWS_ACCELERATOR_TYPE,count=VWS_ACCELERATOR_COUNT Replace the following:
|
| Local SSD | Attaches one or more Local SSDs to your instance. Local SSDs can be used for fast scratch disks or for feeding data into the GPUs while preventing I/O bottlenecks.--local-ssd=interface=nvme \ --local-ssd=interface=nvme \ --local-ssd=interface=nvme ... |
| Network interface | Attaches multiple network interfaces to your instance. Forg4-standard-384 instances, you can attach up to two network interfaces. You can use this flag to create an instance with dual network interfaces (2x 200 Gbps). Each network interface must be in a unique VPC network.--network-interface=network=VPC_NAME_1,subnet=SUBNET_NAME_1,nic-type=GVNIC \ --network-interface=network=VPC_NAME_2,subnet=SUBNET_NAME_2,nic-type=GVNIC Dual network interfaces are only supported on Replace the following:
|
REST
Use theinstances.bulkInsertmethod with therequired parameters to create multiple VMs in a zone. For moreinformation about the parameters and how to use this command, seeCreate VMs in bulk.
Example
This example creates two VMs that have attached GPUs by using the followingspecifications:
- VM names:
my-test-vm-1,my-test-vm-2 Each VM has two GPUs attached, specified by usingthe appropriateaccelerator-optimized machine type
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/instances/bulkInsert{"namePattern":"my-test-vm-#","count":"2","instanceProperties": { "machineType":MACHINE_TYPE, "disks":[ { "type":"PERSISTENT", "initializeParams":{ "diskSizeGb":"200", "sourceImage":SOURCE_IMAGE_URI }, "boot":true } ], "name": "default", "networkInterfaces": [ { "network": "projects/PROJECT_ID/global/networks/default" } ], "scheduling":{ "onHostMaintenance":"TERMINATE", ["automaticRestart":true] }}}
Replace the following:
PROJECT_ID: your project IDREGION: the region for the VMs. This region mustsupportyour selected GPU model.MACHINE_TYPE: the machine type that you selected.Choose from one of the following:- AnA3 High machine type
- AnA3 Mega machine type
- AnA3 Edge machine type
- AnA2 Standard machine type
- AnA2 Ultra machine type
- AG4 machine type
- AG2 machine type.G2 machine types alsosupport custom memory. Memory must be a multiple of 1024 MB andwithin the supported memory range. For example, the machine type namefor an instance with 4 vCPUs and 19 GB of memory would be
g2-custom-4-19456.
SOURCE_IMAGE_URI: the URI for the specificimage or image familythat you want to use.For example:
- Specific image:
"sourceImage": "projects/rocky-linux-cloud/global/images/rocky-linux-8-optimized-gcp-v20220719" - Image family:
"sourceImage": "projects/rocky-linux-cloud/global/images/family/rocky-linux-8-optimized-gcp".
When you specify an image family, Compute Engine creates a VMfrom the most recent, non-deprecated OS image in that family. Formore information about when to use image families, seeImage family best practices.
- Specific image:
Optional flags
To further configure your instance to meet your workload or operating system needs, include one or moreof the following flags when you run theinstances.bulkInsert method.
| Feature | Description |
|---|---|
| Provisioning model | To lower your costs, you can specify a different provisioning model by adding the"provisioningModel": "PROVISIONING_MODEL" field to thescheduling object in your request. If you specify to create Spot VMs, then theonHostMaintenance andautomaticRestart fields are ignored. For more information, see Compute Engine instances provisioning models. "scheduling": { "onHostMaintenance": "terminate", "provisioningModel": "PROVISIONING_MODEL" }Replace
|
| Virtual workstation | Specifies anNVIDIA RTX Virtual Workstations (vWS) for graphics workloads. This feature is supported only for G4 and G2 instances. "guestAccelerators": [ { "acceleratorCount":VWS_ACCELERATOR_COUNT, "acceleratorType": "projects/PROJECT_ID/zones/ZONE/acceleratorTypes/VWS_ACCELERATOR_TYPE" } ]Replace the following:
|
| Local SSD | Attaches one or more Local SSDs to your instance. Local SSDs can be used for fast scratch disks or for feeding data into the GPUs while preventing I/O bottlenecks. { "type": "SCRATCH", "autoDelete": true, "initializeParams": { "diskType": "projects/PROJECT_ID/zones/ZONE/diskTypes/local-nvme-ssd" } } |
| Network interface | Attaches multiple network interfaces to your instance. Forg4-standard-384 instances, you can attach up to two network interfaces. This creates an instance with dual network interfaces (2x 200 Gbps). Each network interface must be in a unique VPC network. "networkInterfaces": [ { "network": "projects/PROJECT_ID/global/networks/VPC_NAME_1", "subnetwork": "projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME_1", "nicType": "GVNIC" }, { "network": "projects/PROJECT_ID/global/networks/VPC_NAME_2", "subnetwork": "projects/PROJECT_ID/regions/REGION/subnetworks/SUBNET_NAME_2", "nicType": "GVNIC" } ]Dual network interfaces are only supported on Replace the following:
|
Create groups of N1-general purpose VMs
You create a group of VMs with attached GPUs by using either theGoogle Cloud CLI, orREST.
This section describes how to create multiple VMs using the following GPU types:
NVIDIA GPUs:
- NVIDIA T4:
nvidia-tesla-t4 - NVIDIA P4:
nvidia-tesla-p4 - NVIDIA P100:
nvidia-tesla-p100 - NVIDIA V100:
nvidia-tesla-v100
NVIDIA RTX Virtual Workstation (vWS) (formerly known as NVIDIA GRID):
- NVIDIA T4 Virtual Workstation:
nvidia-tesla-t4-vws - NVIDIA P4 Virtual Workstation:
nvidia-tesla-p4-vws NVIDIA P100 Virtual Workstation:
nvidia-tesla-p100-vwsFor these virtual workstations, an NVIDIA RTX Virtual Workstation (vWS) licenseis automatically added to your instance.
gcloud
To create a group of VMs, use thegcloud compute instances bulk createcommand.For more information about the parameters and how to use this command, seeCreate VMs in bulk.
Example
The following example creates two VMs with attached GPUs usingthe following specifications:
- VM names:
my-test-vm-1,my-test-vm-2 - VMs created in any zone in
us-central1that supports GPUs - Each VM has two T4 GPUs attached, specified by usingthe accelerator type and accelerator count flags
- Each VM has GPU drivers installed
- Each VM uses the Deep Learning VM image
pytorch-latest-gpu-v20211028-debian-10
gcloud compute instances bulk create \ --name-pattern="my-test-vm-#" \ --count=2 \ --region=us-central1 \ --machine-type=n1-standard-2 \ --accelerator type=nvidia-tesla-t4,count=2 \ --boot-disk-size=200 \ --metadata="install-nvidia-driver=True" \ --scopes="https://www.googleapis.com/auth/cloud-platform" \ --image=pytorch-latest-gpu-v20211028-debian-10 \ --image-project=deeplearning-platform-release \ --on-host-maintenance=TERMINATE --restart-on-failure
If successful, the output is similar to the following:
NAME ZONEmy-test-vm-1 us-central1-bmy-test-vm-2 us-central1-bBulk create request finished with status message: [VM instances created: 2, failed: 0.]
REST
Use theinstances.bulkInsertmethod with therequired parameters to create multiple VMs in a zone.For more information about the parameters and how to use this command, seeCreate VMs in bulk.
Example
The following example creates two VMs with attached GPUs usingthe following specifications:
- VM names:
my-test-vm-1,my-test-vm-2 - VMs created in any zone in
us-central1that supports GPUs - Each VM has two T4 GPUs attached, specified by usingthe accelerator type and accelerator count flags
- Each VM has GPU drivers installed
- Each VM uses the Deep Learning VM image
pytorch-latest-gpu-v20211028-debian-10
ReplacePROJECT_ID with your project ID.
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/us-central1/instances/bulkInsert{ "namePattern":"my-test-vm-#", "count":"2", "instanceProperties": { "machineType":"n1-standard-2", "disks":[ { "type":"PERSISTENT", "initializeParams":{ "diskSizeGb":"200", "sourceImage":"projects/deeplearning-platform-release/global/images/pytorch-latest-gpu-v20211028-debian-10" }, "boot":true } ], "name": "default", "networkInterfaces": [ { "network": "projects/PROJECT_ID/global/networks/default" } ], "guestAccelerators": [ { "acceleratorCount":2, "acceleratorType": "nvidia-tesla-t4" } ], "scheduling":{ "onHostMaintenance":"TERMINATE", "automaticRestart":true }, "metadata":{ "items":[ { "key":"install-nvidia-driver", "value":"True" } ] } } }What's next?
Learn how tomonitor GPU performance.
Learn how touse higher network bandwidth.
Learn how tohandle GPU host maintenance events.
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-12-15 UTC.