Configuring stateful persistent disks in MIGs

Configuring persistent disks to bestateful lets you benefit from VMinstance autohealing and automated updates while preserving the state of thedisks.

You can configure any disk defined in the instance template to be statefulfor all instances in amanaged instance group (MIG)by adding that disk's device name to the MIG'sstateful policy.

You can also configure stateful persistent disks individually for instances in aMIG by settingper-instance configurations;these disks don't need to be defined in the instance template.

Before you begin

Limitations

A MIG withstateful configuration—astateful MIG—has the following limitations:

  • You cannot use autoscaling if your MIG has stateful configuration.
  • If you want to use automated rolling updates, you must set thereplacement methodtoRECREATE.
  • For stateful regional MIGs, you mustdisable proactive redistribution (set the redistribution type toNONE) to prevent deletion of statefulinstances by automatic cross-zone redistribution.

  • If you use anall-instances configurationto override instance template properties, you cannot specify those propertiesin any per-instance configuration and at the same time in the group'sall-instances configuration.

  • A stateful regional MIG does not automatically orchestrate cross-zonefailover. When using a regional MIG, you can make your stateful applicationresilient to zonal failure by deploying redundant replicas tomultiple zones and relying on your application's data replicationfunctionality.

When to use stateful persistent disks

Use stateful persistent disks to take advantage of VMautohealing andautomated updates while still preserving the data on the disks. For more information, seeuse cases for stateful MIGs.

When you configure stateful disks, these disks are preserved through VM instanceautohealing, updates, and recreation. But that also means that stateful diskscannot be recreated from the original image or updated to a new image.

As a best practice, we recommend keeping your boot disks stateless.

Keeping the boot disk stateless has the following benefits:

  • You can repair a boot disk that was corrupted by recreating it from itsoriginal image. Autohealing does such repairs automatically.
  • You can update a boot disk to the latest image with new versions andsecurity patches.

For more information, see howautohealing andupdating handle preserved state.

Configuring stateful persistent disks for all VMs in a MIG

Configure any disk defined in an instance template to be stateful byadding that disk's device name to the MIG'sstateful policy.The MIG treats disks with that device name as stateful for all existing andfuture VM instances.

Note: If an instance template attaches an existing disk in read-only mode, youdon't need to configure it as stateful. The MIG already preserves this disk. Itattaches the disk to all its VMs in read-only mode and detaches the diskwhen a VM is deleted.

Configuring stateful disks on MIG creation

Permissions required for this task

To perform this task, you must have the followingpermissions:

Console

  1. In the Google Cloud console, go to theInstance groups page.

    Go to Instance groups

  2. Select your project and clickContinue.

  3. ClickCreate instance group.

  4. SelectNew managed instance group (stateful).

  5. Specify aName for the instance group.

  6. Select anInstance template. If no templates are available,create an instance template.

  7. In theNumber of instances field, specify the number of instances forthe instance group.

  8. TheStateful configuration section displays the disks specified inthe instance template. Click a disk to edit its stateful configuration.

    1. UnderStateful, selectYes.

    2. From theOn permanent instance deletion drop-down, select theaction to perform on the stateful disk when the VM instance isdeleted. The available options are:

      • Detach disk: (Default.) Never delete the disk; detach the diskwhen the VM is deleted.
      • Delete disk: Delete the stateful disk when its VM is permanentlydeleted from the instance group, for example, when the managedinstance is deleted manually or when the group size is decreased.
      Note: Regardless of the value of the delete rule, stateful disksare always preserved on instance autohealing, update, and recreationoperations.

    3. After you finish the stateful configuration, clickDone.

  9. ClickCreate.

gcloud

To specify which disks from an instance template should be stateful on MIGcreation, use the--stateful-disk flag with thegcloud compute instance-groups managed create command:

gcloud compute instance-groups managed createINSTANCE_GROUP_NAME \    --templateINSTANCE_TEMPLATE \    --sizeSIZE \    --stateful-disk device-name=DEVICE_NAME[,auto-delete=DELETE_RULE]

Replace the following:

  • INSTANCE_GROUP_NAME: The name of the managedinstance group to create.
  • INSTANCE_TEMPLATE: The name of the instancetemplate to use when creating instances.
  • SIZE: The initial number of VMs you needin this group.
  • DEVICE_NAME: The device name of adisk specified in the instance template.

  • DELETE_RULE: A value that prescribeswhat should happen to a stateful disk when a VM is deleted. Availableoptions are:

    • never: (Default.) Never delete the disk; instead, detach thedisk when its VM is deleted.
    • on-permanent-instance-deletion: Delete the disk whenits VM instance is permanentlydeleted from the instance group, forexample, when the managed instance is deleted manually or when thegroup size is decreased.

    Regardless of the value of the delete rule, stateful disks are alwayspreserved on VM autohealing, update, and recreation operations.

Note: You can see the device names of disks that aredefined in an instance template by running thegcloud computeinstance-templates describecommand.

Example

You want to deploy a database with 12 shards, each with a stateless bootdisk that contains the operating system and database binaries, and each witha stateful data disk. Use the following steps:

  1. Create an instance template with a stateless boot disk based on theimageimg-example-db-v01, which has a pre-installed OS and database,and with a stateful data disk:

    gcloud compute instance-templates create example-database-template-v01 \    --image img-example-db-v01 \    --create-disk device-name=data-disk,mode=rw,image=empty10GBext4

    The--create-disk flag instructs the MIG to:

    1. Create a new 10 GB disk for each VM instance froman empty ext4 image, prepared beforehand.
    2. Attach the disk to its VM in read/write mode using device namedata-disk.

  2. Create a MIG from the instance template anddefine the data disk as stateful:

    gcloud compute instance-groups managed create example-database-group \  --template example-database-template-v01 \  --base-instance-name shard \  --size 12 \  --stateful-disk device-name=data-disk,auto-delete=on-permanent-instance-deletion

    The device namedata-disk is taken from the instance template. Thedata disk is configured to be deleted together with the VM instance whenthe VM is permanentlydeleted (either due to manual instance deletion or due to manual decrease of thegroup size). The data disk is preserved on autohealing, updates, and VMrecreation.

  3. Verify that the data disk is configured in the stateful policy:

    gcloud compute instance-groups managed describe example-database-groupbaseInstanceName: shard...name: example-database-group...statefulPolicy:  preservedState:    disks:      data-disk:        autoDelete: ON_PERMANENT_INSTANCE_DELETION...

    You can see that the stateful policy declares disks with device namedata-disk as stateful, with a rule to delete such disks on permanentVM deletion.

Terraform

If you haven't already created an instance template, which specifies themachine type, boot disk image, network, and other VM properties that youwant for each VM in your MIG,create an instance template.

The following sample creates a zonal MIG with a stateful disk. To specifywhich disk from the instance template should be stateful on MIGcreation, include thestateful_disk block. For more information about theresource used in the sample,seegoogle_compute_instance_group_manager resource.To create a regional MIG, use thegoogle_compute_region_instance_group_manager resource.

resource "google_compute_instance_group_manager" "default" {  name               = "example-database-group"  base_instance_name = "shard"  target_size        = 12  zone               = "us-central1-f"  version {    instance_template = google_compute_instance_template.default.id    name              = "primary"  }  stateful_disk {    device_name = "data-disk"    delete_rule = "ON_PERMANENT_INSTANCE_DELETION"  }}

To learn how to apply or remove a Terraform configuration, seeBasic Terraform commands.

REST

To specify which disks from the instance template should be stateful on MIGcreation, include them in thestatefulPolicy field in the request body oftheinstanceGroupManagers.insertmethod:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT/zones/ZONE/instanceGroupManagers{  "name": "NAME",  "versions": [    {      "instanceTemplate": "global/instanceTemplates/TEMPLATE"    }  ],  "targetSize":SIZE,  "statefulPolicy": {    "preservedState": {      "disks": {        "DEVICE_NAME": {"autoDelete": "DELETE_RULE" },        "DEVICE_NAME": {"autoDelete": "DELETE_RULE" }      }    }  }}

Replace the following:

  • PROJECT: The project ID for the request.
  • ZONE: Thezone where the MIG is located(applies to a zonal MIG).
    • For a regional MIG, replacezones/ZONE withregions/REGION and specify the region ofthe MIG.
  • NAME: The name of the MIG tocreate.
  • TEMPLATE: The name of the instance template touse when creating instances.
  • SIZE: The initial number of instances you needin this group.
  • DEVICE_NAME: The device name of adisk specified in the instance template.

  • DELETE_RULE: A value that prescribeswhat should happen to the stateful disk when the VM instance is deleted.The available options are:

    • never: (Default.) Never delete the disk; detach the disk whenthe VM is deleted.
    • on_permanent_instance_deletion: Delete the stateful disk whenits VM is permanentlydeleted from the instance group, for example, when the managed instance isdeleted manually or when the group size is decreased.
    Note: Regardless of the value of the delete rule, stateful disks arealways preserved on instance autohealing, update, and recreationoperations.
Note: You can see the device names of disks that aredefined in an instance template by calling theinstanceTemplates.get methodand reading theinstanceTemplates.disks[].deviceName field inthe response.

Example

You want to deploy a database with 12 shards, each with a stateless bootdisk that contains the operating system and database binaries, and each witha stateful data disk. Use the following steps.

  1. Create an instance template with a stateless boot disk based on theimageimg-example-db-v01, with pre-installed OS and database, andwith a stateful data disk, using theinstanceTemplates.insert method:

    POST https://compute.googleapis.com/compute/v1/projects/example-project/global/instanceTemplates{  "name": "example-database-template-v01",  "properties": {    "machineType":"e2-standard-2",    "disks": [      {        "boot": true,        "deviceName": "boot-disk",        "initializeParams": {          "sourceImage": "projects/example-project/global/images/mg-example-db-v01"        }      },      {        "deviceName": "data-disk",        "mode": "READ_WRITE",        "initializeParams": {          "sourceImage": "projects/example-project/global/images/empty10GBext4"        }      }    ],    "networkInterfaces": [      {        "network": "global/networks/default"      }    ]  }}

    The data disk in the instance template has device namedata-disk andis configured to be created from an empty ext4 image, preparedbeforehand, and to be attached in read/write mode.

  2. Create a MIG from the instance template anddefine the data disk as stateful by using theinstanceGroupManagers.insert method:

    POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers{  "name": "example-database-group",  "baseInstanceName": "shard",  "versions": [    {      "instanceTemplate": "global/instanceTemplates/example-database-template-v01"    }  ],  "targetSize": 12,  "statefulPolicy": {    "preservedState": {      "disks": {        "data-disk": {"autoDelete": "ON_PERMANENT_INSTANCE_DELETION" }      }    }  }}

    The MIG creates 12 instances, each with a disk with the followingproperties:

    • A device name,data-disk, taken from the instance template.
    • A delete rule to delete the data disk when the VM is deleted(either due to manual instance deletion or due to manual decreaseof the group size).
    • An entry in the preserved state from policy(preservedStateFromPolicy)of each managed instance so that the data disk is preserved onautohealing, updates, and instance recreation.

  3. Use theinstanceGroupManagers.get method to verify that the data disk is configured in the stateful policyof the newinstanceGroupManagers resource:

    GET https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-database-group
    {  "name": "example-database-group",  "baseInstanceName": "shard",  ...  "statefulPolicy": {    "preservedState": {      "disks": {        "data-disk": {"autoDelete": "ON_PERMANENT_INSTANCE_DELETION" }      }    }  }  ...}

    You can see that stateful policy declares disks with device namedata-disk as stateful with the rule to delete such disks on permanentinstance deletion.

Setting and updating stateful configuration for disks in an existing MIG

If you run a stateful application on a stateless MIG (a MIG without anystateful configuration), you can configure existing disks that are defined inthe instance template to be stateful for all instances in this MIG. This letsyou preserve the disks on instance recreation, autohealing, and updateoperations, and optionally on deletion operations.

You can do the following operations:

  • Add disks that are defined in the instance template to thestateful policy of an existing MIG to declare them as stateful. This marks disks with thegiven device name as stateful for all existing and future instances in theMIG.
  • Update the stateful policy to change the stateful configurationfor disks.

The MIG applies the updated configuration in the stateful policy automaticallyand asynchronously to all instances. Updates to disk configurations in astateful policy don't disrupt running VMs. For more information, seeapplying stateful policy updates.

For a regional MIG, you mustdisable proactive cross zone instance redistribution before you can configure statefuldisks. For more information, seehow regional groups handle preserved state.

Note: If a MIG has a read-only disk attached toall of its VMs, you don't need to configure this disk asstateful. The MIG already preserves this disk. It attaches the disk to allits instances in read-only mode and detaches the disk when a VM isdeleted.

Permissions required for this task

To perform this task, you must have the followingpermissions:

Console

  1. In the Google Cloud console, go to theInstance groups page.

    Go to Instance groups

  2. Click the name of the instance group in which you want to specifystateful configuration for disk.

  3. ClickEdit to modify the managed instance group.

  4. ClickStateful & per-instance configuration to expand the section.

  5. In theGroup config section, click the disk that you want to makestateful. In the expanded section, do the following:

    1. ForStateful, selectYes.

    2. In theOn permanent instance deletion list, select the action to perform on the stateful disk when the VM instance is deleted.

    3. After you update the stateful configuration, clickDone.

  6. ClickSave to complete the update.

gcloud

To specify which disks from the instance template should be stateful or toupdate the stateful disk configuration for an existing MIG, use one ormultiple--stateful-disk flags with thegcloud compute instance-groups managed update command:

gcloud compute instance-groups managed updateNAME \  --stateful-disk device-name=DEVICE_NAME[,auto-delete=DELETE_RULE]

Replace the following:

If a specified device name is already configured in the stateful policy, thecommand updates the configuration.

Note: You can see the device names of disks that are defined inan instance template by running thegcloud compute instance-templates describe command.

Example

You run a database with multiple shards on a MIG namedexample-database-group. Each VM in the MIG stores a shard on an additional disk withdevice namedata-disk, which is defined by the instance template. The MIGhas no stateful configuration, and you want to preserve the data disks oninstance recreation, autohealing, and updates. You also want to protect thedata disks from deletion when a VM is deleted.

  1. Update the MIG to define the data disk asstateful by using the following command:

    gcloud compute instance-groups managed update example-database-group \  --stateful-disk device-name=data-disk,auto-delete=never

    As a result, the MIG applies the stateful policy configuration updatesautomatically and asynchronously to the data disks for all instances.The data disks are now preserved on autohealing, updates, and instancerecreation, and the data disks are detached on instance deletion becausetheauto-delete rule is set tonever.

  2. Verify that the data disk is configured in the stateful policy byrunning thegcloud compute instance-groups managed describe example-database-group command.

REST

To specify which disks from the instance template should be stateful or toupdate the stateful disk configuration for an existing MIG, configure thedisks in the MIG's stateful policy using theinstanceGroupManagers.patch method:

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT/zones/ZONE/instanceGroupManagers/NAME{  "statefulPolicy": {    "preservedState": {      "disks": {        "DEVICE_NAME": {"autoDelete": "DELETE_RULE" },        "DEVICE_NAME": {"autoDelete": "DELETE_RULE" }      }    }  }}

Replace the following:

  • PROJECT: The project ID for the request.
  • ZONE: Thezone where the MIG is located(applies to a zonal MIG).
    • For a regional MIG, replacezones/ZONE withregions/REGION and specify the region ofthe MIG.
  • NAME: The name of the MIG toupdate.
  • DEVICE_NAME: The device name of adisk, specified in the instance template, for which you would like toupdate stateful configuration.

  • DELETE_RULE: A value that prescribeswhat should happen to the stateful disk when a VM instance is deleted.The available options are:

    • never: (Default.) Never delete the disk, detach the disk whenits instance is deleted.
    • on-permanent-instance-deletion: Delete the stateful disk whenits instance is permanentlydeleted from the instance group, forexample, when the instance is deleted manually or when the groupsize is decreased.
    Note: Regardless of the value of the delete rule, stateful disks arealways preserved on instance autohealing, update, and recreationoperations.

If the specified device name is already configured in the stateful policy,thepatch method updates its configuration.

Note: You can see the device names of disks that are defined inan instance template by executing theinstanceTemplates.get methodand reading theinstanceTemplates.disks[].deviceName field inthe response.

Example

You run a database with multiple shards on a MIG namedexample-database-group. Each VM in the MIG stores a shard on an additional disk withdevice namedata-disk, which is defined by the instance template. The MIGhas no stateful configuration, and you want to preserve the data disks oninstance recreation, autohealing, and updates. You also want to protect thedata disks from deletion when a VM is deleted.

  1. Patch the MIG to define the data disk as stateful:

    PATCH https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-database-group{  "statefulPolicy": {    "preservedState": {      "disks": {        "data-disk": {"autoDelete": "NEVER" }      }    }  }}

    The MIG applies this stateful configuration automatically andasynchronously to the data disks for all instances. The data disks willbe preserved on autohealing, updates, and instance recreation. The datadisks will be detached on instance deletion because theautoDelete rule is set toNEVER.

  2. Verify that the data disk is configured in the stateful policy byviewing theinstanceGroupManagers resource, returned by theinstanceGroupManagers.get method.

Declaring previously stateful persistent disks as stateless

You might need to configure a stateful disk to be treated as stateless. Forexample:

  • If you rearchitect your app to move the state off the disk.
  • If you configured the disk to be stateful by mistake and would like torevert it.

To declare all disks with a given device name as stateless, remove the disk'sconfiguration from the stateful policy.

Note: If disk configuration for the same device name ispresent in a per-instance configuration, the disk remains stateful for theassociated instance even if you remove its configuration from the statefulpolicy. In this case, you must also remove the disk's configuration fromthe per-instance configuration to make the disk stateless.

The MIG applies the change to the stateful policy automatically andasynchronously to all instances. Updates to disk configuration in a statefulpolicy don't disrupt running VM instances.

For more information, seeApplying stateful policy updates.

Permissions required for this task

To perform this task, you must have the followingpermissions:

Console

  1. In the Google Cloud console, go to theInstance groups page.

    Go to Instance groups

  2. Click the name of the instance group from which you want to remove stateful configuration for disks.

  3. ClickEdit to modify the managed instance group.

  4. ClickStateful & per-instance configuration to expand the section.

  5. In theGroup config section, click the stateful disks that you wantto make stateless. In the expanded section, do the following:

    1. Change theStateful option toNo.
    2. ClickDone.

  6. After you make the changes, clickSave.

gcloud

To specify which disks from a MIG's stateful policy to make stateless, usethe--remove-stateful-disks flag with thegcloud compute instance-groups managed update command:

gcloud compute instance-groups managed updateNAME \  --remove-stateful-disksDEVICE_NAME[,DEVICE_NAME,...]

Replace the following:

  • NAME: The name of the MIG toupdate.
  • DEVICE_NAME: The device name of a disk to removefrom the stateful policy and to treat as stateless. You can provide oneor multiple device names in the list.
Note: You can see the device names of disks that are configured ina MIG's stateful policy by running thegcloud compute instance-groups managed describe command.

Example

You run a legacy application with multiple nodes on a MIG namedexample-legacy-group. Each VM in the MIG stores application dataon a boot disk with device nameboot-disk, which you configured asstateful in the MIG's stateful policy. You have moved application data toan additional disk and now want to make the boot disk stateless to make iteasy to update to new images.

To remove the stateful configuration of the boot disk, update the managedinstance group:

gcloud compute instance-groups managed update example-legacy-group \  --remove-stateful-disks boot-disk

The MIG removes the stateful configuration for the device nameboot-diskautomatically and asynchronously for the boot disks of all instances in thegroup. The boot disks remain attached to the instances but are no longerstateful. When you recreate or update the instances, or when instances areautohealed, the MIG recreates the boot disks from the image specified in theinstance template.

REST

To specify which disks from a MIG's stateful policy to make stateless,remove each disk's configuration from the MIG's stateful policy using theinstanceGroupManagers.patch method:

PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT/zones/ZONE/instanceGroupManagers/NAME{  "statefulPolicy": {    "preservedState": {      "disks": {        "DEVICE_NAME": null,        ...      }    }  }}

Replace the following:

  • PROJECT: The project ID for the request.
  • ZONE: Thezone where the MIG is located(applies to a zonal MIG).
    • For a regional MIG, replacezones/ZONE withregions/REGION and specify the region ofthe MIG.
  • NAME: The name of the MIG toupdate.
  • DEVICE_NAME: The device name of a disk that youwant to remove from the stateful policy. Providing anull value leadsto removal of the stateful configuration for that disk. You can provideone or multiple device names to remove.
Note: You can find out the device names of disks configured ina MIG's stateful policy from the output of theinstanceGroupManagers.get method.

Example

You run a legacy application with multiple nodes on a MIG namedexample-legacy-group. Each VM in the MIG stores application dataon a boot disk with device nameboot-disk, which you configuredas stateful in the MIG's stateful policy. You have moved application datato an additional disk and now want to make the boot disk statelessto make it easy to update to new images.

To remove the stateful configuration of the boot disk, patch the managedinstance group:

PATCH https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-legacy-group{  "statefulPolicy": {    "preservedState": {      "disks": {        "boot-disk": null      }    }  }}

The MIG removes the stateful configuration for the device nameboot-diskautomatically and asynchronously for the boot disks of all instances in thegroup. The boot disks remain attached to the instances but are no longerstateful. When you recreate or update the instances, or when instances areautohealed, the MIG recreates the boot disks from the image specified in theinstance template.

Removing stateful persistent disks from a MIG

You might need to completely remove a stateful disk from instances in a MIG,for example, if you re-architected your application and moved the state out ofthat disk.

MIGs don't allow removing stateful disks, so you must dothe following steps:

  1. Remove the stateful configuration of the disk from the stateful policy. This makes disks with the givendevice name stateless.
  2. Detach the disks from the VMs if you still want to keep them.
  3. Roll out a new instance template that no longer defines the disk withthe given device name.

Configuring stateful persistent disks individually for a VM in a MIG

Configure stateful persistent disks for a specific VM in aMIG by adding the disk's device name to that VM'sper-instance configuration.Update the VM toapply the per-instance configurationand make it effective.

Configuring stateful persistent disks individually for specific VMs in a MIG isuseful if you need to:

  • Migrate existing workloads (bring existing disks) from standalone VMs tostateful MIGs to benefit from autohealing and easy updates.
  • Restore backups of disks, configured individually for VMs.
  • Attach additional stateful disks to a specific VM temporarily fortesting, debugging, or copying data.

Pro Tip:Where feasible, consider configuring stateful disks forall VMs ina MIG, instead of configuring stateful disks individually for each VMinstance.

Adding existing stateful disks to new VMs in a MIG

You can add existing stateful disks to new instances that you manually createin a MIG. This is useful for migrating a stateful application from existingstandalone VMs to a stateful MIG, for example:

  1. Create an instance template with common configuration for all VM instances.
  2. Detach the data disks from the standalone instances and delete these instances.You can alsodetach boot disks if they contain state that should be preserved.
  3. Create an empty MIG using the instance template created earlier.

  4. Create instancesin the MIG with the appropriate names and associateddisks from the previous step. The MIG responds to your request with thefollowing actions:

    1. Creates a VM from the instance template using the providedinstance name.
      • A regional MIG creates the VM in the same zone where thedisk is located. If the disk is regional, the regional MIG createsthe VM in any of the disk's replica zones.
    2. Creates aper-instance configuration with the provided stateful configuration for the disks.
    3. Attaches the disks to the new instance.

Add existing stateful disks when manually creating specific instances ina MIG using the gcloud CLI or REST. The MIG applies theconfiguration immediately on VM creation.

gcloud

To create a VM with a custom name and attach one or more existingstateful disks to that VM, use thegcloud compute instance-groups managed create-instancecommand with one or multiple--stateful-disk flags.

gcloud compute instance-groups managed create-instanceNAME \  --instanceVM_NAME \  [--zoneZONE | --regionREGION] \  --stateful-disk device-name=DEVICE_NAME,source=DISK[,mode=MODE][,auto-delete=DELETE_RULE]

Replace the following:

  • NAME: The name of the MIG in whichyou need to create an instance.
  • VM_NAME: The name of the newinstance to create.
  • ZONE: The zone where the MIG is located(applies to a zonal MIG).
  • REGION: The region where the MIG is located(applies to a regional MIG).
  • DEVICE_NAME: The device name to usewhen attaching the disk.
  • DISK: The URI of an existingpersistent disk to attach under the specifiedDEVICE_NAME in the formatprojects/project-id/zones/zone/disks/disk-namefor a zonal disk andprojects/project-id/regions/region/disks/disk-namefor a regional disk.
  • MODE: Specifies the mode of thedisk. Supported options are:
    • ro: Read-only.
    • rw: (Default.) Read/write.

  • DELETE_RULE: A value that prescribeswhat should happen to a stateful disk when a VM instance is deleted. Theavailable options are:

    • never: (Default.) Never delete the disk; instead, detach thedisk when its instance is deleted.
    • on-permanent-instance-deletion: Delete the stateful disk whenits instance is permanentlydeleted from the instance group, forexample, when the instance is deleted manually or when the groupsize is decreased.

    Regardless of the value of the delete rule, stateful disks are alwayspreserved on instance autohealing, update, and recreation operations.

Example

You want to have autohealing for a database server that is currently runningon a standalone VM nameddb-instance and that currently stores data on adisk nameddb-data-disk-1.

Create a stateful MIG with autohealing, create a similar VMinside the MIG, and attach the existing data diskdb-data-disk-1 to thenew instance as a stateful disk:

  1. Stop the VM,db-instance, during a maintenance window.
  2. Create an instance templatenameddb-template using thedb-instance configuration.
  3. Detachdb-data-disk-1 fromdb-instance and deletedb-instance.
  4. Create an empty MIG,example-database-mig, fromdb-template, andconfigure autohealing.

  5. Create a managed instance with the originaldb-instance name andattach thedb-data-disk-1 as a stateful disk:

    gcloud compute instance-groups managed create-instance example-database-mig \  --instance db-instance \  --zone us-east1-c \  --stateful-disk device-name=data-disk,source=projects/example-project/zones/us-east1-c/disks/db-data-disk-1,auto-delete=never

    The command creates an instance,db-instance, in the MIG, creates acorresponding per-instance configuration withdb-data-disk-1 stateful disk,and attaches the disk to the new VM, usingdata-disk as thedevice name.

REST

To create one or multiple instances in a MIG, set custom VM names, andattach one or multiple existing stateful disks to these instances, use theinstanceGroupManagers.createInstances method.

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME/createInstances{  "instances": [    {      "name": "VM_NAME",      "preservedState" : {        "disks": {          "DEVICE_NAME" : {            "source": "DISK",            "mode": "MODE",            "autoDelete": "DELETE_RULE"          },          ...        }      }    },    ...  ]}

Replace the following:

  • PROJECT_ID: Project ID for the request.
  • ZONE: Thezone where the MIG is located(applies to a zonal MIG).
    • For a regional MIG, replacezones/ZONE withregions/REGION and specify the region ofthe MIG.
  • NAME: The name of the MIG in which tocreate an instance.
  • VM_NAME: The name of theinstance to create.
  • DEVICE_NAME: The device name touse when attaching the disk.
  • DISK: The URI of an existing persistentdisk to attach under the specifiedDEVICE_NAMEin the formatprojects/project-id/zones/zone/disks/disk-namefor a zonal disk orprojects/project-id/regions/region/disks/disk-namefor a regional disk.
  • MODE: Specifies the mode of thedisk. Supported options are:
    • READ_ONLY: Read-only.
    • READ_WRITE: (Default.) Read/write.

  • DELETE_RULE: A value that prescribeswhat should happen to a stateful disk when a VM is deleted. Theavailable options are as follows:

    • never: (Default.) Never delete the disk; instead, detach thedisk when its VM is deleted.
    • on-permanent-instance-deletion: Delete the stateful disk whenits instance is permanentlydeleted from the instance group, forexample, when the instance is deleted manually or when the groupsize is decreased.

    Regardless of the value of the delete rule, stateful disks are alwayspreserved on instance autohealing, update, and recreation operations.

Example

You want to have autohealing for a database server that is currently runningon a standalone VM nameddb-instance and that currently stores data on adisk nameddb-data-disk-1.

Create a stateful MIG with autohealing, create a similar instanceinside the MIG, and attach the existing data diskdb-data-disk-1 to thenew VM as a stateful disk:

  1. Stop the VM,db-instance, during a maintenance window.
  2. Create an instance templatenameddb-template using thedb-instance configuration.
  3. Detachdb-data-disk-1 fromdb-instance, and deletedb-instance.
  4. Create an empty MIG,example-database-mig, fromdb-template, andconfigure autohealing.

  5. Create an instance with the originaldb-instance name, and attachthedb-data-disk-1 as a stateful disk:

    POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-database-mig/createInstances{  "instances": [    {      "name": "db-instance",      "preservedState" : {        "disks": {          "data-disk" : {            "source": "projects/example-project/zones/us-east1-c/disks/db-data-disk-1",            "mode": "READ_WRITE",            "autoDelete": "never"          }        }      }    }  ]}

    The method creates an instance,db-instance, in the MIG, creates acorresponding per-instance configuration with thedb-data-disk-1 statefuldisk, and attaches the disk to the new instance, usingdata-disk asthe device name.

Adding, declaring, and replacing stateful disks individually for VMs in a MIG

Configure stateful disks individually for a managed instance by addingor updating a stateful disk configuration in the associated per-instanceconfig. Then update the instance toapply the per-instance configurationto the VM.

Configuring stateful disks individually is useful for the following tasks:

  • Adding a stateful disk from outside of a MIG to a VM in that MIG.
  • Declaring a previously stateless disk as stateful for a VM in a MIG.
  • Replacing a stateful disk with a different disk for a VM in a MIG.

Adding a stateful disk from outside of a MIG to a VM in that MIG.You can attach any disk from outside of a MIG to a managed instance by addingstateful configuration for the disk to the associated per-instance configuration.After you apply the config, the MIG automatically attaches the diskto the instance and treats it as stateful.

Declaring a previously stateless persistent disk as stateful.You can declare a previously stateless disk, currently attached to a VM,as stateful by adding stateful configuration for this disk, including itsdevice name and URI, to the associated per-instance configuration. After you applythe config, the MIG starts preserving the disk as stateful.

Replacing a stateful disk with a different disk.Replacing one stateful disk with another stateful disk can be useful, forexample, if you need access to a recovered backup. You can swap one statefuldisk for another by updating the disk's URI while keeping the same device namein the per-instance configuration. After you apply the updated per-instance configuration,the MIG detaches the old disk and attaches the new one using the samedevice name. Whenapplying the update, choose whether to keep the instance running, restart, orrecreate it. Swapping a boot disk requires at least a VM restart.

gcloud

To configure stateful disks individually for a VM in a MIG, add orupdate stateful disk configuration in the associated per-instanceconfiguration. Then, update the instance toapply the configuration.

If a per-instance configuration doesn't exist for the instance, use thegcloud compute instance-groups managed instance-configs create commandwith one or multiple--stateful-disk flags:

gcloud compute instance-groups managed instance-configs createNAME \--instanceVM_NAME \--stateful-disk device-name=DEVICE_NAME[,source=DISK][,mode=MODE][,auto-delete=DELETE_RULE] \[--no-update-instance | --update-instance] \[--instance-update-minimal-actionMINIMAL_ACTION]

If a per-instance configuration already exists for the instance, use thegcloud compute instance-groups managed instance-configs update commandwith one or multiple--stateful-disk flags.

The--update-instance flag(default) applies the changes immediately to the instance. If you use--no-update-instance, the changes remain unapplied and areapplied when you next recreate or update the instance.

gcloud compute instance-groups managed instance-configs updateNAME \--instanceVM_NAME \--stateful-disk device-name=DEVICE_NAME[,source=DISK][,mode=MODE][,auto-delete=DELETE_RULE] \[--no-update-instance | --update-instance] \[--instance-update-minimal-actionMINIMAL_ACTION]

Replace the following:

  • NAME: The name of the managedinstance group.
  • VM_NAME: The name of theVM instance for which to configure stateful disks.
  • DEVICE_NAME: The device name usedfor attaching the disk.

  • DISK: The URI of an existing persistentdisk to attach under the specifiedDEVICE_NAME,in the formatprojects/project-id/zones/zone/disks/disk-namefor a zonal disk andprojects/project-id/regions/region/disks/disk-namefor a regional disk.

    Thesource=DISK subflag is optional if thedevice is already defined in the instance's per-instance configuration.Otherwise it is required.

    If omitted, the currently configured disk URI remains unchanged.

  • MODE: Specifies the mode of thedisk. You can only specifymode if you also specifysource.Supported options are:

    • ro: Read-only.
    • rw: (Default.) Read/write.

    If omitted, the default value is set for a new stateful diskconfiguration; the value remains unchanged in an existing configuration.

  • DELETE_RULE: A value that prescribeswhat should happen to a stateful disk when a VM is deleted. Theavailable options are as follows:

    • never: (Default.) Never delete the disk; instead, detach thedisk when its instance is deleted.
    • on-permanent-instance-deletion: Delete the stateful disk whenits instance is permanently deleted from the instance group, forexample, when the instance is deleted manually or when thegroup size is decreased.

    If omitted, the default value is set for a new stateful diskconfiguration; the value remains unchanged in an existing configuration.

    Regardless of the value of the delete rule, stateful disks are alwayspreserved on instance autohealing, update, and recreation operations.

  • MINIMAL_ACTION: Perform at least the specifiedaction when applying the per-instance configuration update to the instance.Must be used together with the--update-instance flag. The value mustbe one of:

    • none: No action.
    • refresh: Apply updates that are possible to apply withoutstopping the VM.
    • restart: Stop the VM and then start it again.
    • replace: Recreate the VM.

    If omitted, the least disruptive action required by the update is used.

Example

The data on a currently attached stateful disk,data-disk-1, gotcorrupted, and you want to restore it from the latest backup. You created adisk,data-disk-2, from a snapshot to replace the corrupted disk ininstance,db-instance-1, managed by a stateful MIG,example-database-mig. The original diskdata-disk-1, is attached underthedata-disk device name with an auto-delete rule to never delete thedisk.

To replacedata-disk-1 withdata-disk-2, run the following command:

gcloud compute instance-groups managed instance-configs update example-database-mig \  --instance db-instance-1 \  --stateful-disk device-name=data-disk,source=projects/example-project/zones/us-east1-c/disks/data-disk-2 \  --update-instance \  --instance-update-minimal-action restart

The command does the following:

  1. Updates the per-instance configuration fordb-instance-1:
    1. Updates the source for the disk with device namedata-diskfromdata-disk-1 (last configuration) todata-disk-2 (newconfiguration).
    2. Keeps the auto-delete rule to never delete the disk because theauto-delete parameter is omitted in the--stateful-disk flagand, by default, the delete rule isnever.
  2. Applies the per-instance configuration update to thedb-instance-1 VMimmediately because the--update-instance flag is included.The MIG detachesdata-disk-1 and attachesdata-disk-2 underthe same device name,data-disk.
  3. Because the minimal action is set torestart, the MIG restarts thedb-instance-1 instance to update the VM, which helps thedatabase application to start using the new disk.

Terraform

To configure stateful disks individually for a VM in a MIG, add thestateful disk configuration in the associated per-instanceconfiguration. Then, update the instance toapply the configuration.

To add per-instance configuration for a VM, use thegoogle_compute_per_instance_config resourceand include thepreserved_state block as shown in the following sample.

resource "google_compute_per_instance_config" "default" {  instance_group_manager = google_compute_instance_group_manager.default.name  zone                   = google_compute_instance_group_manager.default.zone  name                   = "db-instance"  preserved_state {    disk {      device_name = "data-disk"      source      = google_compute_disk.default.id      delete_rule = "NEVER"    }  }}

To learn how to apply or remove a Terraform configuration, seeBasic Terraform commands.

REST

To configure stateful disks individually for VMs in a MIG, add orupdate the stateful disk configuration in the associated per-instanceconfigurations. Then, update the instances toapply the configuration.

If per-instance configurations don't yet exist for the given VMs, use theinstanceGroupManagers.updatePerInstanceConfigs methodorregionInstanceGroupManagers.updatePerInstanceConfigs methodwith stateful configuration for one or multiple disks:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME/updatePerInstanceConfigs{  "perInstanceConfigs": [    {      "name": "VM_NAME",      "preservedState" : {        "disks": {          "DEVICE_NAME" : {            "source": "DISK",            "mode": "MODE",            "autoDelete": "DELETE_RULE"          },          ...        }      },      "fingerprint: "FINGERPRINT"    },    ...  ]}
Note: While you can useupdatePerInstanceConfigs for updating existing per-instanceconfigurations, it fully replaces the specified per-instance configurations withnew values. Instead, we recommend that you use thepatchPerInstanceConfigs method to update existingper-instance configurations because patching keeps the omitted configurationunchanged and prevents the risk of accidental deletion of stateful itemsor reset of the values to defaults.

If per-instance configurations already exist for the given VMs, use theinstanceGroupManagers.patchPerInstanceConfigs methodorregionInstanceGroupManagers.patchPerInstanceConfigs methodwith stateful configuration for one or multiple disks:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME/patchPerInstanceConfigs{  "perInstanceConfigs": [    {      "name": "VM_NAME",      "preservedState" : {        "disks": {          "DEVICE_NAME" : {            "source": "DISK",            "mode": "MODE",            "autoDelete": "DELETE_RULE"          },          ...        }      },      "fingerprint: "FINGERPRINT"    },    ...  ]}

Replace the following:

  • PROJECT_ID: The project ID for the request.
  • ZONE: Thezone where the MIG is located(applies to a zonal MIG).
    • For a regional MIG, replacezones/ZONE withregions/REGION and specify the region ofthe MIG.
  • NAME: The name of the MIG.
  • VM_NAME: The name of theVM, for which to configure stateful disks.
  • DEVICE_NAME: The device name usedfor attaching the disk.

  • DISK: The URI of an existing persistentdisk to attach under the specifiedDEVICE_NAME,in the formatprojects/project-id/zones/zone/disks/disk-namefor a zonal disk andprojects/project-id/regions/region/disks/disk-namefor a regional disk.

    The"source": "DISK" field is optional if thedevice is already defined in the instance's per-instance configuration.Otherwise it is required.

    If thesource field is omitted, the currently configured disk URIremains unchanged.

  • MODE: (Optional.) Specifies the mode of thedisk.mode can only be specified ifsource is given. Supportedoptions are:

    • READ_ONLY: Read-only.
    • READ_WRITE: (Default.) Read/write.

    If omitted, the default value is set for a new stateful diskconfiguration; the value remains unchanged in an existing configuration.

  • DELETE_RULE: (Optional.) A value that prescribeswhat should happen to a stateful disk when a VM is deleted. Theavailable options are as follows:

    • never: (Default.) Never delete the disk; instead, detach thedisk when its instance is deleted.
    • on-permanent-instance-deletion: Delete the stateful disk whenits instance is permanently deleted from the instance group, forexample, when the instance is deleted manually or when the groupsize is decreased.

    If theautoDelete field is omitted, the default value is set for a newstateful disk configuration; the value remains unchanged in an existingconfiguration.

    Regardless of the value of the delete rule, stateful disks are alwayspreserved on instance autohealing, update, and recreation operations.

  • FINGERPRINT: (Optional). The fingerprint for thegiven config if it already exists. Used for optimistic locking. To seethe latest fingerprint, make aget() request to retrieve the resource.

TheupdatePerInstanceConfigs andpatchPerInstanceConfigs methods updatethe specified per-instance configurations but don't apply the configurationupdates to the associated managed VMs. The changes are applied to aninstance when the MIG is instructed to recreate or update the instance. Youcan alsoselectively update the instance to apply the changes.

Example

The data on a currently attached stateful disk,data-disk-1, gotcorrupted, and you want to restore it from the latest backup. You created adisk,data-disk-2, from a snapshot to replace the corrupted disk ininstance,db-instance-1, managed by a stateful MIG,example-database-mig. The original diskdata-disk-1, is attached underthedata-disk device name with an auto-delete rule to never delete thedisk.

To update the per-instance configuration fordb-instance-1 with the new disk,call thepatchPerInstanceConfigs method:

POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-database-mig/patchPerInstanceConfigs{  "perInstanceConfigs": [    {      "name": "db-instance-1",      "preservedState" : {        "disks": {          "data-disk" : {            "source": "projects/example-project/zones/us-east1-c/disks/data-disk-2"          }        }      }    }  ]}

The method patches the per-instance configuration fordb-instance-1:

  1. Updates thesource for a disk with device namedata-disk fromdata-disk-1 (last configuration) todata-disk-2 (new configuration).
  2. Keepsmode andautoDelete parameters unchanged because theparameters are omitted in the request.

The config update is not yet applied to thedb-instance-1 VM. TheMIG applies the config update when you recreate or update the instance.

To apply the per-instance configuration update to thedb-instance-1 VM, call theinstanceGroupManagers.applyUpdatesToInstances method for the instance:

POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-database-mig/applyUpdatesToInstances{  "instances": ["/zones/us-east1-c/instances/db-instance-1"],  "minimalAction": "RESTART"}

The method updates the managed instance's preserved state, detachingdata-disk-1 and attachingdata-disk-2 under the same device namedata-disk. Because theminimalAction is set toRESTART, the methodrestarts thedb-instance-1 VM, which lets the database applicationstart using the new disk.

Detaching a stateful disk or declaring it stateless for an individual VM

You might need to detach a stateful disk or configure it to be treated asstateless for an individual VM. For example:

  • If you rearchitect your app to move the state off the disk.
  • If you configure the disk to be stateful by mistake and would like torevert it.

Detach a stateful disk or make it stateless for an individual VM byremoving the disk's stateful configuration from the associated per-instanceconfig or deleting the entire per-instance configuration. When youapply the change:

  • If the disk is not defined in the instance template, the MIG detachesthe disk.
    • The MIG does not delete the disk when you delete itsconfiguration from the per-instance configuration, regardless of theauto-delete rule in the configuration.
  • If the disk is defined by the instance template, the MIG treats the disk asstateless, which means that the MIG recreates the disk from its source inthe instance template on subsequent instance recreation, update, orautohealing events.
Note: If you set a disk configuration for the same devicename in the MIG's stateful policy, the disk remains stateful even if youremove its configuration from the per-instance configuration. Remove the disk'sconfiguration from both the stateful policy and per-instance configuration to makethe disk stateless.

Removing a disk configuration from a per-instance configuration does not restart arunning VM instance, unless you explicitly choose to do so.

For more information, seeApplying per-instance configurations updates.

gcloud

To detach stateful disks or declare them stateless individually for a VMin a MIG, remove the stateful disk configuration from theassociated per-instance configuration ordelete the whole per-instance configuration if it doesn't contain any other state. Update the instance to apply theconfiguration.

To remove a stateful disk configuration from the associated per-instanceconfig, use thegcloud compute instance-groups managed instance-configs update commandwith the--remove-stateful-disks flag. The--update-instance flag(default) applies the changes immediately to the instance. If you use--no-update-instance, the changes remain unapplied and areapplied when you next recreate or update the instance.

gcloud compute instance-groups managed instance-configs updateNAME \  --instanceVM_NAME \  --remove-stateful-disksDEVICE_NAME[,DEVICE_NAME,...] \  [--no-update-instance | --update-instance] \  [--instance-update-minimal-actionMINIMAL_ACTION]

Replace the following:

  • NAME: The name of the MIG.
  • VM_NAME: Name of the VM fromwhich to remove stateful configuration.
  • DEVICE_NAME: The device name usedfor attaching the disk.

  • MINIMAL_ACTION: Perform at least the specifiedaction when updating the VM with its per-instance configuration. Canonly be used together with--update-instance. The value must be onethe following:

    • none: No action.
    • refresh: Apply updates that are possible to apply withoutstopping the VM.
    • restart: Stop the VM and then start it again.
    • replace: Recreate the VM.

    If omitted, the least disruptive action required by the update is used.

Example

You run a legacy application on a MIG namedexample-legacy-group. Each VM in the MIG stores application dataon a boot disk with device name,boot-disk. Using per-instance configurations,you configured each boot disk to be stateful. Now you have moved applicationdata to an additional disk, and you want to make the boot disk stateless foreach VM to facilitate updating to new images.

For each instance, for example, fornode-1, run the command:

gcloud compute instance-groups managed instance-configs update example-legacy-group \  --instance node-1 \  --remove-stateful-disks boot-disk \  --update-instance

The command does the following:

  1. Removes configuration for the disk with device nameboot-disk fromthe per-instance configuration fornode-1.
  2. Applies the per-instance configuration update to thenode-1 VMimmediately because the--update-instance flag is included. The MIGremoves the boot disk from the managed instance'spreservedStateFromConfigand treats the boot disk as stateless, which means that the MIGrecreates the disk from its boot image in the instance template onsubsequent instance recreation, update, or autohealing events.

REST

To detach stateful disks or declare them stateless individually for a VMin a MIG, remove the stateful disk configuration from the associatedper-instance configuration ordelete the whole per-instance configuration if it doesn't contain any other state. Then update the instance toapply the configuration.

To remove a stateful disk configuration from the associated per-instanceconfig, use theinstanceGroupManagers.patchPerInstanceConfigs methodorregionInstanceGroupManagers.patchPerInstanceConfigs method:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instanceGroupManagers/NAME/patchPerInstanceConfigs{  "perInstanceConfigs": [    {      "name": "VM_NAME",      "preservedState" : {        "disks": {          "DEVICE_NAME" : null        },        ...      },      "fingerprint: "FINGERPRINT"      ...    }  ]}

Replace the following:

  • PROJECT_ID: The project ID for the request.
  • ZONE: Thezone where the MIG is located(applies to a zonal MIG).
    • For a regional MIG, replacezones/ZONE withregions/REGION and specify the region ofthe MIG.
  • NAME: The name of the MIG.
  • VM_NAME: The name of the VMfrom which to remove stateful configuration.
  • DEVICE_NAME: The device name usedfor attaching the disk.
  • FINGERPRINT: The fingerprint for thegiven config if it already exists. Used for optimistic locking. To seethe latest fingerprint, make aget() request to retrieve the resource.

ThepatchPerInstanceConfigs method patches the specified per-instanceconfigurations but does not apply the changes to the associated VMs. Thechanges are applied to a VM when you recreate or update the instance.You can apply the changesmanually or useautomated rolling updates.

Example

You run a legacy application on a MIG namedexample-legacy-group. EachVM in the MIG stores application data on a boot disk with device nameboot-disk. You configured the boot disk as stateful in the MIG'sper-instance configurations when migrating the standalone VMs into the MIG.You have moved application data to an additional disk and now want to makethe boot disk stateless for each VM to make it easy to update to newimages.

  1. Call thepatchPerInstanceConfigs method for the instances, for example,fornode-1 with anull value for the boot disk:

    POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-legacy-group/patchPerInstanceConfigs{  "perInstanceConfigs": [    {      "name": "node-1",      "preservedState" : {        "disks": {          "boot-disk" : null        }      }    }  ]}

    The method removes configuration for the disk with device nameboot-disk from the per-instance configuration fornode-1. The config update isnot yet applied to thenode-1 VM instance. The config update will beapplied on the next instance recreation or update.

  2. To apply the per-instance configuration update to thenode-1 VM instance, calltheinstanceGroupManagers.applyUpdatesToInstancesmethod for the instance:

    POST https://compute.googleapis.com/compute/v1/projects/example-project/zones/us-east1-c/instanceGroupManagers/example-legacy-group/applyUpdatesToInstances{  "instances": ["/zones/us-east1-c/instances/node-1"]}

    The MIG removes the boot disk from thepreservedStateFromConfigfor thenode-1instance and treats the disk as stateless. That is, the MIG recreatesthe disk from its boot image in the instance template on subsequentinstance recreation, update, or autohealing events.

Feedback

We want to learn about your use cases, challenges, and feedback about statefulMIGs. You can share your feedback with our team atmig-discuss@google.com.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.