By default, aregional managed instance group (MIG)distributes its managed virtual machine (VM) instances evenly across selectedzones. But if you need hardware that is not available in all zones or that mightbe temporarily unavailable in selected zones, or if you need to prioritize theuse of zonal reservations, you might prefer a different distribution.

To configure how a regional MIG distributes yourmanaged instancesacrossselected zoneswithin a region,set the MIG's target distribution shape.The following options are available:

• EVEN (default): the group creates and deletes VMs to achieve and maintain the same number of VMs across the selected zones. In anEVEN distribution, the number of VMs does not differ by more than 1 between any two zones. Recommended for highly available serving workloads.
• BALANCED: the group prioritizes creation of VMs in zones where resources are available, while distributing VMs as evenly as possible across selected zones to minimize the impact of zonal failure. Recommended for highly available serving or batch workloads.
• ANY: the group picks zones for creating VM instances to fulfill the requested number of VMs within present resource constraints and to maximize utilization of unused zonal reservations. Recommended for batch workloads that do not require high availability.
• ANY SINGLE ZONE: the group creates all VM instances within a single zone. The zone is chosen based on hardware support, current resource and quota availability, and matching reservations. Recommended in combination with a compact instance placement policy for workloads that require extensive communication between VMs.

Choose an option based on your workload requirements and which MIG capabilitiesyou need. See thecomparison table,use cases, andhow distribution shapes work.

Comparison of shapes

For each possible target shape, the following table describes its intendedworkloads, purpose, distribution of managed instances, feature support, anda brief description of MIG behavior when faced with unavailable resources.

	EVEN (default)	BALANCED	ANY	ANY_SINGLE_ZONE
Intended workloads	Highly available serving workloads (stateless or stateful)	Highly available serving workloads (stateless or stateful) Highly available batch workloads	Batch workloads	Batch workloads requiring extensive communication between VMs
Purpose	Minimize the impact of zone-level failure, assuming sufficient availability of resources in each zone.	Minimize the impact of zone-level failure as much as possible considering availability of resources in each zone.	Prioritize resource acquisition and utilization of unusedreservations.	Minimize network latency andcostsbetween VMs by keeping all VMs in one zone.
Target distribution ofmanaged instances across zones	Even. The number of managed instances does not differ by more than 1 between any two zones, regardless of resource availability.* Some managed instances might not be up and running in case of zonal capacity constraints.	As even as possible. No guarantee on discrepancies in the number of VMs across zones, which depends on current resource availability. When resources are available, the distribution is similar toEVEN. In the worst case of resource constraints, the distribution can take any shape.	Any. Each zone can have a different number of managed instances (including all or none).	Single zone. All instances are created within one zone. The MIG chooses the zone whenever it scales out from zero VMs.
Feature support	EVEN (default)	BALANCED	ANY	ANY_SINGLE_ZONE
Autoscaling
Canary updates
Instance flexibility
Proactive instance redistribution			Not applicable
Reservations	Maximally utilized within each zone independently. Reservations don't impact how instances are distributed.	Maximally utilized within each zone independently. If reservations are present they might help arriving at a balanced distribution.	Maximally utilized within the region. The group prioritizes using up reservations in the region.	Maximally utilized within the chosen zone. Whenever the group has no VMs and needs to create one or more VMs, the group prioritizes the zone with the most reservations if that zone also has sufficient resources and supports the hardware to fulfill the request.
Instance template and stateful configuration hardware requirements (machine type, CPU, GPU, existing disks)	Selected hardware must beavailable in all selected zones.	Selected hardware must beavailable in at least one selected zone.	Selected hardware must beavailable in at least one selected zone.	Selected hardware must beavailable in at least one selected zone.
Sole-tenant nodes
Handling failures	EVEN (default)	BALANCED	ANY	ANY_SINGLE_ZONE
Temporary unavailability of resources in a zone	Exposed Creates new managed instances in zones with fewer managed instances. Keeps retrying to create VM instances in a zone where resources are unavailable until it succeeds. Risk: Cannot create VMs in a zone with limited resources.	Resilient Creates new managed instances in zones where resources are available, while distributing instances as evenly as possible across zones. Risk: VMs might not be distributed evenly across zones.	Resilient Creates new managed instances in zones where resources are available and to maximize utilization of unused reservations. Risk: VMs might not be distributed evenly across zones.	Resilient on group creation and resizes from zero Creates new VM instances within a single zone, where resources are available. Risk: Cannot guarantee all additional instances are successfully created during scale-out requests if the chosen zone does not have enough resources.
Zone-level failure	Resilient Impact is minimized because instances in healthy zones keep serving. Impact is further minimized if youprovision extra instances, sufficient to tolerate losing one zone.	Resilient Impact is minimized because instances in healthy zones keep serving. Impact is further minimized if youprovision extra instances, sufficient to tolerate losing one zone.	Exposed Outage might happen if the majority or all instances are concentrated in a failed zone.	Exposed Outage is inevitable if the failure happens in the chosen zone.

Use cases

Review thefeature support, then choose a distributionshape based on your use case.

Prioritize workload resilience with an even distribution

If you run a highly available serving application that must survive zone-levelfailure without degraded performance, use theEVEN target distributionshape with anover-provisioned group size.Overprovisioning the number of instances in a group protects your workload fromzone-level failure.

Depending on your workload, considercreating an autoscalerto automatically add or remove instances to your group when load increases ordecreases.

To learn more about theEVEN target distribution shape, see thecomparison of target shapes and readHow theEVEN target shape works.

For more information about deploying highly available workloads on regionalMIGs, see the following sections:

• If you need special hardware that is only available in specific zones,select zones for your VMs
• Provision the correct managed instance group size.
• If you want to prevent unintended deletion of instances when you manuallydelete or abandon instances from a regional group,turn off cross-zone instance redistribution
• Configure autoscaling for a regional MIG,which tries to preserve an even balance across zones on group resizes andscales out to available zones in case of zone-level failure.
• Configure load balancing, which shifts load tohealthy zones in case of zone-level failure and, if autoscaling is configured,leads to an increase in the number of VMs in zones where load grows.
• Configure a stateful regional MIG,which can preserve the unique state of each of your MIG's instances, includingVM instance name, persistent disks, and metadata, on machine restart,recreation, autohealing, or update events.

Balance resource acquisition with an even distribution

If you run a highly available serving or batch workload and need to balanceacquisition of resources against an even distribution of VM instances acrossselected zones in a region, use theBALANCED target distribution shape.

TheBALANCED shape prioritizes acquisition of resources—the group createsinstances in zones where resources are available—while distributing instancesas evenly as possible across zones to minimize the impact of zone-level failure.

If you run a batch workload that doesn't need to be protected againstzone-level failure, use theANY target shape instead. TheANY shapeprioritizes acquisition of resources as well as use of zonal reservations.

With the shape set toBALANCED or toANY, you don't need to manuallyverify whether specific hardware isavailable in a particular zone. You canselect all zonesin a region and the group automatically deploys instances in zones where yourrequired hardware is available.

To learn more about theBALANCED target distribution shape, see thecomparison of target shapes and readHow theBALANCED target distribution shape works.

Caution: If you set the distribution shape toBALANCED, the MIG attempts tospread instances evenly across selected zones. But an even distribution mightnot be achieved due to constraints in resource availability. This can beespecially true for large machines, orspecialized hardware (such as GPU orCPU platforms). In case of resource constraints, your group might deploy themajority of your instances in a single zone or in two zones. If a zone-levelfailure happens in this zone, your workload might lose the majority of servingVMs, causing an outage. We advise you to monitor your setups and when observingsubstantial imbalance, try to configure your workload in another region or pickhardware configurations that are more commonly available.

Prioritize resource acquisition

If you run batch workloads and if getting the requested number of instances toperform the processing is more important for you thanworkload resilience to zone-level failures,use theANY target distribution shape.

If you have matchingreservations, set yourtarget shape toANY to prioritize the use of zones that contain the matchingreservations. To learn how to configure reservations in an instance template,seeConsuming instances from a specific reservation.

Similar to theBALANCED target shape, theANY shape is useful when yourbatch workload requires any of the following features:

• VMs with special hardware, such as a specific CPU platform or GPU model.The group will deploy instances to the zones that support the requestedhardware, according toresource availability,and with a preference for zones that have matching reservations.
• Preemptible VMs. You won't need to explore which zones have preemptiblecapacity available. The group will deploy to zones with preemptible capacityautomatically.
• VMs with a large number of cores. The group will get large machines wherethey are available, with a preference for zones that have matchingreservations.

You don't need to manually verify whether specific hardware is availablein a particular zone. You canselect all zonesin a region and the group automatically deploys instances in zones where yourrequired hardware is available.

Note: Availability of preemptible capacity is not guaranteed within a region.

You canselectively deletebatch job worker instances that have completed calculations without affectingother workers. Unlike a group with anEVEN target shape and proactiveredistribution, a group withANY target shape doesn't have to achieve aneven balance and won't trigger redistribution.

Caution: Don't use theANY target distribution shape for services and highlyavailable workloads. When the target shape is set toANY, your regional groupmight deploy the majority of your instances in a single zone. If a zone-levelfailure happens in this zone, your workload will lose the majority of servingVMs, causing an outage. To minimize the impact of zone-level failure, use anEVEN orBALANCED target distribution shape.

To learn more details about theANY target distribution shape, see thecomparison of target shapes and readhow theANY target distribution shape works.

Minimize networking across VMs

If you run a batch workload and want to place all VMs in a single zone to reduceVM-to-VM network latency and costs, and if you don't have a specific zonerequirement, set the group's target shape toANY_SINGLE_ZONE. You can alsocreate a compact placement policyandapply it toyour MIG so that the VMs in the MIG are located closer to each other and on thesame network infrastructure.

When you create a MIG with at least one VM, and whenever a MIG with no VMs needsto scale out again, theANY_SINGLE_ZONE shape picks the optimal zone based onyour reservations, quotas, and hardware requirements.

Similar to theBALANCED andANY target shapes, theANY_SINGLE_ZONE shapeis useful when your batch workload requires any of the following features:

• VMs with special hardware, such as a specific CPU platform or GPU model.The group will deploy instances to a zone that supports the requestedhardware, according toresource availability,and with a preference for the zone that has matching reservations.
• Preemptible VMs. You won't need to explore which zones have preemptiblecapacity available. The group will deploy to a zone with preemptible capacityautomatically.
• VMs with a large number of cores. The group will get large machines wherethey are available, with a preference for the zone that has matchingreservations.

You don't need to manually verify whether specific hardware is availablein a particular zone. When creating the MIG,select all zonesin a region and the group automatically deploys instances in a zone where yourrequired hardware is available.

To learn more details about theANY_SINGLE_ZONE target distribution, see thecomparison of target shapes and readhow theANY_SINGLE_ZONE target distribution shape works.

If you have specific zone requirements and don't want your MIG to switch zonesunder any circumstance, use azonal MIG instead.

Caution: Don't use theANY_SINGLE_ZONE target distribution shape for servicesand highly available workloads. When the target shape is set toANY_SINGLE_ZONE, your regional group deploys all of your instances in a singlezone. If a zone-level failure happens in this zone, your workload will lose allserving VMs, causing an outage. The MIG does not attempt to recreate anyunreachable VMs in another zone. To minimize the impact of zone-level failure,use anEVEN orBALANCED target distribution shape instead.

How it works

This section describes how each target distribution shape works in the followingsituations:

• When you resize the MIG
• In case resources are temporarily unavailable in a zone
• In case of zonal failure

TheEVEN distribution shape

With a target distribution shape set toEVEN and proactive redistributionenabled, the number of managed instances in a regional MIG does not differ bymore than 1 between any two zones, regardless of resource availability. But amanaged instance might not be up andrunningif its zone lacks the resources to provision an actual VM.

Resizing a MIG that has anEVEN distribution shape

A group with anEVEN target shape picks zones for adding or deleting instancesin a way that preserves or converges to an even balance of managed instancesacross zones.

For example, the following diagram shows how a group adds and removes managedinstances.

Impact of temporarily unavailable resources

Resources might be temporarily unavailable in a zone when you create the groupor increase the number of instances. For example, if you request preemptibleinstances or specialized hardware in a limited supply, those resources might notbe available at the time of your request.

With the goal of maintaining an even distribution of instances across zones, thegroup continues attempting to create VM instances in zones where the resourcesare temporarily unavailable. Eventually, the group does acquire the full numberof running VM instances after the resources become available.

For example, the following diagram shows what happens if one of the zonescannot fulfill your request due to a temporary unavailability of resources.

Impact of zone-level failure

If you use theEVEN (orBALANCED) target distribution shape, you canprovision extra instancesto minimize impact of a zone-level failure.

In case of zone-level failure, a regional MIG that is deployed to 3 zones withanEVEN (orBALANCED) target distribution shape might lose 1/3 of itsinstances. You can ensure sufficient capacity to serve your load in case ofzone-level failure by provisioning more VMs, 2/3 of which are required by theload.

For example, if you require 8 instances to process requests across 3 zones andyou want to protect your workload against zone-level failure, you should createa regional group with 12 instances. The following diagram shows what happens ifone zone fails.

TheEVEN target distribution shape works well with autoscaling and loadbalancing under such circumstances. In case of a zone-level failure, the loadbalancer starts sending traffic to instances in the two remaining zones toaccommodate traffic from the failed zone.

For more information about how a regional MIG works with an autoscaler, seeAutoscaling a regional MIG.

TheBALANCED distribution shape

A regional MIG with aBALANCED target shape might not achieve an evendistribution across zones, specifically when the requested resources are notavailable in a zone.

The MIG prioritizes provisioning the requested number of VMs by creating VMs inzones where resources are available. When resources are available, thedistribution is similar toEVEN. In the worst case of resource constraints,the distribution can take any shape.

Resizing a MIG that has aBALANCED distribution shape

Increasing group size

With aBALANCED target shape, the group chooses zones for creating newinstances based on the current availability of the resources that you specifiedin the MIG's instance template.

• When resources are sufficiently available in all selected zones, the groupmaintains an even distribution across zones on size increases, the same way astheEVEN target shape.
• When zonal capacity constraints make it impossible to achieve an evendistribution, the group creates instances in the zones where resources areavailable, while still trying to maximize balance.

For example, you might observe capacity constraints and an uneven distributionif you request a specialized CPU platform, GPU model, or preemptible VMs thatare not uniformly available in all zones.

Decreasing group size

When decreasing its size, a Regional MIG with aBALANCED target shape removesinstances in the following sequence in order to limit disruption to yourworkload:

1. Instances that are not running; that is, instances that for any reason can'tbe created or are being created or autohealed.
2. Instances in zones where the group has more VMs, to converge to an evenlydistributed state.

Impact of temporarily unavailable resources or zonal failure

With aBALANCED target distribution shape, the group deploys instances tozones where capacity is available. During temporary zonal capacity constraints,this can lead to an uneven distribution of instances across zones.

If in such a situation a zone with the largest number of VM instances fails,your workload might lose a significant share of your serving capacity. If thehealthy zones have temporary capacity constraints, the group tries to recreatefailed instances in the original location (a failed zone) and this attempt mightfail.

To protect your workload against such an extreme case:

• Overprovisionthe size of your regional MIG, so that your workload has sufficient servingcapacity in the case of a zonal failure.
• Reserve a sufficientamount of resources in each zone to cover peak load, to overprovision, and tomaintain an even distribution across zones. This tactic helps ensure that youcan get an even distribution of instances across zones, which minimizescapacity loss in case of a zonal failure.

The following diagram shows how a scenario with temporary zonal capacityconstraints, followed by a zonal failure, might evolve.

If your request cannot be fulfilled in any zone in the region, the groupschedules VM creation in zones with temporarily unavailable resources. Thegroup continues attempting to create the scheduled instances within the zoneswhere their creation was originally scheduled. If the resources become availablein other zones sooner than in the original zone where a VM was scheduled, thegroup won't try creation in those other zones. You can schedule new instances inzones with available capacity manually by deleting the managed instances thatfailed to create and resizing the group up to its target size.

If VM creation is unsuccessful, you canlist managed instancesto review the error message in the corresponding managed VM instance orlist recent errors.

In case of a zonal failure, theBALANCED target distribution shape works wellwith autoscaling and load balancing. To accommodate traffic from the failedzone, the load balancer sends traffic to instances in the remaining zones. Anautoscaler responds to the increased utilization in the zones and automaticallycreates capacity in healthy zones. For more information,seeAutoscaling a regional MIG.

TheANY distribution shape

With a target distribution shape set toANY, a regional MIG prioritizesresource acquisition by creating managed instances in zones where resources areavailable. This means that all of the instances might be created in one zone, orevenly distributed across all zones, or anything between those two scenarios.

Resizing a MIG that has anANY distribution shape

Increasing group size

When you increase group size, the group picks any zone where capacity isavailable.

If you have matching reservations in one or more zones, the groupprioritizes utilization of those reservations. However, if you decrease thegroup size, it might take a few minutes for anyconsumed reservationto be available again for consumption. During this period when previously consumedreservations are not available yet, if you increase the group size and there areno matching reservations, then the group creates VM instances in a zone whereresources are available.

Decreasing group size

When you decrease group size, the group deletes the VM instances in thefollowing order:

1. VMs that are not running for any reason
2. VMs that are not yet updated to the intended version
3. VMs chosen nondeterministically

If you need to decrease group size in specific zones or remove specific VMinstances, for example workers that finished their job, you candelete specific instancesfrom the group.

Impact of temporarily unavailable resources

With a target distribution shape set toANY, the group schedules VM instancecreation in zones where the requested resources are available and avoids zoneswith temporarily unavailable resources.

If your request cannot be fulfilled in any zone in the region, the groupschedules VM creation in zones with temporarily unavailable resources. Thegroup will keep trying to create the scheduled instances within the zones wheretheir creation was originally scheduled. If the resources become available inother zones sooner than in the original zone where a VM was scheduled, the groupwon't try creation in those other zones. You can manually schedule new instancesin zones with available capacity by deleting the non-running managed instancesand resizing the group up to its target size.

If VM creation is unsuccessful, you canlist managed instancesto review the error message in the corresponding VM instance orlist recent errors.

For example, the following diagram shows how a regional group schedulesinstances when a zone cannot fulfill your request.

Impact of zone-level failure

With its target distribution shape set toANY, the group might deploy themajority or all of its instances in a single zone. In the event of failure inthat zone, most or all of the group's instances could become unavailable for theduration of the failure.

In case of a zone-level failure or resources becoming temporarily unavailable,or when for any reason your VM instances are not running, you can delete theindividual non-running instances then resize the group back to its necessarysize in order to try to get replacement instances in zones with availablecapacity.

Note: If your workload must survive zone-level failure without temporarilydegraded performance, use anEVEN orBALANCED target distribution shape, andprovision extra capacity.

TheANY_SINGLE_ZONE distribution shape

A regional MIG with theANY_SINGLE_ZONE target distribution shapeautomatically selects the optimal zone when the first VM in the group iscreated. After the first VM is created, all other VMs are created in thesame zone.

The MIG can select a different zone only when it is scaled back to zeroVMs and it starts creating its first VM again.

Choosing the optimal zone

When only one of the selected zones supports the group's hardware requirementsthen Compute Engine chooses this zone.

When multiple selected zones support the group's hardware requirements thenCompute Engine chooses a zone that has enough available resources tofit all of the regional MIG's VMs, with a preference for the zone with the mostmatching reservations.

If none of the selected zones has enough available resources or matchingreservations to accommodate all VMs then, in order to createas many VMs as possible, Compute Engine chooses the zone with the mostavailable resources and matching reservations, with a preference for the zonewith the most matching reservations. The group continues to try to create theremainder of the VMs in the same zone even if resources become available soonerin another zone.

Resizing a MIG that has anANY_SINGLE_ZONE distribution shape

Increasing group size

If a MIG already has VMs in it and has its target distribution shape set toANY_SINGLE_ZONE, then for all scale-out operations, the MIG places new VMswithin the same zone as existing VMs. If there are not enough availableresources or reservations to accommodate all additional VMs, the MIG creates asmany as possible.

If the MIG has no VMs, then when it is scaled out, it chooses the optimal zonethat supports the group's hardware requirements and that utilizes matchingreservations.

Decreasing group size

When scaling in, a regional MIG with theANY_SINGLE_ZONE distribution shaperemoves VMs in the following order:

• VMs that are not inRUNNING state are removed first to limit thedisruption to your workload. A not-running VM is a VM that forany reason can't be created or is being created or repaired.
• VMs that don't use the group's latest configuration.
• VMs chosen nondeterministically

Impact of temporarily unavailable resources

A regional MIG with its target distribution shape set toANY_SINGLE_ZONE issusceptible to resource shortages in the selected zone.

If resources become temporarily unavailable in the MIG's chosen zone, the MIGdoes not automatically switch zones. This means that scale-out and updateprocesses can become disrupted until sufficient resources become available.

Impact of zone-level failure

A regional MIG with its target distribution shape set toANY_SINGLE_ZONE issusceptible to zonal failures.

In the unlikely event of a zonal failure in the zone that hosts yourregional MIG's VMs, all of the MIG's VMs might become unable toprocess your workload.

What's next

• To learn how to set the target shape on a MIG, seeSetting a policy for distributing VMs across zones.
• Read more aboutRegional MIGs.
• Learn aboutzonal reservationsof Compute Engine resources.