Hyperdisk Storage Pools can be used with Compute Engine for large-scale storage. AHyperdisk Storage Pool is a pre-purchased collection of capacity, throughput, and IOPSwhich you can then provision to your applications as needed. You can useHyperdisk Storage Pools to create and manage disks in pools and use the disks acrossmultiple workloads. By managing disks in aggregate, you can savecosts while achieving expected capacity and performance growth. By using onlythe storage you need in Hyperdisk Storage Pools, you reduce the complexity of forecastingcapacity and reduce management toil by going from managing hundreds of disks tomanaging a single storage pool.

Storage pools include the following benefits:

• Lower Total Cost of Ownership (TCO)—Hyperdisk Storage Pools usethin provisioning and datareduction to help you store your data efficiently and achieve best in-classTCO.
• Higher efficiency—Hyperdisk Storage Pools can take advantage of thinprovisioning and data reduction to help you achieve higher resourceutilization and lower TCO.
• Less management overhead via Higher Flexibility—Disks inHyperdisk Storage Pools can be provisioned to larger sizes and only use what they need,freeing workload owners from tedious capacity and performance forecasting,and from downtime related to rescaling.
• Transparent to workloads—There is no change to how individual workloadsuse Hyperdisk volumes when using storage pools. There isno need for downtime or any other impact to workloads.

Data on individual disks in a storage pool maintain the same isolationthey would have if the disks were not created in a storage pool.

When to use storage pools

You can use storage pools to solve these problems:

• Difficulty planning the resource requirements when migrating workloadsfrom on-premise workloads that use a SAN to Google Cloud

  Estimating the performance and capacity needs of eachapplication is time consuming and error-prone, and can add weeks to acloud migration or application rollout.

  With Hyperdisk Storage Pools, you can create disks in the storage pool using anoverestimation of the capacity or performance needed, and then consume fromthe storage pool only the amount of disk space that you write data to orthe IOPS or throughput your workload uses.

• Underutilization of resources

  Ensuring that your volumes are well utilized can be complex and painfullydifficult. Block storage is frequently underutilized because you provisionfor peak capacity and performance needs to avoid outages or performanceslowdowns. However, many applications rarely achieve those peaks. Withthin-provisioned capacity and performance, and data reduction, Hyperdisk Storage Poolsmake it easier to increase your block storage utilization while simplifyingyour operations.

• Complex management of the block storage used by your workloads

  Managing hundreds or even thousands of disk volumes is time consuming andtakes resources away from delivering new innovation.

  When you create disks for your Compute Engine instances in astorage pool, you can specify a size or performance limit that is muchlarger than you expect the workload to use. As your workloads write to thedisks, space and performance resourcesare taken from the storage pool provisioned capacity and performance.Only the amount of block storage resources that you use is subtracted fromthe storage pool provisioned capacity and performance, not the amountthat you provisioned when creating the disks. The disk size and performanceyou specify when creating the disks in the storage pool acts as anupper limit on those resources. As a result, you only need to modify thesize or provisioned performance for a disk if that limit is reached.

  If you create disks in the storage pool, and your workloadexceeds your capacity and performance planning across several disks, you canadd more capacity or performance to the storage pool. The additionalcapacity and performance can be used by all the disks created in thestorage pool.

Hyperdisk Storage Pool features

Storage pools have the following features:

• Capacity and performancethin provisioning:Capacity and performance are allocated as needed instead of allocating allthe resources in advance. This helps to avoid low utilization rates ofstorage resources, where large amounts of disk space or performance areallocated, but not used.
• Data reduction: Storage pools use a variety of datareduction technologies to increase storage efficiency. Data reduction dependsgreatly on the type of data stored. Data that is already compressed orencrypted before it is stored in a disk in a Hyperdisk Storage Pool won't yieldadditional reduction.

How Hyperdisk Storage Pools work

You create a storage pool with the aggregate capacity and performance thatyour workloads will need, and then create disks in the storage pool. Youcan then attach the disks to your VMs. When you create the disks, you cancreate them with a much larger size or provisioned performance limit than isneeded. This simplifies planning and provides room for growth later, withoutnecessitating changing the disk's provisioned size or performance at alater date.

If your workloads grow and your disks need more capacity or performance, youcan increase the provisioned capacity and performance of the storage pool.The amount of disk space or performance used by the individual disks in thestorage pool can then increase up to the limit you specified when youcreated the disk. By creating the disks in anAdvanced capacity or Advanced performance storage poolwith a much larger size or performance limit initially, andallocating additional space and performance through the storage pool,you consolidate disk storage management and reduce costs.

If an Advanced capacity storage pool reaches 100% utilization and there is nofree space left in the storage pool, writes to all disks in thestorage pool fail until you reduce the used space in the storage poolby deleting data or disks. Most commercial software interprets the errorsreturned when the pool space is exhausted in a manner similar tohardware failures.

It's important that you actively manage and monitor your storage pool toavoid out of space errors. You should also understand how your workload willrespond if it encounters an out of space error for a disk in astorage pool.

When the aggregate performance utilization of all the disks in thestorage pool reaches the total amount of performance provisioned forthe storage pool, the disks can encounter performance contention.If contention for performance resources is detected in an Advanced performancestorage pool for any disks in the pools, then the auto-grow featureattempts to automatically increase the IOPS available to disks inthe storage pool to prevent performance issues.

Provisioning types for Hyperdisk Storage Pools

When creating a Hyperdisk Storage Pool, you can configure it with either standard oradvanced provisioning for capacity and performance.

Standard capacity storage pools

With Standard capacity provisioning, you create disks in the storage pool untilthe total provisioned capacity of all disks in the storage pool reaches thestorage pool's provisioned capacity. Disks in a Standard capacity storage poolconsume capacity similarly to non-storage pool disks.

Advanced capacity storage pools

Disks in an Advanced capacity storage pool consume capacity differentlythan disks in a Standard capacity storage pool and non-storage pool disks.Advanced capacity storage pools offer thin-provisioning and data reduction forcapacity, enabling you to provision disks with more capacity than you havepurchased. Disks in an Advanced capacity storage pools consume capacity based onlyon the number of bytes written to your disks after data reduction, which meansyou can provision more capacity to your end users and applications than you havepurchased in your storage pool.

With Advanced capacity, you create disks in the storage pool where the cumulativesize of all the disks can exceed the provisioned capacity of thestorage pool by up to 1,000%. The used capacity of the storage pool isdefined by the amount of data written and not by the amount of provisioned diskcapacity.

You can fill disks in an Advanced capacity storage pool up to their provisionedsize as long as the data written to all disks in the storage pool doesn'texceed the storage pool capacity. If the storage pool utilizationreaches 80% of the pool provisioned capacity, the auto-grow feature attempts toautomatically add capacity to the storage pool.If thestorage pool's capacity is fully used, then writes to all storage pooldisks will fail until you delete data or delete disks to lower the used capacityof the storage pool. Most software applications interpret the errorsreturned when the pool space is exhausted in a manner similar to hardwarefailures, so it's important to both:

• Monitor your storage pool to avoid running out of disk space
• Understand how your workload responds if it does happen

Storage pools don't have visibility into your file system, so deleted datais considered to still be in use until your operating system (OS)marks it unused with aDISCARD orTRIM command. All Google-provided OSimages are configured to do this by default, as are most common third-party OSimages, but you should confirm this if you are not using a Google-provided OSimage. More information on verifying or configuring this feature isavailable atDisable lazy initialization and enable DISCARD commands.

Standard performance storage pools

Standard performance provisioning is the best option for the following types of workloads:

• Workloads that can't succeed if performance is limited by storage pool resources
• Workloads where the disks in the storage pool are likely to have correlated performance spikes, for example, data disks for databases that are at peak utilization every morning.

With Standard performance provisioning, thin-provisioning isn't available. Also,disks created in a Standard performance storage pool don't share performanceresources with the rest of the storage pool. The aggregate amount ofperformance of all disks created in the storage pool can't exceed the totalprovisioned IOPS or throughput of the storage pool.

When creating the storage pool, you provision enough performance in thestorage pool to cover the combined peak performance needs of all the disksin the storage pool. When you create disks in the storage pool, youprovision an amount of performance that covers the peak performance requirementsof that disk. You can continue creating disks in the storage pool untilthe total provisioned IOPS of all disks reaches the storage pool'sprovisioned amount.

In a Hyperdisk Balanced Storage Pool with Standard performance, the first 3,000 IOPS and140 MiB/s of throughput of each disk in the storage pool(thebaseline performance) don'tconsume storage pool resources. When you create disks in thestorage pool, any IOPS and throughput in excess of the baseline valuesconsume IOPS and throughput from the storage pool.

Advanced performance storage pools

Advanced performance provisioning is best suited for workloads that don't have highlycorrelated peak usage times. If your workloads all peak at the same time, theAdvanced performance storage pool can reach the performance limits of thestorage pool, resulting in contention for performance resources.

Advanced performance IOPS and throughput provisioning lets you share a pool ofprovisioned performance across all disks in a storage pool. Thestorage pool dynamically allocates performance resources as the disks inthe storage pool read and write data. Only the amount of IOPS andthroughput used by a disk in the storage poolconsumes storage pool performance.

Because Advanced performance storage pools are thinly provisioned, youcan allocate more IOPS or throughput to the disks in the storage poolthan you have provisioned for the storage pool—up to 500% ofthe IOPS or throughput provisioned for the storage pool. For example,if you provision 100,000 IOPS for a storage pool, you could have 10disks in the storage pool, each provisioned with 50,000 IOPS.

In a Hyperdisk Balanced Storage Pool with Advanced performance provisioning, the disks don't have baselineperformance. Every read and write operation of a Hyperdisk Balanced disk in thestorage pool consumes provisioned storage pool resources.

At any given moment, when the aggregate performance used by all the disks in thestorage pool reaches the total amount of performance provisioned for thestorage pool, the disks can contend with each other for performanceresources. As a result, the disks are not able to achieve the maximumperformance levels that you provisioned for the disks. When contention forresources is detected for a prolonged period of time, Compute Engine attemptsto automatically add performance to the storage pool. When usingAdvanced performance storage pools, you should:

• Monitor your storage pools to avoid running out of provisionedperformance
• Understand how your workload responds if it does happen

Example

Assume that you have a Hyperdisk Balanced Storage Pool with 100,000 provisioned IOPS.

With Standard performance provisioning:

• You can provision up to 100,000 of aggregate IOPS when creating Hyperdisk Balanced disks in the storage pool.
• You are charged for the 100,000 IOPS of Hyperdisk Balanced Storage Pool provisioned performance.

• Like disks created outside of a storage pool, Hyperdisk Balanced disks in Standard performance storage pools are automatically provisioned with up to 3,000 baseline IOPS and 140 MiB/s of baseline throughput. This baseline performance isn't counted against the provisioned performance for the storage pool. Only when you add disks to the storage pool with provisioned performance that's above the baseline does it count against the provisioned performance for the storage pool, for example:

  • A disk provisioned with 3,000 IOPS uses 0 pool IOPS and the pool still has 100,000 provisioned IOPS available for other disks.
  • A disk provisioned with 13,000 IOPS uses 10,000 pool IOPS and the pool has 90,000 provisioned IOPS remaining that you can allocate to other disks in the storage pool.

With Advanced performance provisioning:

• You can provision up to 500,000 IOPS of aggregate Hyperdisk performance when creating disks in the storage pool.
• You are charged for 100,000 IOPS provisioned by the storage pool.
• If you create a single disk (Disk1) in the storage pool that has 5,000 IOPS, you don't consume any IOPS from the storage pool provisioned IOPS. However, the amount of IOPS that you can provision to new disks created in the storage pool is now 495,000.
• IfDisk1 starts to read and write data, and if it uses its maximum of 5,000 IOPS in a given minute, then 5,000 IOPS is consumed from the storage pool provisioned IOPS. Any other disks that you created in the same storage pool can use an aggregated maximum of 95,000 IOPS in that same minute without running into contention.

Types of Hyperdisk Storage Pools

The type of Hyperdisk Storage Pool that you create determines the type of disks that youcan create in the storage pool.

• Hyperdisk Throughput Storage Pool: When creating the storage pool, you specify thecapacity and throughput to provision for the storage pool. Each Hyperdisk Throughputdisk you create in the storage pool uses some of the provisionedcapacity and throughput.
• Hyperdisk Balanced Storage Pool: When creating the storage pool, you specify thecapacity, throughput, and IOPS to provision for the storage pool. EachHyperdisk Balanced disk that you create in the storage pool with provisionedcapacity and performance above thebaseline valuesuses some of the storage pool provisioned capacity and performance.

Hyperdisk Throughput Storage Pools

You can use Hyperdisk Throughput Storage Pools to manage your Hyperdisk Throughput disk usage.

Machine type support

Hyperdisk Throughput Storage Pools are supported with the same machine series that support Hyperdisk Throughput.For a list of the supported machine series, seeMachine type support for Hyperdisk Throughput.

Hyperdisk Throughput Storage Pool regional availability

Hyperdisk Throughput Storage Pools can be used in any zone that offers Hyperdisk Throughput disks. For a list ofthe available regions, seeRegional availability for Hyperdisk Throughput.

You can also check for the latest updates to the available regions and zones byusing the following command:

gcloud compute storage-pool-types list --filter="name=hyperdisk-throughput"

Hyperdisk Balanced Storage Pools

You can use Hyperdisk Balanced Storage Pools to manage your Hyperdisk Balanced disk usage.

In Hyperdisk Balanced Storage Pools with Standard performance provisioning, Hyperdisk Balanced disks only consume IOPS andthroughput in excess of the baseline values of 3,000 IOPS and140 MiB/s throughput per disk. For example:

• If you provision a Hyperdisk Balanced disk with 3,000 IOPS and140 MiB/s throughput in a storage pool with Standard performanceprovisioning, the disk doesn't consume any IOPS or throughput from the storage pool.
• If you provision a Hyperdisk Balanced disk with 4,000 IOPS and 180 MiB/s ofthroughput in a storage pool with Standard performance provisioning, the diskconsumes 1,000 IOPS and 40 MiB/s of throughput from thestorage pool.

There's a limit to how much baseline performance disks in Hyperdisk Balanced Storage Pools withStandard capacity, Standard performance, or Advanced capacity provisioning can consume at the same time.Compute Engine enforces a concurrent consumption limit for baseline performance for a project's Hyperdisk Balancedand Hyperdisk Balanced High Availability volumes that are in the same zone. The limits are 500,000 IOPS and 50 GiB/s ofthroughput. For more information, seeConcurrent consumption limits for baseline performance.

Machine type support

Hyperdisk Balanced Storage Pools are supported with the same machine series that support Hyperdisk Balanced.For a list of the supported machine series, seeMachine series support for Hyperdisk Balanced.

Hyperdisk Balanced Storage Pool regional availability

Hyperdisk Balanced Storage Pools can be used in any zone that offers Hyperdisk Balanced. For a list ofthe available regions, seeRegional availability for Hyperdisk Balanced.

You can also check for the latest updates to the available regions and zones byusing the following command:

gcloud compute storage-pool-types list --filter="name=hyperdisk-balanced"

Limits for storage pools

The following tables list the limits for the different storage pool types.

Performance

Performance for disks in a storage pool is the same as for disks that werenot created in a storage pool.

Limitations of storage pools

Hyperdisk Storage Pools have the following limitations:

Resource limits:

• You can create a Hyperdisk Storage Pool with up to 5 PiB of provisioned capacity.
• You can create a maximum of 5 storage pools per hour.
• You can create a maximum of 10 storage pools per day.
• You can create at most 10 storage pools per project.
• You can't change the provisioning model for a pool; you can't change a Standard capacity storage pool to an Advanced capacity storage pool or an Advanced performance storage pool to a Standard performance storage pool.
• Storage pools are a zonal resource.
• You can create up to 10,000 disks in a storage pool.
• You can use Hyperdisk Storage Pools with only Compute Engine. Cloud SQL instances can't use Hyperdisk Storage Pools.
• You can change the provisioned capacity or performance of a storage pool at most two times in a 24 hour period.

Limits for disks in a storage pool:

• Only new disks in the same project and zone can be created in a storage pool.
• Moving disks in or out of a storage pool is not permitted. To move a disk in or out of a storage pool, you have to recreate the disk from a snapshot. For more information, seeChange the disk type.
• To create boot disks in a storage pool, you must use a Hyperdisk Balanced Storage Pool.
• Storage pools don't supportregional disks.
• You can'tclone,create instant snapshots of, orconfigure Asynchronous Replication for disks in a storage pool.

Reservations

You can use Hyperdisk Storage Pool withreservations.You can create a compute instance that consumes a reservation and also uses aHyperdisk in a storage pool. Alternatively, you can createthe instance that consumes a reservation, and afterwards attach aHyperdisk that was created in a storage pool to theinstance. For more information, see how toconsume reservations.

Pricing

Hyperdisk Storage Pools are billed monthly for the provisioned storage pool capacity,throughput and IOPS. You are not billed for the provisioned IOPS, throughput, orcapacity for the disks created in the storage pool.

With a Standard capacity and Standard performance storage pools, capacity and provisionedperformance is priced at the same rate as the underlying disk. For example,the price for Standard capacity for a Hyperdisk Balanced Storage Pool is the same price as standalone Hyperdisk Balanceddisk capacity.

Advanced capacity and Advanced performance storage pools are priced higher because ofthin-provisioning and data reduction. Even with thispremium, the savings from thin-provisioning and data reduction can still reduceyour total block storage costs through increased efficiency and utilization.

For more pricing information, seeDisk pricing.

Committed use discounts with Hyperdisk Storage Pools

Hyperdisk Storage Pools are not eligible for:

• Resource-based committed use discounts (CUDs)
• Sustained use discounts (SUDs)

What's next?

• Learn how tocreate a Hyperdisk Storage Pool.
• Learn how to move existing disks to a storage pool, as described inChange the disk type.
• ReviewDisk pricing information.