Perform failover tests for highly resilient environments

Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page describes how to perform a database failover test forhighly resilient (Highly Available) environments.

Failover tests for your environment simulate a complete outage of a zone in adata center. In such a scenario, a zonal outage of a database might happen. Byperforming a failover test, you can monitor how your highly resilientenvironment performs a failover and check how this affects your DAGs and tasks.

Before you begin

  • To perform failover tests, your Google Account must have the following rolesand permissions:

  • In Cloud Composer 3, the environment's cluster is located in the tenantproject of your environment and it's not possible to simulate a zonaloutage for it.

Check that your environment is healthy

Make sure to perform failover tests only on healthy environments. To check thatyour environment is healthy:

  1. In the Google Cloud console, go to theEnvironments page.

    Go to Environments

  2. In the list of environments, click the name of your environment.TheEnvironment details page opens.

  3. Go to theMonitoring tab.

  4. Make sure that all health metrics are green.

Perform a database failover test

You can perform a database failover test, which simulates a zonal outage, bytriggering it with a Google Cloud CLI command. For example, you might want todo it to measure the amount of time it takes for your environment's databaseto switch to another zone.

Note: The database failover test is not intended for troubleshooting; it is notpossible to fix issues with the database component of an environment with thefailover test.

To perform a database failover test for your environment:

  1. Make sure thatyour environment is healthy.

  2. Get the primary zone of your environment's database:

    gcloudcomposerenvironmentsfetch-database-properties\ENVIRONMENT_NAME\--locationLOCATION

    Replace the following:

    • ENVIRONMENT_NAME: the name of your Cloud Composer environment.
    • LOCATION: the region where the environment is located.

    Example:

    gcloudcomposerenvironmentsfetch-database-properties\example-environment\--locationus-central1

  3. Start the database failover test:

    gcloudcomposerenvironmentsdatabase-failover\ENVIRONMENT_NAME\--locationLOCATION

    Replace the following:

    • ENVIRONMENT_NAME: the name of your Cloud Composer environment.
    • LOCATION: the region where the environment is located.

    Example:

    gcloudcomposerenvironmentsdatabase-failover\example-environment\--locationus-central1

  4. Wait until the database failover test is finished. The process can take upto 3 minutes.

  5. Check that the primary zone of your environment's database is changed:

    gcloudcomposerenvironmentsfetch-database-properties\ENVIRONMENT_NAME\--locationLOCATION

  6. Check your environment's health metrics to make sure thatyour environment is healthy.

  7. Your environment's database becomes ready for another failover whentheDatabase available for failover(composer.googleapis.com/environment/database/available_for_failover)environment metric becomesTrue. For more information about viewing yourenvironment's metrics in Cloud Monitoring, seeMonitor environments.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.