Overview

Database Migration Service uses migration jobs to migrate data from your sourcedatabase instance to the destination database instance.

Creating a migration job for an existing destination instance includes:

• Defining settings for the migration job
• Selecting the source database connection profile
• Selecting the existing destination database instance
• Demoting the existing instance to convert it into a read replica
• Setting up connectivity between the source and destination database instances
• Testing the migration job to ensure that the connection information youprovided for the job is valid

There are certain limitations that you shouldconsider when you want to migrate to a destination instancecreated outside of Database Migration Service. Forexample, your Cloud SQL destination instance must be empty or containonly system configuration data. For more information, seeKnown limitations.

Database Migration Service wizard helps you create a migration job. This wizard consists of the following panes:Get started,Define a source,Create a destination,Define connectivity method,Configure migration databases, andTest and create migration job. Information on how to populate each pane is provided in the following sections of this page.

Create a migration job by using the Google Cloud console

1. Go to theMigration jobsin the Google Cloud console.

2. ClickCreate migration job at the top of the page.

3. Provide a name for the migration job. Choose a friendly name that helpsyou identify the migration job. Don't include sensitive or personallyidentifiable information in the job name.

4. Keep the auto-generatedMigration job ID.

5. Select the source database engine.

6. Select the destination database engine.

7. Select the destination region for your migration. This region must be thesame as the one where your destination database is located.After you choose the destination region, this selection can'tbe changed.Important: If you plan to use the Cloud SQL for PostgreSQL Enterprise Plus edition, make sure your regionis supported for that edition.SeeCloud SQL for PostgreSQL Enterprise Plus edition region support.

8. Specify the migration job type:One-time (snapshot only) orContinuous(snapshot + ongoing changes).

9. In theBefore you continue, review the prerequisites section,clickOpen to view automatically generated instructions that can help guideyou through preparing your source database for the migration.It's best to complete these prerequisites at this step, but you can completethem at any time before you test, or start, the migration job.For more information,seeConfigure your source.

10. ClickSave and continue.

Specify information about the source connection profile

1. If you have created a connection profile, then select it from the list of existing connection profiles.

   If you haven't created a connection profile, then create one by clickingCreate a connection profile at the bottom of the drop-down list, and then perform the same steps as inCreate a source connection profile.

2. In theCustomize data dump configurations section, clickShow data dump configurations.

   The speed of data dump parallelism is related to the amount of load on your source database. You can use the following settings:

   • Optimal (recommended): Balanced performance with optimal load on the source database.
   • Maximum: Provides the highest dump speeds, but might cause increased load on the source database.
   • Minimum: Takes the lowest amount of compute resources on the source database, but might have slower dump throughput.

   If you want to use adjusted data dump parallelism settings, make sure to increase themax_replication_slots,max_wal_senders, andmax_worker_processes parameters on your source database. You can verify your configuration by running the migration job test at the end of migration job creation.

3. ClickSave and continue.

Select the destination instance

1. From theType of destination instance menu, selectExisting instance.
2. In theSelect destination instance section, select your destination instance.
3. Review the information in theInstance details section, and clickSelect and continue.
4. To migrate to an existing destination database, Database Migration Service demotes the target instance and converts it to a replica. To signify that the demotion can be safely performed, in the confirmation window, enter the destination instance identifier.
5. ClickConfirm and continue.

Set up connectivity between the source and destination database instances

1. From theConnectivity method menu, select a network connectivity method. This method defines how the newly created Cloud SQL instance will connect to the source database. Current network connectivity methods includeIP allowlist, reverse SSH tunnel, and VPC peering.

   • If you select the IP allowlist network connectivity method, you need to specify the outgoing IP address of your destination instance. If the Cloud SQL instance you created is a high availability instance, include the outgoing IP addresses for both the primary and the secondary instance.

   • If you select the reverse SSH tunnel network connectivity method, then select the Compute Engine VM instance that will host the tunnel.

     After specifying the instance, Google will provide a script that performs the steps to set up the tunnel between the source and destination databases. You'll need to run the script in theGoogle Cloud CLI.

     Run the commands from a machine that has connectivity to both the source database and to Google Cloud.

   • If you select the VPC peering network connectivity method, then select the VPC network where the source database resides. The Cloud SQL instance will be updated to connect to this network.
   • Learn more about how toConfigure connectivity.

2. After selecting the network connectivity method and providing any additional information for the method, clickCONFIGURE & CONTINUE.

Configure migration databases

You can select the databases that you want to migrate.

1. From theDatabases to migrate list, select one of the following options:
   • All databases: Selects all databases that exist on the source.
   • Specific databases: Lets you select specific databases from all databases that exist on the source.

2. If you want to migrate specific databases, you can filter the list that appears and select the databases that you want Database Migration Service to migrate into your destination.

   If the list doesn't appear and a database discovery error is displayed, clickReload. If database discovery fails, the job migrates all databases. You can continue with creating a migration job and fix connectivity errors later.

3. ClickSave and continue.

Test and create the migration job

On this final step, review the summary of the migration job settings, source,destination, and connectivity method, and then test the validity of the migrationjob setup. If any issues are encountered, then you can modify the migrationjob's settings. Not all settings are editable.

1. ClickTEST JOB to verify that:

   • The source database has been configured correctly, based on the prerequisites.
   • The source and destination instances can communicate with eachother.
   • Any updates to private or public IP addresses needed on the destinationare done.
   • The migration job is valid, and the source and destination versions arecompatible.

   If the test fails, then you can address the problem in the appropriatepart of the flow, and return to re-test.

   For more information about reasons why the testmight fail and how to troubleshoot any issues associated with the testfailing, seeDiagnose issues for PostgreSQL.

   The migration job can be created even if the test fails, but after thejob is started, it may fail at some point during the run.

2. ClickCREATE & START JOB to create the migration job andstart it immediately, or clickCREATE JOB to create the migrationjob without immediatelystarting it.

   Caution: If you used Terraform to provision your destination database,you might experience configuration drift during the migration job execution.Don't try to re-apply Terraform settings before the migration is complete.For more information,seeTerraform configuration drift.

   If the job isn't started at the time that it's created, then it can be started from theMigration jobs page by clickingSTART.

   Regardless of when the migration job starts, your organization ischarged for the existence of the destination instance.

   When you start the migration job, Database Migration Servicebegins the full dump, briefly locking the source database. If your sourceis in Amazon RDS or Amazon Aurora, Database Migration Service additionally requiresa short (approximately under a minute) write downtime at the start ofthe migration. For more information, seeData dump parallelism considerations.

3. The migration job is added to the migration jobs list and can be vieweddirectly.

4. Proceed toReview the migration job.

Create a migration job by using Google Cloud CLI

When you migrate to an existing instance by using Google Cloud CLI,you must manually create the connection profile for the destination instance.This isn't required when you use the Google Cloud console, asDatabase Migration Service takes care of creating and removingthe destination connection profile for you.

Before you begin

Before you use gcloud CLI to create a migration job to anexisting destination database instance, make sure you:

• Create your destination database instance.
• Prepare your source database instance. See:
  • Configure your source
  • Create the source connection profile (Source connection profile identifier is required to create a migration job.)
  • Configure connectivity

Create destination connection profile

Create the destination connection profile for your existing destination instanceby running thegcloud database-migration connection-profiles createcommand:

This sample uses the optional--no-async flag so that all operationsare performed synchronously. This means that some commands might takea while to complete. You can skip the--no-async flag to run commands asynchronously.If you do, you need to use thegcloud database-migration operations describe command to verify if your operationis successful.

Before using any of the command data below, make the following replacements:

• CONNECTION_PROFILE_ID with a machine-readable identifier for your connection profile.
• REGION with the identifier of the region where you want to save the connection profile.
• DESTINATION_INSTANCE_ID with the instance identifier of your destination instance.
• (Optional)CONNECTION_PROFILE_NAME with a human-readable name for your connection profile. This value is displayed in the Google Cloud console.

Execute the following command:

gclouddatabase-migrationconnection-profiles\createpostgresqlCONNECTION_PROFILE_ID\--no-async\--cloudsql-instance=DESTINATION_INSTANCE_ID\--region=REGION\--display-name=CONNECTION_PROFILE_NAME

gclouddatabase-migrationconnection-profiles`createpostgresqlCONNECTION_PROFILE_ID`--no-async`--cloudsql-instance=DESTINATION_INSTANCE_ID`--region=REGION`--display-name=CONNECTION_PROFILE_NAME

gclouddatabase-migrationconnection-profiles^createpostgresqlCONNECTION_PROFILE_ID^--no-async^--cloudsql-instance=DESTINATION_INSTANCE_ID^--region=REGION^--display-name=CONNECTION_PROFILE_NAME

You should receive a response similar to the following:

Waiting for connection profile [CONNECTION_PROFILE_ID]to be created with [OPERATION_ID]Waiting for operation [OPERATION_ID] to complete...done.Created connection profileCONNECTION_PROFILE_ID [OPERATION_ID]

Create the migration job

This sample uses the optional--no-async flag so that all operationsare performed synchronously. This means that some commands might takea while to complete. You can skip the--no-async flag to run commands asynchronously.If you do, you need to use thegcloud database-migration operations describe command to verify if your operationis successful.

Before using any of the command data below, make the following replacements:

• MIGRATION_JOB_ID with a machine-readable identifier for your migration job. You use this value to work with migration jobs by using Database Migration Service Google Cloud CLI commands or API.
• REGION with the region identifier where you want to save the migration job.
• MIGRATION_JOB_NAME with a human-readable name for your migration job. This value is displayed in Database Migration Service in the Google Cloud console.
• SOURCE_CONNECTION_PROFILE_ID with a machine-readable identifier of the source connection profile.
• DESTINATION_CONNECTION_PROFILE_ID with a machine-readable identifier of the destination connection profile.

• Optional: Database Migration Service migrates all databases in your source by default. If you want to migrate only specific databases, use the--databases-filter flag and specify their identifiers as a comma-separated list.

  For example:--databases-filter=my-business-database,my-other-database

  You can later edit migration jobs that you created with the--database-filter flag by using thegcloud database-migration migration-jobs update command.

Execute the following command:

gclouddatabase-migrationmigration-jobs\createMIGRATION_JOB_ID\--no-async\--region=REGION\--display-name=MIGRATION_JOB_NAME\--source=SOURCE_CONNECTION_PROFILE_ID\--destination=DESTINATION_CONNECTION_PROFILE_ID\--type=CONTINUOUS\

gclouddatabase-migrationmigration-jobs`createMIGRATION_JOB_ID`--no-async`--region=REGION`--display-name=MIGRATION_JOB_NAME`--source=SOURCE_CONNECTION_PROFILE_ID`--destination=DESTINATION_CONNECTION_PROFILE_ID`--type=CONTINUOUS`

gclouddatabase-migrationmigration-jobs^createMIGRATION_JOB_ID^--no-async^--region=REGION^--display-name=MIGRATION_JOB_NAME^--source=SOURCE_CONNECTION_PROFILE_ID^--destination=DESTINATION_CONNECTION_PROFILE_ID^--type=CONTINUOUS^

You should receive a response similar to the following:

Waiting for migration job [MIGRATION_JOB_ID]to be created with [OPERATION_ID]Waiting for operation [OPERATION_ID] to complete...done.Created migration jobMIGRATION_JOB_ID [OPERATION_ID]

Demote the destination database

Database Migration Service requires that the destination database instanceworks as a read replica for the time of migration. Before you start themigration job, run thegcloud database-migration migration-jobs demote-destinationcommand to demote the destination database instance.

Before using any of the command data below, make the following replacements:

• MIGRATION_JOB_ID with your migration job identifier.

  If you don't know the identifier, you can use thegcloud database-migration migration-jobs list command to list all migration jobs in a given region and view their identifiers.

• REGION with the identifier of the region where your connection profile is saved.

Execute the following command:

gclouddatabase-migrationmigration-jobs\demote-destinationMIGRATION_JOB_ID\--region=REGION

gclouddatabase-migrationmigration-jobs`demote-destinationMIGRATION_JOB_ID`--region=REGION

gclouddatabase-migrationmigration-jobs^demote-destinationMIGRATION_JOB_ID^--region=REGION

Result

The action is performed in an asynchronous manner. As such, this command returns an Operation entity that represents a long-running operation:

done: falsemetadata:  '@type': type.googleapis.com/google.cloud.clouddms.v1.OperationMetadata  apiVersion: v1  createTime: '2024-02-20T12:20:24.493106418Z'  requestedCancellation: false  target:MIGRATION_JOB_ID  verb: demote-destinationname:OPERATION_ID

To see if your operation is successful, you can query the returned operation object, or check the status of the migration job:

• Use thegcloud database-migration migration-jobs describe command to view the status of the migration job.
• Use thegcloud database-migration operations describe with theOPERATION_ID to see the status of the operation itself.

Manage migration jobs

At this point, your migration job is configured and connected to yourdestination database instance. You can manage it by using thefollowing operations:

1. Optional: Verify the migration job.
   We recommend that you first verify your migration job by running thegcloud database-migration migration-jobs verify command.

   For more information, expand the following section:

   gcloud database-migration migration-jobs verify

   Before using any of the command data below, make the following replacements:

   • MIGRATION_JOB_ID with your migration job identifier.

     If you don't know the identifier, you can use thegcloud database-migration migration-jobs list command to list all migration jobs in a given region and view their identifiers.

   • REGION with the identifier of the region where your connection profile is saved.

   Execute the following command:

   Linux, macOS, or Cloud Shell

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs\verifyMIGRATION_JOB_ID\--region=REGION

   Windows (PowerShell)

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs`verifyMIGRATION_JOB_ID`--region=REGION

   Windows (cmd.exe)

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs^verifyMIGRATION_JOB_ID^--region=REGION

   Result

   The action is performed in an asynchronous manner. As such, this command returns an Operation entity that represents a long-running operation:

   done: falsemetadata:  '@type': type.googleapis.com/google.cloud.clouddms.v1.OperationMetadata  apiVersion: v1  createTime: '2024-02-20T12:20:24.493106418Z'  requestedCancellation: false  target:MIGRATION_JOB_ID  verb: verifyname:OPERATION_ID

   To see if your operation is successful, you can query the returned operation object, or check the status of the migration job:

   • Use thegcloud database-migration migration-jobs describe command with theMIGRATION_JOB_ID to view the status of the migration job.
   • Use thegcloud database-migration operations describe command with theOPERATION_ID to see the status of the operation itself.

2. Optional: Retrieve information about databases selected for migration.
   When you migrate specific databases, Database Migration Service needs to retrieve the details about the databases that you selected for the migration job by using the--database-filter flag.

   Before you start the migration job, run thegcloud database-migration migration-jobs fetch-source-objects command.

   For more information, expand the following section:

   gcloud database-migration migration-jobs fetch-source-objects

   Before using any of the command data below, make the following replacements:

   • MIGRATION_JOB_ID with your migration job identifier.

     If you don't know the identifier, you can use thegcloud database-migration migration-jobs list command to list all migration jobs in a given region and view their identifiers.

   • REGION with the identifier of the region where your connection profile is saved.

   Execute the following command:

   Linux, macOS, or Cloud Shell

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs\fetch-source-objectsMIGRATION_JOB_ID\--region=REGION

   Windows (PowerShell)

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs`fetch-source-objectsMIGRATION_JOB_ID`--region=REGION

   Windows (cmd.exe)

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs^fetch-source-objectsMIGRATION_JOB_ID^--region=REGION

   Result

   The output is similar to the following:

   Waiting for migration jobMIGRATION_JOB_IDto fetch source objects withOPERATION_IDWaiting for operationOPERATION_ID to complete...done.SOURCE_OBJECT                                      STATE         PHASE              ERROR{'database': 'DATABASE_NAME', 'type': 'DATABASE'}  NOT_SELECTED  PHASE_UNSPECIFIED{'database': 'DATABASE_NAME', 'type': 'DATABASE'}  STOPPED       CDC                {'code': 1, 'message': 'Internal error'}

   To see if your operation is successful, you can query the returned operation object, or check the status of the migration job:

   • Use thegcloud database-migration migration-jobs describe command with theMIGRATION_JOB_ID to view the status of the migration job.
   • Use thegcloud database-migration operations describe command with theOPERATION_ID to see the status of the operation itself.

3. Start the migration job.
   Start the migration job by running thegcloud database-migration migration-jobs start command.

   For more information, expand the following section:

   gcloud database-migration migration-jobs start

   Before using any of the command data below, make the following replacements:

   • MIGRATION_JOB_ID with your migration job identifier.

     If you don't know the identifier, you can use thegcloud database-migration migration-jobs list command to list all migration jobs in a given region and view their identifiers.

   • REGION with the identifier of the region where your connection profile is saved.

   Execute the following command:

   Linux, macOS, or Cloud Shell

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs\startMIGRATION_JOB_ID\--region=REGION

   Windows (PowerShell)

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs`startMIGRATION_JOB_ID`--region=REGION

   Windows (cmd.exe)

   Note: Ensure you have initialized the Google Cloud CLI with authentication and a project by running eithergcloud init; orgcloud auth login andgcloud config set project.

   gclouddatabase-migrationmigration-jobs^startMIGRATION_JOB_ID^--region=REGION

   Result

   The action is performed in an asynchronous manner. As such, this command returns an Operation entity that represents a long-running operation:

   done: falsemetadata:  '@type': type.googleapis.com/google.cloud.clouddms.v1.OperationMetadata  apiVersion: v1  createTime: '2024-02-20T12:20:24.493106418Z'  requestedCancellation: false  target:MIGRATION_JOB_ID  verb: startname:OPERATION_ID

   To see if your operation is successful, you can query the returned operation object, or check the status of the migration job:

   • Use thegcloud database-migration migration-jobs describe command with theMIGRATION_JOB_ID to view the status of the migration job.
   • Use thegcloud database-migration operations describe command with theOPERATION_ID to see the status of the operation itself.

Promote the migration job

Once the migration reaches the Change Data Capture (CDC) phase,you can promote the destination databaseinstance from a read replica to a standalone instance.Run thegcloud database-migration migration-jobs promotecommand:

Before using any of the command data below, make the following replacements:

• MIGRATION_JOB_ID with your migration job identifier.

  If you don't know the identifier, you can use thegcloud database-migration migration-jobs list command to list all migration jobs in a given region and view their identifiers.

• REGION with the identifier of the region where your connection profile is saved.

Execute the following command:

gclouddatabase-migrationmigration-jobs\promoteMIGRATION_JOB_ID\--region=REGION

gclouddatabase-migrationmigration-jobs`promoteMIGRATION_JOB_ID`--region=REGION

gclouddatabase-migrationmigration-jobs^promoteMIGRATION_JOB_ID^--region=REGION

Result

The action is performed in an asynchronous manner. As such, this command returns an Operation entity that represents a long-running operation:

done: falsemetadata:  '@type': type.googleapis.com/google.cloud.clouddms.v1.OperationMetadata  apiVersion: v1  createTime: '2024-02-20T12:20:24.493106418Z'  requestedCancellation: false  target:MIGRATION_JOB_ID  verb: startname:OPERATION_ID

• Use thegcloud database-migration migration-jobs describe command with theMIGRATION_JOB_ID to view the status of the migration job.
• Use thegcloud database-migration operations describe command with theOPERATION_ID to see the status of the operation itself.