Overview

Database Migration Service uses migration jobs to migrate data from your source database instance to the destination Cloud SQL database instance.

Creating a migration job includes:

• Defining settings for the migration job
• Specifying information about the connection profile that you created for your source database (source connection profile)
• Defining settings for the destination Cloud SQL database instance and creating the instance
• Setting up connectivity between the source and destination database instances
• Testing the migration job to ensure that the connection information you provided for the job is valid

Database Migration Service wizard helps you create a migration job. The wizard consists of the following panes:Get started,Define a source,Define a destination,Define connectivity method,Configure migration databases, andTest and create migration job. Information on how to populate each pane is provided in the various sections of this page.

You can pause the creation of a migration job by clickingSave and exit. All of the data that was populated until that point is saved in a draft migration job, and you can access this job on theDrafts tab.

To finish creating a migration job, navigate to the tab, and then click the job. The creation flow resumes from where it was left off. The job remains a draft until you clickCreate orCreate and start.

Define settings for the migration job

1. Go to theMigration jobs page in theGoogle Cloud Console.
2. ClickCreate migration job at the top of the page.

3. Provide a name for the migration job.

   Choose a friendly name that helps you identify the migration job. Don'tinclude sensitive or personally identifiable information in the job name.

4. Keep the auto-generatedMigration job ID.

5. Select the source database engine.

6. Select the destination database engine.

7. Select the destinationregion for yourmigration. This is where the Database Migration Service instance is created, and shouldbe selected based on the location of the services that need your data, suchas Compute Engine instances and App Engine apps, and other services. Afteryou choose the destination region, this selection can't be changed.

   Important: If you plan to use the Cloud SQL for PostgreSQL Enterprise Plus edition, make sure your regionis supported for that edition.SeeCloud SQL for PostgreSQL Enterprise Plus edition region support.

8. Specify the migration job type:One-time (snapshot only) orContinuous(snapshot + ongoing changes).

9. Review the required prerequisites that are generated automatically to reflect how the environment must be prepared for a migration job. These prerequisites can include how to configure the source database and how to connect it to the destination Cloud SQL database instance. It's best to complete these prerequisites at this step, but you can complete them at any time before you test the migration job or start it. For more information about these prerequisites, seeConfigure your source.

10. ClickSave and continue.

Specify information about the source connection profile

1. If you have created a connection profile, then select it from the list of existing connection profiles.

   If you haven't created a connection profile, then create one by clickingCreate a connection profile at the bottom of the drop-down list, and then perform the same steps as inCreate a source connection profile.

2. In theCustomize data dump configurations section, clickShow data dump configurations.

   The speed of data dump parallelism is related to the amount of load on your source database. You can use the following settings:

   • Optimal (recommended): Balanced performance with optimal load on the source database.
   • Maximum: Provides the highest dump speeds, but might cause increased load on the source database.
   • Minimum: Takes the lowest amount of compute resources on the source database, but might have slower dump throughput.

   If you want to use adjusted data dump parallelism settings, make sure to increase themax_replication_slots,max_wal_senders, andmax_worker_processes parameters on your source database. You can verify your configuration by running the migration job test at the end of migration job creation.

3. ClickSave and continue.

Define and create the destination Cloud SQL instance

Note:For Cloud SQL sources: If you are migrating from a Cloud SQLinstance that uses a Private IP connection to a Cloud SQL instance that uses anon-RFC 1918 addressIP range, add the non-RFC 1918 range to the network configuration of your source Cloud SQL instance.SeeConfigure authorized networksin Cloud SQL documentation.

1. From theType of destination instance drop-down menu, selectNew instance.

   You can also migrate to an existing instance, seeMigration job for an existing instance.

2. Provide an ID for the Cloud SQL instance or use the auto-generated ID. Don't include sensitive or personally identifiable information in the ID; it's externally visible. There's no need to include the project ID in the instance name. This is done automatically where appropriate (for example, in the log files).

3. Provide an alphanumeric password for the destination Cloud SQL instance. This will be the password for thepostgres administrator account in the instance.

   You can either enter the password manually or clickGenerate to have Database Migration Service create one for you automatically.

4. Choose the database version for the destination instance from the list of supported Database Migration Service versions for the specified database engine.Learn more about cross-version migration support.
5. Select the Cloud SQL for PostgreSQL edition for your destination instance. There are twooptions available:Cloud SQL for PostgreSQL Enterprise edition andCloud SQL for PostgreSQL Enterprise Plus edition.

   Cloud SQL for PostgreSQL editions come with different sets of features,available machine types, and pricing. Make sure you consultthe Cloud SQL documentation to choose the edition that is appropriate for your needs.For more information, seeIntroduction to Cloud SQL for PostgreSQL editions.

6. The instance is created in the region that you selected when youdefined settings for the migration job. Select a zone within that region or leave the zone set toAny for Google to select one automatically.

   If you are configuring your instance forhigh availability, selectMultiple zones (Highly available). You can select both the primary and the secondary zone. The following conditions apply when the secondary zone is used during instance creation:

   • The zones default toAny for the primary zone andAny (different from primary) for the secondary zone.
   • If both the primary and secondary zones are specified, they must be different zones.

7. Choose whether to connect to this instance with private or public IP address.
1. If you're planning to connect with VPC-peering or Reverse SSH tunnel, then select thePrivate IP checkbox.
1. Make sure the following is true to enable private IP:
• Service Networking API is enabled. You can enable theService Networking API using the Google Cloud console.
• You have theservicenetworking.services.addPeering IAM permission.
• You haveconfigured private services access for your project, for which you need to have thecompute.networkAdmin IAM role.
• There's at least one non-legacy VPC network in your project, or a shared VPC network.
• If you are using aShared VPC network, then you also need to do the following:
  • Enable the Service Networking API for the host project.
  • Add your user to the host project.
  • Give your user the compute.networkAdmin IAM role in the host project.
2. Select the associated VPC network to peer. If you plan to connect to the migration source with VPC peering, then choose the VPC where the instance resides.
3. If a managed service network was never configured for the selected VPC, you can choose to either select an IP range and clickConnect or use an automatically selected IP range and clickAllocate & Connect.

2. If you plan to connect with IP allowlisting, selectPublic IP.

   Optionally, clickAuthorized networks, and either authorize a network or a proxy to connect to the Cloud SQL instance. Cloud SQL instances accept connections only from addresses that are authorized. For more information about configuring public access to the instance, seeConfigure public IP.

8. Select the machine type for the Cloud SQL instance. The disk size must be equal to or greater than the source database size.Note: Machines available in theMachine type selection depend on the Cloud SQL edition you use.

   Learn more aboutPostgreSQL machine types.

9. For Cloud SQL for PostgreSQL Enterprise Plus edition: Select theEnable data cache checkboxif you want to use the data cache feature in your destination database.

   Data cache is an optional feature available for Cloud SQL for PostgreSQL Enterprise Plus edition instancesthat adds a high-speed local solid state drive to your destination database.This feature can introduce additional costs to your Cloud SQL.For more information on data cache, seeData cache overview inCloud SQL documentation.

10. Specify the storage type for the Cloud SQL instance. You can choose either a solid-state drive (SSD) or a hard disk drive (HDD).
11. Specify the storage capacity (in GBytes) for the Cloud SQL instance.

    Important: Make sure the instance has enough storage capacity to handle the data from your source database. You can increase this capacity at any time, but you can't decrease it.

12. Optionally, clickSHOW OPTIONAL CONFIGURATIONS, and then:

    1. Specify whether you want to manage the encryption of the data that's migrated from the source to the destination. By default, your data is encrypted with a key that's managed by Google Cloud. If you want to manage your encryption, then you can use a customer-managed encryption key (CMEK). To do so:

       1. Select theUse a customer-managed encryption key (CMEK) check box.
       2. From theSelect a customer-managed key menu, select your CMEK.

       If you don't see your key, then clickENTER KEY RESOURCE NAME to provide the resource name of the key that you want to use. For example, you can enterprojects/my-project-name/locations/my-location/keyRings/my-keyring/cryptoKeys/my-key in theKey resource name field, and then clickSAVE.

       As part of creating the migration job, Database Migration Service will verify that the CMEK exists, and that Database Migration Service has permissions to use the key.

       If Database Migration Service doesn't have these permissions, then information will appear, specifying that the Database Migration Service service account can't use the CMEK. ClickGRANT to give Database Migration Service permissions to use the key.

       For more information about creating a CMEK, seeUsing customer-managed encryption keys (CMEK).

    2. Add any necessary flags that will be applied to the database server. If possible, make sure that the database flags on the created destination Cloud SQL instance are the same as those on the source database. Learn more about supported database flags forPostgreSQL.
    3. Add anylabels that are specific to the Cloud SQL instance.

       Labels help organize your instances. For example, you can organize labels by cost center or environment. Labels are also included in your bill so you can see the distribution of costs across your labels.

    4. ClickCREATE & CONTINUE.

       Creating read replicas andenabling high availability is available only after youpromote the migration.

13. In theCreate destination database window, clickCREATE DESTINATION & CONTINUE to create the new instance. This may take up to several minutes.

    This will create a Cloud SQL instance for which you will be charged according to the configuration specified. If the migration job creation flow is exited before completion, then use theCANCEL button to clean up the Cloud SQL instance. If the browser or tab is closed,Save and exit is clicked, or the screen is navigated away from, then the Cloud SQL instance isn't cleaned up and will need to be deleted manually from the Cloud SQL Instances page.

    After the destination instance is created, the Cloud SQL instance settings can no longer be edited in the Database Migration Service console. You can still access the instance directly in the Cloud SQL console to modify it.

14. Wait for the creation of the destination instance to finish.

Set up connectivity between the source and destination database instances

If you're reusing a source connection profile that was used previously for heterogeneous migrations (for example, from Oracle to Cloud SQL to PostgreSQL), then in this step, you must choose a connectivity method. This method will be used for the homogeneous migration from one Cloud SQL for PostgreSQL database instance to another. You must select a connectivity method because the heterogeneous connectivity method isn't relevant for this migration job.

1. From theConnectivity method drop-down menu, select a network connectivity method. This method defines how the newly created Cloud SQL instance will connect to the source database. Current network connectivity methods includeIP allowlist, reverse SSH tunnel, and VPC peering.

   • If you select the IP allowlist network connectivity method, you need to specify the outgoing IP address of your destination instance. If the Cloud SQL instance you created is a high availability instance, include the outgoing IP addresses for both the primary and the secondary instance.

   • If you select the reverse SSH tunnel network connectivity method, then select the Compute Engine VM instance that will host the tunnel.

     After specifying the instance, Google will provide a script that performs the steps to set up the tunnel between the source and destination databases. You'll need to run the script in theGoogle Cloud CLI.

     Run the commands from a machine that has connectivity to both the source database and to Google Cloud.

   • If you select the VPC peering network connectivity method, then select the VPC network where the source database resides. The Cloud SQL instance will be updated to connect to this network.
   • Learn more about how toConfigure connectivity.

2. After selecting the network connectivity method and providing any additional information for the method, clickCONFIGURE & CONTINUE.

Configure migration databases

You can select the databases that you want to migrate.

1. From theDatabases to migrate list, select one of the following options:
   • All databases: Selects all databases that exist on the source.
   • Specific databases: Allows you to select specific databases from all databases that exist on the source.

2. If you want to migrate specific databases, you can filter the list that appears and select the databases that you want Database Migration Service to migrate into a destination.

   If the list doesn't appear and a database discovery error is displayed, clickReload. If database discovery fails, the job migrates all databases. You can continue with creating a migration job and fix connectivity errors later.

3. ClickSave and continue.

Test and create the migration job

On this final step, review the summary of the migration job settings, source, destination, and connectivity method, and then test the validity of the migration job setup. If any issuesare encountered, then you can modify the migration job's settings. Not all settings are editable.

1. ClickTEST JOB to verify that:

   • The source database has been configured correctly, based on the prerequisites.
   • The source and destination instances can communicate with eachother.
   • Any updates to private or public IP addresses needed on the destinationare done.

   • The migration job is valid, and the source and destination versions arecompatible.

   If the test fails, then you can address the problem in the appropriatepart of the flow, and return to re-test.

   For more information about reasons why the test may fail and how to troubleshoot any issues associated with the test failing, seeDiagnose issues for PostgreSQL.

   The migration job can be created even if the test fails, but after thejob is started, it may fail at some point during the run.

2. ClickCREATE & START JOB to create the migration job andstart it immediately, or clickCREATE JOB to create the migration job without immediatelystarting it.

   If the job isn't started at the time that it's created, then it can be started from theMigration jobs page by clickingSTART.

   Regardless of when the migration job starts, your organization ischarged for the existence of the destination instance.

   Key Point:When you start the migration job, Database Migration Servicebegins the full dump, briefly locking the source database. If your sourceis in Amazon RDS or Amazon Aurora, Database Migration Service additionally requires a short(approximately under a minute) write downtime at the start of the migration.

   For more information, seeKnown limitations.

3. The migration job is added to the migration jobs list and can be vieweddirectly.