Migrate in-cluster to managed control plane on a new cluster

This tutorial shows you how to migrate an application from aGoogle Kubernetes Engine (GKE) cluster using in-cluster Cloud Service Mesh to a new clusterusing managed Cloud Service Mesh — Google's fully managed, Istio-compliant servicemesh.

In this tutorial, you:

1. Create a new Google Kubernetes Engine cluster, and install in-cluster Cloud Service Meshand the Cloud Service Mesh ingress gateway on the cluster. This cluster will actas your existing cluster that you want to migrate away from.
2. Deploy theOnline Boutiquesample application onto the cluster with in-cluster Cloud Service Mesh.
3. Create another Google Kubernetes Engine cluster, in the same Google Cloud project.
4. Provision managed Cloud Service Mesh on the second cluster and deploy theCloud Service Mesh ingress gateway.
5. Deploy Online Boutique onto the cluster with managed Cloud Service Mesh toreplicate the deployment from the cluster with in-cluster Cloud Service Mesh.
6. Shift 50% of user traffic from the cluster with in-cluster Cloud Service Mesh to thecluster with managed Cloud Service Mesh, by using Istio's traffic splittingcapabilities on the cluster with in-cluster Cloud Service Mesh.
7. Complete the migration from in-cluster Cloud Service Mesh to managed Cloud Service Mesh bypointing the domain name system (DNS) entry of the cluster within-cluster Cloud Service Mesh to the cluster with managed Cloud Service Mesh.

Canary deployment

"Canary deployment" is a technique used in software development to test a newversion of some software before releasing that new version to all users. Itinvolvesincrementally increasing the percentage of traffic sent to the newversion. In this tutorial, you will set up a new cluster with managedCloud Service Mesh and incrementally shift user traffic to it. You will startby directing 0% of user traffic to the new cluster, then 50%, and, finally, 100%.In production, you should use smaller and more increments. If at any point younotice that the new cluster is incapable of handling a percentage of traffic,you can rollback by reducing the percentage to 0%.

Canary control plane versus canary cluster

There are two commonly used strategies for migrations from in-cluster Cloud Service Meshto managed Cloud Service Mesh:

• Canary cluster migration: In this strategy, you create a new cluster andthen provision managed Cloud Service Mesh on it.
• Canary control plane migration: In this strategy, you provisionmanaged Cloud Service Mesh on the same cluster in whichin-cluster Cloud Service Mesh is installed. This is not a supported migration pathwith Cloud Service Mesh.

In this tutorial, you will walk through the canary cluster migration strategy.

Costs

This tutorial uses the following billable components of Google Cloud:

• Google Kubernetes Engine
• Cloud Service Mesh

When you finish this tutorial, you can avoid ongoing costs by deleting theresources you created. For more information, seeClean up.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the required APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the required APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

Launch Cloud Shell

In this tutorial you will useCloud Shell, which is a shellenvironment hosted on Google Cloud that lets you manage yourGoogle Cloud resources.

Cloud Shell comes preinstalled with theGoogle Cloud CLI,kubectl, andistioctlcommand-line tools. The gcloud CLI provides theprimary CLI for Google Cloud.

Open a Cloud Shell session from the upper-right corner of this page, clickterminal and then clickAcknowledge. A Cloud Shell session opens inside a frame lower on the page. Complete the following commands in that Cloud Shell session.

Download sample code

Clone the git repositories containing the Kubernetes and Istio resources you will use:

gitclonehttps://github.com/GoogleCloudPlatform/anthos-service-mesh-samples.gitgitclonehttps://github.com/GoogleCloudPlatform/microservices-demo.git

Set up the cluster with in-cluster Cloud Service Mesh

Create the cluster and install in-cluster Cloud Service Mesh

In the section, you create your cluster that uses in-cluster Cloud Service Mesh. Inpractice, this would be the cluster(s) that you are already using.

1. ReplacePROJECT_ID with yourproject ID and create anew cluster:

   gcloudcontainerclusterscreatecluster-with-in-cluster-asm\--project=PROJECT_ID\--zone=us-central1-a\--machine-type=e2-standard-4--num-nodes=2\--workload-pool=PROJECT_ID.svc.id.goog

   Note: This operation can take a few minutes to complete.

2. Rename the cluster context so that the cluster is easier to work with:

   kubectlconfigrename-context\gke_PROJECT_ID_us-central1-a_cluster-with-in-cluster-asm\cluster-with-in-cluster-asm

3. Check that the cluster context has been renamed:

   kubectlconfigget-contexts--output="name"

4. Download the version that installs Cloud Service Mesh 1.28.2 tothe current working directory:

   curlhttps://storage.googleapis.com/csm-artifacts/asm/asmcli_1.28 >asmcli

   You will be asked to type "y" and then press Enter.

   The output is similar to:

   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                              Dload  Upload   Total   Spent    Left  Speed100  167k  100  167k    0     0   701k      0 --:--:-- --:--:-- --:--:--  701k

5. Make theasmcli script executable:

   chmod+xasmcli

6. Install in-cluster Cloud Service Mesh usingasmcli:

   ./asmcliinstall\--project_idPROJECT_ID\--cluster_namecluster-with-in-cluster-asm\--cluster_locationus-central1-a\--output_dir.\--enable_all\--camesh_ca

   It can take several minutes for theasmcli tool to finish. The tool outputsinformational messages so you can follow its progress.

   Upon success, the output is similar to:

   ...asmcli: Successfully installed ASM.

Deploy Cloud Service Mesh's ingress gateway

1. You will deploy the Cloud Service Mesh's ingress gateway into a separatenamespace calledasm-ingress. Create the namespace:

   kubectl\--contextcluster-with-in-cluster-asm\createnamespaceasm-ingress

2. Use theistio.io/rev=asm-1282-4 label to add theasm-ingressnamespace to the service mesh and enable automatic sidecar proxy injection.

   kubectl \  --context cluster-with-in-cluster-asm \  label --overwrite namespace asm-ingress istio.io/rev=asm-1282-4

   The output is similar to:

   namespace/asm-ingress labeled

   Note: You may need to modify theistio.io/rev=asm-1282-4 label tomatch your in-cluster Cloud Service Mesh version. You can see the version suffixedin the name of youristiod-asm-* Service by runningkubectl get service --context=cluster-with-in-cluster-asm --namespace=istio-system

3. Deploy the Cloud Service Meshingress gateway:

   kubectl\--contextcluster-with-in-cluster-asm\--namespace=asm-ingress\apply-fanthos-service-mesh-samples/docs/shared/asm-ingress-gateway/asm-gateway-deployment-svc.yamlkubectl\--contextcluster-with-in-cluster-asm\--namespace=asm-ingress\apply-fanthos-service-mesh-samples/docs/shared/asm-ingress-gateway/gateway.yaml

   The output is similar to:

   serviceaccount/asm-ingressgateway createdservice/asm-ingressgateway createddeployment.apps/asm-ingressgateway createdgateway.networking.istio.io/asm-ingressgateway created

   Note: You can browse the resources in the officialCloud Service Mesh samples GitHub repository.

Deploy Online Boutique

1. You will deploy Online Boutique into a separate namespace calledonlineboutique. Create the namespace:

   kubectl\--contextcluster-with-in-cluster-asm\createnamespaceonlineboutique

2. Use theistio.io/rev=asm-1282-4 label to add theonlineboutiquenamespace to the service mesh and enable automatic sidecar proxy injection.

   kubectl \  --context cluster-with-in-cluster-asm \  label --overwrite namespace onlineboutique istio.io/rev=asm-1282-4

   The output is similar to:

   namespace/onlineboutique labeled

3. Deploy Online Boutique's 12 services, including the load generator thatimitates user traffic:

   kubectl\--contextcluster-with-in-cluster-asm\--namespace=onlineboutique\apply-fanthos-service-mesh-samples/docs/shared/online-boutique/kubernetes-manifests.yamlkubectl\--contextcluster-with-in-cluster-asm\--namespace=onlineboutique\apply-fanthos-service-mesh-samples/docs/shared/online-boutique/virtual-service.yaml

4. Get the external IP address of the Cloud Service Mesh ingress gateway:

   kubectl\--contextcluster-with-in-cluster-asm\--namespaceasm-ingress\getservice--outputjsonpath='{.items[0].status.loadBalancer.ingress[0].ip}'

5. Copy the external IP address of the ingress gateway, and access it throughyour web browser. You will see the Online Boutique sample app.

Set up the new cluster with managed Cloud Service Mesh

Create the cluster and provision managed Cloud Service Mesh

In this section, you create the cluster that you will migrate to. You willprovision managed Cloud Service Mesh, and deploy Online Boutique in order toreplicate the deployments from the cluster that uses in-cluster Cloud Service Mesh.

1. Create a new cluster:

   gcloudcontainerclusterscreatecluster-with-csm\--project=PROJECT_ID--zone=us-central1-a\--machine-type=e2-standard-4--num-nodes=2\--workload-poolPROJECT_ID.svc.id.goog

   Note: This operation can take a few minutes to complete.

2. Rename the cluster context so that the cluster is easier to work with:

   kubectlconfigrename-context\gke_PROJECT_ID_us-central1-a_cluster-with-csm\cluster-with-csm

3. Check that the cluster context has been renamed:

   kubectlconfigget-contexts--output="name"

4. Enable Cloud Service Mesh on your project's fleet. Afleet is a logicalgrouping of Kubernetes clusters and other resources that can be managedtogether.

   gcloudcontainerfleetmeshenable--projectPROJECT_ID

   The output is similar to:

   Waiting for Feature Service Mesh to be created...done.

5. Register the cluster to the project's fleet:

   gcloudcontainerfleetmembershipsregistercluster-with-csm-membership\--gke-cluster=us-central1-a/cluster-with-csm\--enable-workload-identity\--projectPROJECT_ID

   The output is similar to:

   Waiting for membership to be created...done.Finished registering to the Fleet.

6. Enable managed Cloud Service Mesh on the cluster:

   gcloudcontainerfleetmeshupdate\--managementautomatic\--membershipscluster-with-csm-membership\--projectPROJECT_ID

   The output is similar to:

   Waiting for Feature Service Mesh to be updated...done.

   Note:Managed Cloud Service Mesh takes cares of a lot of the manual maintenanceassociated with upgrading, scaling, and securing your service mesh. Learn moreabout aboutManaged Cloud Service Mesh.

7. Verify that managed Cloud Service Mesh has been provisioned for the cluster andis ready to be used:

   gcloudcontainerfleetmeshdescribe--projectPROJECT_ID

   It can take about 10 minutes for Cloud Service Mesh to provision and beready to use on the cluster. If you seecontrolPlaneManagement.state: DISABLED orcontrolPlaneManagement.state: PROVISIONING, you will need to re-run theprevious command every few minutes until you seecontrolPlaneManagement.state: ACTIVE.

   The output is similar to:

   createTime: '2022-07-06T01:05:39.110120474Z'membershipSpecs:  projects/123456789123/locations/global/memberships/cluster-with-csm-membership:    mesh:      management: MANAGEMENT_AUTOMATICmembershipStates:  projects/123456789123/locations/global/memberships/cluster-with-csm-membership:    servicemesh:      controlPlaneManagement:        details:        - code: REVISION_READY          details: 'Ready: asm-managed'        state: ACTIVE      dataPlaneManagement:        details:        - code: OK          details: Service is running.        state: ACTIVE    state:      code: OK      description: 'Revision(s) ready for use: asm-managed.'      updateTime: '2022-07-06T01:19:24.243993678Z'name: projects/your-project-id/locations/global/features/servicemeshresourceState:  state: ACTIVEspec: {}state:  state: {}updateTime: '2022-07-06T01:19:27.475885687Z'

Deploy Cloud Service Mesh's ingress gateway

1. You will deploy the Cloud Service Mesh's ingress gateway into a separatenamespace calledasm-ingress. Create the namespace:

   kubectl\--contextcluster-with-csm\createnamespaceasm-ingress

2. Use theistio.io/rev=asm-managed label to add theasm-ingress namespaceto the service mesh and enable automatic sidecar proxy injection.

   kubectl\--contextcluster-with-csm\labelnamespaceasm-ingress'istio.io/rev=asm-managed'

3. Deploy the Cloud Service Meshingress gateway:

   kubectl\--contextcluster-with-csm\--namespace=asm-ingress\apply-fanthos-service-mesh-samples/docs/shared/asm-ingress-gateway/asm-gateway-deployment-svc.yamlkubectl\--contextcluster-with-csm\--namespace=asm-ingress\apply-fanthos-service-mesh-samples/docs/shared/asm-ingress-gateway/gateway.yaml

   The output is similar to:

   namespace/asm-ingress configuredserviceaccount/asm-ingressgateway configuredservice/asm-ingressgateway configureddeployment.apps/asm-ingressgateway configuredgateway.networking.istio.io/asm-ingressgateway configured

   Note: You can browse the resources in the officialCloud Service Mesh samples GitHub repository.

Deploy Online Boutique

1. You will deploy Online Boutique into a separate namespace calledonlineboutique. Create the namespace:

   kubectl\--contextcluster-with-csm\createnamespaceonlineboutique

2. Use theistio.io/rev=asm-managed label to add theonlineboutiquenamespace to the service mesh and enable automatic sidecar proxy injection.

   kubectl\--contextcluster-with-csm\labelnamespaceonlineboutique'istio.io/rev=asm-managed'

3. Deploy Online Boutique's 12 services, including the load generator thatimitates user traffic:

   kubectl\--contextcluster-with-csm\--namespace=onlineboutique\apply-fanthos-service-mesh-samples/docs/shared/online-boutique/kubernetes-manifests.yamlkubectl\--contextcluster-with-csm\--namespace=onlineboutique\apply-fanthos-service-mesh-samples/docs/shared/online-boutique/virtual-service.yaml

4. Get the external IP address of the Cloud Service Mesh ingress gateway:

   kubectl\--contextcluster-with-csm\--namespaceasm-ingress\getservice--outputjsonpath='{.items[0].status.loadBalancer.ingress[0].ip}'

5. Copy the external IP address of theasm-ingressgateway Service, and accessit through your web browser. You will see the Online Boutique sample app.You will use the external IP address in the next section, so copy it into anenvironment variable:

   exportINGRESS_IP_OF_CLUSTER_WITH_MANAGED_ASM=$(\kubectl\--contextcluster-with-csm\--namespaceasm-ingress\getservice--outputjsonpath='{.items[0].status.loadBalancer.ingress[0].ip}'\)

Test the cluster with Cloud Service Mesh using a canary deployment

In this section, you configure the cluster with in-cluster Cloud Service Mesh such that50% of the user traffic to Online Boutique is shifted to the instance of OnlineBoutique on the cluster with managed Cloud Service Mesh. To achieve this, you deploytwo Istio resources to the cluster with in-cluster Cloud Service Mesh:

• a ServiceEntry to tell in-cluster Cloud Service Mesh about the managed Cloud Service Meshcluster's Online Boutique endpoint
• a VirtualService to tell the in-cluster Cloud Service Mesh ingress gateway to split thetraffic 50-50.

1. Add an A record to the DNS settings to point the domain name (such asexample.com) to the IP address of the ingress gateway running on the clusterwith in-cluster Cloud Service Mesh.

   Note: This operation can take a few minutes to complete.

2. Access Online Boutique by visiting the domain name in your web browser.

3. Minimize DNS record time to live (TTL) to ensure you can quickly revert theDNS entry if you need to rollback.

4. Set the A record of your domain name to the external IP address of theingress gateway of the cluster with managed Cloud Service Mesh.

5. When the migration is successful, delete the cluster within-cluster Cloud Service Mesh:

   gcloudcontainerclustersdeletecluster-with-in-cluster-asm\--zone=us-central1-a\--project=PROJECT_ID

Clean up

To avoid incurring charges to your Google Cloud account for the resourcesused in this tutorial, either delete the project that contains the resources, orkeep the project and delete the individual resources.

Delete project

Caution: Deleting a project has the following effects:
• Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as anappspot.com URL, delete selected resources inside the project instead of deleting the whole project.
1. In the Google Cloud console, go to theManage resources page.

   Go to Manage resources

2. In the project list, select the project that you want to delete, and then clickDelete.
3. In the dialog, type the project ID, and then clickShut down to delete the project.

Delete the resources

Delete the cluster with managed Cloud Service Mesh:

gcloudcontainerclustersdeletecluster-with-managed-asm\--zone=us-central1-a\--project=PROJECT_ID

What's next

• Learn aboutManaged Cloud Service Mesh.
• Learn aboutCloud Service Mesh security best practices.