Configuration updates for modernization

This document describes configuration updates you may need to make to yourmanaged Cloud Service Mesh before modernizing your mesh totheTRAFFIC_DIRECTOR control plane from theISTIOD control plane.

For more information on the modernization workflow, see theManaged control plane modernization page.

Migrate from Istio secrets to multicluster_mode

Multi-cluster secrets are not supported when a cluster is using theTRAFFIC_DIRECTOR control plane. This document describes how youcan modernize from using Istio multi-cluster secrets to usingmulticluster_mode.

Istio secrets versus declarative API overview

Open source istio multi-cluster endpoint discovery works byusingistioctl or other tools to create aKubernetes Secret in acluster. This secret allows a cluster to load balance traffic to another clusterin the mesh. TheISTIOD control plane then reads thissecret and begins routing traffic to that other cluster.

Cloud Service Mesh has adeclarative APIto controlmulti-cluster traffic instead of directly creating Istio secrets. This APItreats Istio secrets as an implementation detail and is more reliablethan creating Istio secrets manually. Future Cloud Service Mesh features willdepend on the declarative API, and you won't be able to use those newfeatures with Istio secrets directly. The declarative API is the onlysupported path forward.

If you are using Istio Secrets, migrate to using the declarative API assoon as possible. Note that themulticluster_mode setting directs each clusterto direct traffic to every other cluster in the mesh. Using secrets allows amore flexible configuration, letting you configure for each cluster which othercluster it should direct traffic to in the mesh.For a full list of the differences between the supportedfeatures of the declarative API and Istio secrets, seeSupported features using Istio APIs.

If you provisioned Cloud Service Mesh using automatic management with thefleet feature API, you don'tneed to follow these instructions.These steps only apply if you onboarded usingasmcli --managed.

Note, this process changes secrets that point to a cluster. During this process,the endpoints are removed and then re-added. In between the endpointsbeing removed and added, the traffic willbriefly revert to routing locally instead of load balancing to other clusters.For more information, see theGitHub issue.

To move from using Istio secrets to the declarative API, follow these steps.Execute these steps at the same time or in close succession:

1. Enable the declarative API for each cluster in the fleet where you want toenable multi cluster endpoint discovery by settingmulticluster_mode=connected. Note that you need to explicitly setmulticluster_mode=disconnected if you don't want the cluster to bediscoverable.

   Use the following command to opt in a cluster for multi cluster endpointdiscovery:

    kubectl patch configmap/asm-options -n istio-system --type merge -p '{"data":{"multicluster_mode":"connected"}}'

   Use the following command to opt a cluster out of endpoint discovery:

    kubectl patch configmap/asm-options -n istio-system --type merge -p '{"data":{"multicluster_mode":"disconnected"}}'

2. Delete old secrets.

   After settingmulticluster_mode=connected on your clusters, eachcluster will have a new secret generated for every other cluster that alsohasmulticluster_mode=connected set.The secret is placed in the istio-system namespace and have the followingformat:

   istio-remote-secret-projects-PROJECT_NAME-locations-LOCATION-memberships-MEMBERSHIPS

   Each secret will also have the labelistio.io/owned-by: mesh.googleapis.com applied.

   Once the new secrets are created, you can delete any secrets manuallycreated withistioctl create-remote-secret:

   kubectldeletesecretSECRET_NAME-nistio-system

Once migrated, check your request metrics to make sure they're routed asexpected.