Create a GKE cluster

In this section, you create aregional GKE cluster.Unlike a zonal cluster, a regional cluster's control plane is replicated intoseveral zones, so an outage in a single zone doesn't make the control planeunavailable.

To create a GKE cluster, follow these steps:

Deploy MySQL StatefulSets

In this section, you deploy one MySQLStatefulSet. A StatefulSet is a Kubernetes controller that maintains a persistent unique identity for each of its Pods.

Each StatefulSetconsists of three MySQL replicas.

To deploy the MySQL StatefulSet, follow these steps:

1. Create a namespace for the StatefulSet.

   kubectlcreatenamespacemysql1

2. Create the MySQL secret.

   apiVersion:v1kind:Secretmetadata:name:mysql-secrettype:Opaquedata:password:UGFzc3dvcmQkMTIzNDU2# Password$123456admin-password:UGFzc3dvcmQkMTIzNDU2# Password$123456

   kubectlapply-nmysql1-fsecret.yaml

   The password is deployed with each Pod, and is used by management scriptsand commands for MySQL InnoDB Cluster and ClusterSet deploymentin this tutorial.

3. Create the StorageClass.

   apiVersion:storage.k8s.io/v1kind:StorageClassmetadata:name:fast-storageclassprovisioner:pd.csi.storage.gke.iovolumeBindingMode:WaitForFirstConsumerreclaimPolicy:RetainallowVolumeExpansion:trueparameters:type:pd-balanced

   kubectlapply-nmysql1-fstorageclass.yaml

   This storage class uses thepd-balanced Persistent Disk type thatbalances performance and cost. ThevolumeBindingMode field is set toWaitForFirstConsumer meaning that GKE delays provisioningof a PersistentVolume until the Pod is created. This setting ensures thatthe disk is provisioned in the same zone where the Pod is scheduled.

4. Deploy the StatefulSet of MySQL instance Pods.

   apiVersion:apps/v1kind:StatefulSetmetadata:name:dbc1labels:app:mysqlspec:replicas:3selector:matchLabels:app:mysqlserviceName:mysqltemplate:metadata:labels:app:mysqlspec:topologySpreadConstraints:-maxSkew:1topologyKey:"topology.kubernetes.io/zone"whenUnsatisfiable:DoNotSchedulelabelSelector:matchLabels:app:mysqlaffinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:-labelSelector:matchExpressions:-key:appoperator:Invalues:-mysqltopologyKey:"kubernetes.io/hostname"containers:-name:mysqlimage:mysql/mysql-server:8.0.28command:-/bin/bashargs:--c->-/entrypoint.sh--server-id=$((20 +  $(echo $HOSTNAME | grep -o '[^-]*$') + 1))--report-host=${HOSTNAME}.mysql.mysql1.svc.cluster.local--binlog-checksum=NONE--enforce-gtid-consistency=ON--gtid-mode=ON--default-authentication-plugin=mysql_native_passwordenv:-name:MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name:mysql-secretkey:password-name:MYSQL_ADMIN_PASSWORDvalueFrom:secretKeyRef:name:mysql-secretkey:admin-password-name:MYSQL_ROOT_HOSTvalue:'%'ports:-name:mysqlcontainerPort:3306-name:mysqlxcontainerPort:33060-name:xcomcontainerPort:33061resources:limits:cpu:"500m"ephemeral-storage:"1Gi"memory:"1Gi"requests:cpu:"500m"ephemeral-storage:"1Gi"memory:"1Gi"volumeMounts:-name:mysqlmountPath:/var/lib/mysqlsubPath:mysqlreadinessProbe:exec:command:-bash-"-c"-|mysql -h127.0.0.1 -uroot -p$MYSQL_ROOT_PASSWORD -e'SELECT 1'initialDelaySeconds:30periodSeconds:2timeoutSeconds:1livenessProbe:exec:command:-bash-"-c"-|mysqladmin -uroot -p$MYSQL_ROOT_PASSWORD pinginitialDelaySeconds:30periodSeconds:10timeoutSeconds:5updateStrategy:rollingUpdate:partition:0type:RollingUpdatevolumeClaimTemplates:-metadata:name:mysqllabels:app:mysqlspec:storageClassName:fast-storageclassvolumeMode:FilesystemaccessModes:-ReadWriteOnceresources:requests:storage:10Gi

   kubectlapply-nmysql1-fc1-mysql.yaml

   This command deploys the StatefulSet consisting of three replicas. In thistutorial, the primary MySQL cluster is deployed across three zones inus-west1. The output is similar to the following:

   service/mysql createdstatefulset.apps/dbc1 created

   In this tutorial, the resource limits and requests are set to minimalvalues to save cost. When planning for a production workload,make sure to set these values appropriately for your organization's needs.

5. Verify the StatefulSet is created successfully.

   kubectlgetstatefulset-nmysql1--watch

   It can take about 10 minutes for the StatefulSet to be ready.

6. When all three pods are in a ready state, exit the command usingCtrl+C. Ifyou seePodUnscheduleable errors due to insufficient CPU or memory, wait afew minutes for the control plane to resize to accommodate the large workload.

   The output is similar to the following:

   NAME   READY   AGEdbc1   1/3     39sdbc1   2/3     50sdbc1   3/3     73s

7. To inspect the placement of your Pods on the GKE cluster nodes,run this script:

   bash../scripts/inspect_pod_node.shmysql1mysql

   The output shows the Pod name, the GKE node name, and thezone where the node is provisioned, and looks similar to the following:

   gke-gkemulti-west-5-default-pool-4bcaca65-jch0 us-west1-b dbc1-0gke-gkemulti-west-5-default-pool-1ac6e8b5-ddjx us-west1-c dbc1-1gke-gkemulti-west-5-default-pool-1f5baa66-bf8t us-west1-a dbc1-2

   The columns in the output represent the hostname, cloud zone, and Pod name,respectively.

   ThetopologySpreadConstraints policy in the StatefulSet specification(c1-mysql.yaml) directs the scheduler to place the Pods evenly across thefailure domain (topology.kubernetes.io/zone).

   ThepodAntiAffinity policy enforces the constraint that Pods are required tonot be placed on the same GKE cluster node (kubernetes.io/hostname). For theMySQL instance Pods, this policy results in the Pods being deployedevenly across the three zones in the Google Cloud region. This placement enableshigh availability of the MySQL InnoDB Cluster by placing each database instancein a separate failure domain.

Prepare the primary MySQL InnoDB Cluster

To configure a MySQL InnoDB Cluster, follow these steps:

1. In theCloud Shell terminal, set the group replicationconfigurations for the MySQL instances to be added to your cluster.

   bash../scripts/c1-clustersetup.sh

   POD_ORDINAL_START=${1:-0}POD_ORDINAL_END=${2:-2}foriin$(seq${POD_ORDINAL_START}${POD_ORDINAL_END});doecho"Configuring pod mysql1/dbc1-${i}"cat<<'  EOF'|kubectl-nmysql1exec-idbc1-${i}--bash-c'mysql -uroot -proot --password=${MYSQL_ROOT_PASSWORD}'INSTALLPLUGINgroup_replicationSONAME'group_replication.so';RESETPERSISTIFEXISTSgroup_replication_ip_allowlist;RESETPERSISTIFEXISTSbinlog_transaction_dependency_tracking;SET@@PERSIST.group_replication_ip_allowlist='mysql.mysql1.svc.cluster.local';SET@@PERSIST.binlog_transaction_dependency_tracking='WRITESET';EOFdone

   The script will remotely connect into each of the three MySQL instances toset and persist the following environment variables:

   • group_replication_ip_allowlist: allows the instance within the clusterto connect to any instance in the group.
   • binlog_transaction_dependency_tracking='WRITESET': allows parallelizedtransactions which won't conflict.

   In MySQL versions earlier than 8.0.22, usegroup_replication_ip_whitelistinstead ofgroup_replication_ip_allowlist.

2. Open a second terminal, so that you do not need to create a shell for each Pod.

3. Connect to MySQL Shell on the Poddbc1-0.

   kubectl-nmysql1exec-itdbc1-0--\/bin/bash\-c'mysqlsh --uri="root:$MYSQL_ROOT_PASSWORD@dbc1-0.mysql.mysql1.svc.cluster.local"'

4. Verify the MySQL group replication allowlist for connecting to other instances.

   \sqlSELECT@@group_replication_ip_allowlist;

   The output is similar to the following:

   +----------------------------------+| @@group_replication_ip_allowlist |+----------------------------------+| mysql.mysql1.svc.cluster.local   |+----------------------------------+

5. Verify theserver-id is unique on each of the instances.

   \sqlSELECT@@server_id;

   The output is similar to the following:

   +-------------+| @@server_id |+-------------+|          21 |+-------------+

6. Configure each instance for MySQL InnoDB Cluster usage and create anadministrator account on each instance.

   \jsdba.configureInstance('root@dbc1-0.mysql.mysql1.svc.cluster.local',{password:os.getenv("MYSQL_ROOT_PASSWORD"),clusterAdmin:'icadmin',clusterAdminPassword:os.getenv("MYSQL_ADMIN_PASSWORD")});dba.configureInstance('root@dbc1-1.mysql.mysql1.svc.cluster.local',{password:os.getenv("MYSQL_ROOT_PASSWORD"),clusterAdmin:'icadmin',clusterAdminPassword:os.getenv("MYSQL_ADMIN_PASSWORD")});dba.configureInstance('root@dbc1-2.mysql.mysql1.svc.cluster.local',{password:os.getenv("MYSQL_ROOT_PASSWORD"),clusterAdmin:'icadmin',clusterAdminPassword:os.getenv("MYSQL_ADMIN_PASSWORD")});

   All instances must have the same username and password in order for theMySQL InnoDB Cluster to function properly. Each command producesoutput similar to the following:

   ...The instance 'dbc1-2.mysql:3306' is valid to be used in an InnoDB cluster.Cluster admin user 'icadmin'@'%' created.The instance 'dbc1-2.mysql.mysql1.svc.cluster.local:3306' is alreadyready to be used in an InnoDB cluster.Successfully enabled parallel appliers.

7. Verify that the instance is ready to be used in a MySQL InnoDB Cluster.

   dba.checkInstanceConfiguration()

   The output is similar to the following:

   ...The instance 'dbc1-0.mysql.mysql1.svc.cluster.local:3306' is valid to be used in an InnoDB cluster.{    "status": "ok"}

   Optionally, you can connect to each MySQL instance and repeat this command.For example, run this command to check the status on thedbc1-1 instance:

   kubectl-nmysql1exec-itdbc1-0--\/bin/bash\-c'mysqlsh --uri="root:$MYSQL_ROOT_PASSWORD@dbc1-1.mysql.mysql1.svc.cluster.local" \    --js --execute "dba.checkInstanceConfiguration()"'

Create the primary MySQL InnoDB Cluster

Next, create the MySQL InnoDB Cluster using the MySQL AdmincreateCluster command. Start with thedbc1-0 instance, which will bethe primary instance for the cluster, then add two additional replicas tothe cluster.

To initialize the MySQL InnoDB Cluster, follow these steps:

1. Create the MySQL InnoDB Cluster.

   varcluster=dba.createCluster('mycluster');

   Running thecreateCluster command triggers these operations:

   • Deploy the metadata schema.
   • Verify that the configuration is correct for Group Replication.
   • Register it as the seed instance of the new cluster.
   • Create necessary internal accounts, such as the replication user account.
   • Start Group Replication.

   This command initializes a MySQL InnoDB Cluster with the hostdbc1-0 as theprimary. The cluster reference is stored in the cluster variable.

   The output looks similar to the following:

   A new InnoDB cluster will be created on instance 'dbc1-0.mysql:3306'.Validating instance configuration at dbc1-0.mysql:3306...This instance reports its own address as dbc1-0.mysql.mysql1.svc.cluster.local:3306Instance configuration is suitable.NOTE: Group Replication will communicate with other instances using'dbc1-0.mysql:33061'. Use the localAddressoption to override.Creating InnoDB cluster 'mycluster' on'dbc1-0.mysql.mysql1.svc.cluster.local:3306'...Adding Seed Instance...Cluster successfully created. Use Cluster.addInstance() to add MySQLinstances.At least 3 instances are needed for the cluster to be able to withstandup to one server failure.

2. Add the second instance to the cluster.

   cluster.addInstance('icadmin@dbc1-1.mysql',{password:os.getenv("MYSQL_ROOT_PASSWORD"),recoveryMethod:'clone'});

3. Add the remaining instance to the cluster.

   cluster.addInstance('icadmin@dbc1-2.mysql',{password:os.getenv("MYSQL_ROOT_PASSWORD"),recoveryMethod:'clone'});

   The output is similar to the following:

   ...The instance 'dbc1-2.mysql:3306' was successfully added to the cluster.

4. Verify the cluster's status.

   cluster.status()

   This command shows the status of the cluster. The topology consists ofthree hosts, one primary and two secondary instances. Optionally, you can callcluster.status({extended:1}).

   The output is similar to the following:

   {    "clusterName": "mysql1",    "defaultReplicaSet": {        "name": "default",        "primary": "dbc1-0.mysql:3306",        "ssl": "REQUIRED",        "status": "OK",        "statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",        "topology": {            "dbc1-0.mysql:3306": {                "address": "dbc1-0.mysql:3306",                "memberRole": "PRIMARY",                "mode": "R/W",                "readReplicas": {},                "replicationLag": null,                "role": "HA",                "status": "ONLINE",                "version": "8.0.28"            },            "dbc1-1.mysql:3306": {                "address": "dbc1-1.mysql:3306",                "memberRole": "SECONDARY",                "mode": "R/O",                "readReplicas": {},                "replicationLag": null,                "role": "HA",                "status": "ONLINE",                "version": "8.0.28"            },            "dbc1-2.mysql:3306": {                "address": "dbc1-2.mysql:3306",                "memberRole": "SECONDARY",                "mode": "R/O",                "readReplicas": {},                "replicationLag": null,                "role": "HA",                "status": "ONLINE",                "version": "8.0.28"            }        },        "topologyMode": "Single-Primary"    },    "groupInformationSourceMember": "dbc1-0.mysql:3306"}

   Optionally, you can callcluster.status({extended:1}) to obtain additionalstatus details.

Create a sample database

To create a sample database, follow these steps:

1. Create a database and load data into the database.

   \sqlcreatedatabaseloanapplication;useloanapplicationCREATETABLEloan(loan_idINTunsignedAUTO_INCREMENTPRIMARYKEY,firstnameVARCHAR(30)NOTNULL,lastnameVARCHAR(30)NOTNULL,statusVARCHAR(30));

2. Insert sample data into the database. To insert data, you must be connectedto the primary instance of the cluster.

   INSERTINTOloan(firstname,lastname,status)VALUES('Fred','Flintstone','pending');INSERTINTOloan(firstname,lastname,status)VALUES('Betty','Rubble','approved');

3. Verify that the table contains the three rows inserted in the previous step.

   SELECT*FROMloan;

   The output is similar to the following:

   +---------+-----------+------------+----------+| loan_id | firstname | lastname   | status   |+---------+-----------+------------+----------+|       1 | Fred      | Flintstone | pending  ||       2 | Betty     | Rubble     | approved |+---------+-----------+------------+----------+2 rows in set (0.0010 sec)

Create a MySQL InnoDB ClusterSet

You can create a MySQL InnoDB ClusterSet to manage replication from your primarycluster to replica clusters, using a dedicated ClusterSet replication channel.

A MySQL InnoDB ClusterSet provides disaster tolerance for MySQL InnoDB Clusterdeployments by linking a primary MySQL InnoDB Cluster with one or more replicas ofitself in alternate locations, such as multiple zones and multiple regions.

If you closed MySQL Shell, create a new shell by running thiscommand in a new Cloud Shell terminal:

kubectl-nmysql1exec-itdbc1-0--\/bin/bash-c'mysqlsh \      --uri="root:$MYSQL_ROOT_PASSWORD@dbc1-0.mysql.mysql1.svc.cluster.local"'

To create a MySQL InnoDB ClusterSet, follow these steps:

1. In your MySQL Shell terminal, obtain a cluster object.

   \jscluster=dba.getCluster()

   The output is similar to the following:

   <Cluster:mycluster>

2. Initialize a MySQL InnoDB ClusterSet with the existing MySQL InnoDB Clusterstored in the cluster object as the primary.

   clusterset=cluster.createClusterSet('clusterset')

   The output is similar to the following:

   A new ClusterSet will be created based on the Cluster 'mycluster'.* Validating Cluster 'mycluster' for ClusterSet compliance.* Creating InnoDB ClusterSet 'clusterset' on 'mycluster'...* Updating metadata...ClusterSet successfully created. Use ClusterSet.createReplicaCluster() to add Replica Clusters to it.<ClusterSet:clusterset>

3. Check the status of your MySQL InnoDB ClusterSet.

   clusterset.status()

   The output is similar to the following:

   {    "clusters": {        "mycluster": {            "clusterRole": "PRIMARY",            "globalStatus": "OK",            "primary": "dbc1-0.mysql:3306"        }    },    "domainName": "clusterset",    "globalPrimaryInstance": "dbc1-0.mysql:3306",    "primaryCluster": "mycluster",    "status": "HEALTHY",    "statusText": "All Clusters available."}

   Optionally, you can callclusterset.status({extended:1}) to obtain additionalstatus details, including information about the cluster.

4. Exit MySQL Shell.

   \q

Deploy a MySQL Router

You can deploy a MySQL Router to direct client application traffic to theproper clusters. Routing is based on the connection port of the applicationissuing a database operation:

• Writes are routed to the primary Cluster instance in the primary ClusterSet.
• Reads can be routed to any instance in the primary Cluster.

When you start a MySQL Router, it is bootstrapped against theMySQL InnoDB ClusterSet deployment. The MySQL Router instances connectedwith the MySQL InnoDB ClusterSet are aware of any controlled switchovers oremergency failovers and direct traffic to the new primary cluster.

To deploy a MySQL Router, follow these steps:

1. In the Cloud Shell terminal, deploy the MySQL Router.

   kubectlapply-nmysql1-fc1-router.yaml

   The output is similar to the following:

   configmap/mysql-router-config createdservice/mysql-router createddeployment.apps/mysql-router created

2. Check the readiness of the MySQL Router deployment.

   kubectl-nmysql1getdeploymentmysql-router--watch

   When all three Pods are ready, the output is similar to the following:

   NAME           READY   UP-TO-DATE   AVAILABLE   AGEmysql-router   3/3     3            0           3m36s

   If you see aPodUnschedulable error in the console, wait a minute or twowhile GKE provisions more nodes. Refresh, and you should see3/3 OK.

3. Start MySQL Shell on any member of the existing cluster.

   kubectl-nmysql1exec-itdbc1-0--\/bin/bash-c'mysqlsh --uri="root:$MYSQL_ROOT_PASSWORD@dbc1-0.mysql"'

   This command connects to thedbc1-0 Pod, then starts a shell connected tothedbc1-0 MySQL instance.

4. Verify the router configuration.

   clusterset=dba.getClusterSet()clusterset.listRouters()

   The output is similar to the following:

   {  "domainName": "clusterset",  "routers": {    "mysql-router-7cd8585fbc-74pkm::": {        "hostname": "mysql-router-7cd8585fbc-74pkm",        "lastCheckIn": "2022-09-22 23:26:26",        "roPort": 6447,        "roXPort": 6449,        "rwPort": 6446,        "rwXPort": 6448,        "targetCluster": null,        "version": "8.0.27"    },    "mysql-router-7cd8585fbc-824d4::": {      ...    },    "mysql-router-7cd8585fbc-v2qxz::": {      ...    }  }}

5. Exit MySQL Shell.

   \q

6. Run this script to inspect the placement of the MySQL Router Pods.

   bash../scripts/inspect_pod_node.shmysql1|sort

   The script shows the node and Cloud Zone placement of the all of the Pods inthemysql1 namespace, where the output is similar to the following:

   gke-gkemulti-west-5-default-pool-1ac6e8b5-0h9v us-west1-c mysql-router-6654f985f5-df97qgke-gkemulti-west-5-default-pool-1ac6e8b5-ddjx us-west1-c dbc1-1gke-gkemulti-west-5-default-pool-1f5baa66-bf8t us-west1-a dbc1-2gke-gkemulti-west-5-default-pool-1f5baa66-kt03 us-west1-a mysql-router-6654f985f5-qlfj9gke-gkemulti-west-5-default-pool-4bcaca65-2l6s us-west1-b mysql-router-6654f985f5-5967dgke-gkemulti-west-5-default-pool-4bcaca65-jch0 us-west1-b dbc1-0

   You can observe that the MySQL Router Pods are distributed equally acrossthe zones; that is, not placed on the same node as a MySQL Pod, or onthe same node as another MySQL Router Pod.

Manage GKE and MySQL InnoDB Cluster upgrades

Updates for both MySQL and Kubernetes are released on a regular schedule. Followoperational best practices to update your software environment regularly. Bydefault, GKE manages cluster and node pool upgrades for you.Kubernetes and GKE also provide additional features tofacilitate MySQL software upgrades.

Plan for GKE upgrades

You can take proactive steps and set configurations to mitigate risk andfacilitate a smoother cluster upgrade when you are running stateful services,including:

• Standard clusters: FollowGKE best practices for upgrading clusters. Choose an appropriateupgrade strategy to ensure the upgrades happen during the period of the maintenance window:

  • Choosesurge upgrades if cost optimization is important and if your workloadscan tolerate a graceful shutdown in less than 60 minutes.
  • Chooseblue-green upgrades if your workloads are less tolerant of disruptions, and atemporary cost increase due to higher resource usage is acceptable.

  To learn more, seeUpgrade a cluster running a stateful workload.Autopilot clusters areautomatically upgraded, based on the release channel you selected.

• Usemaintenance windowsto ensure upgrades happen when you intend them. Before the maintenance window,ensure your database backups are successful.

• Before allowing traffic to the upgraded MySQL nodes, use Readiness Probes andLiveness Probes to ensure they are ready for traffic.

• Create Probes that assess whether replication is in sync before acceptingtraffic. This can be done through custom scripts, depending on the complexityand scale of your database.

Set a Pod Disruption Budget (PDB) policy

When a MySQL InnoDB Cluster is running on GKE, there must be asufficient number of instances running at any time to meet the quorum requirement.

In this tutorial, given a MySQL cluster of three instances, two instances mustbe available to form a quorum. APodDisruptionBudget policy allows you tolimit the number of Pods that can be terminated at any given time. This isuseful for both steady state operations of your stateful services and forcluster upgrades.

To ensure that a limited number of Pods are concurrently disrupted, you set thePDB for your workload tomaxUnavailable: 1. This ensures that at any point inthe service operation, no more than one Pod is not running.

apiVersion:policy/v1kind:PodDisruptionBudgetmetadata:name:mysql-pdbspec:maxUnavailable:1selector:matchLabels:app:mysql

kubectlrolloutundostatefulset/dbc1-nmysql1

statefulset.apps/dbc1 rolled back

To add nodes to your Standard cluster, follow the instructions below forCloud Shell or the Google Cloud console. For detailed steps,seeResize a node pool

gcloudcontainerclustersresize${CLUSTER_NAME}\--node-pooldefault-pool\--num-nodes=8

Add MySQL Pods to the primary cluster

To deploy additional MySQL Pods to scale your cluster horizontally, follow these steps:

1. In Cloud Shell, update the number of replicas in the MySQL deploymentfrom three replicas to five replicas.

   kubectlscale-nmysql1--replicas=5-fc1-mysql.yaml

2. Verify the progress of the deployment.

   kubectl-nmysql1getpods--selector=app=mysql-owide

   To determine whether the Pods are ready, use the--watch flag to watch thedeployment. If you are using Autopilot clusters and seePod Unschedulable errors, this might indicate GKE isprovisioning nodes to accommodate the additional Pods.

3. Configure the group replication settings for the new MySQL instances to addto the cluster

   bash../scripts/c1-clustersetup.sh34

   The script submits the commands to the instances running on the Pods withordinals 3 through 4.

4. Open MySQL Shell.

   kubectl-nmysql1\exec-itdbc1-0--\/bin/bash\-c'mysqlsh \        --uri="root:$MYSQL_ROOT_PASSWORD@dbc1-0.mysql"'

5. Configure the two new MySQL instances.

   dba.configureInstance('root:$MYSQL_ROOT_PASSWORD@dbc1-3.mysql',{password:os.getenv("MYSQL_ROOT_PASSWORD"),clusterAdmin:'icadmin',clusterAdminPassword:os.getenv("MYSQL_ADMIN_PASSWORD")});dba.configureInstance('root:$MYSQL_ROOT_PASSWORD@dbc1-4.mysql',{password:os.getenv("MYSQL_ROOT_PASSWORD"),clusterAdmin:'icadmin',clusterAdminPassword:os.getenv("MYSQL_ADMIN_PASSWORD")});

   The commands check if the instance is configured properly for MySQL InnoDB Clusterusage and perform the necessary configuration changes.

6. Add one of the new instances to the primary cluster.

   cluster=dba.getCluster()cluster.addInstance('icadmin@dbc1-3.mysql',{password:os.getenv("MYSQL_ROOT_PASSWORD"),recoveryMethod:'clone'});

7. Add a second new instance to the primary cluster.

   cluster.addInstance('icadmin@dbc1-4.mysql',{password:os.getenv("MYSQL_ROOT_PASSWORD"),recoveryMethod:'clone'});

8. Obtain the ClusterSet status, which also includes the Cluster status.

   clusterset=dba.getClusterSet()clusterset.status({extended:1})

   The output is similar to the following:

   "domainName": "clusterset","globalPrimaryInstance": "dbc1-0.mysql:3306","metadataServer": "dbc1-0.mysql:3306","primaryCluster": "mycluster","status": "HEALTHY","statusText": "All Clusters available."

9. Exit MySQL Shell.

   \q