This document outlines the steps to decommission an org from a hybrid deployment. Decommissioning an org means deleting all data related to the org across all Cassandra pods in all Kubernetes clusters.

1. Back up the org
   1. If it's not already enabled, enable backups on the hybrid deployment. In multi-region setups, use the hybrid deployment operating in the primary region. SeeCassandra backup overview for information on hybrid backups.
   2. Trigger a hybrid backup job using the following command:

      kubectl create job -nAPIGEE_NAMESPACE --from=cronjob/apigee-cassandra-backupBACKUP_JOB_NAME

      TheBACKUP_JOB_NAME can be any valid container name.

   3. Once the backup job completes, use the "Check the status of the backup job" and "Check the backup logs" instructions inMonitoring backups to verify the backup was successful.

2. (Optional) If you have configured Apigee Telemetry (Metrics and Logger) on the org to be deleted, follow these steps to reconfigure them so that the metrics and log data apply to a new org/project.
1. Run this command for the org where you want to send the data. Be sure to use theoverrides.yaml file for the org. For example, if the org to be decommissioned is "test-dev" theoverrides.yaml file should contain anorg: test-dev org field.

   helm upgrade telemetry apigee-telemetry/ \  --namespaceAPIGEE_NAMESPACE \  --atomic \  -fOVERRIDES_FILE.yaml

   Note: If you see an error sayingError: UPGRADE FAILED: "ORG_NAME" has no deployed releases, replaceupgrade withinstall and try the command again.
2. Run this command, making sure to use the correct org/project:

   kubectl -nAPIGEE_NAMESPACE get apigeetelemetry apigee-telemetry -oyaml | grep `gcpProjectID:`

3. Delete the org's Kubernetes resources from the hybrid deployment.
   1. Verify the current context is the correct context for the source Kubernetes cluster:

      kubectl config current-context

      If needed, set the Kubernetes context.

      List your current contexts to see the context name for each cluster:

      kubectl config get-contexts

      Set the context to the cluster and region you want to decommission:

      kubectl config use-contextCONTEXT_NAME

      WhereCONTEXT_NAME is the context name for the cluster and region.

      For example:

      kubectl config get-contexts    CURRENT   NAME                                                   CLUSTER                                                AUTHINFO                                               NAMESPACE              gke_example-org-1_us-central1_example-cluster-1        gke_example-org-1_us-central1_example-cluster-1        gke_example-org-1_us-central1_example-cluster-1        apigee*         gke_example-org-1_us-central1_example-cluster-2        gke_example-org-1_us-central1_example-cluster-2        gke_example-org-1_us-central1_example-cluster-2        apigee              gke_example-org-1_us-west1_example-cluster-2           gke_example-org-1_us-west1_example-cluster-2           gke_example-org-1_us-west1_example-cluster-2           apigeekubectl config use-context gke_example-org-1_us-west1_example-cluster-2

   2. Delete the virtualhost.

      Repeat this for each environment group:

      helm -nAPIGEE_NAMESPACE deleteENV_GROUP_NAME

   3. Delete the environments. Repeat this for each environment:

      helm -nAPIGEE_NAMESPACE deleteENV_NAME

   4. Delete the Apigee org.

      helm -nAPIGEE_NAMESPACE deleteORG_NAME

4. Delete the org data from the hybrid deployment.Once this step is complete all org data will be gone from the hybrid deployment.
   1. Exec into theapigee-cassandra-default-0 pod:

      kubectl exec -it -nAPIGEE_NAMESPACE apigee-cassandra-default-0 -- /bin/bash

   2. Execute the following command. Copy the list of all the names that are shown in the output. This list will be needed later.

      find /opt/apigee/data/apigee-cassandra/ -iname '*ORG_NAME_hybrid' -type d -maxdepth 2 -printf "%f\n"

      SeeGet org name for instructions on how to find and prepare theORG_NAME.

      Exit from theapigee-cassandra-default-0 pod.

   3. Create a Cassandra debug client pod as described inCreate a client container for debugging. Move on to the next step after getting acqlsh prompt.
   4. Execute the following commands in thecqlsh prompt:

      desc keyspaces;

      Make sure this command returns no errors.

      For each name in the list created earlier from theapigee apigee-cassandra-default-0, run the following commands:

      drop keyspaceKEYSPACE_NAME;

      Exit from the Cassandra debug client pod.

   5. Perform a rolling restart of all Cassandra pods. Restarting the Cassandra pods can be done in any order as long as only one Cassandra pod is restarted at a time. For multi-region deployments, perform a rolling restart on all Cassandra pods in each hybrid region.

      Run the following command and verify the state shows "Running":

      kubectl get apigeeds -nAPIGEE_NAMESPACE

      Restart a single Cassandra pod with the following command:

      kubectl delete pod -nAPIGEE_NAMESPACECASSANDRA_POD_NAME

      Wait for the pod to reachRunning state using:

      kubectl get pods -nAPIGEE_NAMESPACE

      Restart the next Cassandra pod.

   6. After executing thecqlsh commands, run the following commands on all Cassandra pods in the hybrid deployment. For multi-region hybrid deployments, run the commands on all Cassandra pods in all hybrid regions.

      kubectl exec -it -nAPIGEE_NAMESPACECASSANDRA_POD_NAME -- /bin/bash

      find /opt/apigee/data/apigee-cassandra/ -iname '*ORG_NAME_hybrid' -type d -maxdepth 2

      SeeGet org name for instructions on how to find and prepare theORG_NAME.

      find /opt/apigee/data/apigee-cassandra/ -iname '*ORG_NAME_hybrid' -type d -maxdepth 2 -exec rm -rf {} +

   7. Exit from the Cassandra pod.