Upgrading Apigee hybrid to version 1.5

You are currently viewing version 1.5 of the Apigee hybrid documentation.This version is end of life. You should upgrade to a newer version. For more information, seeSupported versions.
Note:This procedure covers upgrading from Apigee hybrid version 1.4.x to Apigee hybrid version 1.5.10 only.

If you are upgrading from Apigee hybrid version 1.3 or older, you must first upgrade to hybrid version 1.4 before upgrading to version 1.5.10. See the instructions forUpgrading Apigee hybrid to version 1.4.

Upgrading to version 1.5.10 overview.

Upgrading to Apigee hybrid version 1.5.10 may require downtime:

The procedures for upgrading Apigee hybrid are organized in the following sections:

  1. Backup your hybrid installation.
  2. Check your Kubernetes version and upgrade as appropriate.
  3. Upgrade ASM.
  4. Install hybrid runtime version 1.5.

Prerequisite

Upgrade to version 1.5

  1. These instructions use the environment variableAPIGEECTL_HOME for the directory in your file system where you have installedapigeectl. If needed,cd into yourapigeectl directory and define the variable with the following command:

    Linux

    export APIGEECTL_HOME=$PWD
    echo $APIGEECTL_HOME

    Mac OS

    export APIGEECTL_HOME=$PWD
    echo $APIGEECTL_HOME

    Windows

    set APIGEECTL_HOME=%CD%
    echo %APIGEECTL_HOME%
  2. (Recommended) Make a backup copy of your version 1.4$APIGEECTL_HOME/ directory. For example:
    tar -czvf $APIGEECTL_HOME/../apigeectl-v1.4-backup.tar.gz $APIGEECTL_HOME
  3. (Recommended) Backup your Cassandra database following the instructions inCassandra backup and recovery
  4. Upgrade your Kubernetes platform to the versions supported by hybrid 1.5. Follow your platform's documentation if you need help.

    Click to expand a list of supported platforms

    Apigee hybrid versions

    Platforms

    		1.51.6
    Anthos(3) (Google Cloud)1.18.x
    1.19.x    		1.19.x
    1.20.x
    1.21.x
    Anthos(3) (AWS)1.6.x
    1.7.x
    1.8.x(1)    		1.7.x
    1.8.x(1)
    1.9.3+
    1.10.x
    Anthos(3) (Azure)N/A1.8.x
    Anthos(3) (on-premises - VMware)1.6.x
    1.7.x
    1.8.x(1)    		1.7.x
    1.8.x(1)
    1.9.3+
    1.10.x
    Anthos(3) (Bare Metal)1.6.x
    1.7.x    		1.7.x
    1.8.2+
    1.9.3+
    1.10.x
    Anthos(3) (attached clusters - multi-cloud)
    Kubernetes version (EKS)    		1.18.x
    1.19.x    		1.19.x
    1.20.x
    1.21.x
    Anthos(3) (attached clusters - multi-cloud)
    Kubernetes version (AKS)    		1.18.x
    1.19.x    		1.19.x
    1.20.x
    1.21.x
    Anthos(3) (attached clusters - multi-cloud)
    OpenShift version    		4.5
    4.6
    4.7    		4.6
    4.7
    4.8
    Anthos(3) (attached clusters - multi-cloud)
    Konvoy version    		1.7.x1.7.x

    Components

    		1.51.6
    Anthos Service Mesh (ASM)(2)1.7.x
    1.8.x
    1.9.x
    1.12.x(for Apigee hybrid 1.5 starting with version 1.5.9)    		1.9.x
    1.10.x
    1.12.x(Apigee hybrid version 1.6.6 and newer)
    JDKJDK 11JDK 11
    cert-manager1.2.01.2.0
    1.5.4
    Cassandra3.11.63.11.6

    (1) On version 1.8.2 and above, follow the instructions in this document:Conflict with cert-manager when upgrading to version 1.8.2 or above.

    (2) Install only Supported versions of ASM.

    (3) Install only Supported versions of Anthos.

  5. If you are running a version ofcert-manager prior to v1.2.0, you need to upgrade it to v1.2.0.

    1. Check the currentcert-manager version using the following command:

      kubectl -n cert-manager get deployment -o yaml | grep 'image:'

      Something similar to the following is returned:

      image:quay.io/jetstack/cert-manager-controller:v1.2.0image:quay.io/jetstack/cert-manager-cainjector:v1.2.0image:quay.io/jetstack/cert-manager-webhook:v1.2.0
    2. Remove the deployments using the following command:
      kubectl delete -n cert-manager deployment cert-manager cert-manager-cainjector cert-manager-webhook
    3. Upgradecert-manager to v1.2.0 version using the following command:
      kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.2.0/cert-manager.yaml

Upgrade ASM to version 1.28

You may experience a brief interruption of incoming traffic (typically less than two seconds) when you upgrade ASM. If you are running hybrid in two or more clusters, you can avoid this interruption by upgrading one cluster at a time. Divert all production traffic away from the cluster you are about to upgrade, take the cluster offline, and then proceed with the upgrade process. Repeat the process for each cluster.

Perform the upgrade using the ASM documentation appropriate for your platform:

The instructions to install and configure ASM are different depending on your platform. The platforms are divided into the following categories:

  • GKE: Google Kubernetes Engine clusters running on Google Cloud.
  • Outside Google Cloud: Anthos clusters running on:
    • Anthos clusters on VMware (GKE on-prem)
    • Anthos on bare metal
    • Anthos clusters on AWS
    • Amazon EKS
  • Other Kubernetes Platforms: Conformant clusters created and running on:
    • AKS
    • EKS
    • OpenShift

GKE

The sequence for upgrading to ASM version 1.8.x for your hybrid installation is as follows:

  1. Prepare for the upgrade.
  2. Install the new version of ASM.
  3. Delete the previous ASM version's deployments, services, and webhooks from your current installation.
  4. Upgrade your gateways and configure the new webhooks.

To upgrade to ASM version 1.8.x for hybrid on GKE:

Note: You can upgrade directly from ASM version 1.10 or higher to ASM version 1.8.x. If you are running a version of ASM older than 1.10, you must first upgrade to version 1.10. Follow the instructions inUpgrading from earlier versions
  1. Review the requirements inUpgrade Anthos Service Mesh, but do not perform the upgrade yet.
  2. Before installing the new version, determine the current revision. You will need this information to delete the previous ASM version's deployments, services, and webhooks from your current installation. Use the following command to store the current istiod revision to an environment variable:
    export DELETE_REV=$(kubectl get deploy -n istio-system -l app=istiod -o jsonpath={.items[].metadata.labels.'istio\.io\/rev'}'{"\n"}')echo ${DELETE_REV}
  3. Create a newoverlay.yaml file or verify that your existingoverlay.yaml contains the following contents:
    apiVersion:install.istio.io/v1alpha1kind:IstioOperatorspec:revision:asm-1282-4components:ingressGateways:-name:istio-ingressgatewayenabled:truek8s:nodeSelector:# default node selector, if different or not using node selectors, change accordingly.cloud.google.com/gke-nodepool:apigee-runtimeresources:requests:cpu:1000mservice:type:LoadBalancerloadBalancerIP:STATIC_IP# If you do not have a reserved static IP, leave this out.ports:-name:http-status-portport:15021-name:http2port:80targetPort:8080-name:httpsport:443targetPort:8443meshConfig:accessLogFormat:'{"start_time":"%START_TIME%","remote_address":"%DOWNSTREAM_DIRECT_REMOTE_ADDRESS%","user_agent":"%REQ(USER-AGENT)%","host":"%REQ(:AUTHORITY)%","request":"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%","request_time":"%DURATION%","status":"%RESPONSE_CODE%","status_details":"%RESPONSE_CODE_DETAILS%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","upstream_address":"%UPSTREAM_HOST%","upstream_response_flags":"%RESPONSE_FLAGS%","upstream_response_time":"%RESPONSE_DURATION%","upstream_service_time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","upstream_cluster":"%UPSTREAM_CLUSTER%","x_forwarded_for":"%REQ(X-FORWARDED-FOR)%","request_method":"%REQ(:METHOD)%","request_path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","request_protocol":"%PROTOCOL%","tls_protocol":"%DOWNSTREAM_TLS_VERSION%","request_id":"%REQ(X-REQUEST-ID)%","sni_host":"%REQUESTED_SERVER_NAME%","apigee_dynamic_data":"%DYNAMIC_METADATA(envoy.lua)%"}'
  4. Follow the instructions in the following sections in the ASM documentation:Important: Make sure to follow the instructions to upgrade ASM with optional features, and to include youroverlay.yaml.
    1. Download asmcli
    2. Grant cluster admin permissions
    3. Validate project and cluster
    4. Upgrade with optional features. Stop before starting the "Upgrade Gateways section"
  5. Delete the mutating webhook and validating webhook:
    1. cd into the directory where you installedasmcli.
    2. Store the current new revision in an environment variable to use in the script to delete the webhooks:
      UPGRADE_REV="asm-1282-4"
    3. create a shell script containing the following commands:
      #!/bin/bashset -exPROJECT_ID="YOUR_PROJECT_ID"CLUSTER_NAME="YOUR_CLUSTER_NAME"CLUSTER_LOCATION="YOUR_CLUSTER_LOCATION"kubectl label namespace istio-system istio.io/rev=${UPGRADE_REV} istio-injection- --overwritekubectl rollout restart deployment -n istio-systemkubectl apply -n istio-system -fPATH_TO_INGRESSGATEWAYistio-ingressgatewaykubectl apply -n istio-system -fPATH_TO_INGRESSGATEWAY/istio-ingressgateway-connectorsif [[ "${DELETE_REV}" != "${UPGRADE_REV}" ]]; then  kubectl apply -f out/asm/istio/istiod-service.yaml  kubectl delete deploy -l app=istio-ingressgateway,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete deploy -l app=istio-ingressgateway-connectors,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete ValidatingWebhookConfiguration -l app=istiod,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete MutatingWebhookConfiguration -l app=sidecar-injector,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete Service,Deployment,HorizontalPodAutoscaler,PodDisruptionBudget istiod-${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete IstioOperator installed-state-${DELETE_REV} -n istio-system --ignore-not-found=truefi
    4. Execute the script to delete the current webhooks.
  6. Follow the steps inUpgrade gateways to create the new webhooks and switch traffic to the new gateways.

Outside Google Cloud

These instructions cover upgrading ASM on:

  • Anthos clusters on VMware (GKE on-prem)
  • Anthos on bare metal
  • Anthos clusters on AWS
  • Amazon EKS
Note: You can upgrade directly from ASM version 1.10 or higher to ASM version 1.8.x. If you are running an older version of ASM, you must first upgrade to version 1.10. Follow the instructions inUpgrading from earlier versions
  1. Review the requirements inUpgrade Anthos Service Mesh, but do not perform the upgrade yet.
  2. Before installing the new version, determine the current revision. You will need this information to delete thevalidating webhook andmutating webhook from your current ASM installation. Use the following command to store the current istiod revision to an environment variable:
    export DELETE_REV=$(kubectl get deploy -n istio-system -l app=istiod -o jsonpath={.items[].metadata.labels.'istio\.io\/rev'}'{"\n"}')echo ${DELETE_REV}
  3. Create a newoverlay.yaml file or verify that your existingoverlay.yaml contains the following contents:
    apiVersion:install.istio.io/v1alpha1kind:IstioOperatorspec:revision:asm-1282-4components:ingressGateways:-name:istio-ingressgatewayenabled:truek8s:nodeSelector:# default node selector, if different or not using node selectors, change accordingly.cloud.google.com/gke-nodepool:apigee-runtimeresources:requests:cpu:1000mservice:type:LoadBalancerloadBalancerIP:STATIC_IP# If you do not have a reserved static IP, leave this out.ports:-name:http-status-portport:15021-name:http2port:80targetPort:8080-name:httpsport:443targetPort:8443values:gateways:istio-ingressgateway:runAsRoot:truemeshConfig:accessLogFormat:'{"start_time":"%START_TIME%","remote_address":"%DOWNSTREAM_DIRECT_REMOTE_ADDRESS%","user_agent":"%REQ(USER-AGENT)%","host":"%REQ(:AUTHORITY)%","request":"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%","request_time":"%DURATION%","status":"%RESPONSE_CODE%","status_details":"%RESPONSE_CODE_DETAILS%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","upstream_address":"%UPSTREAM_HOST%","upstream_response_flags":"%RESPONSE_FLAGS%","upstream_response_time":"%RESPONSE_DURATION%","upstream_service_time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","upstream_cluster":"%UPSTREAM_CLUSTER%","x_forwarded_for":"%REQ(X-FORWARDED-FOR)%","request_method":"%REQ(:METHOD)%","request_path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","request_protocol":"%PROTOCOL%","tls_protocol":"%DOWNSTREAM_TLS_VERSION%","request_id":"%REQ(X-REQUEST-ID)%","sni_host":"%REQUESTED_SERVER_NAME%","apigee_dynamic_data":"%DYNAMIC_METADATA(envoy.lua)%"}'
  4. Follow the instructions in the following sections in the ASM documentation:Important: Make sure to follow the instructions to upgrade ASM with optional features, and to include youroverlay.yaml.
    1. Download asmcli
    2. Grant cluster admin permissions
    3. Validate project and cluster
    4. Upgrade with optional features. Stop before starting the "Upgrade Gateways section"
  5. Delete the mutating webhook and validating webhook:
    1. cd into the directory where you installedasmcli.
    2. Store the current new revision in an environment variable to use in the script to delete the webhooks:
      UPGRADE_REV="asm-1282-4"
    3. create a shell script containing the following commands:
      #!/bin/bashset -exPROJECT_ID="YOUR_PROJECT_ID"CLUSTER_NAME="YOUR_CLUSTER_NAME"CLUSTER_LOCATION="YOUR_CLUSTER_LOCATION"gcloud config configurations activate ${PROJECT_ID}gcloud container clusters get-credentials ${CLUSTER_NAME} --region ${CLUSTER_LOCATION} --project ${PROJECT_ID}kubectl label namespace istio-system istio.io/rev=${UPGRADE_REV} istio-injection- --overwritekubectl rollout restart deployment -n istio-systemkubectl apply -n istio-system -fPATH_TO_INGRESSGATEWAYistio-ingressgatewaykubectl apply -n istio-system -fPATH_TO_INGRESSGATEWAY/istio-ingressgateway-connectorsif [[ "${DELETE_REV}" != "${UPGRADE_REV}" ]]; then  kubectl apply -f out/asm/istio/istiod-service.yaml  kubectl delete deploy -l app=istio-ingressgateway,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete deploy -l app=istio-ingressgateway-connectors,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete ValidatingWebhookConfiguration -l app=istiod,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete MutatingWebhookConfiguration -l app=sidecar-injector,istio.io/rev=${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete Service,Deployment,HorizontalPodAutoscaler,PodDisruptionBudget istiod-${DELETE_REV} -n istio-system --ignore-not-found=true  kubectl delete IstioOperator installed-state-${DELETE_REV} -n istio-system --ignore-not-found=truefi
    4. Execute the script to delete the current webhooks.
  6. Follow the steps inUpgrade gateways to create the new webhooks and switch traffic to the new gateways.

AKS / EKS

In these instructions the process of upgrading Anthos Service Mesh (ASM) version istio-1.28.2-asm.4 on Anthos attached clusters is the same as performing a fresh install.

Preparing to install Anthos Service Mesh

    Linux

  1. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-linux-amd64.tar.gz
  2. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-linux-amd64.tar.gz.1.sig
    openssldgst-verify/dev/stdin-signature istio-1.28.2-asm.4-linux-amd64.tar.gz.1.sig istio-1.28.2-asm.4.tar.gz <<'EOF'-----BEGINPUBLICKEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZwQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==-----ENDPUBLICKEY-----EOF
  3. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.28.2-asm.4-linux-amd64.tar.gz

    The command creates an installation directory in your current working directory namedistio-1.28.2-asm.4 that contains:

    • Sample applications in thesamples directory.
    • Theistioctl command-line tool that you use to install Anthos Service Mesh is in thebin directory.
    • The Anthos Service Mesh configuration profiles are in themanifests/profiles directory.
  4. Ensure that you're in the Anthos Service Mesh installation's root directory:
    cd istio-1.28.2-asm.4
  5. For convenience, add the tools in the /bin directory to your PATH:
    export PATH=$PWD/bin:$PATH

    6. Mac OS

  6. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-osx.tar.gz
  7. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-osx.tar.gz.1.sig
    openssldgst-sha256-verify/dev/stdin-signature istio-1.28.2-asm.4-osx.tar.gz.1.sig istio-1.28.2-asm.4.tar.gz <<'EOF'-----BEGINPUBLICKEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZwQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==-----ENDPUBLICKEY-----EOF
  8. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.28.2-asm.4-osx.tar.gz

    The command creates an installation directory in your current working directory namedistio-1.28.2-asm.4 that contains:

    • Sample applications in thesamples directory.
    • Theistioctl command-line tool that you use to install Anthos Service Mesh is in thebin directory.
    • The Anthos Service Mesh configuration profiles are in themanifests/profiles directory.
  9. Ensure that you're in the Anthos Service Mesh installation's root directory:
    cd istio-1.28.2-asm.4
  10. For convenience, add the tools in the /bin directory to your PATH:
    export PATH=$PWD/bin:$PATH

    11. Windows

  11. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-win.zip
  12. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-win.zip.1.sig
    openssldgst-verify--signature istio-1.28.2-asm.4-win.zip.1.sig istio-1.28.2-asm.4.win.zip <<'EOF'-----BEGINPUBLICKEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZwQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==-----ENDPUBLICKEY-----EOF
  13. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.28.2-asm.4-win.zip

    The command creates an installation directory in your current working directory namedistio-1.28.2-asm.4 that contains:

    • Sample applications in thesamples directory.
    • Theistioctl command-line tool that you use to install Anthos Service Mesh is in thebin directory.
    • The Anthos Service Mesh configuration profiles are in themanifests\profiles directory.
  14. Ensure that you're in the Anthos Service Mesh installation's root directory:
    cd istio-1.28.2-asm.4
  15. For convenience, add the tools in the \bin directory to your PATH:
    set PATH=%CD%\bin:%PATH%
  16. Now that ASM Istio is installed, check the version ofistioctl:
    istioctl version
  17. Create a namespace called istio-system for the control plane components:
    kubectl create namespace istio-system

Configure the validating webhook

When you install Anthos Service Mesh, you set a revision label on istiod. You need to set the same revision on the validating webhook.

  1. Create a file calledistiod-service.yaml with the following contents:
    apiVersion:v1kind:Servicemetadata:name:istiodnamespace:istio-systemlabels:istio.io/rev:asm-1282-4app:istiodistio:pilotrelease:istiospec:ports:-port:15010name:grpc-xds#plaintextprotocol:TCP-port:15012name:https-dns#mTLSwithk8s-signedcertprotocol:TCP-port:443name:https-webhook#validationandinjectiontargetPort:15017protocol:TCP-port:15014name:http-monitoring#prometheusstatsprotocol:TCPselector:app:istiodistio.io/rev:asm-1282-4meshConfig:accessLogFormat:'{"start_time":"%START_TIME%","remote_address":"%DOWNSTREAM_DIRECT_REMOTE_ADDRESS%","user_agent":"%REQ(USER-AGENT)%","host":"%REQ(:AUTHORITY)%","request":"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%","request_time":"%DURATION%","status":"%RESPONSE_CODE%","status_details":"%RESPONSE_CODE_DETAILS%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","upstream_address":"%UPSTREAM_HOST%","upstream_response_flags":"%RESPONSE_FLAGS%","upstream_response_time":"%RESPONSE_DURATION%","upstream_service_time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","upstream_cluster":"%UPSTREAM_CLUSTER%","x_forwarded_for":"%REQ(X-FORWARDED-FOR)%","request_method":"%REQ(:METHOD)%","request_path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","request_protocol":"%PROTOCOL%","tls_protocol":"%DOWNSTREAM_TLS_VERSION%","request_id":"%REQ(X-REQUEST-ID)%","sni_host":"%REQUESTED_SERVER_NAME%","apigee_dynamic_data":"%DYNAMIC_METADATA(envoy.lua)%"}'
  2. Usekubectl to apply the validating webhook configuration:
    kubectl apply -f istiod-service.yaml
  3. Verify that the configuration was applied:
    kubectl get svc -n istio-system

    The response should look similar to:

    NAME     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                 AGEistiod   ClusterIP   172.200.18.133   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP   22s

Installing Anthos Service Mesh

  1. Install Anthos Service Mesh withistioctl using theasm-multicloud profile:
    istioctl install \    --set profile=asm-multicloud \    --set revision="asm-1282-4"

    Your output should look something like:

    kubectl get pods -n istio-systemNAME                                   READY   STATUS    RESTARTS   AGEistio-ingressgateway-88b6fd976-flgp2   1/1     Running   0          3m13sistio-ingressgateway-88b6fd976-p5dl9   1/1     Running   0          2m57sistiod-asm-1282-4-798ffb964-2ls88       1/1     Running   0          3m21sistiod-asm-1282-4-798ffb964-fnj8c       1/1     Running   1          3m21s

    The--set revision argument adds a revision label in the formatistio.io/rev=asm-1282-4 to istiod. The revision label is used by the automatic sidecar injector webhook to associate injected sidecars with a particular istiod revision. To enable sidecar auto-injection for a namespace, you must label it with a revision that matches the label on istiod.

  2. Verify that your install completed:
    kubectl get svc -n istio-system

    Your output should look something like:

    NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                                                                      AGEistio-ingressgateway   LoadBalancer   172.200.48.52    34.74.177.168   15021:30479/TCP,80:30030/TCP,443:32200/TCP,15012:32297/TCP,15443:30244/TCP   3m35sistiod                 ClusterIP      172.200.18.133   <none>          15010/TCP,15012/TCP,443/TCP,15014/TCP                                        4m46sistiod-asm-1282-4       ClusterIP      172.200.63.220   <none>          15010/TCP,15012/TCP,443/TCP,15014/TCP                                        3m43s

OpenShift

In these instructions the process of upgrading Anthos Service Mesh (ASM) version istio-1.28.2-asm.4 on Anthos attached clusters is the same as performing a fresh install.

Preparing to install Anthos Service Mesh

    Linux

  1. Grant theanyuid security context constraint (SCC) to the istio-system with the following OpenShift CLI (oc) command:
    oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system
  2. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-linux-amd64.tar.gz
  3. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-linux-amd64.tar.gz.1.sig
    openssldgst-verify/dev/stdin-signature istio-1.28.2-asm.4-linux-amd64.tar.gz.1.sig istio-1.28.2-asm.4.tar.gz <<'EOF'-----BEGINPUBLICKEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZwQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==-----ENDPUBLICKEY-----EOF
  4. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.28.2-asm.4-linux-amd64.tar.gz

    The command creates an installation directory in your current working directory namedistio-1.28.2-asm.4 that contains:

    • Sample applications in thesamples directory.
    • Theistioctl command-line tool that you use to install Anthos Service Mesh is in thebin directory.
    • The Anthos Service Mesh configuration profiles are in themanifests/profiles directory.
  5. Ensure that you're in the Anthos Service Mesh installation's root directory:
    cd istio-1.28.2-asm.4
  6. For convenience, add the tools in the /bin directory to your PATH:
    export PATH=$PWD/bin:$PATH

    7. Mac OS

  7. Grant theanyuid security context constraint (SCC) to the istio-system with the following OpenShift CLI (oc) command:
    oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system
  8. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-osx.tar.gz
  9. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-osx.tar.gz.1.sig
    openssldgst-sha256-verify/dev/stdin-signature istio-1.28.2-asm.4-osx.tar.gz.1.sig istio-1.28.2-asm.4.tar.gz <<'EOF'-----BEGINPUBLICKEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZwQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==-----ENDPUBLICKEY-----EOF
  10. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.28.2-asm.4-osx.tar.gz

    The command creates an installation directory in your current working directory namedistio-1.28.2-asm.4 that contains:

    • Sample applications in thesamples directory.
    • Theistioctl command-line tool that you use to install Anthos Service Mesh is in thebin directory.
    • The Anthos Service Mesh configuration profiles are in themanifests/profiles directory.
  11. Ensure that you're in the Anthos Service Mesh installation's root directory:
    cd istio-1.28.2-asm.4
  12. For convenience, add the tools in the /bin directory to your PATH:
    export PATH=$PWD/bin:$PATH

    13. Windows

  13. Grant theanyuid security context constraint (SCC) to the istio-system with the following OpenShift CLI (oc) command:
    oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system
  14. Download the Anthos Service Mesh installation file to your current working directory:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-win.zip
  15. Download the signature file and use openssl to verify the signature:
    curl -LO https://storage.googleapis.com/gke-release/asm/istio-1.28.2-asm.4-win.zip.1.sig
    openssldgst-verify--signature istio-1.28.2-asm.4-win.zip.1.sig istio-1.28.2-asm.4.win.zip <<'EOF'-----BEGINPUBLICKEY-----MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEWZrGCUaJJr1H8a36sG4UUoXvlXvZwQfk16sxprI2gOJ2vFFggdq3ixF2h4qNBt0kI7ciDhgpwS8t+/960IsIgw==-----ENDPUBLICKEY-----EOF
  16. Extract the contents of the file to any location on your file system. For example, to extract the contents to the current working directory:
    tar xzf istio-1.28.2-asm.4-win.zip

    The command creates an installation directory in your current working directory namedistio-1.28.2-asm.4 that contains:

    • Sample applications in thesamples directory.
    • Theistioctl command-line tool that you use to install Anthos Service Mesh is in thebin directory.
    • The Anthos Service Mesh configuration profiles are in themanifests\profiles directory.
  17. Ensure that you're in the Anthos Service Mesh installation's root directory:
    cd istio-1.28.2-asm.4
  18. For convenience, add the tools in the \bin directory to your PATH:
    set PATH=%CD%\bin:%PATH%
  19. Now that ASM Istio is installed, check the version ofistioctl:
    istioctl version
  20. Create a namespace called istio-system for the control plane components:
    kubectl create namespace istio-system

Configure the validating webhook

When you install Anthos Service Mesh, you set a revision label on istiod. You need to set the same revision on the validating webhook.

  1. Create a file calledistiod-service.yaml with the following contents:
    apiVersion:v1kind:Servicemetadata:name:istiodnamespace:istio-systemlabels:istio.io/rev:asm-1282-4app:istiodistio:pilotrelease:istiospec:ports:-port:15010name:grpc-xds#plaintextprotocol:TCP-port:15012name:https-dns#mTLSwithk8s-signedcertprotocol:TCP-port:443name:https-webhook#validationandinjectiontargetPort:15017protocol:TCP-port:15014name:http-monitoring#prometheusstatsprotocol:TCPselector:app:istiodistio.io/rev:asm-1282-4meshConfig:accessLogFormat:'{"start_time":"%START_TIME%","remote_address":"%DOWNSTREAM_DIRECT_REMOTE_ADDRESS%","user_agent":"%REQ(USER-AGENT)%","host":"%REQ(:AUTHORITY)%","request":"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%","request_time":"%DURATION%","status":"%RESPONSE_CODE%","status_details":"%RESPONSE_CODE_DETAILS%","bytes_received":"%BYTES_RECEIVED%","bytes_sent":"%BYTES_SENT%","upstream_address":"%UPSTREAM_HOST%","upstream_response_flags":"%RESPONSE_FLAGS%","upstream_response_time":"%RESPONSE_DURATION%","upstream_service_time":"%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%","upstream_cluster":"%UPSTREAM_CLUSTER%","x_forwarded_for":"%REQ(X-FORWARDED-FOR)%","request_method":"%REQ(:METHOD)%","request_path":"%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%","request_protocol":"%PROTOCOL%","tls_protocol":"%DOWNSTREAM_TLS_VERSION%","request_id":"%REQ(X-REQUEST-ID)%","sni_host":"%REQUESTED_SERVER_NAME%","apigee_dynamic_data":"%DYNAMIC_METADATA(envoy.lua)%"}'
  2. Usekubectl to apply the validating webhook configuration:
    kubectl apply -f istiod-service.yaml
  3. Verify that the configuration was applied:
    kubectl get svc -n istio-system

    The response should look similar to:

    NAME     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                 AGEistiod   ClusterIP   172.200.18.133   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP   22s

Installing Anthos Service Mesh

  1. Install Anthos Service Mesh withistioctl using theasm-multicloud profile:
    istioctl install \    --set profile=asm-multicloud \    --set revision=istio-1.28.2-asm.4

    Your output should look something like:

    kubectl get pods -n istio-systemNAME                                   READY   STATUS    RESTARTS   AGEistio-ingressgateway-88b6fd976-flgp2   1/1     Running   0          3m13sistio-ingressgateway-88b6fd976-p5dl9   1/1     Running   0          2m57sistiod-asm-1282-4-798ffb964-2ls88       1/1     Running   0          3m21sistiod-asm-1282-4-798ffb964-fnj8c       1/1     Running   1          3m21s

    The--set revision argument adds a revision label in the formatistio.io/rev=1.8.6-asm.1 to istiod. The revision label is used by the automatic sidecar injector webhook to associate injected sidecars with a particular istiod revision. To enable sidecar auto-injection for a namespace, you must label it with a revision that matches the label on istiod.

  2. Verify that your install completed:
    kubectl get svc -n istio-system

    Your output should look something like:

    NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                                                                      AGEistio-ingressgateway   LoadBalancer   172.200.48.52    34.74.177.168   15021:30479/TCP,80:30030/TCP,443:32200/TCP,15012:32297/TCP,15443:30244/TCP   3m35sistiod                 ClusterIP      172.200.18.133   <none>          15010/TCP,15012/TCP,443/TCP,15014/TCP                                        4m46sistiod-asm-1282-4       ClusterIP      172.200.63.220   <none>          15010/TCP,15012/TCP,443/TCP,15014/TCP                                        3m43s

Install the hybrid 1.5.10 runtime

Note:Do not create new environments during the upgrade process.This can lead to an error state that is difficult to diagnose.

  1. Download the release package for your operating system:

    Mac 64 bit:

    curl -LO \    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.5.10/apigeectl_mac_64.tar.gz

    Linux 64 bit:

    curl -LO \    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.5.10/apigeectl_linux_64.tar.gz

    Mac 32 bit:

    curl -LO \    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.5.10/apigeectl_mac_32.tar.gz

    Linux 32 bit:

    curl -LO \    https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/1.5.10/apigeectl_linux_32.tar.gz
  2. Rename your currentapigeectl/ directory to a backup directory name. For example:
    mv $APIGEECTL_HOME/ $APIGEECTL_HOME-v1.2/

  3. Extract the downloaded gzip file contents into your hybrid base directory. For example:

    tar xvzfFILENAME.tar.gz -CHYBRID_BASE_DIRECTORY
  4. cd to the base directory.

  5. The tar contents are, by default, expanded into a directory with the version and platform in its name. For example:./apigeectl_1.5.0-d591b23_linux_64. Rename that directory toapigeectl:

    mvapigeectl_1.5.0-d591b23_linux_64 apigeectl
  6. In thenewapigeectl/ directory, runapigeectl init,apigeectl apply, andapigeectl check-ready:
    1. Initialize hybrid 1.5.10:
      apigeectl init -fOVERRIDES.yaml

      WhereOVERRIDES.yaml is your editedoverrides.yaml file.

    2. Check that it initialized correctly with the following commands:
      apigeectl check-ready -fOVERRIDES.yaml
      kubectl describe apigeeds -n apigee

      Your output should look something like:

      Status:  Cassandra Data Replication:  Cassandra Pod Ips:    10.8.2.204  Cassandra Ready Replicas:  1  Components:    Cassandra:      Last Successfully Released Version:        Revision:  v1-f8aa9a82b9f69613        Version:   v1      Replicas:        Available:  1        Ready:      1        Total:      1        Updated:    1      State:        running  Scaling:    In Progress:         false    Operation:    Requested Replicas:  0  State:                 running
    3. The syntax of theapigeectl--dry-run flag depends on the version ofkubectl you are running. Check the version ofkubectl:
      gcloud version
    4. Check for errors with a dry run with the command appropriate to your version ofkubectl:

      kubectl version 1.17 and older:

      apigeectl apply -fOVERRIDES.yaml --dry-run=true

      kubectl version 1.18 and newer:

      apigeectl apply -fOVERRIDES.yaml --dry-run=client
    5. Apply your overrides. Select and follow the instructions for production environments or demo/experimental environments, depending on your installation.During the upgrade process each component will perform a rolling restart. Therefore, for production environments, it is best to apply the upgrade to one component at a time.

      Production

      For production environments you should upgrade each hybrid component individually, and check the status of the upgraded component before proceeding to the next component.

      1. Apply your overrides to upgrade Cassandra:
        apigeectl apply -fOVERRIDES.yaml --datastore
      2. Check completion:
        apigeectl check-ready -fOVERRIDES.yaml
        Tip: Ifcheck-ready fails, you can get more information about your pods with:
        kubectl -nNAMESPACE get pods

        WhereNAMESPACE is your Apigee hybrid namespace.

        Proceed to the next step only when the pods are ready.

      3. Apply your overrides to upgrade Telemetry components and check completion:
        apigeectl apply -fOVERRIDES.yaml --telemetry
        apigeectl check-ready -fOVERRIDES.yaml
      4. Bring up Redis components:Note: Redis support is new with hybrid v1.5.
        apigeectl apply -fOVERRIDES.yaml --redis
      5. Apply your overrides to upgrade the org-level components (MART, Watcher and Apigee Connect) and check completion:
        apigeectl apply -fOVERRIDES.yaml --org
        apigeectl check-ready -fOVERRIDES.yaml
      6. Apply your overrides to upgrade your environments. You have two choices:
        • Environment by environment: Apply your overrides to one environment at a time and check completion. Repeat this step for each environment:
          apigeectl apply -fOVERRIDES.yaml --envENV_NAME
          apigeectl check-ready -fOVERRIDES.yaml

          WhereENV_NAME is the name of the environment you are upgrading.

        • All environments at one time: Apply your overrides to all environments at once and check completion:
          apigeectl apply -fOVERRIDES.yaml --all-envs
          apigeectl check-ready -fOVERRIDES.yaml

      Demo/Experimental

      In most demo or experimental environments, you can apply the overrides to all components at once. If your demo/experimental environment large and complex or closely mimics a production environment, you may want to use the instructions for upgradingproduction environments

      1. apigeectl apply -fOVERRIDES.yaml
      2. Check the status:
        apigeectl check-ready -fOVERRIDES.yaml
Congratulations! You have successfully upgraded to Apigee hybrid version 1.5.10.

Rolling back an upgrade

Follow these steps to roll back a previous upgrade:

  1. Clean up completed jobs for the hybrid runtime namespace, whereNAMESPACE is the namespace specified in your overrides file, if you specified a namespace. If not, the default namespace isapigee:
    kubectl delete job -nNAMESPACE \  $(kubectl get job -nNAMESPACE \  -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')
  2. Clean up completed jobs for theapigee-system namespace:
    kubectl delete job -n apigee-system \  $(kubectl get job -n apigee-system \  -o=jsonpath='{.items[?(@.status.succeeded==1)].metadata.name}')
  3. Change theAPIGEECTL_HOME variable to point to the directory that contains the previous version ofapigeectl. For example:
    export APIGEECTL_HOME=PATH_TO_PREVIOUS_APIGEECTL_DIRECTORY
  4. In the root directory of the installation you want to roll back to, runapigeectl apply, check the status of your pods, delete the Redis component (new in hybrid v1.5.0), and then runapigeectl init. Be sure to use the original overrides file for the version you wish to roll back to:
    • Runapigeectl apply:
      $APIGEECTL_HOME/apigeectl apply -f overrides/ORIGINAL_OVERRIDES.yaml
    • Check the status of your pods:
      kubectl -nNAMESPACE get pods

      WhereNAMESPACE is your Apigee hybrid namespace.

      Proceed to the next step only when theapigeedspod is running.

    • Since Redis is a new component in hybrid v1.5, run the following command to delete it:

      apigeectl_1.5.0 delete --redis -fORIGINAL_OVERRIDES.yaml
    • Runapigeectl init:
      $APIGEECTL_HOME/apigeectl init -f overrides/ORIGINAL_OVERRIDES.yaml

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-18 UTC.