Cloud Run and Kubernetes both use standard container images asdeployment artifacts, they both use a declarative API model with resources thatcan be represented in YAML files with the same standard structure.

Introduction

TheCloud Run Admin API v1is designed to maximize portability with Kubernetes,for example, the Cloud Run Admin API resources share the same structureconventions and attribute names as Kubernetes resources. See theCloud Run service YAML reference.

The Cloud Run Admin API v1 implements theKnative Serving API specification,but you donot need to migrate to Knative to move yourCloud Run workloads to a Kubernetes cluster like GKE.

Quickstart

This quickstart is an example of a simple migration.

Simple resource comparison

Compare the following simple Cloud Run service namedmy-app with theequivalent Kubernetesdeployment.Note how the YAML files are almost identical.

However, the parts inblueare different and need to be changed.The parts ingreenshould be added.

apiVersion:serving.knative.dev/v1kind:Servicemetadata:  name: my-app  namespace: 'PROJECT_NUMBER'spec:  template:    spec:      containers:      - image: gcr.io/cloudrun/hello        env:        - name: HELLO          value: world

apiVersion:apps/v1kind:Deploymentmetadata:  name: my-app  namespace:defaultlabels:app: my-appspec:  template:    metadata:labels:app: my-app    spec:      containers:      - image: gcr.io/cloudrun/hello        env:        - name: HELLO          value: worldreplicas: 3selector:matchLabels:app: my-app

Migrate a Cloud Run service to GKE

1. Download service YAML file into the current directory:

   gcloudrunservicesdescribemy-app--formatexport>my-app.yaml

2. Modify the YAML to match a Kubernetes deployment:

   • For the "kind" attribute: replace the value "Service" with "Deployment"
   • For the "apiVersion" attribute: replace the value "serving.knative.dev/v1" with"apps/v1"
   • Replacemetadata.namespace with the namespace of the GKEcluster you want to deploy to, for exampledefault.
   • Add a new label to undermetadata andspec.template.metadata.
   • Set a fixed number of instances ("replicas") usingspec.template.spec.replicas andset a label selector inspec.template.spec.selector.

3. Install anduse thekubectl command line tool to deploy themy-app.yaml fileto your GKE cluster:

   kubectlapply-f./my-app.yaml

4. Expose the deployment as a service:

   kubectlexposedeploymentmy-app--typeLoadBalancer--port80--target-port8080

Considerations when migrating from Cloud Run to GKE

Cluster:

• Cloud Run is a fully managed platform, while GKErequires more platform management.If you have not yet created a GKE cluster, useGKE Autopilot.

• The scalability of GKE workloads is restricted by the size of thecluster. If you are not using an Autopilot cluster, consider usingnode auto-provisioningand acluster autoscalerto resize your cluster.

• Cloud Run has abuilt-in zonal redundancy,so migrate to aregional clusterand provision enough replicas to ensure your service is resilient to a zonaloutage in the selected Google Cloud region.

Pricing

Cloud Runcharges for used resources, whileGKEcharges for provisionedresources.

Security:

• Contrary to Cloud Run, invoking a GKE service is notsubject to anIAM invoker permission.

• Because GKE does not provide strong isolation between containers,consider usingGKE Sandboxif you need to execute unknown or untrusted code.

Networking

Cloud Run requires a Serverless VPC Access connector to access other resources in aVPC. GKE workloads are directly in a VPC and do not need aconnector.

Features not supported by Google Kubernetes Engine

The following Cloud Run features are not available on GKE:

• Configuration versioning viarevisions
• Traffic splitting forgradual rollouts and rollbacks of revisions
• Request-based billing
• CPU boost
• Session affinity
• Labels on GKEworkloads are not Google Cloud labels
• Tags on workloads

Migrate Cloud Run resources

The following sections describe migrating resources used in Cloud Run, such asCloud Run services, jobs, and secrets.

Migrate Cloud Run services

You can migrate a Cloud Run service to the following resources onGKE:

1. Kubernetes deployment to create instances (called "pods" in Kubernetes).
2. Kubernetes services to expose the deployment at a specific endpoint.
3. Kubernetes Horizontal Pod Autoscaler:to automatically scale the deployment.

The attributes of a Kubernetes deployment are a superset of the attributes of aCloud Run services.As shown in thequickstart, after you change theapiVersion andkind attributes toapps/v1 andDeployment, you also need to change the following:

• Replacenamespace with the GKE cluster namespaceto deploy to, for exampledefault.
• serviceAccountName should reference a Kubernetes service account, which canoptionally act as an IAM service account with Workload Identity Federation for GKE.
• Add aLABEL atmetadata.labels andspec.template.metadata.labelsthat will be used to select the deployment and pods. For example:app:NAME
• Underspec.template:
  • Add areplicas attribute to specify a number of"instances".
  • Add aselector.matchLabels attribute that selects on the labelLABEL.
• If your Cloud Run service mounts secrets, seeMigrate secrets.
• If the migrated Cloud Run service was accessing resources on a Virtual Private Cloud,you do not need to use a Serverless VPC Access connector.

After creating the Kubernetes deployment, create a Kubernetes services to exposeit:

apiVersion: v1kind: Servicemetadata:  name:NAMEspec:  selector:LABEL  ports:    - protocol: TCP      port: 80      targetPort:PORT

Replace:

• NAME: with the name of your service.
• LABEL: with the label defined in your deployment. For exampleapp:NAME.
• PORT: with thecontainerPort of the container receiving requestsin the Cloud Run service, which defaults to8080.

You can then optionally create a Kubernetes Horizontal Pod Autoscaler in order toautomatically scale the number of pods.Follow the KubernetesHorizontal Pod Autoscaling documentation to create anHorizontalPodAutoscaler.Use theminimum instances(autoscaling.knative.dev/minScale)andmaximum instances(autoscaling.knative.dev/maxScale)values of your Cloud Run service as values for theminReplicas andmaxReplicas attributesHorizontalPodAutoscaler.

Migrate Cloud Run worker pools

You can migrate aCloud Run worker poolto aKubernetes deployment on GKE.

The following samples show the structural difference betweena Cloud Run worker pool and a Kubernetes deployment.

apiVersion:run.googleapis.com/v1kind:WorkerPoolmetadata:  name: my-app  annotations:    run.googleapis.com/manualInstanceCount:'1'spec:template:    metadata:      labels:      app: my-app    spec:      containers:      - image: gcr.io/cloudrun/hello        env:        - name: HELLO          value: world

apiVersion:apps/v1kind:Deploymentmetadata:  name: my-app  namespace: default  labels:    app: my-appspec:  template:    metadata:      labels:        app: my-app    spec:      containers:      - image: gcr.io/cloudrun/hello        env:        - name: HELLO          value: worldreplicas: 3selector:matchLabels:app: my-app

Migrate Cloud Run jobs

You can migrate aCloud Run job to aKubernetes jobon GKE.

Contrary to Cloud Run jobs, Kubernetes jobs are executed when they arecreated. If you want to execute the job again, you need to create a new job.

The following samples show the structural difference between aCloud Run job and a Kubernetes job:

apiVersion:run.googleapis.com/v1kind: Jobmetadata:  name: my-jobspec:  template:    spec:      template:        spec:          containers:          - image: us-docker.pkg.dev/cloudrun/container/job

apiVersion:batch/v1kind: Jobmetadata:  name: my-jobspec:  template:    spec:      containers:      - image: us-docker.pkg.dev/cloudrun/container/job

Migrate secrets

You can keep existing secrets in Secret Manager or you can migratethem to Kubernetes secrets.

If you choose to keep secrets in Secret Manager, you will need toupdate how you use them on GKE

If you choose to migrate from Secret Manager to Kubernetes secrets,consider these differences between Secret Manager secretsand Kubernetes secrets:

• Allowed characters in names:
  • Kubernetes secrets:[a-z0-9-.]{1,253}
  • Secret Manager secrets:[a-zA-Z0-9_-]{1,255}
• Versioning: Secrets from Secret Manager are versioned, whileKubernetes secrets aren't.
• Payload: Secrets from Secret Manager contain a single[]byte, while Kubernetes secrets contain amap<string, string>.

Migration strategy

After youcreate the equivalent resources, exposing external endpoints behind a global external Application Load Balancerenables you to gradually migrate traffic between Cloud Run andGoogle Kubernetes Engine (GKE).