This page describes how to deploy KubernetesGatewayresources for load balancingingress traffic to a single Google Kubernetes Engine (GKE) cluster.

For deploying Gateways to load balance ingress traffic across multiple clusters(or fleet), seeDeploying Multi-Cluster Gateways.

For more specific Gateway configurations such as cross-namespace routing andHTTP traffic splitting, see theGateway API user guides.

Before you begin

Before you start, make sure that you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• If you want to use the Google Cloud CLI for this task,install and theninitialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running thegcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.Note: For existing gcloud CLI installations, make sure to set thecompute/regionproperty. If you use primarily zonal clusters, set thecompute/zone instead. By setting a default location, you can avoid errors in the gcloud CLI like the following:One of [--zone, --region] must be supplied: Please specify location. You might need to specify the location in certain commands if the location of your cluster differs from the default that you set.

• Ensure that you have an existing Autopilot or Standardcluster. To create a new cluster, seeCreate an Autopilot cluster.

GKE Gateway controller requirements

• Gateway API is supported onVPC-nativeclusters only.
• If you are using the regional or cross-region GatewayClasses, you must enable aproxy-only subnet.
• Your cluster must have theHttpLoadBalancing add-on enabled.
• If you are using Istio, you must upgrade Istio to one of the followingversions:
  • 1.15.2 or later
  • 1.14.5 or later
  • 1.13.9 or later.
• If you are using Shared VPC, then in the host project, you need to assign theCompute Network User role to the GKE Service account for the service project.

Restrictions and limitations

When using GKE Gateway, understand the following limitations andrestrictions:

• GKE GatewayClasses support different capabilities depending onthe load balancer they use. To learn more about the different featuressupported with each GatewayClass, seeGatewayClass capabilities.

  Best practice:

  For optimal performance, limit the number of Gateways to a maximum of 100.Exceeding this limit can affect performance or result in increased latency.

• You cannot use aFrontendConfigor aBackendConfigto configure a Gateway. You must use aPolicy.

• GKE Gateway behaves differently than Ingress, in that Gatewaydoes not infer health check parameters. If your Service does not return 200for requests toGET /, or you have other tuned pod readiness checks, youneed toconfigure a HealthCheckPolicyfor your service.

• You cannot specify a port number directly in the hostname (for example,web.example.com:80) for traffic routing.

• You can view the load balancer resources that GKE creates forGateways in the Google Cloud console, but these resources don't reference theGateway or GKE cluster they are attached to.

• You can't automatically generate a Google-managed SSL certificate withGateways but you can manually create and reference a Google-managed SSLcertificate. For more information, seeSecure a Gateway.

• HTTPRoute is the only Route type supported. TCPRoutes, UDPRoutes, andTLSRoutes are not supported. To see a list of fields the GKEGateway controller supports, seeGatewayClass capabilities.

• Custom request and response headers with Gateway or path redirects and URLrewrites with Gateway is only available on GKE version 1.27 orlater.

• For custom request and response headers with Gateway and path redirects andURL rewrites with Gateway, the GatewayClassgke-l7-gxlb is not supported.

• When configuring HTTPRoute custom request and response headers,the followingGoogle Cloud variablesare not supported:

  • cdn_cache_id (Cloud CDN is not supported with GKE Gateway)
  • cdn_cache_status (Cloud CDN is not supported with GKE Gateway)
  • origin_request_header (CORS policiesare not supported with GKE Gateway)

• GKE Gateway does not support the Cloud CDN load balancingfeature.

• Mutual TLS custom headers are not supported (mTLS with GKEGateway is not supported)

• Google Cloud classic Application Load Balancer limitations apply to theGKE Gateway. In addition, you can't configure a custom Host response header in the backend service.

• Path redirects and URL rewrites are mutually exclusive, you can't use bothfilters at the same time in the same rules.

• Redirecting traffic to a different port is not supported with Cloud Load Balancing.To see the list of fields the GKE Gateway controller supports, seeGatewayClass capabilities.

• GKE Gateway does not supportWildcards, regular expressions, and dynamic URLs.

• If you specify a Gateway with a regional external gateway class,the controller provisions an internal IP address instead of the externaladdress. To learn how to use a named address with the regional external Application Load Balancer,seedeploy a regional external Gateway.

• Gateway utilizesStandalone NEGsfor provisioning Network Endpoint Groups. To ensure that the Gatewaycontroller properly reconciles the load balancer configuration, you cannotmodify thecloud.google.com/neg annotation for a Service that is part ofthe Gateway.

• GKE Gateway does not support referencing a Service that is alsoreferenced by a GKE Ingress.

• When aGateway is configured to provision an IP address,changing theGateway.spec.gatewayClass is not supported. To ensure that theGateway controller properly reconciles the load balancer, delete theexisting Gateway and re-deploy the manifest with the updatedgatewayClassvalue.

• Thenetworking.gke.io/app-protocols annotation is not supported. Use theappProtocol fieldinstead to achieve the same result.

• If you use GKE Gateway withexternal-dns and thehealth state of the Gateway is unhealthy, by default, all DNS recordsassociated with the Gateway are deleted from your DNS zones.

  Best practice:

  When runningexternal-dns, set thepolicy=upsert-only flag. This configuration helps to prevent the deletion of existing DNS records.

• If a port is removed from aService that GKE Gatewayreferences through a route, theStandalone NEGannotation on the Service youmust also update the Standalone NEG controller on the Service to remove thatport. If you don't, the NEG controller eventually stops syncing Pod endpointsfor this Service. For details, seeNEG Controller stops managing endpointswhen port removed fromService.

• Deployments using Multi-Cluster Gateways (MCG) that usezonal clusters are susceptible to service disruptions during cluster upgrades.This issue occurs because a legacy Network Endpoint Group (NEG) discovery mechanism might incorrectly report zero backends when the zonal cluster's control plane is temporarily unavailable.A long-term fix is in progress. To avoid this single point of failure, useregional GKE clusters with MCG.

Enable Gateway API in your cluster

Before using Gateway resources in GKE, your cluster must haveGateway API enabled.

Before you update an existing GKE cluster toenable Gateway API, make sure that theminimum requirementsare met before proceeding with the update.

To enable Gateway API on an existing GKE cluster (Autopilot orStandard), use the following command. This operation might take up to 45 minutesfor the cluster to reconcile and install the CRDs.

gcloudcontainerclustersupdateCLUSTER_NAME\--location=CLUSTER_LOCATION\--gateway-api=standard

Replace the following:

• CLUSTER_NAME: the name of the existing cluster.

• CLUSTER_LOCATION: theCompute Engine region or zoneof the cluster.

The--gateway-api=standard flag instructs GKE to install thev1beta1 CRDs with the cluster.

Verify your cluster

After creating or upgrading your cluster, theGKE Gateway controller automatically installs GatewayClasses. It mighttake a few minutes for the controller to recognize the CRDs and install theGatewayClasses.

1. Confirm Gateway API is enabled in the GKE control plane:

   gcloudcontainerclustersdescribeCLUSTER_NAME\--location=CLUSTER_LOCATION\--formatjson

   The output is similar to the following. If this output is empty, re-run thecluster update command.

   "networkConfig": {  ...  "gatewayApiConfig": {    "channel": "CHANNEL_STANDARD"  },  ...},

2. Confirm the GatewayClasses are installed in your cluster:

   kubectlgetgatewayclass

   The output is similar to the following:

   NAME                             CONTROLLER                  ACCEPTED   AGEgke-l7-global-external-managed   networking.gke.io/gateway   True       16hgke-l7-regional-external-managed networking.gke.io/gateway   True       16hgke-l7-gxlb                      networking.gke.io/gateway   True       16hgke-l7-rilb                      networking.gke.io/gateway   True       16h

To understand the capabilities of each GatewayClass, seeGatewayClass capabilities.

Only single-cluster GatewayClasses are installed automatically. Toinstall and use the multi-cluster GatewayClasses for internal and externalmulti-cluster load balancing, seeEnabling multi-cluster Gateways.

Deploy an internal Gateway

An internal Gateway exposes applications that are only reachable from within theVPC or networks connected to the VPC.

Deploy a regional internal Gateway

The following example shows you how to deploy a regional internal Gateway thatenables efficient and secure communication between services within a specificgeographic region.

Configure a proxy-only subnet

You mustconfigure a proxy-only subnetbefore you create a Gateway that uses an internal Application Load Balancer. Each region of aVPC in which you use internal Application Load Balancers must have a proxy-onlysubnet. This subnet provides internal IP addresses to the load balancer proxies.

1. Create a proxy-only subnet:

   gcloudcomputenetworkssubnetscreateSUBNET_NAME\--purpose=REGIONAL_MANAGED_PROXY\--role=ACTIVE\--region=COMPUTE_REGION\--network=VPC_NETWORK_NAME\--range=CIDR_RANGE

   Replace the following:

   • SUBNET_NAME: the name of the proxy-only subnet.
   • COMPUTE_REGION: the region of the proxy-only subnet.
   • VPC_NETWORK_NAME: the name of the VPCnetwork in which you create this proxy-only subnet. Ensure this is the sameVPC network where your GKE cluster residesand where you deploy the Gateway. This is important for seamless communicationbetween the load balancer and your backend services.
   • CIDR_RANGE: the primary IP address range of the subnet.You must use a subnet mask no longer than/26 so that at least 64 IPaddresses are available for proxies in the region. The recommended subnet maskis/23.

2. Verify your proxy-only subnet:

   gcloudcomputenetworkssubnetsdescribeSUBNET_NAME\--region=COMPUTE_REGION

   The output is similar to the following:

   ...gatewayAddress: 10.1.1.1ipCidrRange: 10.1.1.0/24kind: compute#subnetworkname: proxy-subnetnetwork: https://www.googleapis.com/compute/v1/projects/PROJECT_NAME/global/networks/defaultprivateIpGoogleAccess: falseprivateIpv6GoogleAccess: DISABLE_GOOGLE_ACCESSpurpose: REGIONAL_MANAGED_PROXYregion: https://www.googleapis.com/compute/v1/projects/PROJECT_NAME/regions/REGIONrole: ACTIVEselfLink: https://www.googleapis.com/compute/v1/projects/PROJECT_NAME/regions/REGION/subnetworks/proxy-subnetstate: READY

Create a Gateway

A Gateway resource represents a data plane that routes traffic in Kubernetes. AGateway can represent many different kinds of load balancing and routingdepending on the GatewayClass it is derived from. To learn more about theGateway resource, see theGateway resourcedescription or theAPI specification.

In this case, the administrator of the GKE cluster wants tocreate a Gateway that can be used by different teams to expose theirapplications internally. The administrator deploys the Gateway, and applicationteams deploy their Routes independently and attach them to this Gateway.

1. Save the following Gateway manifest to a file namedgateway.yaml:

   kind:GatewayapiVersion:gateway.networking.k8s.io/v1metadata:name:internal-httpspec:# Specify an existing GatewayClass.gatewayClassName:gke-l7-rilblisteners:# Listen for HTTP traffic on port 80.-name:httpprotocol:HTTPport:80

   This manifest includes the following fields:

   • gatewayClassName: gke-l7-rilb: specifies the GatewayClass that thisGateway is derived from.gke-l7-rilb corresponds to theinternal Application Load Balancer.
   • port: 80: specifies that the Gateway exposes only port 80 for listeningfor HTTP traffic.

   This Gateway is configured to handle HTTP traffic only on port 80. It doesn'tsupport HTTPS (port 443) by default, and if you attempt to connectover HTTPS, the request might fail.

2. Deploy the Gateway in your cluster:

   kubectlapply-fgateway.yaml

3. Validate that the Gateway has deployed correctly. It might take a few minutesfor it to deploy all of its resources.

   kubectldescribegateways.gateway.networking.k8s.iointernal-http

   The output is similar to the following:

   Name:         internal-httpNamespace:    defaultSpec:  Gateway Class Name:  gke-l7-rilb  Listeners:    Allowed Routes:      Kinds:        Group:  gateway.networking.k8s.io        Kind:   HTTPRoute      Namespaces:        From:  Same    Name:      http    Port:      80    Protocol:  HTTP Status:   Addresses:     Type:   IPAddress     Value:  192.168.1.14   Conditions:     Last Transition Time:  2025-03-19T19:53:46Z     Message:               The OSS Gateway API has deprecated this condition, do not depend on it.     Observed Generation:   1     Reason:                Scheduled     Status:                True     Type:                  Scheduled     Last Transition Time:  2025-03-19T19:53:46Z     Message:     Observed Generation:   1     Reason:                Accepted     Status:                True     Type:                  AcceptedLast Transition Time:  2025-03-19T19:53:46Z     Message:     Observed Generation:   1     Reason:                Programmed     Status:                True # Indicates that the Gateway is ready.     Type:                  Programmed     Last Transition Time:  2025-03-19T19:53:46Z     Message:               The OSS Gateway API has altered the "Ready" condition semantics and reserved it for future use.  GKE Gateway will stop emitting it in a future update, use "Programmed" instead.     Observed Generation:   1     Reason:                Ready     Status:                True     Type:                  Ready     Last Transition Time:  2025-03-19T19:53:46Z     Message:     Observed Generation:   1     Reason:                Healthy     Status:                True     Type:                  networking.gke.io/GatewayHealthyEvents:  Type    Reason  Age                From                       Message  ----    ------  ----               ----                       -------  Normal  ADD     92s                networking.gke.io/gateway  test/internal-http  Normal  UPDATE  45s (x3 over 91s)  networking.gke.io/gateway  test/internal-http  Normal  SYNC    45s                networking.gke.io/gateway  SYNC on test/internal-http was a success

   In this output, the status ofTrue for theProgrammed condition indicatesthat the Gateway is ready.

   At this point, there is a Gateway deployed in your cluster that hasprovisioned a load balancer and an IP address. The Gateway has no Routes,however, and so it doesn't know how it should send traffic to backends.Without Routes, all traffic goes to a default backend, which returns an HTTP 404.Next, you deploy an application and Routes, which tell the Gateway howto get to application backends.

Deploy the demo applications

Application teams can deploy their applications and Routes independently fromthe deployment of Gateways. In some cases the application team might want toown the Gateway as well and deploy it themselves as a resource dedicated totheir applications. SeeRoutebindingfor different ownership models of Gateways and Routes. In this example however,the store team deploys their application and an accompanying HTTPRoute to exposetheir app through theinternal-http Gateway created in the previous section.

The HTTPRoute resource has many configurable fields for traffic matching.For an explanation of HTTPRoute's fields, see theAPI specification.

1. Deploy the store application (store-v1, store-v2, and store-german deployments) to your cluster:

   kubectlapply-fhttps://raw.githubusercontent.com/GoogleCloudPlatform/gke-networking-recipes/main/gateway/gke-gateway-controller/app/store.yaml

   This creates three Deployments and three Services which are named store-v1,store-v2, and store-german.

2. Validate that the application has deployed successfully:

   kubectlgetpod

   The output is similar to the following after the application is running:

   NAME                        READY   STATUS    RESTARTS   AGEstore-german-66dcb75977-5gr2n   1/1     Running   0          38sstore-v1-65b47557df-jkjbm       1/1     Running   0          14mstore-v2-6856f59f7f-sq889       1/1     Running   0          14m

3. Validate that the Services have been deployed:

   kubectlgetservice

   The output shows a Service for each store Deployment:

   NAME           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGEstore-german   ClusterIP   10.48.3.183   <none>        8080/TCP   4sstore-v1       ClusterIP   10.48.2.224   <none>        8080/TCP   5sstore-v2       ClusterIP   10.48.4.48    <none>        8080/TCP   5s

Deploy the HTTPRoute

Route resources define protocol-specific rules for mapping traffic from aGateway to Kubernetes backends. TheHTTPRouteresource does HTTP andHTTPS traffic matching and filtering and is supported by all of thegke-l7GatewayClasses.

In this section, you deploy an HTTPRoute, which programs the Gateway with therouting rules needed to reach your store application.

1. Save the following HTTPRoute manifest to a file namedstore-route.yaml:

   kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:storespec:# Attach the HTTPRoute to a Gateway.parentRefs:-kind:Gatewayname:internal-http# Route requests that have `store.example.com` in the Host header.hostnames:-"store.example.com"rules:# Send requests with the `env: canary` header to the `store-v2` Service.-matches:-headers:-name:envvalue:canarybackendRefs:-name:store-v2port:8080# Send requests with `/de` in the path to the `store-german` Service.-matches:-path:value:/debackendRefs:-name:store-germanport:8080# Send unmatched requests to the store-v1 Service.-backendRefs:-name:store-v1port:8080

2. Deploy the HTTProute in your cluster:

   kubectlapply-fstore-route.yaml

   Thestore HTTPRoute is bound to theinternal-http Gateway by using theparentRefs property. These routing rules are configured on the underlyingload balancer as in this diagram:

   

   These routing rules process HTTP traffic in the following manner:

   • Traffic tostore.example.com/de goes to the Servicestore-german.
   • Traffic tostore.example.com with the HTTP header"env: canary" goes tothe Servicestore-v2.
   • The remaining traffic tostore.example.com goes to the Servicestore-v1.

3. Verify that the HTTPRoute has been deployed:

   kubectldescribehttproutestore

   The output is similar to the following:

   Name:         storeNamespace:    defaultLabels:       <none>Annotations:  <none>API Version:  gateway.networking.k8s.io/v1Kind:         HTTPRoute# Multiple lines are omitted here.Spec:  Hostnames:    store.example.com  Parent Refs:    Group:  gateway.networking.k8s.io    Kind:   Gateway    Name:   internal-http  Rules:    Backend Refs:      Group:      Kind:    Service      Name:    store-v1      Port:    8080      Weight:  1    Matches:      Path:        Type:   PathPrefix        Value:  /    Backend Refs:      Group:      Kind:    Service      Name:    store-v2      Port:    8080      Weight:  1    Matches:      Headers:        Name:   env        Type:   Exact        Value:  canary      Path:        Type:   PathPrefix        Value:  /    Backend Refs:      Group:      Kind:    Service      Name:    store-german      Port:    8080      Weight:  1    Matches:      Path:        Type:   PathPrefix        Value:  /deStatus:  Parents:    Conditions:      Last Transition Time:  2022-11-01T04:18:52Z      Message:      Reason:                Accepted      Status:                True      Type:                  Accepted      Last Transition Time:  2022-11-01T04:18:52Z      Message:      Reason:                ReconciliationSucceeded      Status:                True      Type:                  Reconciled    Controller Name:         networking.gke.io/gateway    Parent Ref:      Group:  gateway.networking.k8s.io      Kind:   Gateway      Name:   internal-httpEvents:  Type    Reason  Age                From                   Message  ----    ------  ----               ----                   -------  Normal  ADD     24m                sc-gateway-controller  default/store  Normal  SYNC    16m (x4 over 23m)  sc-gateway-controller  Bind of HTTPRoute "default/store" to ParentRef {Group:       gateway.networking.k8s.io",  # Multiple lines are omitted here.

4. Verify that the HTTPRoute is bound to the Gateway:

   kubectldescribegateway

   The output is similar to the following:

   Name:         internal-httpNamespace:    defaultLabels:       <none><...>Status:  Addresses:    Type:   IPAddress    Value:  10.128.15.203  Conditions:    Last Transition Time:  2022-11-01T03:47:01Z    Message:    Reason:                Scheduled    Status:                True    Type:                  Scheduled    Last Transition Time:  2022-11-01T03:47:01Z    Message:    Reason:                Ready    Status:                True    Type:                  Ready  Listeners:    Attached Routes:  1    Conditions:      Last Transition Time:  2022-11-01T03:47:01Z      Message:      Reason:                Ready      Status:                True      Type:                  Ready    Name:                    http    Supported Kinds:      Group:  gateway.networking.k8s.io      Kind:   HTTPRoute      <...>

Send traffic to your application

Now that your Gateway, Route, and application are deployed in your cluster, youcan pass traffic to your application. The Gateway is configured to serve HTTPtraffic only on port 80. Requests made with HTTPS might fail unless TLS hasbeen separately configured.

1. Retrieve the IP address from the Gateway so that you can send trafficto your application:

   kubectlgetgateways.gateway.networking.k8s.iointernal-http-o=jsonpath="{.status.addresses[0].value}"

   The output is an IP address.

2. Send traffic to this IP address from shell on a virtual machine (VM) instancewith connectivity to the cluster. You cancreate a VM for this purpose. Thisis necessary because the Gateway has an internal IP address and is onlyaccessible from within your VPC network.Because theinternal-http is a regional load balancer, the client shellmust be within the same region as the GKE cluster.

   Make a request to store.example.com:

   curlhttp://store.example.com--resolvestore.example.com:80:GATEWAY_IP_ADDRESS-v

   ReplaceGATEWAY_IP_ADDRESS with the IP address from the previous step.

   The output from the demo app shows information about the location where theapp is running:

   {"cluster_name":"gke1","host_header":"store.example.com","metadata":"store-v1","node_name":"gke-gke1-pool-2-bd121936-5pfc.c.gateway-demo-243723.internal","pod_name":"store-v1-84b47c7f58-pmgmk","pod_name_emoji":"💇🏼‍♀️","project_id":"gateway-demo-243723","timestamp":"2022-10-25T13:31:17","zone":"ZONE_NAME"}

3. Test the path match by going to the German version of the store service atstore.example.com/de:

   curlhttp://store.example.com/de--resolvestore.example.com:80:GATEWAY_IP_ADDRESS-v

   The output confirms that the request was served by astore-german Pod:

   {"cluster_name":"gke1","host_header":"store.example.com","metadata":"Gutentag!","node_name":"gke-gke1-pool-2-bd121936-n3xn.c.gateway-demo-243723.internal","pod_name":"store-german-5cb6474c55-lq5pl","pod_name_emoji":"🧞‍♀","project_id":"gateway-demo-243723","timestamp":"2022-10-25T13:35:37","zone":"ZONE_NAME"}

4. Finally, use theenv: canary HTTP header to send traffic to the canaryversion of the store Service:

   curl-H"env: canary"http://store.example.com--resolvestore.example.com:80:GATEWAY_IP_ADDRESS-v

   The output confirms that the request was served by astore-v2 Pod:

   {"cluster_name":"gke1","host_header":"store.example.com","metadata":"store-v2","node_name":"gke-gke1-pool-2-bd121936-5pfc.c.gateway-demo-243723.internal","pod_name":"store-v2-5788476cbd-s9thb","pod_name_emoji":"🦰","project_id":"gateway-demo-243723","timestamp":"2022-10-25T13:38:26","zone":"ZONE_NAME"}

Deploy an external Gateway

An external Gateway exposes applications that are reachable from the internet ornetworks outside of your VPC network. The deployment is similarto aninternal Gateway deployment except you must secureyour applications because the Gateway is accessible to the public internet.

You have two options to create an external Gateway: a global external Gateway ora regional external Gateway.

• Global external Gateway: This Gateway uses a global IP address (orAnycast IP address) as the frontend of the Gateway. This IP address isadvertised in all Google Cloud Compute regions. Clients sending traffic tothis Anycast IP address are routed to the closest Google location where theIP address is advertised. The global external Gateway is only available in thePremium Network Service Tier.
• Regional external Gateway: This Gateway uses a regional IP address as thefrontend of the Gateway. The behavior of this Gateway depends on the NetworkService Tier you select:
  • If you choose the Standard Network Service Tier (default), the regional IP addressis advertised only in the local Google Cloud Compute region where theregional external Gateway is deployed. Clients sending traffic to this IPaddress are routed by their local ISP and over the public internet beforereaching the Google region where the IP address is advertised.
  • If you choose the Premium Network Service Tier, the regional IP addressis advertised across Google's global network. This means client trafficenters Google's high-quality global backbone at a Google edge peeringpoint as close as possible to the client, even if the destination is aregional IP address. This location significantly reduces latency and improves performanceby minimizing the distance traffic travels over the public internet.

Deploy a global external Gateway

The following example shows you how to expose a store application with multiplecertificates attached to the global external Gateway and grouped in a certificatemap using Certificate Manager and an HTTPRoute.

Create a certificate map

Google recommends that you useCertificate Manager to manage certificates when you need 15 or more certificates per Gateway or youneed to use wildcard certificates.

You can also secure your external Gateway using Kubernetes Secrets orGoogle-managed SSL certificates. For more information, seeGateway security.

In this section, you create certificates usingCertificate Manager to secure the applications running on thecluster.

1. Enable the Certificate Manager API:

   gcloudservicesenablecertificatemanager.googleapis.com

2. Create a certificate map:

   gcloudbetacertificate-managermapscreatestore-example-com-map

3. Load your Google-managed certificate and keys into a Certificate:

   gcloudbetacertificate-managercertificatescreatestore-example-com-cert\--certificate-file="CERTIFICATE_FILE"\--private-key-file="PRIVATE_KEY_FILE"

   Replace the following:

   • CERTIFICATE_FILE: the name of your certificate file. The file must have the.pem extension. For example,cert.pem.
   • PRIVATE_KEY_FILE: the name of your private keyfile.

   For more information, seeCreate a private key and certificate.

4. Create aCertificateMapEntry which assigns the certificate to thecertificate map:

   gcloudbetacertificate-managermapsentriescreatestore-example-com-map-entry\--map=store-example-com-map\--hostname=store.example.com\--certificates=store-example-com-cert

To learn how to secure a Gateway using other sources for certificates, such asKubernetes Secrets or SSL certificates, seeSecure a Gateway.

Create a Gateway

A Gateway resource represents a data plane that routes traffic in Kubernetes. AGateway can represent many different kinds of load balancing and routingdepending on the GatewayClass it uses.

To learn more about the Gateway resource, see theGateway resource description or theAPI specification.

In this section, you create a Gateway. Application teams can use the Gateway toexpose their applications to the internet by deploying Routes independently andattaching them securely to the Gateway.

1. Save the following manifest to a file namedgateway.yaml:

   kind:GatewayapiVersion:gateway.networking.k8s.io/v1metadata:name:external-httpannotations:networking.gke.io/certmap:store-example-com-mapspec:# This GatewayClass uses a global external Application Load Balancer.gatewayClassName:gke-l7-global-external-managedlisteners:-name:httpsprotocol:HTTPSport:443

   This manifest describes a Gateway with the following fields:

   • gatewayClassName: gke-l7-global-external-managed: specifies theGatewayClass for this Gateway. This gateway class uses a global external Application Load Balancer.
   • protocol: HTTPS andport: 443: specify that the Gateway exposes port443 for HTTPS traffic. These fields enables TLS.
   • networking.gke.io/certmap: store-example-com-map: specifies the name ofthe certificate map in Certificate Manager.

   There is no TLS section because TLS isconfigured with Certificate Managerusing the annotationnetworking.gke.io/certmap.

2. Apply the manifest to your cluster:

   kubectlapply-fgateway.yaml

   It might take a few minutes for GKE to deploy the resources.

3. Verify that the Gateway has deployed successfully:

   kubectldescribegateway

   The output is similar to the following:

   Name:         external-httpNamespace:    defaultLabels:       <none>...Spec:  Gateway Class Name:  gke-l7-global-external-managed  Listeners:    Allowed Routes:      Namespaces:        From:  Same    Name:      https    Port:      443    Protocol:  HTTPS    Tls:      Certificate Refs:        Group:        Kind:   Secret        Name:   store-example-com      Mode:     Terminate ...

   This output shows that the Gateway deployed in your cluster has a loadbalancer and a public IP address. The Gateway has no Routes, which means itcan't send traffic to backends. Without Routes, all traffic goes to adefault backend, which returns an HTTP 404 response. In the next section,you deploy Routes, which instruct the Gateway to send traffic to backends.

Deploy the demo applications

Application teams can deploy their applications and Routes independently fromthe deployment of Gateways. In some cases the application team might want to ownthe Gateway as well and deploy it themselves as a resource dedicated to theirapplications. SeeRoute binding for different ownership models of Gateways and Routes. In this example, thestore team deploys their application and an accompanying HTTPRoute to exposetheir app through theexternal-http Gateway created in the previous section.

For more information about HTTPRoute fields, see theAPI specification.

1. Deploy the sample application to your cluster:

   kubectlapply-fhttps://raw.githubusercontent.com/GoogleCloudPlatform/gke-networking-recipes/main/gateway/gke-gateway-controller/app/store.yaml

   This sample application creates three Deployments and three Services whichare namedstore-v1,store-v2, andstore-german.

2. Verify that the application has deployed successfully:

   kubectlgetpod

   The output is similar to the following:

   NAME                            READY   STATUS    RESTARTS   AGEstore-german-66dcb75977-5gr2n   1/1     Running   0          38sstore-v1-65b47557df-jkjbm       1/1     Running   0          14mstore-v2-6856f59f7f-sq889       1/1     Running   0          14m

3. Verify that the Services have deployed successfully:

   kubectlgetservice

   The output is similar to the following:

   NAME           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGEstore-german   ClusterIP   10.48.3.183   <none>        8080/TCP   4sstore-v1       ClusterIP   10.48.2.224   <none>        8080/TCP   5sstore-v2       ClusterIP   10.48.4.48    <none>        8080/TCP   5s

Create an HTTPRoute

Route resources define protocol-specific rules for mapping traffic from aGateway to Kubernetes backends. TheHTTPRoute resource does HTTP and HTTPS traffic matching and filtering and is supported by all ofthegke-l7-* GatewayClasses.

In this section, you deploy an HTTPRoute, which configures the Gateway withrouting rules required to reach the sample application.

1. Save the following manifest to a file namedstore-route-external.yaml:

   kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:store-externalspec:parentRefs:# Bind the route to the 'external-http' Gateway.-kind:Gatewayname:external-httphostnames:-"store.example.com"rules:# Default rule for store.example.com that sends traffic to the store-v1 service.-backendRefs:-name:store-v1port:8080# Match requests with the "env: canary" header and send them to the store-v2 service.-matches:-headers:-name:envvalue:canarybackendRefs:-name:store-v2port:8080# Match requests with the path "/de" and sends them to the store-german service.-matches:-path:value:/debackendRefs:-name:store-germanport:8080

   This manifest describes an HTTPRoute that references theexternal-httpGateway.

2. Apply the manifest to your cluster:

   kubectlapply-fstore-route-external.yaml

   Thestore HTTPRoute is bound to theexternal-http Gateway by using theparentRefs property. These following diagram shows the routing rulesconfigured on the underlying load balancer:

   

   The routing rules process HTTP traffic as follows:

   • Traffic tostore.example.com/de routes to Servicestore-german.
   • Traffic tostore.example.com with the HTTP header"env: canary" routesto Servicestore-v2.
   • The remaining traffic tostore.example.com routes to Servicestore-v1.

3. Verify that the HTTPRoute has been deployed:

   kubectldescribehttproutestore-external

   The output is similar to the following:

   Name:         store-externalNamespace:    defaultLabels:       <none>Annotations:  <none>API Version:  gateway.networking.k8s.io/v1Kind:         HTTPRoute# Multiple lines are omitted here.Spec:  Hostnames:    store.example.com  Parent Refs:    Group:  gateway.networking.k8s.io    Kind:   Gateway    Name:   external-http  Rules:    Backend Refs:      Group:      Kind:    Service      Name:    store-v1      Port:    8080      Weight:  1    Matches:      Path:        Type:   PathPrefix        Value:  /    Backend Refs:      Group:      Kind:    Service      Name:    store-v2      Port:    8080      Weight:  1    Matches:      Headers:        Name:   env        Type:   Exact        Value:  canary      Path:        Type:   PathPrefix        Value:  /    Backend Refs:      Group:      Kind:    Service      Name:    store-german      Port:    8080      Weight:  1    Matches:      Path:        Type:   PathPrefix        Value:  /deStatus:  Parents:    # This section shows the status of this route in relation to each Gateway attached.    Conditions:      Last Transition Time:  2022-11-01T05:42:31Z      Message:      Reason:                Accepted      Status:                True # Means that the Gateway has validated and accepted this route's configuration.      Type:                  Accepted      Last Transition Time:  2022-11-01T05:43:18Z      Message:      Reason:                ReconciliationSucceeded      Status:                True      Type:                  Reconciled    Controller Name:         networking.gke.io/gateway    Parent Ref:      Group:  gateway.networking.k8s.io      Kind:   Gateway      Name:   external-httpEvents:  Type     Reason  Age    From                   Message  ----     ------  ----   ----                   -------  Normal   ADD     2m48s  sc-gateway-controller  default/store-external  Normal  SYNC  61s (x3 over 2m27s)  sc-gateway-controller  Bind of HTTPRoute "default/store-external" to ParentRef Group:       "gateway.networking.k8s.io",  ...

4. Verify that the HTTPRoute is bound to the Gateway:

   kubectldescribegatewayexternal-http

   The output is similar to the following:

   Name:         external-httpNamespace:    defaultLabels:       <none># Multiple lines are omitted here.Status:  Addresses:    Type:   IPAddress    Value:  34.149.207.45  Conditions:    Last Transition Time:  2022-11-01T05:37:21Z    Message:    Reason:                Scheduled    Status:                True    Type:                  Scheduled    Last Transition Time:  2022-11-01T05:43:18Z    Message:    Reason:                Ready    Status:                True    Type:                  Ready  Listeners:    Attached Routes:  1    Conditions:      Last Transition Time:  2022-11-01T05:43:18Z      Message:      Reason:                Ready      Status:                True      Type:                  Ready    Name:                    https    Supported Kinds:      Group:  gateway.networking.k8s.io      Kind:   HTTPRoute      # Multiple lines are omitted here.

Send traffic to your application

Now that your Gateway, Route, and application are deployed in your cluster, youcan pass traffic to your application.

1. Get the IP address of the Gateway:

   kubectlgetgateways.gateway.networking.k8s.ioexternal-http-o=jsonpath="{.status.addresses[0].value}"

   The output is an IP address.

2. Create a VM:

   gcloudcloud-shellssh

3. Send traffic to the Gateway IP address from the VM. You must set thehost header manually because you don't own theexample.com hostname.

   curlhttps://store.example.com--resolvestore.example.com:443:GATEWAY_IP_ADDRESS--cacertcacert.pem-v

   ReplaceGATEWAY_IP_ADDRESS with the IP address ofthe Gateway from the previous step.

   cacert.pem: the certificate file that you generated. You must save this file on the machine that you use to connect to the Gateway.

   The output shows information from the demo app about the location where theapp is running:

   {  "cluster_name": "gke1",  "host_header": "store.example.com",  "metadata": "store-v1",  "node_name": "gke-gke1-pool-2-bd121936-5pfc.c.gateway-demo-243723.internal",  "pod_name": "store-v1-84b47c7f58-pmgmk",  "pod_name_emoji": "💇🏼‍♀️",  "project_id": "gateway-demo-243723",  "timestamp": "2022-09-25T13:31:17",  "zone": "us-central1-a"}

4. Test the path match by going to the German version of thestore serviceatstore.example.com/de:

   curlhttps://store.example.com/de--resolvestore.example.com:443:GATEWAY_IP_ADDRESS--cacertcacert.pem-v

   The output confirms that the request was served by astore-german Pod:

   {  "cluster_name": "gke1",  "host_header": "store.example.com",  "metadata": "Gutentag!",  "node_name": "gke-gke1-pool-2-bd121936-n3xn.c.gateway-demo-243723.internal",  "pod_name": "store-german-5cb6474c55-lq5pl",  "pod_name_emoji": "🧞‍♀",  "project_id": "gateway-demo-243723",  "timestamp": "2022-09-25T13:35:37",  "zone": "us-central1-a"}

5. Send traffic to the canary version of thestore Service using theenv: canary HTTP header:

   curl-H"env: canary"https://store.example.com--resolvestore.example.com:443:GATEWAY_IP_ADDRESS--cacertcacert.pem-v

   The output confirms that the request was served by astore-v2 Pod:

   {  "cluster_name": "gke1",  "host_header": "store.example.com",  "metadata": "store-v2",  "node_name": "gke-gke1-pool-2-bd121936-5pfc.c.gateway-demo-243723.internal",  "pod_name": "store-v2-5788476cbd-s9thb",  "pod_name_emoji": "👩🏿",  "project_id": "gateway-demo-243723",  "timestamp": "2022-09-25T13:38:26",  "zone": "us-central1-a"}

Deploy a regional external Gateway

The following example shows you how to expose a store application with multiplecertificates attached to the regional external Gateway using self-managedcertificates and an HTTPRoute.

Create a proxy subnet for your regional Gateway

You mustconfigure a proxy-only subnetbefore you create a Gateway that uses aregional external Application Load Balancer. Each region ofa VPC in which you use regional external Application Load Balancer must have anexternal_managed_proxy subnet. This subnet provides internal IPaddresses to the load balancer proxies.

Create a certificate to secure your client traffic

You can use a certificate issued and validated by your certificate authority (CA)or create a self-signed certificate. For more information on how to create acertificate, seeStore a certificate in a Kubernetes Secret.

To secure traffic between your clients and your regional Gateway, you can usethe following methods:

• Google-managed or self-managed Certificate Managercertificates
• Self-managed regional SSL certificates
• Kubernetes Secrets

CertificateMap or Google-managed SSL certificates aren't supported withregional Gateways.

For more information, seeCertificates and Google Cloud load balancers

Create a regional external HTTP(S) Gateway

1. Create a regional static IP address for the external load balancer.

   gcloudcomputeaddressescreateIP_ADDRESS_NAME\--region=COMPUTE_REGION\--network-tier=STANDARD

   Replace the following:

   • IP_ADDRESS_NAME: the name of the new static IPaddress.
   • COMPUTE_REGION: The Compute Engineregion where your cluster is running.

2. Create a regional external Application Load Balancer Gateway using a self-managed certificate asfollows and save the manifest asregional-gateway.yaml:

   kind:GatewayapiVersion:gateway.networking.k8s.io/v1metadata:name:external-regional-httpspec:# Name of an existing GatewayClass.gatewayClassName:gke-l7-regional-external-managed# Listen for HTTPS traffic on port 443listeners:-name:httpsprotocol:HTTPSport:443tls:# Terminate the TLS session with the client at the Gateway.mode:Terminate# Certificates for the Gateway to use to create a new TLS session.certificateRefs:-name:store-example-com# The name of the static IP address of the external load balancer.# You can also use the `IPAddress` type to specify the actual IP address.addresses:-type:NamedAddressvalue:IP_ADDRESS_NAME

3. Apply theregional-gateway manifest:

   kubectlapply-fregional-gateway.yaml

4. Verify your configuration.

   kubectlgetgateway

   The output is similar to the following:

   NAME            CLASS                              ADDRESS         READY   AGEexternal-http   gke-l7-regional-external-managed   35.118.32.224   True    49s

   To get more details, use a describe command:

   kubectldescribegateway

   The output is similar to the following:

   Name:         external-regional-httpNamespace:    defaultLabels:       <none>...Spec:  Gateway Class Name:  gke-l7-regional-external-managed  Listeners:    Allowed Routes:      Namespaces:        From:  Same    Name:      https    Port:      443    Protocol:  HTTPS    Tls:      Certificate Refs:        Group:        Kind:   Secret        Name:   store-example-com      Mode:     Terminate  ...

Deploy the demo application

You can deploy your applications and routes independently from the deployment ofGateways.

For more information on how to deploy the demo applications, seeDeploy the demo applications.

Create an HTTPRoute

You mustcreate an HTTPRoute to do HTTP and HTTPStraffic matching and filtering.

Send Traffic to your application

After you have deployed your application and created HTTPRoutes, you can passtraffic to your application.

For more information on how to send traffic to yourapplication, seeSend traffic to your application.

Use shared Gateways

Gateway API uses separate resources, Gateways and Route resources, todeploy load balancers and routing rules. This differs from Ingress, whichcombines everything in one resource. By splitting responsibility amongresources, Gateway enables the load balancer and its routing rules to bedeployed separately and to be deployed by different users or teams. Thisenables Gateways to become shared Gateways that attach with many different Routesthat can be fully owned and managed by independent teams, even acrossdifferent namespaces.

Deploy routes against a shared Gateway

This example builds on theinternal-http Gateway deployed inDeploy an internal Gateway.

In this example, the site team deploys their application, Services, and anHTTPRoute to match traffic from the Gateway to those Services.

1. Deploy the example application:

   kubectlapply-fhttps://raw.githubusercontent.com/GoogleCloudPlatform/gke-networking-recipes/main/gateway/gke-gateway-controller/app/site.yaml

2. Save the following manifest to a file namedsite-route-internal.yaml:

   kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:site-internalspec:# Attach the HTTPRoute to the `internal-http` Gateway.parentRefs:-kind:Gatewayname:internal-http# Route requests that have `site.example.com` in the Host header.hostnames:-"site.example.com"# Send all requests to the `site-v1` Service.rules:-backendRefs:-name:site-v1port:8080

   This manifest describes an HTTPRoute that matches all traffic forsite.example.com and routes it to thesite-v1 Service.

3. Apply the manifest to your cluster:

   kubectlapply-fsite-route-internal.yaml

4. Verify that the HTTPRoute is attached to the Gateway:

   kubectldescribehttproute.gateway.networking.k8s.iosite-internal

   The output is similar to the following:

   Status:  Parents:    Conditions:      Last Transition Time:  2023-01-09T15:05:43Z      Message:      Reason:                Accepted      Status:                True      Type:                  Accepted      Last Transition Time:  2023-01-09T15:05:43Z      Message:      Reason:                ReconciliationSucceeded      Status:                True      Type:                  Reconciled    Controller Name:         networking.gke.io/gateway    Parent Ref:      Group:  gateway.networking.k8s.io      Kind:   Gateway      Name:   internal-http      ...

   If theAccepted condition for the Gateway isTrue, the HTTPRoute hassuccessfully bound to the Gateway. To learn more about the Statusfield, seeroute status.

5. Verify that traffic to the Gateway is routed correctly:

   curl-H"host: site.example.com"GATEWAY_IP_ADDRESScurl-H"host: store.example.com"GATEWAY_IP_ADDRESS

   ReplaceGATEWAY_IP_ADDRESS with the IP addressof the internal Gateway.

   You must use a virtual machine (VM) in the same VPCas the Gateway.

   The output is similar to the following:

   {  "cluster_name": "CLUSTER_NAME",  "host_header": "site.example.com",  "metadata": "site-v1",  "pod_name": "site-v1-5d64fc4d7d-fz6f6",  "pod_name_emoji": "👩🏼‍🍳",  "project_id": "PROJECT_ID",  "timestamp": "2022-11-02T19:07:01",  "zone": "ZONE_NAME"}...{  "cluster_name": "CLUSTER_NAME",  "host_header": "store.example.com",  "metadata": "store-v1",  "pod_name": "store-v1-6d8d58d78-vz8pn",  "pod_name_emoji": "🧝🏻‍♂️",  "project_id": "PROJECT_ID",  "timestamp": "2022-11-02T19:07:01",  "zone": "ZONE_NAME"}

   This output indicates that requests with thesite.example.com Host headerreach thesite-v1 Service, while requests with thestore.example.comHost header reach thestore-v1 Service.

Configure network tier

You can specify the network tier for a regional external Application Load Balancer's listeneraddress by using thetype field within theaddresses[] array in the Gatewaydefinition. If you don't specify a network tier, the Gateway defaults to usinga Standard Tier ephemeral IP address.

Use the following values for thetype field:

• networking.gke.io/premium-ephemeral-ipv4-address: assigns a Premium Tier IP address.
• networking.gke.io/standard-ephemeral-ipv4-address: assigns a Standard Tier IP address.

To assign IP addresses from both network tiers, specify both types in theaddresses field.

The following example shows you how to make Google Cloud assign a Premium Tier IPaddress to a Gateway. To provision a Standard Tier IP address, usenetworking.gke.io/standard-ephemeral-ipv4-address.

1. Save the following sample manifest asexternal-regional-http.yaml:

   kind:GatewayapiVersion:gateway.networking.k8s.io/v1metadata:name:external-regional-httpspec:# Name of an existing GatewayClass.gatewayClassName:gke-l7-regional-external-managed# Listen for HTTPS traffic on port 443.listeners:-name:httpsprotocol:HTTPSport:443tls:# Terminate the TLS session with the client at the Gateway.mode:Terminate# Certificates for the Gateway to use to create a new TLS session.certificateRefs:-name:store-example-comaddresses:# Request a Premium Tier ephemeral IPv4 address for the Gateway.-type:networking.gke.io/premium-ephemeral-ipv4-address

   Note: to provision a Standard Tier IP address, usenetworking.gke.io/standard-ephemeral-ipv4-address.

2. Apply the sample manifest:

   kubectlapply-fexternal-regional-http.yaml

kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:custom-default-backendspec:parentRefs:-kind:Gatewayname:my-internal-gateway# Omit the `hostnames` field to route all unmatched traffic from the# attached Gateway.rules:-backendRefs:-name:my-custom-default-backend-serviceport:8080

This HTTPRoute matches all traffic from a particular Gateway. You can only haveone such rule for each Gateway or else the rules conflict andprecedence ordering applies.

You can use a default backend to prevent someone from creating a default routeBackend that routes all Gateway traffic. An explicit HTTPRoute always takesprecedence over new HTTPRoutes with conflicting routing rules.

Configure a static IP address for a Gateway

Every Gateway has an IP address it uses to listen for traffic. If you don'tspecify an IP address on the Gateway, then the Gateway controller automaticallyprovides an IP address. You can also create a static IP address so that the IPaddress exists independent of the Gateway lifecycle.

After a Gateway is deployed, its IP address shows in the status field:

kind:Gateway...status:addresses:-value:10.15.32.3

Depending on the GatewayClass, the IP address is allocated from the following subnets:

The fieldaddresses.NamedAddress lets you specify an IP address independentlyof the Gateway. You can create a static IP address resource prior to Gatewaydeployment and the resource is referenced by theNamedAddress. You can reusethe static IP address even if the Gateway is deleted.

Use a named IP address

You can configure an IPv4 or IPv6 address by specifying aNamedAddress. Youmust provision a static IP address before you create a Gateway.

1. Create a static IP address resource:

   gcloudcomputeaddressescreateIP_ADDRESS_NAME\--purpose=SHARED_LOADBALANCER_VIP\--region=COMPUTE_REGION\--subnet=SUBNET\--project=PROJECT_ID

   Replace the following:

   • IP_ADDRESS_NAME: the name of the new static IPaddress
   • COMPUTE_REGION: for regional Gateways, the Compute Engineregion where your cluster is running. This flag is not needed for global, external Gateways.
   • SUBNET: the subnet for the IP address. This flag is not needed for global, external Gateways.
   • PROJECT_ID: the project where yourGKE cluster is running.

2. Save the following manifest to a file namednamed-ip-gateway.yaml:

   kind:GatewayapiVersion:gateway.networking.k8s.io/v1metadata:name:internal-httpspec:gatewayClassName:gke-l7-rilblisteners:-name:httpprotocol:HTTPport:80addresses:-type:NamedAddressvalue:IP_ADDRESS_NAME

   This manifest describes a Gateway that references the named IP address.

3. Apply the manifest to your cluster:

   kubectlapply-fnamed-ip-gateway.yaml

4. Verify your Gateway IP address:

   kubectldescribegatewayinternal-http

   The output is similar to the following:

   Name:         internal-httpNamespace:    defaultLabels:       <none>...Spec:  Addresses:    Type:              NamedAddress    Value:IP_ADDRESS_NAME  Gateway Class Name:  gke-l7-rilb  Listeners:    Allowed Routes:      Namespaces:        From:  Same    Name:      http    Port:      80    Protocol:  HTTPStatus:  Addresses:    Type:   IPAddress    Value:  10.15.32.103

Configure HTTP-to-HTTPS redirects

Cloud Load Balancing offers HTTP to HTTPS redirect functionality. Anexternal Application Load Balancer redirects unencrypted HTTP requests to an HTTPS load balancerthat uses the same IP address. When you create a Gateway with HTTP-to-HTTPSredirects enabled, both of these load balancers are created automatically.Requests to the external IP address of the Gateway on port 80 are automaticallyredirected to the same external IP address on port 443.

By default, HTTP to HTTPS redirects are not defined on the Gateway.

To redirect HTTP traffic to HTTPS, configure a Gateway to handle bothHTTP and HTTPS traffic. If you disable either HTTP or HTTPS, the Gateway doesnot redirect traffic.

The following example shows you how you can use HTTP-to-HTTPS redirect as a meansto ensure that traffic from your clients going to your web applications is alwaysbeing redirected to a secure page.

HTTP-to-HTTPS redirects are not supported with thegke-l7-gxlb andgke-l7-gxlb-mc GatewayClasses. To learn more about the different featuressupported with each GatewayClass, seeGatewayClass capabilities.

Note that thehostnames section in an HTTPRoute is optional. You can create acatch-all redirect for all hostnames on a specific HTTP port by omitting thehostnames section. Additionaly, you can use a wildcard redirect, forexample:

...hostnames:-"*.example.com"# Matches all subdomains of example.com...

Redirect HTTP traffic from an infrastructure namespace

In some cases, there isn't a clear distinction between the infrastructure orplatform admin team and the application teams and preventing misuse of theGateway can become a challenge.

The following example further restricts the use of the HTTP listener to preventunintentional use of non-secure protocol from the application teams. This exampleconfigures the Gateway to allow an HTTPRoute to use the HTTP listener only if theroute is in a namespace that has theotherInfra: httpToHttps label. However,the Gateway allows HTTPRoutes in any namespace to use the HTTPS listener. Youcan restrict the http-redirect namespace using KubernetesRBAC so that application teams cannot create an HTTPRoute in this namespace bymistake.

1. Create the namespace of a Gateway. Save the manifest asgateway-namespace.yaml:

   apiVersion:v1kind:Namespacemetadata:name:gateway-infra

2. Apply the manifest:

   kubectlapply-fgateway-namespace.yaml

3. Create the namespace of a Gateway and save the manifest asredirect-namespace.yaml:

   apiVersion:v1kind:Namespacemetadata:name:http-redirectlabels:otherInfra:httpToHttps

   Thishttp-redirect namespace has theotherInfra: httpToHttps label.

4. Apply the manifest:

   kubectlapply-fredirect-namespace.yaml

5. To restrict the http listener usage, create a Gateway using the followingmanifest. Save the manifest asexternal-gateway.yaml:

   kind:GatewayapiVersion:gateway.networking.k8s.io/v1metadata:name:external-httpnamespace:gateway-infraspec:# Name of an existing GatewayClass.gatewayClassName:gke-l7-global-external-managedlisteners:-name:httpprotocol:HTTPport:80#  Allow only HTTPRoutes from namespaces that have the# `otherInfra: httpToHttps` label to use this listener.allowedRoutes:kinds:-kind:HTTPRoutenamespaces:from:Selectorselector:matchLabels:otherInfra:httpToHttps-name:httpsprotocol:HTTPSport:443# Allow HTTPRoutes from any namespace to use this listener.allowedRoutes:kinds:-kind:HTTPRoutenamespaces:from:Alltls:mode:Terminateoptions:networking.gke.io/pre-shared-certs:store-example-com

   • Thenamespace field specifies that the Gateway is created in thegateway-infra namespace.

   • Thenamespaces field in theallowedRoutes section restricts thehttp listener to the namespace matching the labelotherInfra: httpToHttps.

6. Apply the manifest:

   kubectlapply-fexternal-gateway.yaml

7. To force the HTTPS redirect, create a default HTTPRoute using the followingmanifest. Save the manifest ashttp-redirect.yaml:

   kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:redirect# Create the HTTPRoute in the `http-redirect` namespace.namespace:http-redirectspec:# Attach the HTTPRoute to the `http` listener in the `external-http`# Gateway.parentRefs:-namespace:gateway-infraname:external-httpsectionName:httprules:# Respond to the request with an HTTPS redirection.-filters:-type:RequestRedirectrequestRedirect:scheme:https

   • ThesectionName field instructs the Gateway to match only on the httplistener. TheRequestRedirect filter forces the redirection to thehttps listener.

8. Apply the manifest:

   kubectlapply-fhttp-redirect.yaml

9. Create a Service for an application using the following manifest. Save themanifest asservice-deployment.yaml:

   apiVersion:v1kind:Servicemetadata:name:store-v1spec:selector:app:storeversion:v1ports:-port:8080targetPort:8080---apiVersion:apps/v1kind:Deploymentmetadata:name:store-v1spec:replicas:2selector:matchLabels:app:storeversion:v1template:metadata:labels:app:storeversion:v1spec:containers:-name:whereamiimage:us-docker.pkg.dev/google-samples/containers/gke/whereami:v1ports:-containerPort:8080env:-name:METADATAvalue:"store-v1"

10. Apply the manifest:

    kubectlapply-fservice-deployment.yaml

11. Create an HTTPRoute for an application that only allows HTTPS using thefollowing manifest. Save the manifest ashttp-route.yaml:

    kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:store-externallabels:gateway:external-httpspec:# Attach the HTTPRoute to the HTTPS listener in the `external-http`# Gateway.parentRefs:-name:external-httpnamespace:gateway-infrasectionName:https# Match requests that have the `store.example.com` Host header.hostnames:-"store.example.com"# Route requests to the `store-v1` Service.rules:-backendRefs:-name:store-v1port:8080

12. Apply the manifest:

    kubectlapply-fhttp-route.yaml

Configure path redirects and URL rewrites

Path redirectsinvolve redirecting an incoming request from one URL path to another. Pathredirects let you change the structure of the URL when you need to handleoutdated or deprecated URLs.

URL rewriteshelp modify the incoming URL before processing it on the server. It allows you tochange the structure or format of the URL without actually changing the underlyingcontent or file structure. URL rewriting is beneficial for creating user-friendlyand SEO-friendly URLs that are easy to remember and understand.By default, path redirects and URL rewrites are not configured, you need toexplicitly configure those redirects or rewrites using a filter in your HTTPRoute.

GKE Gateway supports path redirects and URL rewrites. For moreinformation, seeHTTP path redirects and rewrites.

Configure path redirects

You can configure path redirects to either replace the entire path or only aprefix in the URL.

Replace entire path

1. To replace an entire path, configure a filter in an HTTPRoute that replacesany URL that contains the prefix/any-path in the URL path by the strictvalue/new-path.

2. Create anHTTPRoute manifest as follows and name it asstore.yaml:

   apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:storespec:parentRefs:-kind:Gatewayname:external-httphostnames:-store.example.comrules:-matches:-path:# Match requests by a prefix, like `/any-path` and `/any-path/home`.type:PathPrefixvalue:/any-pathfilters:-type:RequestRedirectrequestRedirect:path:# Replace the full path with `/new-path`. For example, both# `/any-path/home` and `/any-path` become `/new-path`.type:ReplaceFullPathreplaceFullPath:/new-pathstatusCode:302

   For example, this manifest sets a routing rule for an HTTPRoute as follows:Any route to the URLhttps://store.example.com/any-path/...should be redirected to a new location,https://store.example.com/new-path/ (strict).

3. Apply the manifest:

   kubectlapply-fstore.yaml

This routing rule follows a strict redirection rule, which means that the browserdoes not attempt to cache the redirect, instead, redirects to the latestversion.

Replace a prefix only

1. To replace a prefix only, configure a filter in an HTTPRoute that replacesany URL that contains the prefix/any-prefix in the URL path by the strictvalue/new-prefix.

2. Create anHTTPRoute manifest as follows and name it asstore.yaml:

   apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:storespec:parentRefs:-kind:Gatewayname:external-httphostnames:-store.example.comrules:-matches:-path:# Match requests by prefix, like `/any-prefix` and# `/any-prefix/home`.type:PathPrefixvalue:/any-prefixfilters:-type:RequestRedirectrequestRedirect:path:# Replace the matched prefix with `/new-prefix`. For example,# `/any-prefix` becomes `/new-prefix` and `/any-prefix/home`# becomes `/new-prefix/home`.type:ReplacePrefixMatchreplacePrefixMatch:/new-prefixstatusCode:302

   For example, this manifest sets a routing rule for an HTTPRoute as follows:Any route to the URLhttps://store.example.com/any-path/v1/...should be redirected to a new location,https://store.example.com/new-path/v1/... (only).

3. Apply the manifest:

   kubectlapply-fstore.yaml

This routing rule follows the only redirection rule, which ensures that thebrowser always redirects you to the same intended page.

Configure URL rewrites

Set URL rewrites to change the way a URL appears to users. You can use URLrewrites to make URLs more user-friendly, to improve SEO, or to redirect usersto a new page.

Rewrite the entire hostname

To rewrite the entire hostname:

1. Configure a filter in an HTTPRoute that instructs the Gateway to replace theHost information in the request header fromwww.example.com tostore.example.combefore forwarding the request to the backend service.

2. Create anHTTPRoute manifest as follows and name it aswww.yaml:

   apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:wwwspec:parentRefs:-kind:Gatewayname:external-httphostnames:-www.example.comrules:-filters:-type:URLRewrite# Replace the hostname in the URL with `store.example.com`.urlRewrite:hostname:store.example.combackendRefs:-name:store-v1port:8080

   For example, with the above configuration, any request tohttps://www.example.com is forwarded tothe backend service with theHost: store.example.com header, instead ofHost: www.example.com.

3. Apply the manifest:

   kubectlapply-fwww.yaml

Rewrite using path modifiers

You can combine rewrites with path modifiers to provide advanced URL and pathmodifications before relaying the request to the backend service.

To rewrite using path modifiers:

1. Configure a filter in an HTTPRoute that instructs the Gateway to replacethe 'Host' information in the request header fromwww.example.comto store.example.comand replace the value/store by/ before forwarding the request to thebackend service.

2. Create anHTTPRoute manifest as follows and name it aswww.yaml:

   apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:wwwspec:parentRefs:-kind:Gatewayname:external-httphostnames:-www.example.comrules:-matches:-path:type:PathPrefixvalue:/storefilters:-type:URLRewrite# For URLs that have `/store` in the path, replace the hostname with# store.example.com and replace the `/store` prefix with `/de`.urlRewrite:hostname:store.example.compath:type:ReplacePrefixMatchreplacePrefixMatch:/debackendRefs:-name:store-germanport:8080

   For example, with the above configuration, any request tohttps://www.example.com/store/... isforwarded to the backend service withHost: store.example.com in therequest header (instead ofHost: www.example.com) and the/store isrewritten to/de.

3. Apply the manifest:

   kubectlapply-fwww.yaml

Verify your configuration

To verify that the filter was applied after creating your HTTPRoute with URLrewrite or path redirects filters, do the following:

kubectlgethttproutewww-oyaml

The output is similar to the following:

  apiVersion: gateway.networking.k8s.io/v1  kind: HTTPRoute  metadata:    annotations:      kubectl.kubernetes.io/last-applied-configuration: |        {"apiVersion":"gateway.networking.k8s.io/v1","kind":"HTTPRoute","metadata":{"annotations":{},"name":"www","namespace":"default"},"spec":{"hostnames":["www.example.com"],"parentRefs":[{"kind":"Gateway","name":"external-http"}],"rules":[{"backendRefs":[{"name":"store-german","port":8080}],"filters":[{"type":"URLRewrite","urlRewrite":{"hostname":"store.example.com","path":{"replacePrefixMatch":"/de","type":"ReplacePrefixMatch"}}}],"matches":[{"path":{"type":"PathPrefix","value":"/store"}}]}]}}    creationTimestamp: "2023-06-22T01:00:42Z"    generation: 3    name: www    namespace: default    resourceVersion: "51268631"    uid: e516493e-806d-44d6-ae0d-1c9ff25682cf  spec:    hostnames:    - www.example.com    parentRefs:    - group: gateway.networking.k8s.io      kind: Gateway      name: external-http    rules:    - backendRefs:      - group: ""        kind: Service        name: store-german        port: 8080        weight: 1      filters:      - type: URLRewrite        urlRewrite:          hostname: store.example.com          path:            replacePrefixMatch: /de            type: ReplacePrefixMatch      matches:      - path:          type: PathPrefix          value: /store  status:    parents:    - conditions:      - lastTransitionTime: "2023-06-22T01:11:26Z"        message: ""        observedGeneration: 2        reason: Accepted        status: "True"        type: Accepted      - lastTransitionTime: "2023-06-22T01:11:26Z"        message: ""        observedGeneration: 2        reason: ReconciliationSucceeded        status: "True"        type: Reconciled      controllerName: networking.gke.io/gateway      parentRef:        group: gateway.networking.k8s.io        kind: Gateway        name: external-http

To get more details, use the describe command:

kubectldescribehttproute

Configure custom request and response headers

Custom request and response headers let you specify additional headers toHTTP(S) requests and responses. Depending on the information detected by theload balancer, these headers can include the following information:

• Latency to the client
• Geographic location of the client's IP address
• Parameters of the TLS connection

By default, there are no custom headers added to the request sent/received to/fromyour backend services, you need to explicitly configure custom headers using afilter in your HTTPRoute.

You can configure custom headers by adding a filter section in your HTTPRoute'srules as follows:

Configure custom request headers

Create a HTTPRoute manifest with a RequestHeaderModifier filter and save it ashttp-route-request.yaml:

apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:storespec:<...>rules:filters:-type:RequestHeaderModifierrequestHeaderModifier:<...>

Apply the manifest:

kubectlapply-fhttp-route-request.yaml

Configure custom response headers

Create a HTTPRoute manifest with a ResponseHeaderModifier filter and save it ashttp-route-response.yaml:

apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:storespec:<...>rules:filters:-type:ResponseHeaderModifierresponseHeaderModifier:<...>

Apply the manifest:

kubectlapply-fhttp-route-response.yaml

You can add, set, and remove headers as described in theGateway API implementation.You can configure your HTTPRoute with a custom header usingGoogle Cloud supported variables.

Example 1:

To configure an HTTPRoute that adds client location information to the HTTPrequest before sending it to the backend service, create a HTTPRoute manifestand name it asexternal-http-request.yaml:

apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:storespec:parentRefs:-kind:Gatewayname:external-httphostnames:-store.example.comrules:-matches:-path:type:PathPrefixvalue:/frfilters:# Add custom headers to requests that have `/fr` in the path.-type:RequestHeaderModifierrequestHeaderModifier:add:-name:X-Client-Geo-Locationvalue:"{client_region},{client_city}"backendRefs:-name:store-frenchport:8080

For example, for clients located in Strasbourg, France, the Gateway adds aheader asX-Client-Geo-Location:FR,Strasbourg.

Example 2:

To configure an HTTPRoute that adds a custom response header to support HTTPStrict Transport Security, create a HTTPRoute manifest and name it asexternal-http-response.yaml:

apiVersion:gateway.networking.k8s.io/v1kind:HTTPRoutemetadata:name:storespec:parentRefs:-kind:Gatewayname:external-httphostnames:-store.example.comrules:-matches:-path:type:PathPrefixvalue:/defilters:# Add custom headers to responses to requests that have `/de` in the# path.-type:ResponseHeaderModifierresponseHeaderModifier:add:-name:Strict-Transport-Securityvalue:max-age=63072000backendRefs:-name:store-germanport:8080

Verify your configuration

1. To verify your configuration after configuring custom request and responseheaders, do the following:

   kubectlgethttproute

   The output is similar to the following:

     NAME    HOSTNAMES               AGE  store   ["store.example.com"]   4d23h

2. To get more details, use the describe command:

   kubectldescribehttproute

   The output is similar to the following:

     Name:         store  Namespace:    default  Labels:       <none>  Annotations:  <none>  API Version:  gateway.networking.k8s.io/v1  Kind:         HTTPRoute  Metadata:    Creation Timestamp:  2023-05-27T00:51:01Z    Generation:          5    Resource Version:    25418887    UID:                 2e07a1b8-420b-41b4-acd1-cecbfcd39f42  Spec:    Hostnames:      store.example.com    Parent Refs:      Group:  gateway.networking.k8s.io      Kind:   Gateway      Name:   external-http    Rules:      Backend Refs:        Group:        Kind:    Service        Name:    store-v1        Port:    8080        Weight:  1      Matches:        Path:          Type:   PathPrefix          Value:  /      Backend Refs:        Group:        Kind:    Service        Name:    store-v2        Port:    8080        Weight:  1      Matches:        Headers:          Name:   env          Type:   Exact          Value:  canary        Path:          Type:   PathPrefix          Value:  /      Backend Refs:        Group:        Kind:    Service        Name:    store-german        Port:    8080        Weight:  1      Filters:        Request Header Modifier:          Add:            Name:   X-Client-Geo-Location            Value:  {client_region},{client_city}        Type:       RequestHeaderModifier      Matches:        Path:          Type:   PathPrefix          Value:  /de  Status:    <...>

Route status

HTTPRoute resources emit conditions and events to help users understand if aHTTPRoute has successfully bound with one or more Gateways or if it was rejected.

HTTPRoute conditions

HTTPRoute conditions indicate the status of the Route and the Gateways it isbound to. Because a Route can be bound to multiple Gateways, this is a list ofGateways and the individual conditions between the Route and each Gateway.

• Accepted=True indicates that the HTTPRoute is successfully bound to aGateway.
• Accepted=False indicates that the HTTPRoute has been rejected from bindingwith this Gateway.

If there are no Gateways listed under theGateway bindings heading, then yourHTTPRoute labels and Gateway label selectors might not match. This can occur ifyour Route is not being selected by any Gateways.

HTTPRoute events

HTTPRoute events provide details about the status of the HTTPRoute. Events aregrouped by the following reasons:

• ADD events are triggered by a resource being added.
• UPDATE events are triggered by a resource being updated.
• SYNC events are triggered by periodic reconciliation.

Route merging, precedence, and validation

Route precedence

Gateway API defines strictprecedence rules for how traffic is matched by Routes that have overlapping routing rules. Theprecedence between two overlapping HTTPRoutes is as follows:

1. Hostname merge: The longest/most specific hostname match.
2. Path merge: The longest/most specific path match.
3. Header merge: The largest number of HTTP headers that match.
4. Conflict: If the previous three rules don't establish precedence, thenprecedence goes to the HTTPRoute resource with the oldest timestamp.

Route merging

Forgke-l7 GatewayClasses, all HTTPRoutes for a given Gateway are merged intothe sameURL map resource. How the HTTPRoutesare merged together depends on the type of overlap between HTTPRoutes. TheHTTPRoute fromthe earlier example can be split into three separateHTTPRoutes to illustrate route merging and precedence:

1. Route merge: All three HTTPRoutes attach with the sameinternal-httpGateway, so they are merged together.
2. Hostname merge: All three Routes match forstore.example.com, so theirhostname rules are merged.
3. Path merge: store-german-route has a more specific path/de, so thisis not merged further. store-v1-route and store-v2-route both match on the same/* path as well, so they are merged on the path.
4. Header merge: store-v2-route has a more specific set of HTTP header matchesthan store-v1-route, so they are not merged further.
5. Conflict: Because the Routes are able to be merged on hostname, path, andheaders, there are no conflicts, and all of the routing rules are applied totraffic.

The single HTTPRoute used inthe earlier example are equivalentto these three separate routes:

kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:store-v1-routespec:parentRefs:-kind:Gatewayname:internal-httphostnames:-"store.example.com"rules:-backendRefs:-kind:Servicename:store-v1port:8080---kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:store-v2-routespec:parentRefs:-kind:Gatewayname:internal-httphostnames:-"store.example.com"rules:-matches:-headers:-type:Exactname:envvalue:canarybackendRefs:-kind:Servicename:store-v2port:8080---kind:HTTPRouteapiVersion:gateway.networking.k8s.io/v1metadata:name:store-german-routespec:parentRefs:-kind:Gatewayname:internal-httphostnames:-"store.example.com"rules:-matches:-path:type:PathPrefixvalue:/debackendRefs:-kind:Servicename:store-germanport:8080

Kubernetes Gateways and Istio Gateways

Note that the Kubernetes Gateway API and the Istio API both have a resourcenamedGateway. While they perform similar functions, they are not the sameresource. If you are using Istio and Gateway API in the same Kubernetescluster, these names overlap when using kubectl on the command line.kubectl get gateway might return the Kubernetes Gateway resources and not theIstio Gateway resources or vice versa.

$kubectlapi-resourcesNAMESHORTNAMESAPIGROUPNAMESPACEDKINDgatewaysgwnetworking.istio.io/v1beta1trueGatewaygatewaysgtwnetworking.k8s.io/v1beta1trueGateway

If you are using Istio and upgrade to GKE 1.20 and later it isrecommended to start using the Gateway resource shortname or specify the APIgroup. The shortname for a Kubernetes Gateway isgtw and the shortname for anIstio Gateway isgw. The following commands return the Kubernetes Gateway andIstio Gateway resources respectively.

# Kubernetes Gateway$kubectlgetgtwNAMECLASSmulti-cluster-gatewaygke-l7-global-external-managed-mc$kubectlgetgateway.networking.x-k8s.ioNAMECLASSmulti-cluster-gatewaygke-l7-global-external-managed-mc# Istio Gateway$kubectlgetgwNAMEAGEbookinfo-gateway64m$kubectlgetgateway.networking.istio.ioNAMEAGEbookinfo-gateway64m

Troubleshooting

Proxy-only subnet missing in the region

Symptom:

The following issue might occur when you create a regional Gateway(internal or external):

generic::invalid_argument: error ensuring load balancer: Insert: Invalid value for field 'resource.target': 'regions/[REGION_NAME]/targetHttpProxies/gkegw-x5vt-default-internal-http-[ID]'. A reserved managed proxy subnetwork with purpose REGIONAL_MANAGED_PROXY is required.

Reason:

This error message indicates that no proxy-only subnet exists in theregion for your Gateway.

Workaround:

To resolve this issue, configure aproxy-only subnet.

Proxy-only subnet already exists in the region with the wrong purpose

Symptom:

The following issue might occur when you create a proxy-only subnet for yourregional Gateway (internal or external):

ERROR: (gcloud.compute.networks.subnets.create) Could not fetch resource: - The resource 'projects/[PROJECT_NAME]/regions/[REGION_NAME]/subnetworks/[PROXY_ONLY_SUBNET_NAME]' already exists

Reason:

This error message indicates that you attempted to create a regional proxy-onlysubnet in a region that already has a proxy-only subnet.

Workaround:

To resolve this issue, use the following steps:

1. Check that a proxy-only subnet already exists in the region and verify thatit has the correct purpose:

   1. List your subnets to find which one is the proxy-only subnet in the region:

      gcloudcomputenetworkssubnetslist--regions=COMPUTE_REGION

      ReplaceCOMPUTE_REGION with the Compute Engineregion where you want to create your regional Gateway.

   2. Describe your proxy-only subnet in the region to find its purpose:

      gcloudcomputenetworkssubnetsdescribePROXY_ONLY_SUBNET\--regionCOMPUTE_REGION|grep-E'name|purpose'

      ReplacePROXY_ONLY_SUBNET with the proxy-only subnet.

   GKE Gateway only supportsREGIONAL_MANAGED_PROXY proxy-onlysubnets for regional Gateways (internal or regional).

2. If the existing proxy-only subnet in the region was created with anINTERNAL_HTTPS_LOAD_BALANCER purpose,migrate its purpose toREGIONAL_MANAGED_PROXY.

No healthy upstream

Symptom:

The following issue might occur when you create a Gateway butcannot access the backend services (503 response code):

no healthy upstream

Reason:

This error message indicates that the health check prober cannot find healthybackend services. It is possible that your backend services are healthybut you might need to customize the health checks.

Workaround:

To resolve this issue,customize your health check based on your application'srequirements (for example,/health) using aHealthCheckPolicy.

What's next

• Learn more about theGateway controller.
• Learn how toConfigure Gateway resources using Policies.
• Learn about other Gateway configurations in theGateway API documentation.