This page shows you how to create anexternal Application Load Balancerto route requests toserverless backends. Here the term serverless refers tothe following serverless compute products:

• App Engine
• Cloud Run functions
• Cloud Run

Serverless NEGsallow you to useGoogle Cloud serverless appswithexternal Application Load Balancers. After youconfigure a load balancer with the serverless NEG backend, requests to the loadbalancer are routed to the serverless app backend.

To learn more about serverless NEGs, read theServerless NEGsoverview.

Before you begin

1. Deploy an App Engine, Cloud Run functions, or Cloud Run service.
2. If you haven't already done so, install the Google Cloud CLI.
3. Configure permissions.
4. Add an SSL certificate resource.

Deploy an App Engine, Cloud Run functions, or Cloud Run service

The instructions on this page assume you already have aCloud Run, Cloud Run functions,or App Engine service running.

For the example on this page, we have used theCloud Run Pythonquickstart to deploy a Cloud Runservice in theus-central1 region. The rest of this page shows you how to setup an external Application Load Balancer that uses a serverless NEG backend to route requests tothis service.

If you haven't already deployed a serverless app, or if you want to try aserverless NEG with a sample app, use one of the following quickstarts. You cancreate a serverless app in any region, but you must use thesame region lateron to create the serverless NEG and load balancer.

Cloud Run

To create a simple Hello World application, package it into a container image,and then deploy the container image to Cloud Run, seeQuickstart: Build and Deploy.

If you already have a sample container uploaded to the Container Registry, seeQuickstart: Deploy a Prebuilt Sample Container.

Cloud Run functions

SeeCloud Run functions: Python Quickstart.

App Engine

See the following App Engine quickstart guides for Python 3:

• Standard Environment
• Flexible Environment

Install the Google Cloud CLI

Install the Google Cloud CLI. Seegcloud Overview forconceptual and installation information about the tool.

If you haven't run the gcloud CLI previously, first rungcloud init to initialize your gcloud directory.

Configure permissions

To follow this guide, you need to create a serverless NEG and create an externalHTTP(S) load balancer in a project. You should be either a projectowner oreditor, or you should have the followingCompute Engine IAM roles:

Optional: Use BYOIP addresses

With bring your own IP (BYOIP), you can import your own public addresses toGoogle Cloud to use the addresses with Google Cloud resources. Forexample, if you import your own IPv4 addresses, you can assign one to theforwarding rule when you configure your load balancer. When you follow theinstructions in this document tocreate the load balancer, provide the BYOIP address as theIP address.

For more information about using BYOIP, seeBring your own IP addresses.

Reserve an external IP address

Now that your services are up and running, set up aglobal static external IPaddress that your customers useto reach your load balancer.

gcloud compute addresses createEXAMPLE_IP \    --network-tier=PREMIUM \    --ip-version=IPV4 \    --global

gcloud compute addresses describeEXAMPLE_IP \    --format="get(address)" \    --global

Create an SSL certificate resource

To create anHTTPS load balancer, you must add an SSL certificateresource to the load balancer's front end. Create an SSL certificate resourceusing either aGoogle-managed SSLcertificateor aself-managed SSL certificate.

• Google-managed certificates. Using Google-managed certificates isrecommended because Google Cloud obtains, manages, and renews thesecertificates automatically. To create a Google-managed certificate,you must have a domain and the DNS records for that domain in order forthe certificate to be provisioned.

  Additionally, you need to update the domain's DNS A record to point tothe load balancer's IP address created in the previous step.If you have multiple domains in a Google-managed certificate, you mustupdate DNS records for all the domains and subdomains to point to your loadbalancer's IP address. For detailed instructions, seeUsing Google-managedcertificates.

• Self-signed certificates. If you do not want to set up a domain at thistime, you can use a self-signed SSL certificate for testing.

In the following diagram, the load balancer uses a serverless NEG backendto direct requests to a serverless Cloud Run service. For thisexample, we have used theCloud Run Pythonquickstart to deploy a Cloud Run service.

Because health checks are not supported for backend services with serverless NEGbackends, you don't need to create a firewall rule allowing health checksif the load balancer has only serverless NEG backends.

Connect your domain to your load balancer

After the load balancer is created, note the IP address that is associated withthe load balancer—for example,30.90.80.100. To point your domain to yourload balancer, create anA record by using your domain registration service. Ifyou added multiple domains to your SSL certificate, you must add anA recordfor each one, all pointing to the load balancer's IP address. For example, tocreateA records forwww.example.com andexample.com, use the following:

NAME                  TYPE     DATAwww                   A        30.90.80.100@                     A        30.90.80.100

If you use Cloud DNS as your DNS provider, seeAdd, modify, and delete records.

Test the load balancer

Now that you have configured your load balancer, you can start sendingtraffic to the load balancer's IP address. If you configured a domain, you cansend traffic to the domain name as well. However, DNS propagation can take timeto complete so you can start by using the IP address for testing.

1. In the Google Cloud console, go to theLoad balancing page.

   Go to Load balancing

2. Click on the load balancer you just created.

3. Note theIP Address of the load balancer.

4. For anHTTP load balancer, you can test your load balancerusing a web browser by going tohttp://IP_ADDRESS.ReplaceIP_ADDRESS with theload balancer's IP address. You should be directed to thehelloworld service homepage.
   

5. For anHTTPS load balancer, you can test your load balancerusing a web browser by going tohttps://IP_ADDRESS.ReplaceIP_ADDRESS with theload balancer's IP address. You should be directed to thehelloworld service homepage.
   If that does not work and you are using a Google-managed certificate,confirm that your certificate resource's status is ACTIVE. For moreinformation, seeGoogle-managed SSL certificate resourcestatus.
   If you used a self-signed certificate for testing, your browser displaysa warning. You must explicitly instruct your browser to accept aself-signed certificate. Click through the warning to see the actual page.

Restrict ingress on default endpoint

Without the proper ingress settings, users can use your serverless application'sdefault URL to bypass the load balancer, Cloud Armor securitypolicies, SSL certificates, and private keys that are passed through the loadbalancer.

Ensure external traffic only reaches your serverless application from the loadbalancer by using setting ingress to "Internal and Cloud Load Balancing".

• Set network ingress to "Internal and Cloud Load Balancing" forApp Engine
• Set network ingress to "Internal and Cloud Load Balancing" forCloud Run functions
• Set network ingress to "Internal and Cloud Load Balancing" forCloud Run

Additionally, you candisable the default URLfor Cloud Run.

Additional configuration options

This section expands on the configuration example to provide alternative andadditional configuration options. All of the tasks are optional. You can performthem in any order.

Set up multi-region load balancing

This section builds on the load-balancer setup described earlier on this page,in which you created one serverless NEG in theus-central1 region that pointsto a Cloud Run service in the same region. It alsoassumes that you've created a second Cloud Run service intheeurope-west1 region. The second serverless NEG that you create will pointto this Cloud Run service in theeurope-west1 region.

In this example, you will complete the following steps:

1. Create a second serverless NEG in theeurope-west1 region.
2. Attach the second serverless NEG to the backend service.

To add a second serverless NEG to an existing backend service, follow these steps.

Use an authenticated Pub/Sub push subscription with a multi-region Cloud Run deployment

For authenticated push requests, Cloud Run expects aregion-specific audience field by default. In case of a multi-regionCloud Run deployment, if the push request is routed to aCloud Run service in a different region, the JWT tokenverification fails due to an audience mismatch.

To work around this region-specific restriction:

1. Configure acustom audiencethat's the same for service deployments in different regions.
2. Configure the Pub/Sub push messages to use the custom audience as theaudience in the JWT token.

Set up regional routing

A common reason for serving applications from multipleregions is to meet data locality requirements. For example, you might want toensure that requests made by European users are always served from a regionlocated in Europe. To set this up you need a URL schema with separate URLs forEU and non-EU users, and direct your EU users to the EU URLs.

In such a scenario, you would use the URL map to route requests from specificURLs to their corresponding regions. With such a setup, requests meant for oneregion are never delivered to a different region. This provides isolationbetween regions. On the other hand, when a region fails, requests are not routedto a different region. So this setup does not increase your service'savailability.

To set up regional routing, you will need to use the Premium network tier sothat you can combine different regions in a single forwarding rule.

To set up a load balancer with regional routing:

1. Set up two Cloud Run services in different regions. Let's assume you havedeployed two Cloud Run services: hello-world-eu to a region in Europe,and hello-world-us to a region in the US.
2. Create an external Application Load Balancer with the following setup:
   1. Set up a backend service with a serverless NEG in Europe. The serverlessNEG must be created in the same region as the Cloud Run servicedeployed in Europe.
   2. Set up a second backend service with another serverless NEG in theUS. This serverless NEG must be created in the same region as the CloudRun service deployed in the US.
   3. Set up your URL map with the appropriate host and path rules so thatone set of URLs routes to the European backend service while all therequests route to the US backend service.
   4. Set up your frontend configuration with the Premium network tier.

The rest of the setup can be the same as described previously. Your resultingsetup should look like this:

Use a URL mask

When creating a serverless NEG, instead of selecting a specific Cloud Runservice, you can use a URL mask to point to multiple services serving at thesame domain. A URL mask is a template of your URL schema. The serverless NEGwill use this template to extract the service name from the incomingrequest's URL and map the request to the appropriate service.

URL masks are particularly useful if your service ismapped to a customdomain rather than the default address thatGoogle Cloud provides for the deployed service. A URL mask allows you totarget multiple services and versions with a single rule even when yourapplication is using a custom URL pattern.

If you haven't already done so, make sure you readServerless NEGs overview:URL Masks.

Construct a URL mask

To construct a URL mask for your load balancer, start with the URL of yourservice. For this example, we will use a sample serverless app running athttps://example.com/login. This is the URL where the app'slogin servicewill be served.

1. Remove thehttp orhttps from the URL. You are left withexample.com/login.
2. Replace the service name with a placeholder for the URL mask.
   1. Cloud Run: Replace the Cloud Run service name with theplaceholder<service>. If the Cloud Run service has a tag associated with it,replace the tag name with the placeholder<tag>.In this example, the URL mask you are left with is,example.com/<service>.
   2. Cloud Run functions: Replace the function name with the placeholder<function>.In this example, the URL mask you are left with is,example.com/<function>.
   3. App Engine: Replace the service name with the placeholder<service>.If the service has a version associated with it, replace the version withthe placeholder<version>.In this example, the URL mask you are left with is,example.com/<service>.
   4. API Gateway: Replace the gateway name with the placeholder<gateway>.In this example, the URL mask you are left with is,example.com/<gateway>.

3. (Optional) If the service name (or function, version, or tag) can beextracted from the path portion of the URL,the domain can be omitted. Thepath part of the URL mask is distinguishedby the first/ character. If a/ is not present in the URL mask,the mask is understood to represent the host only. Therefore, for thisexample, the URL mask can be reduced to/<service>,/<gateway>, or/<function>.

   Similarly, if the service name can be extracted from the host part of theURL, you can omit the path altogether from the URL mask.

   You can also omitany host or subdomain components that come before the first placeholder aswell as any path components that come after the last placeholder. In suchcases, the placeholder captures the required information for the component.

Here are a few more examples that demonstrate these rules:

Create a serverless NEG with a URL mask

gcloud compute network-endpoint-groups create helloworld-neg-mask \  --region=us-central1 \  --network-endpoint-type=serverless \  --cloud-run-url-mask="example.com/<service>"

gcloud compute network-endpoint-groups create helloworld-neg-mask \ --region=us-central1 \ --network-endpoint-type=serverless \ --cloud-function-url-mask="example.com/<service>"

gcloud compute network-endpoint-groups create helloworld-neg-mask \  --region=us-central1 \  --network-endpoint-type=serverless \  --app-engine-url-mask="example.com/<service>"

gcloud beta compute network-endpoint-groups create helloworld-neg-mask \  --region=us-central1 \  --network-endpoint-type=serverless \  --serverless-deployment-platform=apigateway.googleapis.com \  --serverless-deployment-resource=my-gateway \  --serverless-deployment-url-mask="example.com/<gateway>"

To learn how the load balancer handles issues with URL mask mismatches, seeTroubleshooting issues with serverlessNEGs.

Move your custom domain to be served by the external Application Load Balancer

If your serverless compute apps are being mapped to custom domains, you mightwant to update your DNS records so that traffic sent to the existingCloud Run, Cloud Run functions,API Gateway, or App Engine custom domain URLsis routed through the load balancer instead.

For example, if you have a custom domain calledexample.com and all your CloudRun services are mapped to this domain, you should update the DNS record forexample.com to point to the load balancer's IP address.

Before updating the DNS records, you can test your configuration locally byforcing local DNS resolution of the custom domain to theload balancer's IPaddress. To test locally, either modify the/etc/hosts/ file onyour local machine to pointexample.com to the load balancer's IP address, or,use thecurl --resolve flag to forcecurl to use the load balancer's IPaddress for the request.

When the DNS record forexample.com resolves to the HTTP(S) load balancer's IPaddress, requests sent toexample.com begin to be routed via the loadbalancer. The load balancer dispatches them to the relevant backend serviceaccording to its URL map. Additionally, if the backend service is configured witha URL mask, the serverless NEG uses the mask to route the request to theappropriate Cloud Run, Cloud Run functions,API Gateway, or App Engine service.

Enable Cloud CDN

EnablingCloud CDN for your Cloud Run serviceallows you to optimize content delivery by caching content close to your users.

You can enable Cloud CDN on backend services used byglobal external Application Load Balancers by using thegcloud computebackend-services updatecommand.

gcloud compute backend-services update helloworld-backend-service
    --enable-cdn
    --global

Cloud CDN is supported for backend services withCloud Run, Cloud Run functions,API Gateway,and App Engine backends.

Enable IAP on the external Application Load Balancer

You can configure IAP to beenabled or disabled (default). If enabled, you must provide values foroauth2-client-id andoauth2-client-secret.

To enable IAP, update the backend serviceto include the--iap=enabled flag with theoauth2-client-id andoauth2-client-secret.

gcloud compute backend-services updateBACKEND_SERVICE_NAME \    --iap=enabled,oauth2-client-id=ID,oauth2-client-secret=SECRET \    --global

Optionally, you canenable IAPfor a Compute Engine resource by using the Google Cloud console,gcloud CLI, or API.

Enable Google Cloud Armor

Cloud Armor is a security product that providesprotection against distributed denial-of-service (DDoS) attacks to allGCLB proxy load balancers. Cloud Armor also provides configurable securitypolicies to services accessed through an external Application Load Balancer. To learn aboutCloud Armor security policies for external Application Load Balancers, seeCloud Armor security policy overview.

Optional: Configure a default backend security policy. The default security policy throttles traffic over a user-configured threshold. For more information about default security policies, see theRate limiting overview.

1. To opt out of the Cloud Armor default security policy, selectNone in theCloud Armor backend security policy list.
2. To configure the Cloud Armor default security policy, selectDefault security policy in theCloud Armor backend security policy list.
3. In thePolicy name field, accept the automatically generated name or enter a name for your security policy.
4. In theRequest count field, accept the default request count or enter an integer between1 and10,000.
5. In theInterval field, select an interval.
6. In theEnforce on key field, choose one of the following values:All,IP address, orX-Forwarded-For IP address. For more information about these options, seeIdentifying clients for rate limiting.

Enable logging and monitoring

You can enable, disable, and view logs for an external Application Load Balancerbackend service. When using the Google Cloud console, logging is enabled by defaultfor backend services with serverless NEG backends. You can usegcloud todisable logging for each backend service as needed. For instructions, seeLogging.

The load balancer also exports monitoring data to Cloud Monitoring.Monitoring metrics can be used to evaluate a load balancer's configuration,usage, and performance. Metrics can also be used to troubleshoot problemsand improve resource utilization and user experience. For instructions, seeMonitoring.

Delete a serverless NEG

A network endpoint group cannot be deleted if it is attached to a backendservice. Before you delete a NEG, ensure that it is detached from thebackend service.

gcloud compute backend-services remove-backend helloworld-backend-service \    --network-endpoint-group=helloworld-serverless-neg \    --network-endpoint-group-region=us-central1 \    --global

gcloud compute network-endpoint-groups delete helloworld-serverless-neg \    --region=us-central1

What's next

• Using logging and monitoring
• Troubleshooting serverless NEGs issues
• Clean up the load balancer setup
• Using a Terraform module for an external HTTPS load balancer with a Cloud Runbackend