This page shows how to useServerless VPC Accessto connect your App Engine standard environment app directly to yourVPC network, allowing access to Compute Engine VM instances,Memorystore instances, and any other resources with an internalIP address.

Before you begin

• If you don't already have a VPC network in your project,create one.

• If you use Shared VPC, seeConnecting to a Shared VPC network.

• In the Google Cloud console, ensure that the Serverless VPC AccessAPI is enabled for your project.

  Enable API

Create a Serverless VPC Access connector

To send requests to your VPC network and receive thecorresponding responses without using the public internet, you can use aServerless VPC Access connector.

If your connector is located in the same project as its VPCnetwork, you can either create a connector using an existing subnet or createa connector and a new subnet.

If your connector is located in a service project and uses a Shared VPCnetwork, the connector and its associated VPC network are indifferent projects. When a connector and its VPC network are indifferent projects, a Shared VPC network administrator must create theconnector's subnet in the Shared VPC networkbefore you can create theconnector,and you must create the connector using an existing subnet.

To learn more about subnet requirements, seeconnector subnetrequirements.

To learn about connector throughput, including machine type and scaling, seeThroughput and scaling.

You can create a connector by using the Google Cloud console, Google Cloud CLI, orTerraform.

resource "google_project_service" "vpcaccess-api" {  project = var.project_id # Replace this with your project ID in quotes  service = "vpcaccess.googleapis.com"}

module "test-vpc-module" {  source       = "terraform-google-modules/network/google"  version      = "~> 13.0"  project_id   = var.project_id # Replace this with your project ID in quotes  network_name = "my-serverless-network"  mtu          = 1460  subnets = [    {      subnet_name   = "serverless-subnet"      subnet_ip     = "10.10.10.0/28"      subnet_region = "us-central1"    }  ]}module "serverless-connector" {  source     = "terraform-google-modules/network/google//modules/vpc-serverless-connector-beta"  version    = "~> 13.0"  project_id = var.project_id  vpc_connectors = [{    name        = "central-serverless"    region      = "us-central1"    subnet_name = module.test-vpc-module.subnets["us-central1/serverless-subnet"].name    # host_project_id = var.host_project_id # Specify a host_project_id for shared VPC    machine_type  = "e2-standard-4"    min_instances = 2    max_instances = 7    }    # Uncomment to specify an ip_cidr_range    #   , {    #     name          = "central-serverless2"    #     region        = "us-central1"    #     network       = module.test-vpc-module.network_name    #     ip_cidr_range = "10.10.11.0/28"    #     subnet_name   = null    #     machine_type  = "e2-standard-4"    #     min_instances = 2    #   max_instances = 7 }  ]  depends_on = [    google_project_service.vpcaccess-api  ]}

Configure your service to use a connector

After you have created a Serverless VPC Access connector,you must configure each service in your App Engine app that you want toconnect to your VPC network.

To specify a connector for a service in your app:

1. In order to useServerless VPC Access, discontinue the use of theApp Engine URL Fetch service. Serverless VPC Accessdoes not support URL Fetch, and requests made using URL Fetch will ignoreServerless VPC Access settings. Make outbound connections withsocketsinstead.

2. Add thevpc_access_connector field to your service'sapp.yaml file:

   vpc_access_connector:  name: projects/PROJECT_ID/locations/REGION/connectors/CONNECTOR_NAME

   WherePROJECT_ID is your Google Cloud project ID,REGION is the region your connector is in,andCONNECTOR_NAME is the name of your connector.

3. Deploy the service:

   gcloudappdeploy

After you deploy your service, it is able to send requests to internal IPaddresses in order to access resources in your VPC network.

Restrict access to VPC resources

Required firewall rules for connectors in service projects

If you create a connector in a standalone VPC network or in thehost project of a Shared VPC network, Google Cloud creates allnecessary firewall rules for the connector's operation. For more information,seeFirewall rules for connectors in standalone VPC networks or Shared VPC host projects.

However, if you create a connector in a service project and the connector targetsa Shared VPC network in the host project, you must add firewall rulesto allow necessary traffic for the connector's operation from the followingranges:

• Serverless infrastructure IP range:35.199.224.0/19
• Health check probe IP ranges:35.191.0.0/16,35.191.192.0/18, and130.211.0.0/22

These ranges are used by the Google infrastructure underlyingCloud Run, Cloud Run functions, and App Engine standard environment. All requests fromthese IP addresses originate from Google infrastructure to make surethat each serverless resource only communicates with theconnector that it's connected to.

You must also allow traffic from the connector's subnet to resources in yourVPC network.

To perform these steps, you must have one of the following roles on thehost project:

• Owner role (roles/owner)
• Compute Security Admin role (roles/compute.securityAdmin)
• CustomIdentity and Access Management (IAM) role with thecompute.firewalls.create permissionenabled

For a basic configuration, apply the rules to allow serverless resources in anyservice project connected to the Shared VPC network to send requests toany resource in the network.

To apply these rules, run the following commands in the host project:

1. Create firewall rules that allow requests from Google's serverlessinfrastructure and health check probes to reach all connectors in thenetwork. In these commands, UDP and TCP ports are used as proxies and forHTTP health checks, respectively. Don't change the specified ports.

   gcloudcomputefirewall-rulescreateserverless-to-vpc-connector\    --allowtcp:667,udp:665-666,icmp\    --source-ranges=35.199.224.0/19\    --direction=INGRESS\    --target-tagsvpc-connector\    --network=VPC_NETWORK

   gcloudcomputefirewall-rulescreatevpc-connector-to-serverless\    --allowtcp:667,udp:665-666,icmp\    --destination-ranges=35.199.224.0/19\    --direction=EGRESS\    --target-tagsvpc-connector\    --network=VPC_NETWORK

   gcloudcomputefirewall-rulescreatevpc-connector-health-checks\    --allowtcp:667\    --source-ranges=35.191.0.0/16,35.191.192.0/18,130.211.0.0/22\    --direction=INGRESS\    --target-tagsvpc-connector\    --network=VPC_NETWORK

   ReplaceVPC_NETWORK with the name of theVPC network to attach your connector to.

2. Create an ingress firewall rule on your VPC network to allowrequests from connectors that target this network:

   gcloudcomputefirewall-rulescreatevpc-connector-requests\    --allowtcp,udp,icmp\    --direction=INGRESS\    --source-tagsvpc-connector\    --network=VPC_NETWORK

   This rule gives the connector access to every resource in the network. Tolimit the resources that your serverless environment can reach by usingServerless VPC Access, seeRestrict connector VM access to VPC network resources.

Create firewall rules for specific connectors

Following the procedure inRequired firewall rules for connectors in service projectsresults in firewall rules that apply toall connectors, both current onesand ones created in the future. If you don't want this, but instead want tocreate rules for specific connectors only, you can scope the rules so thatthey apply only to those connectors.

To limit the scope of the rules to specific connectors, you can use one of thefollowing mechanisms:

• Network tags: Every connector has two network tags:vpc-connector andvpc-connector-REGION-CONNECTOR_NAME.Use the latter format to limit the scope of your firewall rules to a specificconnector.
• IP ranges: Use this for the egress rules only, because it doesn't work foringress rules. You can use the IP range of the connectorsubnet to limit the scope of your firewall rules to a single VPCconnector.

Restrict connector VM access to VPC network resources

You can restrict your connector's access to resources in its targetVPC network by usingVPC firewallrules or rules infirewallpolicies. You can accomplish theserestrictions using one of the following strategies:

• Create ingress rules whose targets represent the resources that you wantto limit connector VM access to and whose sources represent the connector VMs.
• Create egress rules whose targets represent the connector VMs and whosedestinations represent the resources that you want to limit connector VMaccess to.

The following examples illustrate each strategy.

Restrict access using ingress rules

Choose eithernetwork tags orCIDR ranges to control the incoming traffic to your VPC network.

Restrict access using egress rules

The following steps show how to create egress rules to restrict connector access.

1. Ensure that you have the required permissions to insert firewall rules. Youmust have one of the followingIdentity and Access Management (IAM)roles:

   • Compute Security Admin role
   • Custom IAM role with thecompute.firewalls.createpermission enabled

2. Deny egress traffic from your connector.

   Create an egress firewall rule on your Serverless VPC Accessconnector to prevent it from sending outgoing traffic, with the exception ofestablished responses, to any destination.

   gcloudcomputefirewall-rulescreateRULE_NAME\--action=DENY\--rules=PROTOCOL\--direction=EGRESS\--target-tags=VPC_CONNECTOR_NETWORK_TAG\--network=VPC_NETWORK\--priority=PRIORITY

   Replace the following:

   • RULE_NAME: the name of your new firewall rule. For example,deny-vpc-connector.

   • PROTOCOL: one or more protocols that you want to allow from yourVPC connector. Supported protocols aretcp orudp. For example,tcp:80,udpallows TCP traffic through port 80 and UDP traffic. For more information, seethe documentation for theallowflag.

     For security and validation purposes, you can alsoconfigure deny rules to block traffic for the following unsupportedprotocols:ah,all,esp,icmp,ipip, andsctp.

   • VPC_CONNECTOR_NETWORK_TAG: the universal VPCconnector network tag if you want the rule to apply to all existingVPC connectors and any VPC connectors madein the future. Or, the unique VPC connector network tagif you want to control a specific connector.

   • VPC_NETWORK: the name of your VPC network

   • PRIORITY: an integer between 0-65535. For example, 0 sets thehighest priority.

3. Allow egress traffic when the destination is in the CIDR range that you wantyour connector to access.

   Use theallow anddestination-ranges flags to create a firewall ruleallowing egress traffic from your connector for a specific destination range.Set the destination range to the CIDR range of the resource in yourVPC network that you want your connector to be able to access.Set the priority for this rule to be a lower value than the priority of therule you made in the previous step.

   gcloudcomputefirewall-rulescreateRULE_NAME\--allow=PROTOCOL\--destination-ranges=RESOURCE_CIDR_RANGE\--direction=EGRESS\--network=VPC_NETWORK\--target-tags=VPC_CONNECTOR_NETWORK_TAG\--priority=PRIORITY

   Replace the following:

   • RULE_NAME: the name of your new firewall rule. For example,allow-vpc-connector-for-select-resources.

   • PROTOCOL: one or more protocols that you want to allow from yourVPC connector. Supported protocols aretcp orudp. For example,tcp:80,udpallows TCP traffic through port 80 and UDP traffic. For more information, seethe documentation for theallowflag.

   • RESOURCE_CIDR_RANGE: the CIDR range for the connector whoseaccess you are restricting

   • VPC_NETWORK: the name of your VPC network

   • VPC_CONNECTOR_NETWORK_TAG: the universal VPCconnector network tag if you want the rule to apply to all existingVPC connectors and any VPC connectors madein the future. Or, the unique VPC connector network tagif you want to control a specific connector. If you used the uniquenetwork tag in the previous step, use the unique network tag.

   • PRIORITY: an integer less than the priority you set inthe previous step. For example, if you set the priority for the ruleyou created in the previous step to 990, try 980.

For more information about the required and optional flags for creating firewallrules, refer to thedocumentation forgcloud compute firewall-rules create.

Manage your connector

Controlling egress traffic from a service

By default, only requests to internal IP addresses and internal DNS names arerouted through a Serverless VPC Access connector. You canspecify the egress setting for your service in yourapp.yaml file.

Egress settings are not compatible with the URL Fetch service. If you have notalready done so,disable the URL Fetch defaultand discontinue any explicit use of theurlfetchlibrary. Using theurlfetch library ignores egress settings, and requests will not route through a Serverless VPC Access connector.

To configure the egress behavior of your App Engine service:

1. Add theegress_setting attribute to thevpc_access_connector field of yourservice'sapp.yaml file:

   vpc_access_connector:name:projects/PROJECT_ID/locations/REGION/connectors/CONNECTOR_NAMEegress_setting:EGRESS_SETTING

   Replace:

   • PROJECT_ID with your Google Cloud project ID
   • REGION with the region your connector is in
   • CONNECTOR_NAME with the name of your connector
   • EGRESS_SETTING with one of the following:
     • private-ranges-only Default. Only requests toRFC 1918 andRFC 6598 IP addressranges or internal DNS names are routed to your VPCnetwork. All other requests are routed directly to the internet.
     • all-traffic All outbound requests from your service arerouted to your VPC network. Requests are then subject tothe firewall, DNS, and routing rules of your VPC network.Note that routing all outbound requests to your VPC networkincreases the amount of egress handled by theServerless VPC Access connector and canincur charges.

2. Deploy the service:

   gcloudappdeploy

Disconnect a service from a VPC network

To disconnect a service from a VPC network, remove thevpc_access_connector field from theapp.yaml file and re-deploy the service.

Connectors continue to incur charges even if they have no traffic and aredisconnected. For details, seepricing.If you no longer need your connector, be sure todelete it to avoidcontinued billing.

Update a connector

You can update and monitor the following attributes of your connector by usingthe Google Cloud console, Google Cloud CLI, or the API:

• Machine (instance) type
• Minimum and maximum number of instances
• Recent throughput, number of instances, and CPU utilization

Update machine type

Decrease minimum and maximum number of instances

Find the current attribute values

To find the current attribute values for your connector, run the following inyour terminal:

gcloudcomputenetworksvpc-accessconnectorsdescribeCONNECTOR_NAME--region=REGION--project=PROJECT

• CONNECTOR_NAME: the name of your connector
• REGION: the name of your connector's region
• PROJECT: the name of your Google Cloud project

Monitor connector usage

Monitoring usage over time can help you determine when to adjust a connector'ssettings. For example, if CPU utilization spikes, you might try increasingthe maximum number of instances for better results. Or if you are maxing outthroughput, you might decide to switch to a larger machine type.

To display charts for the connector's throughput, number of instances, and CPUutilization metrics over time by using the Google Cloud console:

1. Go to the Serverless VPC Access overview page.

   Go to Serverless VPC Access

2. Click the name of the connector you want to monitor.

3. Select the number of days you want to display between 1 and 90 days.

4. In theThroughput chart, hold the pointer over the chart to view the connector'srecent throughput.

5. In theNumber of instances chart, hold the pointer over the chart to view thenumber of instances recently used by the connector.

6. In theCPU Utilization chart, hold the pointer over the chart to view theconnector's recent CPU usage. The chart displays the CPU usage distributedacross instances for the 50th, 95th, and 99th percentiles.

Delete a connector

Before you delete a connector, you must remove it from any serverless resourcesthat still use it. Deleting a connector before removing it from your serverlessresources prohibits you from deleting the VPC networklater.

For Shared VPC users who set up connectors in the Shared VPChost project, you can use the commandgcloud compute networks vpc-access connectors describeto list the projects in which there are serverless resources that use a givenconnector.

To delete a connector, use the Google Cloud console or the Google Cloud CLI:

Troubleshooting

Service account permissions

To perform operations in your Google Cloud project,Serverless VPC Access uses theServerless VPC Access Service Agent service account. This serviceaccount's email address has the following form:

service-PROJECT_NUMBER@gcp-sa-vpcaccess.iam.gserviceaccount.com

By default, this service account has theServerless VPC Access Service Agent role(roles/vpcaccess.serviceAgent). Serverless VPC Accessoperations may fail if you change this account's permissions.

Poor network performance or high idle CPU utilization

Using a single connector for thousands of instances can cause performancedegradation and elevated idle CPU utilization. To fix this, shard your servicesbetween multiple connectors.

Issues with custom MTU

If you experience issues with a custom MTU, ensure that youuse the default MTU setting for Cloud Run.

Errors

Service account needs Service Agent role error

If you use theRestrict Resource Service Usageorganization policy constraintto block Cloud Deployment Manager (deploymentmanager.googleapis.com), youmight see the following error message:

Serverless VPC Access service account (service-<PROJECT_NUMBER>@gcp-sa-vpcaccess.iam.gserviceaccount.com) needs Serverless VPC Access Service Agent role in the project.

Set the organization policyto either remove Deployment Manager from the denylist or add itto the allowlist.

Connector creation error

If creating a connector results in an error, try the following:

• Specify anRFC 1918 internal IP range that does not overlap with any existing IP addressreservations in the VPC network.
• Grant your project permission to use Compute Engine VM images from theproject with IDserverless-vpc-access-images. For more information abouthow to update your organization policy accordingly, seeSet image access constraints.

Unable to access resources

If you specified a connector but still cannot access resources in yourVPC network, make sure that there are no firewall rules on yourVPC network with a priority lower than 1000 that deny ingressfrom your connector's IP address range.

If youconfigure a connector in a Shared VPC service project,make sure that your firewall rulesallow ingress from your serverless infrastructure to the connector.

Connection refused error

If you receiveconnection refused orconnection timeout errors that degradenetwork performance, your connections could be growing without limit acrossinvocations of your serverless application. To limit the maximum number ofconnections used per instance, use a client library that supports connectionpools. For detailed examples of how to use connection pools, seeManage database connections.

Resource not found error

When deleting a VPC network or a firewall rule, you might see amessage that is similar to the following:The resource"aet-uscentral1-subnet--1-egrfw" was not found.

For information about this error and its solution, seeResource not found error in theVPC firewall rules documentation.

Next steps

• Monitor admin activity withServerless VPC Access audit logging.
• Protect resources and data bycreating a service perimeterwith VPC Service Controls.
• Learn about theIdentity and Access Management (IAM)roles associated with Serverless VPC Access. SeeServerless VPC Accessroles in theIAM documentation for a list of permissions associated witheach role.
• Learn how toconnect to Memorystorefrom the App Engine standard environment.