Serverless architectures let you develop software and services withoutprovisioning or maintaining servers. You can use serverless architectures tobuild applications for a wide range of services.

This document provides opinionated guidance for DevOps engineers, securityarchitects, and application developers on how to help protect serverlessapplications that useCloud Run functions (2nd gen).The document is part of a security blueprint that consists of the following:

• AGitHub repository that contains a set of Terraform configurations and scripts.
• A guide to the architecture, design, and security controls that youimplement with the blueprint (this document).

Though you can deploy this blueprint without deploying theGoogle Cloud enterprise foundations blueprint first, this document assumes that you've already configured a foundational setof security controls as described in the Google Cloud enterprise foundationsblueprint. The architecture that's described in this document helps you to layeradditional controls onto your foundation to help protect your serverlessapplications.

To help define key security controls that are related to serverlessapplications, theCloud Security Alliance (CSA) publishedTop 12 Critical Risks for Serverless Applications.The security controls used in this blueprint are designed to address the risksthat are relevant to the various use cases described in this document.

Serverless use cases

The blueprint supports the following use cases:

• Deploying a serverless architecture using Cloud Run functions (thisdocument)
• Deploying a serverless architecture using Cloud Run

Differences between Cloud Run functions and Cloud Runinclude the following:

• Cloud Run functions is triggered by events, such as changes to datain a database or the receipt of a message from a messaging system such asPub/Sub. Cloud Run is triggered by requests, such asHTTP requests.
• Cloud Run functions is limited to aset of supported runtimes.You can use Cloud Run with any programming language.
• Cloud Run functions manages containers and the infrastructure thatcontrols the web server or language runtime so that you can focus on yourcode. Cloud Run provides the flexibility for you to runthese services yourself, so that you have control of the containerconfiguration.

For more information about differences between Cloud Run andCloud Run functions, seeChoosing a Google Cloud compute option.

Architecture

This blueprint uses aShared VPC architecture, in which Cloud Run functions is deployed in a service projectand can access resources that are located in other VPC networks.

The following diagram shows a high-level architecture, which is furtherdescribed in the example architectures that follow it.

The architecture that's shown in the preceding diagram uses a combination ofthe following Google Cloud services and features:

• Cloud Run functions lets you run functions as a service and manages the infrastructure on yourbehalf. By default, this architecture deploys Cloud Run functions withan internal IP address only and without access to the public internet.
• The triggering event is the event that triggers Cloud Run functions.As further described in the example architectures, this can be aCloud Storage event, a scheduled interval, or a change in BigQuery.
• Artifact Registry stores the source containers for your Cloud Run functions application.
• Shared VPC lets you connect a Serverless VPC Access connector in your serviceproject to the host project. You deploy a separate Shared VPCnetwork for each environment (production, non-production, and development).Thisnetworking design provides network isolation between the different environments. AShared VPC network lets you centrally manage network resources in acommon network while delegating administrative responsibilities for theservice project.

• The Serverless VPC Access connector connects yourserverless application to your VPC network usingServerless VPC Access.Serverless VPC Access helps to ensure that requests fromyour serverless application to the VPC network aren't exposed to theinternet. Serverless VPC Access lets Cloud Run functionscommunicate with other services, storage systems, and resources thatsupport VPC Service Controls.

  You can configure Serverless VPC Access in theShared VPC host project or a service project. By default, thisblueprint deploys Serverless VPC access in the Shared VPC hostproject to align with the Shared VPC model of centralizing networkconfiguration resources. For more information, seeComparison of configuration methods.

• VPC Service Controls creates a security perimeter that isolates your Cloud Run functionsservices and resources by setting up authorization, access controls, andsecure data exchange.This perimeter is designed to isolate your application and managed servicesby setting up additional access controls and monitoring, and to separateyour governance of Google Cloud from the application. Your governanceincludes key management and logging.

• The consumer service is the application that is acted on byCloud Run functions. The consumer service can be an internal server oranother Google Cloud service such as Cloud SQL. Depending on youruse case, this service might be behind Cloud Next Generation Firewall, in anothersubnet, in the same service project as Cloud Run functions, or inanother service project.

• Secure Web Proxy is designed to secure the egress web traffic, if required. It enablesflexible and granular policies based on cloud identities and webapplications. This blueprint uses Secure Web Proxy for granular access policiesto egress web traffic during the build phase of Cloud Run functions.The blueprint adds an allowed list of URLs to theGateway Security Policy Rule.

• Cloud NAT provides outbound connection to the internet, if required. Cloud NATsupports source network address translation (SNAT) for compute resourceswithout public IP addresses. Inbound response packets use destinationnetwork address translation (DNAT). You can disable Cloud NAT ifCloud Run functions doesn't require access to the internet.Cloud NAT implements the egress network policy that is attached toSecure Web Proxy.

• Cloud Key Management Service (Cloud KMS) stores thecustomer-managed encryption keys (CMEKs) that are used by the services in this blueprint, including your serverlessapplication, Artifact Registry, and Cloud Run functions.

• Secret Manager stores the Cloud Run functions secrets. The blueprint mountssecrets as a volume to provide a higher level of security than passing secrets asenvironment variables.

• Identity and Access Management (IAM) andResource Manager help to restrict access and isolate resources. The access controls andresource hierarchy follow the principle of least privilege.

• Cloud Logging collects all the logs from Google Cloud services for storage andretrieval by your analysis and investigation tools.

• Cloud Monitoring collects and stores performance information and metrics aboutGoogle Cloud services.

Example architecture with a serverless application using Cloud Storage

The following diagram shows how you can run a serverless application thataccesses an internal server when a particular event occurs inCloud Storage.

In addition to the services described inArchitecture,this example architecture uses a combination of the following Google Cloudservices and features:

• Cloud Storage emits an event when any cloud resource, application, or user creates a webobject on a bucket.
• Eventarc routes events from different resources. Eventarc encrypts eventsin transit and at rest.
• Pub/Sub queues events that are used as the input and a trigger forCloud Run functions.
• Virtual Private Cloud (VPC) firewall rules control the flow of data into the subnet that hosts your resources, such asan internal server.
• The internal server runs on Compute Engine or Google Kubernetes Engine andhosts your internal application. If you deploy theSecure Cloud Run functions with Internal Server Example,you deploy an Apache server with aHello World HTML page. This examplesimulates access to an internal application that runs VMs or containers.

Example architecture with Cloud SQL

The following diagram shows how you can run a serverless application thataccesses a Cloud SQL hosted service at a regular interval that isdefined in Cloud Scheduler. You can use this architecture when you mustgather logs, aggregate data, and so on.

In addition to the services described inArchitecture,this example architecture uses a combination of the following Google Cloudservices and features:

• Cloud Scheduler emits events on a regular basis.
• Pub/Sub queues events that are used as the input and a trigger forCloud Run functions.
• Virtual Private Cloud (VPC) firewall rules control the flow of data into the subnet that hosts your resources, such ascompany data stored in Cloud SQL.
• Cloud SQL Auth Proxy controls access to Cloud SQL.
• Cloud SQL hosts a service that is peered to the VPC network and that the serverlessapplication can access. If you deploy theSecure Cloud Run functions with Cloud SQL example,you deploy a MySQL database with a sample database.

Example architecture with BigQuery data warehouse

The following diagram shows how you can run a serverless application that istriggered when an event occurs in BigQuery (for example, data isadded or a table is created).

In addition to the services described inArchitecture,this example architecture uses a combination of the following Google Cloudservices and features:

• BigQuery hosts a data warehouse. If you deploy theSecure Cloud Run functions triggered by BigQuery example,you deploy a sample BigQuery dataset and table.
• Eventarc triggers Cloud Run functions when aparticular event occurs in BigQuery.

Organization structure

Resource Manager lets you logically group resources by project, folder, andorganization.

The following diagram shows a resource hierarchy with folders that representdifferent environments such as bootstrap, common, production, non-production (ortesting), and development. This resource hierarchy is based on the hierarchythat's described in theenterprise foundations blueprint.You deploy the projects that the blueprint specifies into the following folders:Common,Production,Non-production, andDev.

The following sections describe this diagram in more detail.

Folders

You use folders to isolate your production environment and governance servicesfrom your non-production and testing environments. The following table describesthe folders from the enterprise foundations blueprint that are used by thisblueprint.

You can change the names of these folders to align with your organization'sfolder structure, but we recommend that you maintain a similar structure. Formore information, seeOrganization structure.For other folder structures, seeDecide a resource hierarchy for your Google Cloud landing zone.

Projects

You isolate resources in your environment usingprojects.The following table describes the projects that are needed within theorganization. You can change the names of these projects, but we recommend thatyou maintain a similar project structure.

Mapping roles and groups to projects

You must give different user groups in your organization access to the projectsthat make up the serverless architecture. The following table describes theblueprint recommendations for user groups and role assignments in the projectsthat you create. You can customize the groups to match your organization'sexisting structure, but we recommend that you maintain a similar segregation ofduties and role assignment.

Security controls

This section discusses the security controls in Google Cloud that you useto help secure your serverless architecture. The key security principles toconsider are as follows:

• Secure access according to the principle of least privilege, givingprincipals only the privileges required to perform tasks.
• Secure network connections through trust boundary design, which includesnetwork segmentation, organization policies, and firewall policies.
• Secure configuration for each of the services.
• Identify any compliance or regulatory requirements for theinfrastructure that hosts serverless workloads and assign a risk level.
• Configure sufficient monitoring and logging to support audit trails forsecurity operations and incident management.

Build system controls

When you deploy your serverless application, you use Artifact Registry to storethe container images and binaries. Artifact Registry supports CMEK so that youcan encrypt the repository using your own encryption keys.

Network and firewall rules

Virtual Private Cloud (VPC) firewall rules control the flow of data into the perimeters. You create firewall rules thatdeny all egress, except for specific TCP port 443 connections fromrestricted.googleapis.com special domain names. Using therestricted.googleapis.com domain has the following benefits:

• It helps to reduce your network attack surface by usingPrivate Google Access when workloads communicate with Google APIs andservices.
• It ensures that you use only services that support VPC Service Controls.

In addition, you create a DNS record to resolve*.googleapis.com to restricted.googleapis.com.

For more information, seeConfiguring Private Google Access.

Perimeter controls

As shown in theArchitecture section, you place the resources for the serverless application in a separateVPC Service Controls security perimeter. This perimeter helps reduce thebroad impact from a compromise of systems or services. However, this securityperimeter doesn't apply to theCloud Run functions build process when Cloud Build automatically builds your code into a container image andpushes that image to Artifact Registry. In this scenario, create aningress rule for the Cloud Build service account in the service perimeter.

Access policy

To help ensure that only specific principals (users or services) can accessresources and data, you enable IAM groups and roles.

To help ensure that only specific resources can access your projects, you enableanaccess policy for your Google organization. For more information, seeAccess level attributes.

Service accounts and access controls

Service accounts are accounts for applications or compute workloads instead offor individual end users. To implement the principle of least privilege and theprinciple of separation of duties, you create service accounts with granularpermissions and limited access to resources. The service accounts are asfollows:

• A Cloud Run functions service account (cloudfunction_sa) thathas the following roles:

  • Compute Network Viewer (roles/compute.networkViewer)
  • Eventarc Event Receiver (roles/eventarc.eventReceiver)
  • Cloud Run Invoker (roles/run.invoker)
  • Secret Manager Secret Assessor (roles/secretmanager.secretAccessor)

  For more information, seeAllow Cloud Run functions to access a secret.

  Cloud Run functions uses this service account to grant permission tospecific Pub/Sub topics only and to restrict theEventarc event system from Cloud Run functions computeresources inExample architecture with a serverless application using Cloud Storage andExample architecture with BigQuery data warehouse.

• A Serverless VPC Access connector account(gcp_sa_vpcaccess) that has theCompute Network User (roles/compute.networkUser) role.

• A second Serverless VPC Access connector account (cloud_services) thathas the Compute Network User (roles/compute.networkUser) role.

  These service accounts for the Serverless VPC Accessconnector are required so that the connector can create the firewallingress and egress rules in the host project. For more information, seeGrant permissions to service accounts in your service projects.

• A service identity to run Cloud Run functions (cloudfunction_sa)that has the[Serverless VPC Access User (roles/vpcaccess.user)](/iam/docs/understanding-roles#vpcaccess.user)and theService Account User (roles/iam.serviceAccountUser) roles.

• Aservice account for the Google APIs (cloud_services_sa) that has the Compute Network User(roles/compute.networkUser) role to run internal Google processes on yourbehalf.

• Aservice identity for Cloud Run functions (cloud_serverless_sa) that has theArtifact Registry Reader (roles/artifactregistry.reader) role. Thisservice account provides access to Artifact Registry and CMEKs.

• A service identity for Eventarc (eventarc_sa) that has theCloud KMS CryptoKey Decrypter (roles/cloudkms.cryptoKeyDecrypter) and the Cloud KMS CryptoKey Encrypter(roles/cloudkms.cryptoKeyEncrypter) roles.

• A service identity for Artifact Registry (artifact_sa) with theCryptoKey Decrypter (roles/cloudkms.cryptoKeyDecrypter) and theCloud KMS CryptoKey Encrypter(roles/cloudkms.cryptoKeyEncrypter) roles.

Key management

To validate integrity and help protect your data at rest, you useCMEKs with Artifact Registry, Cloud Run functions, Cloud Storage, andEventarc. CMEKs provides you with greater control over yourencryption key. The following CMEKs are used:

• A software key for Artifact Registry that attests the code for yourserverless application.
• Anencryption key to encrypt the container images that Cloud Run functions deploys.
• Anencryption key for Eventarc events that encrypts the messaging channelat rest.
• Anencryption key to help protect data in Cloud Storage.

When you apply the Terraform configuration, you specify theCMEK location,which determines the geographical location where the keys are stored. You mustensure that your CMEKs are in the sameregion as your resources. By default, CMEKs are rotated every 30 days.

Secret management

Cloud Run functions supportsSecret Manager to store the secrets that your serverless application might require. Thesesecrets can include API keys and database usernames and passwords. To expose thesecret as a mounted volume, use theservice_configs object variables in themain module.

When you deploy this blueprint with the enterprise foundations blueprint, you mustadd your secrets to the secrets project before you apply the Terraform code. Theblueprint will grant the Secret Manager Secret Assessor(roles/secretmanager.secretAccessor) role to the Cloud Run functionsservice account. For more information, seeUsing secrets.

Organization policies

This blueprint adds constraints to theorganization policy constraints that the enterprise foundations blueprint uses. For more information about theconstraints that the enterprise foundations blueprint uses, seeOrganization policy constraints.

The following table describes the additional organization policy constraintsthat are defined in theSecure Cloud Run functions Security module of this blueprint.

Operational controls

You can enable logging andSecurity Command Center Premium tier features such as security health analytics and threat detection. These controls help youto do the following:

• Monitor data access.
• Ensure that proper auditing is in place.
• Support security operations and incident management capabilities of yourorganization.

Logging

To help you meet auditing requirements and get insight into your projects, youconfigure theGoogle Cloud Observability with data logs for the services that you want to track. DeployCloud Logging in the projects before you apply the Terraform code to ensure that the blueprintcan configure logging for the firewall, load balancer, and VPC network.

After you deploy the blueprint, we recommend that you configure the following:

• Create an aggregated log sink across all projects.
• Add CMEKs to your logging sink.

For all services within the projects, ensure that your logs include informationabout data writes and administrative access. For more information about loggingbest practices, seeDetective controls.

Monitoring and alerts

After you deploy the blueprint, you can set up alerts to notify your securityoperations center (SOC) that a security event has occurred. For example, you canuse alerts to let your security analysts know when a permission was changed onan IAM role. For more information about configuring Security Command Center alerts, seeSetting up finding notifications.

TheCloud Run functions Monitoring dashboard helps you to monitor the performance and health of your Cloud Run functions.It provides a variety of metrics and logs, which you can use to identify andtroubleshoot problems. The dashboard also includes a number of features that canhelp you to improve the performance of your functions, such as the ability toset alerts and quotas.

For more information, seeMonitoring Cloud Run functions.

To export alerts, see the following documents:

• Introduction to alerting
• Cloud Monitoring metric export

Debugging and troubleshooting

You can runConnectivity Tests to help you debug network configuration issues between Cloud Run functionsand the resources within your subnet. Connectivity Tests simulates theexpected path of a packet and provides details about the connectivity, includingresource-to-resource connectivity analysis.

Connectivity Tests isn't enabled by the Terraform code; you must setit up separately. For more information, seeCreate and run Connectivity Tests.

Terraform deployment modes

The following table describes the ways that you can deploy this blueprint, andwhich Terraform modules apply for each deployment mode.

Bringing it all together

To implement the architecture described in this document, do the following:

1. Review theREADME for the blueprint to ensure that you meet all the prerequisites.
2. In your testing environment, to see the blueprint in action, deploy oneof theexamples.These examples match the architecture examples described inArchitecture.As part of your testing process, consider doing the following:
   1. Use Security Command Center to scan the projects against commoncompliance requirements.
   2. Replace the sample application with a real application (forexample 1) and run through a typical deployment scenario.
   3. Work with the application engineering and operations teams inyour enterprise to test their access to the projects and to verifywhether they can interact with the solution in the way that they would expect.
3. Deploy the blueprint into your environment.

What's next

• Review theGoogle Cloud enterprise foundations blueprint for a baseline secure environment.
• To see the details of the blueprint, read theTerraform configuration README.
• To deploy a serverless application using Cloud Run, seeDeploy a secured serverless architecture using Cloud Run.