The architecture of thegated ingress pattern is based on exposing selectAPIs of workloads running in Google Cloud to the private computing environmentwithout exposing them to the public internet. This pattern is the counterpart tothegated egress pattern and is well suited foredge hybrid,tiered hybrid,andpartitioned multicloud scenarios.

Like with thegated egress pattern, you can facilitate this limited exposurethrough an API gateway or load balancer thatserves as a facade for existing workloads or services. Doing so makes it accessible to privatecomputing environments, on-premises environments, or on other cloud environment,as follows:

• Workloads that you deploy in the private computing environment or othercloud environments are able to communicate with the API gateway or loadbalancer by using internal IP addresses. Other systems deployed inGoogle Cloud can't be reached.
• Communication from Google Cloud to the private computingenvironment or to other cloud environments isn't allowed. Traffic is onlyinitiated from the private environment or other cloud environments to theAPIs in Google Cloud.

Architecture

The following diagram shows a reference architecture that meets therequirements of the gated ingress pattern.

The description of the architecture in the preceding diagram is as follows:

• On the Google Cloud side, you deploy workloads into anapplication VPC (or multiple VPCs).
• The Google Cloud environment network extends to other computingenvironments (on-premises or on another cloud) by using hybrid ormulticloud network connectivity to facilitate the communication betweenenvironments.
• Optionally, you can use a transit VPC to accomplish the following:
  • Provide additional perimeter security layers to allow access tospecific APIs outside of your application VPC.
  • Route traffic to the IP addresses of the APIs. You can createVPC firewall rules to prevent some sources from accessing certain APIsthrough an endpoint.
  • Inspect Layer 7 traffic at the transit VPC by integrating anetwork virtual appliance (NVA).
• Access APIs through an API gateway or a load balancer (proxy orapplication load balancer) to provide a proxy layer, and an abstractionlayer or facade for your service APIs. If you need to distribute trafficacross multiple API gateway instances, you could use aninternal passthrough Network Load Balancer.
• Provide limited and fine-grained access to a publishedservice through a Private Service Connect endpoint by using a load balancer through Private Service Connectto expose an application or service.
• All environments should use an overlap-free RFC 1918 IP address space.

The following diagram illustrates the design of this pattern usingApigee as the API platform.

In the preceding diagram, using Apigee as the API platform provides thefollowing features and capabilities to enable thegated ingress pattern:

• Gateway or proxy functionality
• Security capabilities
• Rate limiting
• Analytics

In the design:

• The northbound networking connectivity (for traffic coming from otherenvironments) passes through a Private Service Connectendpoint in your application VPC that'sassociated with the Apigee VPC.
• At the application VPC, an internal load balancer is used to expose theapplication APIs through a Private Service Connect endpointpresented in the Apigee VPC. For more information, seeArchitecture with VPC peering disabled.

• Configure firewall rules and traffic filtering at the application VPC.Doing so provides fine-grained and controlled access. It also helps stopsystems from directly reaching your applications without passing throughthe Private Service Connect endpoint and API gateway.

  Also, you can restrict the advertisement of the internal IPaddress subnet of the backend workload in the application VPC to theon-premises network to avoid direct reachability without passingthrough the Private Service Connect endpoint and the APIgateway.

Certain security requirements might require perimeter security inspectionoutside the application VPC, including hybrid connectivity traffic. In suchcases, you can incorporate a transit VPC to implement additional securitylayers. These layers, like next generation firewalls (NGFWs) NVAs withmultiple network interfaces,or Cloud Next Generation Firewall Enterprise withintrusion prevention service (IPS), perform deep packet inspection outside of yourapplication VPC, as illustrated in the following diagram:

As illustrated in the preceding diagram:

• The northbound networking connectivity (for traffic coming from otherenvironments) passes through a separate transit VPC toward thePrivate Service Connect endpointin the transit VPC that's associated with the Apigee VPC.
• At the application VPC, an internal load balancer (ILB in the diagram) isused to expose the application through aPrivate Service Connect endpoint in the ApigeeVPC.

You can provision several endpoints in the same VPC network, as shown in thefollowing diagram. To cover differentuse cases,you can control the different possible network paths using Cloud Router andVPC firewall rules. For example, If you're connecting your on-premises networkto Google Cloud using multiple hybrid networking connections, you couldsend some traffic from on-premises to specific Google APIs or published servicesover one connection and the rest over another connection. Also, you can usePrivate Service Connect global access to provide failover options.

Variations

Thegated ingress architecture pattern can be combined with other approachesto meet different design requirements, while still considering the communicationrequirements of the pattern. The pattern offers the following options:

• Access Google APIs from other environments

• Expose application backends to other environments using Private Service Connect

• Use a hub and spoke architecture to expose application backends to other environments

Access Google APIs from other environments

For scenarios requiring access to Google services, like Cloud Storage orBigQuery, without sending traffic over the public internet,Private Service Connect offers a solution. As shown in thefollowing diagram, itenables reachability to thesupported Google APIs and services (including Google Maps, Google Ads, andGoogle Cloud) from on-premises or other cloud environmentsthrough a hybrid network connection using the IP address of the Private Service Connect endpoint. For more information aboutaccessing Google APIs through Private Service Connect endpoints,seeAbout accessing Google APIs through endpoints.

In the preceding diagram, your on-premises network must be connected to thetransit (consumer) VPC network using either Cloud VPN tunnels or aCloud Interconnect VLAN attachment.

Google APIs can be accessed by usingendpoints orbackends.Endpoints let you target abundle of Google APIs.Backends let you target a specificregional Google API.

Note: Private Service Connect endpoints are registered withService Directory for Google APIs where you can store, manage, and publish services.

Expose application backends to other environments using Private Service Connect

In specific scenarios, as highlighted by thetiered hybrid pattern, you mightneed to deploy backends in Google Cloud while maintaining frontends inprivate computing environments. While less common, this approach is applicablewhen dealing with heavyweight, monolithic frontends that might rely on legacycomponents. Or, more commonly, when managing distributed applications acrossmultiple environments, including on-premises and other clouds, that requireconnectivity to backends hosted in Google Cloud over a hybrid network.

In such an architecture, you can use a local API gateway or load balancer in theprivate on-premises environment, or other cloud environments, to directly exposethe application frontend to the public internet. UsingPrivate Service Connect in Google Cloud facilitates privateconnectivity to the backends that are exposed through aPrivate Service Connect endpoint, ideally using predefined APIs,as illustrated in the following diagram:

The design in the preceding diagram uses anApigee Hybrid deployment consisting of a management plane in Google Cloud and a runtimeplane hosted in your other environment. You can install and manage the runtimeplane on a distributed API gateway on one of the supportedKubernetes platforms in your on-premises environment or in other cloud environments. Based on yourrequirements for distributed workloads across Google Cloud and otherenvironments, you can use Apigee on Google Cloud withApigee Hybrid. For more information, seeDistributed API gateways.

Use a hub and spoke architecture to expose application backends to other environments

Exposing APIs from application backends hosted in Google Cloud acrossdifferent VPC networks might be required in certain scenarios. As illustrated inthe following diagram, a hub VPC serves as a central point of interconnectionfor the various VPCs (spokes), enabling secure communication over private hybridconnectivity. Optionally, local API gateway capabilities in other environments,such asApigee Hybrid,can be used to terminate client requests locally where the application frontendis hosted.

As illustrated in the preceding diagram:

• To provide additional NGFW Layer 7 inspection abilities, the NVA withNGFW capabilities is optionally integrated with the design. You mightrequire these abilities to comply with specific security requirements andthe security policy standards of your organization.

• This design assumes that spoke VPCs don't require direct VPC to VPCcommunication.

  • If spoke-to-spoke communication is required, you can use theNVA to facilitate such communication.
  • If you have different backends in different VPCs, you can usePrivate Service Connect to expose these backends to the Apigee VPC.
  • If VPC peering is used for the northbound and southboundconnectivity between spoke VPCs and hub VPC, you need to consider thetransitivity limitation of VPC networking over VPC peering. To overcome this limitation, youcan use any of the following options:
    • To interconnect the VPCs, usean NVA.
    • Where applicable, consider the Private Service Connectmodel.
    • To establish connectivity between theApigee VPC and backends that are located in otherGoogle Cloud projects in the same organization withoutadditional networking components, useShared VPC.

• If NVAs are required for traffic inspection—including traffic from yourother environments—the hybrid connectivity to on-premises or other cloudenvironments should be terminated on the hybrid-transit VPC.

• If the design doesn't include the NVA, you can terminate the hybridconnectivity at the hub VPC.

• If certain load-balancing functionalities or security capabilities arerequired, like adding Google Cloud Armor DDoS protection or WAF, you canoptionally deploy an external Application Load Balancer at the perimeter through an externalVPC before routing external client requests to the backends.

Best practices

• For situations where client requests from the internet need to bereceived locally by a frontend hosted in a private on-premises or othercloud environment, consider using Apigee Hybrid as an APIgateway solution. This approach also facilitates a seamless migration ofthe solution to a completely Google Cloud-hosted environment whilemaintaining the consistency of the API platform (Apigee).
• Use Apigee Adapter for Envoy with anApigee Hybrid deployment with Kubernetes architecture where applicable to your requirements and the architecture.
• The design of VPCs and projects in Google Cloud should follow theresource hierarchy and secure communication model requirements, asdescribed in this guide.
• Incorporating a transit VPC into this design provides the flexibility toprovision additional perimeter security measures and hybrid connectivityoutside the workload VPC.
• Use Private Service Connect to access Google APIs andservices from on-premises environmentsor other cloud environments using the internal IP address ofthe endpoint over a hybrid connectivity network. For more information, seeAccess the endpoint from on-premises hosts.
• To help protect Google Cloud services in your projects and helpmitigate the risk of data exfiltration, use VPC Service Controls to specifyservice perimeters at the project or VPC network level.
  • When needed, you canextend service perimeters to a hybrid environment over a VPN or Cloud Interconnect. For moreinformation about the benefits of service perimeters, seeOverview of VPC Service Controls.
• UseVPC firewall rules orfirewall policies to control network-level access to Private Service Connect resources through the Private Service Connect endpoint.For example,outbound firewall rules at the application (consumer) VPC can restrict access from VM instances tothe IP address or subnet of your endpoints. For more information about VPCfirewall rules in general, seeVPC firewall rules.
• When designing a solution that includes NVAs, it's important to considerthe high availability (HA) of the NVAs to avoid a single point of failurethat could block all communication. Follow the HA and redundancy design andimplementation guidance provided by your NVA vendor.
• To strengthen perimeter security and secure your API gateway that'sdeployed in the respective environment, you can optionally implement loadbalancing and web application firewall mechanisms in your other computingenvironment (hybrid or other cloud). Implement these options at theperimeter network that's directly connected to the internet.
• If instances require internet access, useCloud NAT in the application VPC to allow workloads to access the internet. Doing solets you avoid assigning VM instances with external public IP addresses insystems that are deployed behind an API gateway or a load balancer.
• For outbound web traffic, useSecure Web Proxy.The proxy offersseveral benefits.
• Review thegeneral best practices for hybrid and multicloud networking patterns.