This document is part of a series that describes networking and securityarchitectures for enterprises that are migrating data center workloads to Google Cloud.

The series consists of the following documents:

• Designing networks for migrating enterprise workloads: Architectural approaches
• Networking for secure intra-cloud access: Reference architectures
• Networking for internet-facing application delivery: Reference architectures
• Networking for hybrid and multi-cloud workloads: Reference architectures (thisdocument)

This document discusses networking for a scenario where workloads run in morethan one place, such as on-premises and the cloud, or in multiple cloudenvironments.

Lift-and-shift architecture

The first hybrid workload access scenario is a lift-and-shiftarchitecture.

Establishing private connectivity

You can establish connectivity to on-premises networks using eitherDedicated Interconnect or Partner Interconnect.The topology illustrated in figure 1 shows how you can use four Dedicated Interconnectconnections in two different metros and different edge availability domains toachieve 99.99% availability.(You can also achieve99.99% availability using Partner Interconnect.)

For connectivity between your Google Cloud networks and your networks hosted by another cloud provider, useCross-Cloud Interconnect.

For more information and detailed recommendations, seeHybrid connectivity between an on-premises environment and Google Cloud in the enterprise foundations blueprint.

Figure 1. Configuration of redundant Dedicated Interconnectconnections for 99.99% availability.

NCC lets you use Google's network fordata transfer between multiple on-premises or cloud-hosted sites.This approach lets you take advantage of the reach and reliability of Google'snetwork when you need to move data. You can use your existing Cloud VPN, Cloud Interconnect, SD-WAN router appliances, and VPC networks asNCC spokes to support data transfer between the on-premisesnetworks, branch sites, other cloud providers, andGoogle Cloud VPC networks as shown in figure 2.

Figure 2. NCC configuration connecting differenton-premises enterprise and other cloud networks outside of Google Cloud using the Googlebackbone network.

For more information about setting up NCC, seeConsiderations in the NCC documentation.

SD-WAN appliances

NCC lets youuse a third-party router appliance as a NCC spoke to establish connectivity between an external siteand your VPC network resources. A router appliance could be athird-partySD-WAN router supported by one of our partners or another virtual appliance that lets you exchange routes with theCloud Router instance. These appliance-based solutions are in addition to thecurrent site-to-cloud connectivity options that are available withCloud VPN and Cloud Interconnect as spokes. Figure 3 shows atopology that uses SD-WAN appliances.

Figure 3. NCC configuration using router appliance forintegrating your SD-WAN implementation with Google's network.

You can use third-party appliances to perform security functions. The securitycapabilities of the appliance can be integrated in the router appliance as shownin figure 3. It's also a common pattern to use a networking virtual appliance,where traffic from on-premises lands in a transit VPC network,and the appliance establishes the connectivity with the workloadVPC network, as shown in figure 4.

For more information about setting up NCC, seeConsiderations in the NCC documentation.

Hybrid services architecture

As in the intra-cloud use case discussed inNetworking for secure intra-cloud access: Reference architectures,NCC enables connectivity from yourbranch sites and on-premises networks to Google Cloud.Private Service Connect provides private access to Google-managedservices or lets you consume other services that are built and deployed in thecloud.

You can also implement network security by using a combination ofVPC Service Controls, Google Cloud firewalls, and network virtualappliances, as shown in figure 4.

Figure 4. Networks with an architecture that uses both a lift-and-shiftpattern and a hybrid services design pattern that is designed to provide asecure data plane.

Zero Trust Distributed Architecture

In a hybrid environment, microservices run inservice meshes that are deployed across differentcloud providers and on-premises environments. You can help secure communication between themicroservices by using mutual Transport Layer Security (mTLS) and authorization policies. It's a common practicefor enterprises to build service meshes in the cloud and to extend the meshes toon-premises. Figure 5 shows an example in which services that are deployedon-premises access the services in the cloud. End-to-end mTLS between theservices is enabled using the east-west gateway and Server Name Indication (SNI)-based routing.Cloud Service Mesh helps you secure service-to-service communications, lettingyou configure authorization policies for the services and deploy certificatesand keys that are provided by a managedcertificate authority.

Hybrid environments typically feature multiple mesh deployments, such asmultiple GKE clusters. An important component in this flowis SNI routing, which is used at the GKEeast-west gateway for each cluster. Thisconfiguration allows direct-to-workload mTLS routing by the gateway while preserving end-to-end mTLS connectivity.

Figure 5. Zero-trust service mesh deployed across an on-premisesenvironment and Google Cloud.

Enterprises can use Cloud Service Mesh to deploy across clouds. To addresschallenges in managing identity and certificates across cloud providers,Cloud Service Mesh providesworkload identity and an intermediate in-cluster certificate authority (CA), usingCA Service (CA Service). The intermediate CA canbe chained to an external CA or can be hosted in Google. You can customize CAattributes like region and the signature algorithm, using both enterprise-ownedHSMs andCloud HSM.

Workload identity lets you assign distinct, fine-grained identities andauthorization for each microservice in your cluster. Cloud Service Meshmanages the process of issuing certificates and of automatically rotating keys andcertificates, without disrupting communications. It also provides a single rootof trust across GKE clusters.

Figure 6 shows an architecture that uses Cloud Service Mesh to manageidentity and authorization.

Services in the mesh can access Google Cloud services usingworkload identity federation.This feature lets services act with the authority of a Google service accountwhen they invoke Google Cloud APIs.Workload identity federation also lets the service mesh that's installed in other cloud providers access theGoogle cloud APIs.

Figure 6. Zero-trust service mesh deployed across clouds.

You can use Cloud Service Mesh to route traffic from the mesh to on-premises orto any other cloud.

For example, you can create services in Cloud Service Mesh calledon-prem-service andother-cloud-service andadd hybrid connectivity networkendpoint groups (NEGs)that have endpoints10.1.0.1:80 and10.2.0.1:80.Cloud Service Mesh then sends the traffic to its clients, which are mesh sidecarproxies that run alongside your applications. Thus, when your application sendsa request to theon-prem-service service, the Cloud Service Mesh clientinspects the request and directs it to the10.2.0.1:80 endpoint. Figure 7illustrates this configuration.

Figure 7. Traffic steered from a service mesh using Cloud Service Mesh.

You can also incorporate advanced functionality such as weight-based trafficsteering, as shown in figure 8. This capability lets you enable criticalenterprise needs such as cloud migration. Cloud Service Mesh serves as aversatile, globally managed control plane for your service meshes.

Figure 8. Weighted traffic steered using Cloud Service Mesh.

What's next

• Networking for secure intra-cloud access: Reference architectures.
• Networking for internet-facing application delivery: Reference architectures
• Migration to Google Cloud can help you to plan, design, and implement the process of migrating yourworkloads to Google Cloud.
• Landing zone design in Google Cloudhas guidance for creating a landing zone network.
• For more reference architectures, diagrams, and best practices, explore theCloud Architecture Center.