This document highlights the core design considerations that play apivotal role in shaping your overall hybrid and multicloud architecture.Holistically analyze and assess these considerations across your entire solutionarchitecture, encompassing all workloads, not just specific ones.

Refactor

In a refactor migration, you modify your workloads to take advantage of cloudcapabilities, not just to make them work in the new environment. You can improveeach workload for performance, features, cost, and user experience. Ashighlighted inRefactor: move and improve,some refactor scenarios let you modifyworkloads before migrating them to the cloud. This refactoring approach offersthe following benefits, especially if your goal is to build a hybridarchitecture as a long term targeted architecture:

• You can improve the deployment process.
• You can help speed up the release cadence and shortenfeedback cycles by investing in continuous integration/continuous deployment(CI/CD) infrastructure and tooling.
• You can use refactoring as a foundation to build and manage hybridarchitecture with application portability.

To work well, this approach typically requires certain investments inon-premises infrastructure and tooling. For example, setting up a localContainer Registry and provisioning Kubernetes clusters to containerizeapplications. Google Kubernetes Engine (GKE) Enterprise edition can be useful in this approach forhybrid environments.More information about GKE Enterprise is covered in the followingsection. You can also refer to theGKE Enterprise hybridenvironment reference architecture for more details.

Workload portability

With hybrid and multicloud architectures, you might want to be able to shiftworkloads between the computing environments that host your data. To help enablethe seamless movement of workloads between environments, consider the followingfactors:

• You can move an application from one computing environment to anotherwithout significantly modifying the application and its operational model:
  • Application deployment and management are consistent acrosscomputing environments.
  • Visibility, configuration, and security are consistent acrosscomputing environments.
• The ability to make a workload portable shouldn't conflict with theworkload being cloud-first.

Infrastructure automation

Infrastructure automation is essential for portability in hybrid and multicloudarchitectures. One common approach to automating infrastructure creation isthrough infrastructure as code (IaC). IaC involves managing your infrastructurein files instead of manually configuring resources—like a VM, a security group,or a load balancer—in a user interface.Terraform is a popular IaC tool to define infrastructure resources in a file. Terraformalso lets you automate the creation of those resources in heterogeneousenvironments.

For more information about Terraform core functions that can help youautomate provisioning and managing Google Cloud resources, seeTerraform blueprints and modules for Google Cloud.

You can use configuration management tools such asAnsible,Puppet,orChef to establish a common deployment and configuration process. Alternatively, youcan use an image-baking tool likePacker to create VM images for different platforms. By using a single, sharedconfiguration file, you can usePacker and Cloud Build to create a VM image for use on Compute Engine. Finally, you can use solutionssuch as Prometheus and Grafana to help ensure consistent monitoring acrossenvironments.

Based on these tools, you can assemble a common tool chain as illustrated in thefollowing logical diagram. This common tool chain abstracts away the differencesbetween computing environments. It also lets you unify provisioning, deployment,management, and monitoring.

Although a common tool chain can help you achieve portability, it's subject toseveral of the following shortcomings:

• Using VMs as a common foundation can make it difficult to implementtrue cloud-first applications. Also, using VMs only can prevent you fromusing cloud-managed services. You might miss opportunities to reduceadministrative overhead.
• Building and maintaining a common tool chain incurs overhead andoperational costs.
• As the tool chain expands, it can develop unique complexities tailoredto the specific needs of your company. This increased complexity cancontribute to rising training costs.

Before deciding to develop tooling and automation, explore the managed servicesyour cloud provider offers. When your provider offers managed services thatsupport the same use case, you can abstract away some of its complexity. Doingso lets you focus on the workload and the application architecture rather thanthe underlying infrastructure.

For example, you can use theKubernetes Resource Model to automate the creation of Kubernetes clusters using a declarativeconfiguration approach. You can useDeployment Manager convert to convert your Deployment Manager configurations and templatesto other declarative configuration formats that Google Cloud supports(like Terraform and the Kubernetes Resource Model) so they're portable when youpublish.

You can also considerautomating the creation of projects and the creation of resources within those projects.This automation can help you adopt an infrastructure-as-code approach forproject provisioning.

Containers and Kubernetes

Using cloud-managed capabilities helps to reduce the complexity of building andmaintaining a custom tool chain to achieve workload automation and portability.However, only using VMs as a common foundation makes it difficult to implementtruly cloud-first applications. One solution is to use containers and Kubernetesinstead.

Containers help your software to run reliably when you move it from oneenvironment to another. Because containers decouple applications from theunderlying host infrastructure, they facilitate the deployment across computingenvironments, such as hybrid and multicloud.

Kubernetes handles the orchestration, deployment, scaling, and management ofyour containerized applications. It's open source and governed by theCloud Native Computing Foundation.Using Kubernetes provides the services that form the foundation of a cloud-firstapplication. Because you can install and run Kubernetes on many computingenvironments, you can also use it to establish a common runtime layer acrosscomputing environments:

• Kubernetes provides the same services and APIs in a cloud or privatecomputing environment. Moreover, the level of abstraction is much higherthan when working with VMs, which generally translates into less requiredgroundwork and improved developer productivity.
• Unlike a custom tool chain, Kubernetes is widely adopted for bothdevelopment and application management, so you can tap into existingexpertise, documentation, and third-party support.
• Kubernetes supports all container implementations that:
  • Support the KubernetesContainer Runtime Interface (CRI)
  • Are industry-adopted for application
  • Aren't tied to any specific vendor

When a workload is running on Google Cloud, you can avoid the effort ofinstalling and operating Kubernetes by using a managed Kubernetes platform suchas Google Kubernetes Engine (GKE). Doing so can help operations staff shift theirfocus from building and maintaining infrastructure to building and maintaining applications.

You can also useAutopilot,a GKE mode of operation that manages your clusterconfiguration, including your nodes, scaling, security, and other preconfiguredsettings. When using GKE Autopilot, consider yourscaling requirements and itsscaling limits.

Technically, you can install and run Kubernetes on many computing environmentsto establish a common runtime layer. Practically, however, building andoperating such an architecture can create complexity. The architecture gets evenmore complex when you require container-level security control (servicemesh).

To simplify managing multi-cluster deployments, you can useGKE Enterprise to run modern applications anywhere at scale. GKE includespowerful managed open source components to secure workloads, enforce compliancepolicies, and provide deep network observability and troubleshooting.

As illustrated in the following diagram, using GKE Enterprisemeans you canoperate multi-cluster applications as fleets.

GKE Enterprise helps with the following design options to supporthybrid and multicloud architectures:

• Design and build cloud-like experiences on-premises or unifiedsolutions for transitioning applications to GKE Enterprise hybrid environment. For more information,see theGKE Enterprise hybrid environment reference architecture.

• Design and build a solution to solve multicloud complexity with aconsistent governance, operations, and security posture withGKE Multi-Cloud. For more information, see theGKE Multi-Cloud documentation.

GKE Enterprise also provides logical groupings of similarenvironments with consistent security, configuration, and service management.For example, GKE Enterprise powerszero trust distributed architecture.In a zero trust distributed architecture, services that are deployedon-premises or in another cloud environment can communicate across environmentsthrough end-to-end mTLS secure service-to-service communications.

Workload portability considerations

Kubernetes and GKE Enterprise provide a layer of abstraction forworkloads that can hide the many intricacies and differences between computingenvironments. The following list describes some of those abstractions:

• An application might be portable to a different environment withminimal changes, but that doesn't mean that the application performsequally well in both environments.
  • Differences in underlying compute, infrastructure securitycapabilities, or networking infrastructure, along with proximity todependent services, might lead to substantially different performance.
• Moving a workload between computing environments might also require youto move data.
  • Different environments can have different data storage andmanagement services and facilities.
• The behavior and performance of load balancers provisioned withKubernetes or GKE Enterprise might differ between environments.

Data movement

Because it can be complex to move, share, and access data at scale betweencomputing environments, enterprise-level companies might hesitate to build ahybrid or multicloud architecture. This hesitation might increase if they arealready storing most of their data on-premises or in one cloud.

However, the variousdata movement options offered by Google Cloud, provide enterprises with a comprehensive set ofsolutions to help move, integrate, and transform their data. These options helpenterprises to store, share, and access data across different environments in away that meets their specific use cases. That ability ultimately makes it easierfor business and technology decision-makers to adopt hybrid and multicloudarchitectures.

Data movement is an important consideration for hybrid and multicloud strategyand architecture planning. Your team needs to identify your different businessuse cases and the data that powers them. You should also think about storagetype, capacity, accessibility, and movement options.

If an enterprise has a data classification for regulated industries, thatclassification can help to identify storage locations and cross-region datamovement restrictions for certain data classes. For more information, seeSensitive Data Protection.Sensitive Data Protection is a fully managed service designed to help you discover, classify, and protect your data assets.

To explore the process, from planning a data transfer to using best practices inimplementing a plan, seeMigration to Google Cloud: Transferring your large datasets.

Security

As organizations adopt hybrid and multicloud architectures, their attacksurface can increase depending on the way their systems and data are distributedacross different environments. Combined with the constantly evolving threatlandscape, increased attack surfaces can lead to an increased risk ofunauthorized access, data loss, and other security incidents. Carefully considersecurity when planning and implementing hybrid or multicloud strategies.

For more information, seeAttack Surface Management for Google Cloud.

When architecting for a hybrid architecture, it's not always technicallyfeasible or viable to extend on-premises security approaches to the cloud.However, many of the networking security capabilities of hardwareappliances are cloud-first features and they operate in a distributedmanner. For more information about the cloud-first network security capabilitiesof Google Cloud, seeCloud network security.

Hybrid and multicloud architectures can introduce additional securitychallenges, such as consistency and observability. Every public cloud providerhas its own approach to security, including different models, best practices,infrastructure and application security capabilities, compliance obligations,and even the names of security services. These inconsistencies can increasesecurity risk. Also, theshared responsibility model of each cloud provider can differ. It's essential to identify and understand theexact demarcation of responsibilities in a multicloud architecture.

Observability is key to gaining insights and metrics from the differentenvironments. In a multicloud architecture, each cloud typically provides toolsto monitor forsecurity posture and misconfigurations. However, using these tools results in siloed visibility,which prevents building advanced threat intelligence across the entireenvironment. As a result, the security team must switch between tools anddashboards to keep the cloud secure. Without an overarching end-to-end securityvisibility for the hybrid and multicloud environments, it's difficult toprioritize and mitigate vulnerabilities.

To obtain the fullvisibility and posture of all your environments, prioritize your vulnerabilities,and mitigate the vulnerabilities you identify. We recommend a centralizedvisibility model. A centralized visibility model avoids the need for manualcorrelation between different tools and dashboards from different platforms. Formore information, seeHybrid and multicloud monitoring and logging patterns.

As part of your planning to mitigate security risks and deploy workloads onGoogle Cloud, and to help you plan and design your cloud solution formeeting your security and compliance objectives, explore the Google Cloudsecurity best practices center and theenterprise foundations blueprint.

Compliance objectives can vary, as they are influenced byboth industry-specific regulations and the varying regulatory requirements ofdifferent regions and countries. For more information, see the Google Cloudcompliance resource center.The following are some of the primary recommendedapproaches for architecting secure hybrid and multicloud architecture:

• Develop a unified tailored cloud security strategy and architecture. Hybrid andmulticloud security strategies should be tailored to the specific needs andobjectives of your organization.

  It's essential to understand the targetedarchitecture and environment before implementing security controls, becauseeach environment can use different features, configurations, and services.

• Consider a unified security architecture across hybrid and multicloudenvironments.

• Standardize cloud design and deployments, especially security design andcapabilities. Doing so can improve efficiency and enable unified governanceand tooling.

• Use multiple security controls.

  Typically, no single security control canadequately address all security protection requirements. Therefore,organizations should use a combination of security controls in a layereddefense approach, also known as defense-in-depth.

• Monitor and continuously improve security postures: Your organization shouldmonitor its different environments for security threats and vulnerabilities.It should also try to continuously improve its security posture.

• Consider using cloud security posture management (CSPM) to identify andremediate security misconfigurations and cybersecurity threats. CSPM alsoprovides vulnerability assessments across hybrid and multicloudenvironments.

Security Command Center is a built-in security and risk management solution for Google Cloudthat helps to identify misconfigurations and vulnerabilities and more.Security Health Analytics is a managed vulnerability assessment scanning tool. It's a feature ofSecurity Command Center that identifies security risks and vulnerabilities in yourGoogle Cloud environment and provides recommendations for remediatingthem.

Mandiant Attack Surface Management for Google Cloud lets your organization better see their multicloud or hybrid cloud environmentassets. It automatically discovers assets frommultiple cloud providers, DNS, and the extended external attack surface to giveyour enterprise a deeper understanding of its ecosystem. Use this informationto prioritize remediation on the vulnerabilities and exposuresthat present the most risk.

• Cloud security information and event management (SIEM) solution: Helps to collect and analyze security logs from hybrid and multicloud environments to detect and respond to threats.Google Security Operations SIEM from Google Cloud helps to provide security Information and event management by collecting, analyzing, detecting, and investigating all of your security data in one place.