This document provides a reference architecture for a highly availableenterprise application that is hosted on Compute Engine virtual machines(VMs) with low-latency connectivity to Oracle Cloud Infrastructure (OCI) Exadatadatabases that run in Google Cloud. The intended audience for thisdocument is cloud architects and Oracle database administrators. The documentassumes that you're familiar with Compute Engine and Oracle ExadataDatabase Service.

If you use Oracle Exadata or Oracle Real Application Clusters (Oracle RAC) torun Oracle databases on-premises, you can efficiently migrate your applicationsto Google Cloud and run your databases onOracle Database@Google Cloud.Oracle Database@Google Cloud is a Google Cloud Marketplace offering that lets you runOracle Exadata Database Service andOracle Autonomous Database directly inside Google Cloud.

If you don't need the Oracle RAC capability or if you need an Oracle Databaseversion other than 19c and 23ai, then you can run self-managed Oracle databaseson Compute Engine VMs. For more information, seeEnterprise application with Oracle Database on Compute Engine.

Architecture

The following diagram shows a high-level view of the architecture:

In the preceding diagram, an external load balancer receives requests fromusers of a public-facing application and it distributes the requests to frontendweb servers. The web servers forward the user requests through an internal loadbalancer to application servers. The application servers read data from andwrite to databases in Oracle Database@Google Cloud. Administrators and OCI servicescan connect and interact with the Oracle databases.

The following diagram shows a detailed view of the architecture:

In this architecture, the web tier and application tier run in active-activemode on Compute Engine VMs that are distributed across two zones withina Google Cloudregion.The application uses Oracle Exadata databases in the same Google Cloudregion.

All the components in the architecture are in a single Google Cloudregion. This architecture is aligned with theregional deployment archetype.You can adapt this architecture to build a topology that is robust againstregional outages by using the multi-regional deployment archetype. For moreinformation, seeMulti-regional deployment on Compute Engine and also the guidance in theReliability section later in this document.

The architecture that's shown in the preceding diagram includes the followingcomponents:

Products used

This reference architecture uses the following Google Cloud products:

• Compute Engine: A secure and customizable compute service that lets youcreate and run VMs on Google's infrastructure.
• Cloud Load Balancing: A portfolio of high performance, scalable, global andregional load balancers.
• Virtual Private Cloud (VPC): A virtual system that provides global, scalablenetworking functionality for your Google Cloud workloads. VPC includesVPC Network Peering, Private Service Connect, private services access, andShared VPC.
• Google Cloud Armor: A network security service that offers web applicationfirewall (WAF) rules and helps to protect against DDoS and application attacks.
• Cloud NAT: A service that provides Google Cloud-managedhigh-performance network address translation.
• Cloud Monitoring: A service that provides visibility into theperformance, availability, and health of your applications and infrastructure.
• Cloud Interconnect: A service that extends your external network to theGoogle network through a high-availability, low-latency connection.
• Partner Interconnect: A service that provides connectivity betweenyour on-premises network and your Virtual Private Cloud networks and other networksthrough a supported service provider.
• Cloud VPN: A service that securely extends your peer network to Google'snetwork through an IPsec VPN tunnel.

This reference architecture uses the following OCI products:

• Exadata Database Service on Dedicated Infrastructure: A service that letsyou run Oracle Database instances on Exadata hardware that's dedicated for you.
• Object Storage: A service for storing large amounts of structured andunstructured data as objects.
• VCN and subnets: A VCN is a virtual and private network for resources in an OCIregion. A subnet is a contiguous range of IP addresses with a VCN.
• Dynamic Routing Gateway: A virtual router for traffic between a VCN and externalnetworks.
• Service Gateway: A gateway to let resources in a VCN access specific Oracleservices privately.

Design considerations

This section describes design factors, best practices, and designrecommendations that you should consider when you use this referencearchitecture to develop a topology that meets your specific requirements forsecurity, reliability, operational efficiency, cost, and performance.

The guidance in this section isn't exhaustive. Depending on the specificrequirements of your application and the Google Cloud and third-partyproducts and features that you use, there might be additional design factors andtrade-offs that you should consider.

System design

This section provides guidance to help you to choose Google Cloud regionsfor your deployment and to select appropriate Google Cloud services.

Region selection

When you choose the Google Cloud regions where your applications must bedeployed, consider the following factors and requirements:

• Availability of Google Cloud services in each region. For moreinformation, seeProducts available by location.
• Availability of Compute Engine machine types in each region. Formore information, seeRegions and zones.
• End-userlatency requirements.
• Cost of Google Cloud resources.
• Cross-regional data transfer costs.
• Regulatory requirements.
• Sustainability requirements.

Some of these factors and requirements might involve trade-offs. Forexample, the most cost-efficient region might not have the lowestcarbon footprint. For more information, seeBest practices for Compute Engine regions selection.

Compute infrastructure

The reference architecture in this document uses Compute Engine VMs forcertain tiers of the application. Depending on the requirements of yourapplication, you can choose from other Google Cloud compute services:

• Containers: You can runcontainerized applications inGoogle Kubernetes Engine (GKE) clusters. GKE is a container-orchestration engine thatautomates deploying, scaling, and managing containerized applications.
• Serverless: If you prefer to focus your IT efforts on your data andapplications instead of setting up and operating infrastructure resources,then you can useserverless services likeCloud Run.

The decision of whether to use VMs, containers, or serverless services involvesa trade-off between configuration flexibility and management effort. VMs andcontainers provide more configuration flexibility, but you're responsible formanaging the resources. In a serverless architecture, you deploy workloads to apreconfigured platform that requires minimal management effort. For moreinformation about choosing appropriate compute services for your workloads inGoogle Cloud, seeHosting Applications on Google Cloud.

Storage options

For the Compute Engine VMs in the architecture, you can useHyperdisk orPersistent Disk boot volumes. Hyperdisk volumes provide better performance,flexibility, and efficiency than Persistent Disk. WithHyperdisk Balanced,you can provision IOPS and throughput separately and dynamically, which lets youtune the volume to a wide variety of workloads.

To store application binaries, useFilestore.Files that you store in aFilestore Regional instance are replicated synchronously across three zones within theregion.This replication helps to ensurehigh availability and robustness against zone outages. For robustness against region outages, youcan replicate a Filestore instance to a different region. Formore information, seeInstance replication.

When you design storage for your workloads, consider the functionalcharacteristics of the workloads, resilience requirements, performanceexpectations, and cost goals. For more information, seeDesign an optimal storage strategy for your cloud workload.

Oracle Database@Google Cloud network design

Choose a network design that meets your business and technical requirements. Forexample, you can use a single VPC network or multiple VPC networks. For moreinformation, seeLearn about selecting network topologies for Oracle Database@Google Cloud.

When you assign IP address ranges for the client and backup subnets to be usedfor the Exadata VM Clusters, consider the minimum subnet size requirements. Formore information, seePlan for IP Address Space in Oracle Database@Google Cloud.

Database migration

When you plan to migrate on-premises databases to Oracle Database@Google Cloud,assess your current database environment and get configuration and sizingrecommendations by using theDatabase Migration Assessment (DMA) tool.

For information about the procedure and tools that you can use to migrate Oracledatabases to Google Cloud, see theOracle Migration Methods Advisor.

Before you use the migrated databases in a production environment, verifyconnectivity from your applications to the databases.

Security, privacy, and compliance

This section describes factors to consider when you use this referencearchitecture to design a topology in Google Cloud that meets the security, privacy,and compliance requirements of your workloads.

Protection against external threats

To protect your application against threats like distributed-denial-of-service(DDoS) attacks and cross-site scripting (XSS), you can use Google Cloud Armorsecurity policies. Each policy is a set of rules that specifies certainconditions that should be evaluated and actions to take when the conditions aremet. For example, a rule could specify that if the source IPaddress of the incoming traffic matches a specific IP address or CIDR range,then the traffic must be denied. You can also apply preconfigured webapplication firewall (WAF) rules. For more information, seeSecurity policy overview.

External access for VMs

In the reference architecture that this document describes, theCompute Engine VMs don't need inbound access from the internet. Don'tassignexternal IP addresses to the VMs. Google Cloud resources that have only a private, internal IPaddress can still access certain Google APIs and services by usingPrivate Service Connect or Private Google Access. For moreinformation, seePrivate access options for services.

To enable secure outbound connections from Google Cloud resources thathave only private IP addresses, like the Compute Engine VMs in thisreference architecture, you can useSecure Web Proxy orCloud NAT.

For the subnets that are used by the Exadata VMs, Oracle recommends that youassign private IP address ranges.

Service account privileges

For the Compute Engine VMs in the architecture, instead of using thedefault service accounts, we recommend that you create dedicated serviceaccounts and specify the resources that the service account can access. Thedefault service account has a broad range of permissions, including some thatmight not be necessary. You can tailor dedicated service accounts tohave only the essential permissions. For more information, seeLimit service account privileges.

SSH security

To enhance the security of SSH connections to the Compute Engine VMs inyour architecture, implementIdentity-Aware Proxy (IAP) andCloud OS Login API.IAP lets you control network access based on user identity andIdentity and Access Management (IAM) policies. Cloud OS Login API lets you controlLinux SSH access based on user identity and IAM policies. Formore information about managing network access, seeBest practices for controlling SSH login access.

Data encryption

By default, the data that's stored in Hyperdisk,Persistent Disk, and Filestore is encrypted usingGoogle-owned and Google-managed encryption keys. As an additional layer of protection,you can choose to encrypt the Google-owned and managed key by usingkeys that you own and manage in Cloud Key Management Service (Cloud KMS). For moreinformation, seeAbout disk encryption for Hyperdisk and Persistent Disk volumes andEncrypt data with customer-managed encryption keys for Filestore.

By default, Exadata databases useTransparent Data Encryption (TDE),which lets you encrypt sensitive data that's stored in tables and tablespaces.

Network security

To control network traffic between the resources in the architecture, you mustconfigure appropriateCloud Next Generation Firewall (NGFW) policies.

Oracle Exadata security and compliance

Oracle Exadata Database Service includes Oracle Data Safe, which helps youmanage security and compliance requirements for Oracle databases. You can useOracle Data Safe to evaluate security controls, monitor user activity, and masksensitive data. For more information, seeManage Database Security with Oracle Data Safe.

More security considerations

When you build the architecture for your workload, consider the platform-levelsecurity best practices and recommendations that are provided in theEnterprise foundations blueprint andGoogle Cloud Well-Architected Framework: Security, privacy, and compliance.

Reliability

This section describes design factors to consider when you use this referencearchitecture to build and operate reliable infrastructure for your deployment inGoogle Cloud.

Robustness against VM failures

In the architecture that's shown in this document, if a Compute EngineVM in the web tier or application tier crashes, the relevantMIG recreates the VM automatically.The load balancers forward requests to only the available web serverinstances and application server instances.

VM autohealing

Sometimes the VMs that host your application might be running and available, butthere might be issues with the application itself. The application might freeze,crash, or not have sufficient memory. To verify whether an application isresponding as expected, you can configure application-based health checks aspart of the autohealing policy of your MIGs. If the application on a particularVM isn't responding, the MIG autoheals (repairs) the VM. For more informationabout configuring autohealing, seeAbout repairing VMs for high availability.

Robustness against region outages

If a region outage occurs, then the application is unavailable. To reduce thedowntime caused by region outages, you can implement the following approach:

• Maintain a passive (failover) replica of the web tier and applicationtier in another Google Cloud region.
• Create a standby Exadata Infrastructure instance with the requiredExadata VM Clusters in the same region that has the passive replica of theapplication stack. UseOracle Data Guard for data replication and automatic failover to the standby Exadatadatabases. If your application needs a lower recovery point objective(RPO), you can backup and recover the databases by usingOracle Autonomous Recovery Service.
• If an outage occurs in the primary region, use the database replica orbackup to restore the database to production and to activate theapplication in the failover region.
• UseDNS routing policies to route traffic to an external load balancer in the failover region.

For business-critical applications that must continue to be available even whena region outage occurs, consider using themulti-regional deployment archetype.You can use Oracle Active Data Guard to provide a read-only standby database inthe failover region.

Oracle manages the infrastructure in Oracle Database@Google Cloud. For informationabout the service level objectives (SLOs) for Oracle Exadata Database Service onDedicated Infrastructure, seeService Level Objectives for Oracle PaaS and IaaS Public Cloud Services.

MIG autoscaling

The architecture in this document uses regional MIGs for the web tier andapplication tier. The autoscaling capability of stateless MIGs ensures that theCompute Engine VMs that host the web tier and application tier aren'taffected by single-zone outages.

To control the autoscalingbehavior of your stateless MIGs, you can specify target utilization metrics,such as average CPU utilization. You can also configure schedule-basedautoscaling for stateless MIGs.Stateful MIGs can't be autoscaled. For more information, seeAutoscaling groups of instances.

MIG size limit

When you decide the size of your MIGs, consider the default and maximum limitson the number of VMs that can be created in a MIG. For more information, seeAdd and remove VMs from a MIG.

VM placement

In the architecture that this document describes, the application tier and webtier run on Compute Engine VMs that are distributed across multiplezones. This distribution helps to ensure that your web tier and your applicationtier are robust against single-zone outages.

To improve the robustness of the architecture, you can create aspread placement policy and apply it to the MIG template. When the MIG creates VMs, it places the VMswithin each zone on different physical servers (calledhosts), so your VMs arerobust against failures of individual hosts. For more information, seeCreate and apply spread placement policies to VMs.

VM capacity planning

To make sure that capacity for Compute Engine VMs is available when VMsneed to be provisioned, you can createreservations. A reservation providesassured capacity in a specific zone for a specified number of VMs of a machinetype that you choose. A reservation can be specific to a project, or sharedacross multiple projects. For more information about reservations, seeChoose a reservation type.

Stateful storage

A best practice in application design is to avoid the need for stateful localdisks. But if the requirement exists, you can configure your persistent disks tobe stateful to ensure that the data is preserved when the VMs are repaired orrecreated. However, we recommend that you keep the boot disks stateless, so thatyou can update them to the latest images with new versions and securitypatches. For more information, seeConfiguring stateful persistent disks in MIGs.

Oracle Exadata capacity

You can scale Exadata Infrastructure by adding database servers and storageservers as needed. After you add the required database servers or storageservers to Exadata Infrastructure, to be able to use the additional CPU orstorage resources, you must add the capacity to the associated Exadata VMcluster. For more information, seeScaling Exadata Compute and Storage.

Data durability

You can use Backup and DR Service to create, store, and manage backups ofCompute Engine VMs. Backup and DR stores backup data in itsoriginal, application-readable format. When required, you can restore workloadsto production by directly using data from long-term backup storage withouttime-consuming data-movement or preparation activities. For more information,seeBackup and DR for Compute Engine instance backups.

To ensure the durability of data in your Filestore instances, youcan createbackups and snapshots of the instance or useBackup and DR for Filestore and file systems.

By default, backups of databases in Oracle Exadata Database Service onDedicated Infrastructure are stored in OCI Object Storage. To achieve a lowerRPO, you can backup and recover the databases by usingOracle Autonomous Recovery Service.

More reliability considerations

When you build the cloud architecture for your workload, review thereliability-related best practices and recommendations that are provided in thefollowing documentation:

• Google Cloud infrastructure reliability guide
• Patterns for scalable and resilient apps
• Designing resilient systems
• Google Cloud Well-Architected Framework: Reliability

Cost optimization

This section provides guidance to optimize the cost of setting up and operatinga Google Cloud topology that you build by using this referencearchitecture.

VM machine types

To help you optimize the resource utilization of your VM instances,Compute Engine providesmachine type recommendations.Use the recommendations to choose machine types that match your workload'scompute requirements. For workloads with predictable resource requirements, youcan customize the machine type to your needs and save money by usingcustom machine types.

VM provisioning model

If your application is fault tolerant, thenSpot VMs can help to reduce your Compute Engine costs for the VMs in theapplication and web tiers. The cost of Spot VMs is significantly lowerthan regular VMs. However, Compute Engine might preemptively stop ordelete Spot VMs to reclaim capacity.

Spot VMs are suitable forbatch jobs that can tolerate preemption and don't have high availabilityrequirements. Spot VMs offer the same machine types, options, andperformance as regular VMs. However, when the resource capacity in a zone islimited, MIGs might not be able to scale out (that is, create VMs) automaticallyto the specified target size until the required capacity becomes availableagain.

VM resource utilization

Theautoscaling capability of stateless MIGs enables your application to handle increases intraffic gracefully, and it helps you to reduce cost when the need for resourcesis low.Stateful MIGs can't be autoscaled.

Oracle product licenses

You're responsible for procuring licenses for the Oracle products that youdeploy on Compute Engine, and you're responsible for complying with theterms and conditions of the Oracle licenses. For more information, seeLicensing Oracle Software in the Cloud Computing Environment.

Oracle Exadata database licenses

When you create an Exadata VM Cluster, you can either bring your own license(BYOL) or use a license that you purchased as part of yourGoogle Cloud Marketplace order for Oracle Database@Google Cloud.

Networking charges for data transfer between your applications and OracleExadata databases that are within the same region are included in the price ofthe Oracle Database@Google Cloud offering.

More cost considerations

When you build the architecture for your workload, also consider the generalbest practices and recommendations that are provided inGoogle Cloud Well-Architected Framework: Cost optimization.

Operational efficiency

This section describes the factors to consider when you use this referencearchitecture to design a Google Cloud topology that you can operateefficiently.

VM configuration updates

To update the configuration of the VMs in a MIG (such as the machine type orboot-disk image), you create a new instance template with the requiredconfiguration and then apply the new template to the MIG. The MIG updates theVMs by using the update method that you choose: automatic or selective. Choosean appropriate method based on your requirements for availability andoperational efficiency. For more information about these MIG update methods, seeApply new VM configurations in a MIG.

Oracle Linux images

For your VMs, you can useOracle Linux images that are available in Compute Engine or you canimport Oracle Linux images that you build and maintain.

You can also create and usecustom OS images that include the configurations and software that your applications require.Group your custom images into a custom image family. An image family alwayspoints to the most recent image in that family, so your instance templates andscripts can use that image without you having to update references to a specificimage version. Regularly update your custom images to include the securityupdates and patches that are provided by the OS vendor.

Deterministic instance templates

If the instance templates that you use for your MIGs include startup scripts toinstall third-party software, make sure that the scripts explicitly specifysoftware-installation parameters such as the software version. Otherwise, whenthe MIG creates the VMs, the software that's installed on the VMs might not beconsistent. For example, if your instance template includes a startup script toinstall Apache HTTP Server 2.0 (theapache2 package), then make sure that thescript specifies the exactapache2 version that should be installed, such asversion2.4.53. For more information, seeDeterministic instance templates.

Oracle Exadata database administration

Oracle manages the physical database servers, storage servers, and networkinghardware in Oracle Exadata Database Service on Dedicated Infrastructure. You canmanage the Exadata Infrastructure instances and the Exadata VM Clusters throughthe OCI or Google Cloud interfaces. You create and manage databasesthrough the OCI interfaces. The Google Cloud console pages forOracle Database@Google Cloud include links that you can use to go directly to therelevant pages in the OCI console. To avoid the need to sign in again to OCI,you can configureidentity federation between OCI and Google Cloud.

Observability for Oracle applications

To implement observability for Oracle workloads deployed in Google Cloud,you can useGoogle Cloud Observability services orOracle Enterprise Manager.Choose an appropriate monitoring strategy depending on your requirements andconstraints. For example, if you run other workloads in Google Cloud inaddition to Oracle workloads, then you can use Google Cloud Observability services tobuild a unified monitoring dashboard for all of the workloads.

Oracle documentation and support

Oracle products that run on Compute Engine VMs have similar operationalconcerns as Oracle products that run on-premises. However, you don't need tomanage the underlying compute, networking, and storage infrastructure. Forguidance about operating and managing Oracle products, see the relevant Oracledocumentation.

For information about Oracle's support policy for Oracle Database instancesthat you deploy in Google Cloud, seeOracle Database Support for Non-Oracle Public Cloud Environments (Doc ID 2688277.1).

More operational considerations

When you build the architecture for your workload, consider the general bestpractices and recommendations for operational efficiency that are described inGoogle Cloud Well-Architected Framework: Operational excellence.

Performance optimization

This section describes the factors to consider when you use this referencearchitecture to design a topology in Google Cloud that meets theperformance requirements of your workloads.

Compute performance

Compute Engine offers a wide range of predefined and customizablemachine types for the workloads that you run on VMs. Choose an appropriatemachine type based on your performance requirements. For more information, seeMachine families resource and comparison guide.

VM multithreading

Each virtual CPU (vCPU) that you allocate to a Compute Engine VM isimplemented as a single hardware multithread. By default, two vCPUs share aphysical CPU core. For applications that involve highly parallel operations or that performfloating point calculations (such as genetic sequence analysis, and financialrisk modeling), you can improve performance by reducing the number of threadsthat run on each physical CPU core. For more information, seeSet the number of threads per core.

VM multithreading might have licensing implications for some third-partysoftware, like databases. For more information, read the licensing documentationfor the third-party software.

Network performance

Compute Engine has a per-VM limit for egressnetwork bandwidth.This limit depends on the VM's machine type and whether traffic is routedthrough the same VPC network as the source VM. For VMs withcertain machine types, you can get a higher maximum egress bandwidth by enablingTier_1 networking. For more information, seeConfigure per VM Tier_1 networking performance.

Network traffic between the application VMs and the Oracle Exadatanetwork is routed through a low-latency Partner Interconnectconnection that Google sets up.

Exadata Infrastructure usesRDMA over Converged Ethernet (RoCE) for high bandwidth and low latency networking among its database servers andstorage servers. The servers exchange data directly in main memory withoutinvolving the processor, cache, or operating system.

More performance considerations

When you build the architecture for your workload, consider the general bestpractices and recommendations that are provided inGoogle Cloud Well-Architected Framework: Performance optimization.

What's next

• Accelerating cloud transformation with Google Cloud and Oracle
• Oracle documentation
  • Overview of Oracle Database@Google Cloud
  • Plan for IP Address Space in Oracle Database@Google Cloud
  • Learn about selecting network topologies for Oracle Database@Google Cloud
  • Deploy Oracle Database@Google Cloud
• Google documentation
  • Oracle Database@Google Cloud overview
  • Available configurations for Oracle Database@Google Cloud
• For more reference architectures, diagrams, and best practices, explore theCloud Architecture Center.

Contributors

Authors:

• Kumar Dhanagopal | Cross-Product Solution Developer
• Samantha He | Technical Writer

Other contributors:

• Andy Colvin | Database Black Belt Engineer, Oracle on Google Cloud
• Jeff Welsch | Director, Product Management
• Lee Gates | Group Product Manager
• Marc Fielding | Data Infrastructure Architect
• Mark Schlagenhauf | Technical Writer, Networking
• Michelle Burtoft | Senior Product Manager
• Rajesh Kasanagottu | Engineering Manager
• Roberto Mendez | Staff Network Implementation Engineer
• Samantha He | Technical Writer
• Sekou Page | Outbound Product Manager
• Souji Madhurapantula | Group Product Manager
• Victor Moreno | Product Manager, Cloud Networking