US20180062993A1 - Stateful connection optimization over stretched networks using specific prefix routes - Google Patents

Abstract

Techniques for stateful connection optimization over stretched networks are disclosed. Such stretched networks may extend across both a data center and a cloud. In one embodiment, configuration changes are made to cloud layer 2 (L2) concentrators used by extended networks and a cloud router such that the L2 concentrators block packets with the cloud router's source MAC address and block address resolution protocol (ARP) requests for a gateway IP address from/to cloud networks that are part of the extended networks. Further, the cloud router is configured with the same gateway IP address as that of a default gateway router in the data center and responds to ARP requests for the gateway IP address with its own MAC address. In addition, specific prefix routes (e.g., /32 routes) for virtual computing instances on route optimized networks in the cloud are injected into the cloud router and propagating to a data center router.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This application claims the benefit of U.S. Provisional Patent Application No. 62/380,991, filed Aug. 29, 2016, the entire contents of which are incorporated by reference herein.
• This application is related to the patent applications entitled “Stateful Connection Optimization Over Stretched Networks Using Packet Introspection” (Attorney Docket No. D221.02), “Stateful Connection Optimization Over Stretched Networks with Temporary Tromboning (D221.03), “CreatingLayer 2 Extension Networks in a Hybrid Cloud Computing System” (Attorney Docket No. D222.02), and “Low Downtime Software-Defined Wide Area Network Service Upgrade” (Attorney Docket No. D222.03), which are assigned to the assignee of this application and filed on the same day as this application and which are herein incorporated by reference in their entireties.
BACKGROUND
• Cloud architectures are used in cloud computing and cloud storage systems for offering infrastructure-as-a-service (IaaS) cloud services. Examples of cloud architectures include the VMware vCloud Director® cloud architecture software, Amazon EC2™ web service, and OpenStack™ open source cloud computing service. IaaS cloud service is a type of cloud service that provides access to physical and/or virtual resources in a cloud environment. These services provide a tenant application programming interface (API) that supports operations for manipulating IaaS constructs such as virtual machines (VMs) and logical networks. However, the use of such public cloud services is typically kept separate from the use of existing computing resources in data centers managed by an enterprise.
• With an increasing trend in migrating data centers to cloud platforms, a hybrid model has been developed that combines public cloud services and traditional on-premise data center computing resources. Such hybrid cloud computing systems permit workloads, such as virtual machines, to be rapidly moved from the data center to the cloud. Network extension, in which networks are created that stretch across both the data center and the cloud, enables seamless workload migration by allowing endpoint network configuration and connections to existing networks in the data center to be kept when workloads are moved to the cloud. However, traditional network extension can result in undesirable traffic patterns such as tromboning, in which traffic between sources and destinations that coexist within the cloud is routed from the cloud to the physically-separate data center and then back again to the cloud.
SUMMARY
• One embodiment disclosed herein provides a computer-implemented method for optimizing connections over an extended network created by bridging a first network in a first computing system, a second network in a second computing system, and a wide area network using a first bridging appliance in the first computing system and a second bridging appliance in the second computing system. The method generally includes configuring the first bridging appliance to: block packets with a source media access control (MAC) address belonging to a first router in the first computing system, and block address resolution protocol (ARP) requests for a gateway Internet Protocol (IP) address from and to the first network. The method further includes configuring the first router to have a same gateway IP address as a second router in the second computing system that is a default gateway of the extended network. In addition, the method includes injecting, into the first router, one or more specific prefix routes for one or more respective virtual computing instances on the first network, and propogating the one or more specific prefix routes to the second router.
• Further embodiments of the present invention include computer systems configured to carry out the above methods, and non-transitory computer-readable storage media comprising instructions that cause the computer system to carry out the above methods.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a block diagram of a hybrid cloud computing system in which one or more embodiments of the present disclosure may be utilized.
• FIG. 2A illustrates an example of inefficient traffic routing.
• FIG. 2B illustrates an example of asymmetric routing.
• FIG. 3 illustrates an approach for routing optimization in a hybrid cloud computing system using specific prefix routes, according to an embodiment.
• FIG. 4 illustrates a method of optimizing traffic flows in a hybrid cloud computing system using specific prefix routes, according to an embodiment.
• FIG. 5 illustrates an approach for routing optimization in a hybrid cloud computing system based on packet introspection, according to an embodiment.
• FIG. 6 illustrates a host system in which a hypervisor filter module may be implemented, according to an embodiment.
• FIG. 7 illustrates a method of optimizing traffic flows in a hybrid cloud computing system using packet introspection, according to an embodiment.
• FIG. 8 illustrates an approach for tromboning traffic of live-migrated virtual machines to preserve active sessions, according to an embodiment.
• FIG. 9 illustrates a method of temporarily tromboning traffic of live-migrated virtual machines, according to an embodiment.
DETAILED DESCRIPTION
• Embodiments presented herein provide techniques for stateful connection optimization over stretched networks (also referred to herein as “extended networks”) that eliminate undesirable traffic patterns such as tromboning. As described, extended networks may stretch across both a data center and a cloud. In one embodiment, configuration changes are made to cloud layer 2 (L2) concentrators used by extended networks and a cloud router such that the L2 concentrators block packets with the cloud router's source MAC address and block address resolution protocol (ARP) requests for a gateway Internet Protocol (IP) address from/to cloud networks that are part of the extended networks. Further, the cloud router is configured with the same gateway IP address as that of a default gateway router in the data center and will respond to the ARP requests for the gateway IP address with its own MAC address. As a result, traffic internal to the data center and cloud that would otherwise trombone from the cloud back to the default gateway router in the data center may instead be routed efficiently via the cloud router. In addition, to eliminate inefficient routing of external traffic, specific prefix routes, such as /32 routes, for virtual computing instances (e.g., virtual machines (VMs)) that are on route optimized networks in the cloud are injected into the cloud router and propagating to a router in the data center. As a result, the cloud router's routing table will include connected routes to those virtual computing instances' addresses, while the data center router will announce out that those addresses are reachable via the cloud router.
• Although described herein primarily with respect to virtual computing instances that are VMs, it should be understood that techniques disclosed herein are also applicable for providing stateful connection optimization over stretched networks to other types of virtualized computing instances, such as containers that implement operating system—level virtualization, wherein an abstraction layer is provided on top of the kernel of an operating system on a host computer (see, e.g., www.docker.com).
System Overview
• FIG. 1 is a block diagram of a hybridcloud computing system100 in which one or more embodiments of the present disclosure may be utilized. As shown, hybridcloud computing system100 includes avirtualized computing system102 and acloud computing system150, and is configured to provide a common platform for managing and executing virtual workloads seamlessly betweenvirtualized computing system102 andcloud computing system150. In one embodiment,virtualized computing system102 may be a data center controlled and administrated by a particular enterprise or business organization, whilecloud computing system150 is operated by a cloud computing service provider and exposed as a service available to account holders, such as the particular enterprise in addition to other enterprises. As such,virtualized computing system102 may sometimes be referred to as an on-premise data center(s), andcloud computing system150 may be referred to as a “public” cloud service. In some embodiments,virtualized computing system102 itself may be configured as a private cloud service providing cloud services to various organizations within the enterprise.
• As used herein, an internal cloud or “private” cloud is a cloud in which a tenant and a cloud service provider are part of the same organization, while an external or “public” cloud is a cloud that is provided by an organization that is separate from a tenant that accesses the external cloud. For example, the tenant may be part of an enterprise, and the external cloud may be part of a cloud service provider that is separate from the enterprise of the tenant and that provides cloud services to different enterprises and/or individuals. In embodiments disclosed herein, a hybrid cloud is a cloud architecture in which a tenant is provided with seamless access to both private cloud resources and public cloud resources.
• Virtualizedcomputing system102 includes one or morehost computer systems104.Hosts104 may each be constructed on a servergrade hardware platform106, such as an x86 architecture platform, a desktop, or a laptop. As shown,hardware platform106 of eachhost104 may include conventional components of a computing device, such as one or more processors (CPUs)108,system memory110, anetwork interface112,storage114, and other I/O devices such as, for example, a mouse and keyboard (not shown).Processor108 is configured to execute instructions, for example, executable instructions that perform one or more operations described herein and may be stored inmemory110 and in local storage.Memory110 is a device allowing information, such as executable instructions, cryptographic keys, virtual disks, configurations, and other data, to be stored and retrieved.Memory110 may include, for example, one or more random access memory (RAM) modules.Network interface112 enableshost104 to communicate with another device via a communication medium, such as anetwork122 withinvirtualized computing system102.Network interface112 may be one or more network adapters, also referred to as a Network Interface Card (NIC).Storage114 represents local storage devices (e.g., one or more hard disks, flash memory modules, solid state disks, and optical disks) and/or a storage interface that enableshost104 to communicate with one or more network data storage systems. Examples of a storage interface are a host bus adapter (HBA) thatcouples host104 to one or more storage arrays, such as a storage area network (SAN) or a network-attached storage (NAS), as well as other network data storage systems.
• Eachhost104 is configured to provide a virtualization layer that abstracts processor, memory, storage, and networking resources ofhardware platform106 into multiple virtual machines120₁to120_N(collectively referred to as VMs120) that run concurrently on the same hosts. VMs120 run on top of a software interface layer, referred to herein as ahypervisor116, that enables sharing of the hardware resources ofhost104 by VMs120. One example ofhypervisor116 that may be used in an embodiment described herein is a VMware ESXi hypervisor provided as part of the VMware vSphere® solution made commercially available from VMware, Inc.Hypervisor116 may run on top of the operating system ofhost104 or directly on hardware components ofhost104. In addition,hypervisor116 may provide a virtual switch (not shown), which is a software-based switch acting as a layer 2 (L2) forwarding engine and may be capable of performing VLAN tagging, stripping, filtering, L2 security, checksum, segmentation offload units, and other tasks typically performed by physical switches. The virtual switch may include uplink ports which connect to physical network adapters, as well as VM ports which connect to virtual network adapters and provide connections forhypervisor116 and VMs. In one embodiment, the virtual switch may be part of a distributed virtual switch that is an abstraction of a switch across multiple host servers and that permits virtual switches on the multiple host servers to be managed as if ports of those virtual switches belonged to a single switch, the distributed virtual switch.
• Virtualized computing system102 includes a virtualization management module (depicted inFIG. 1 as virtualization manager130) that may communicate to the plurality ofhosts104 via a network, sometimes referred to as amanagement network126. In one embodiment,virtualization manager130 is a computer program that resides and executes in a central server, which may reside invirtualized computing system102, or alternatively, running as a VM in one ofhosts104. One example of a virtualization management module is the vCenter Server® product made available from VMware, Inc.Virtualization manager130 is configured to carry out administrative tasks forcomputing system102, including managinghosts104, managing VMs120 running within eachhost104, provisioning VMs, migrating VMs from one host to another host, and load balancing betweenhosts104.
• In one embodiment,virtualization manager130 includes a hybrid cloud management module (depicted as hybrid cloud manager132) configured to manage and integrate virtualized computing resources provided bycloud computing system150 with virtualized computing resources ofcomputing system102 to form a unified “hybrid” computing platform.Hybrid cloud manager132 is configured to deploy VMs incloud computing system150, transfer VMs fromvirtualized computing system102 tocloud computing system150, and perform other “cross-cloud” administrative tasks, as described in greater detail later. In one implementation,hybrid cloud manager132 is a module or plug-in complement tovirtualization manager130, although other implementations may be used, such as a separate computer program executing in a central server or running in a VM in one ofhosts104.
• In one or more embodiments,cloud computing system150 is configured to dynamically provide an enterprise (or users of an enterprise) with one or morevirtual data centers170 in which a user may provision VMs120, deploy multi-tier applications on VMs120, and/or execute workloads.Cloud computing system150 includes aninfrastructure platform154 upon which acloud computing environment170 may be executed. In the particular embodiment ofFIG. 1,infrastructure platform154 includeshardware resources160 having computing resources (e.g., hosts162₁to162_N), storage resources (e.g., one or more storage array systems, such as SAN164), and networking resources, which are configured in a manner to provide avirtualization environment156 that supports the execution of a plurality ofvirtual machines172 across hosts162. It is recognized thathardware resources160 ofcloud computing system150 may in fact be distributed across multiple data centers in different locations.
• In one embodiment,cloud computing environment170 may be configured as a dedicated cloud service for a single tenant comprised of dedicated hardware resources160 (i.e., physically isolated from hardware resources used by other users of cloud computing system150). In other embodiments,cloud computing environment170 may be configured as a multi-tenant cloud service with logically isolated virtualized computing resources on a shared physical infrastructure. It is recognized thatcloud computing system150 may support multiplecloud computing environments170, available to multiple enterprises in single-tenant and multi-tenant configurations.
• In one embodiment,virtualization environment156 includes an orchestration component158 (e.g., implemented as a process running in a VM) that provides infrastructure resources tocloud computing environment170 responsive to provisioning requests. For example, if enterprise required a specified number of virtual machines to deploy a web applications or to modify (e.g., scale) a currently running web application to support peak demands,orchestration component158 can initiate and manage the instantiation of virtual machines (e.g., VMs172) on hosts162 to support such requests. In one embodiment,orchestration component158 instantiates virtual machines according to a requested template that defines one or more virtual machines having specified virtual computing resources (e.g., compute, networking, storage resources). Further,orchestration component158 monitors the infrastructure resource consumption levels and requirements ofcloud computing environment170 and provides additional infrastructure resources tocloud computing environment170 as needed or desired. In one example, similar tovirtualized computing system102,virtualization environment156 may be implemented by running on hosts162 VMware ESX™-based hypervisor technologies provided by VMware, Inc. of Palo Alto, Calif. (although it should be recognized that any other virtualization technologies, including Xen® and Microsoft Hyper-V virtualization technologies may be utilized consistent with the teachings herein).
• In one embodiment,cloud computing system150 may include a cloud director152 (e.g., run in one or more virtual machines) that manages allocation of virtual computing resources to an enterprise for deploying applications.Cloud director152 may be accessible to users via a REST (Representational State Transfer) API (Application Programming Interface) or any other client-server communication protocol.Cloud director152 may authenticate connection attempts from the enterprise using credentials issued by the cloud computing provider.Cloud director152 maintains and publishes acatalog166 of available virtual machine templates and virtual machine packages that represent virtual machines that may be provisioned incloud computing environment170. A virtual machine template is a virtual machine image that is loaded with a pre-installed guest operating system, applications, and data, and is typically used to repeatedly create a VM having the pre-defined configuration. A virtual machine package is a logical container of one or more pre-configured virtual machines that package applications and parameters that define operational details of the package. An example of a VM package is vApp™ technology made available by VMware, Inc., of Palo Alto, Calif., although other technologies may be utilized.Cloud director152 receives provisioning requests submitted (e.g., via REST API calls) and may propagates such requests toorchestration component158 to instantiate the requested virtual machines (e.g., VMs172).
• In the embodiment ofFIG. 1,cloud computing environment170 supports the creation of avirtual data center180 having a plurality ofvirtual machines172 instantiated to, for example, host deployed multi-tier applications. Avirtual data center180 is a logical construct that provides compute, network, and storage resources to an organization.Virtual data centers180 provide an environment whereVM172 can be created, stored, and operated, enabling complete abstraction between the consumption of infrastructure service and underlying resources.VMs172 may be configured similarly to VMs120, as abstractions of processor, memory, storage, and networking resources ofhardware resources160.
• Virtual data center180 includes one or morevirtual networks182 used to communicate betweenVMs172 and managed by at least one networking gateway component (e.g., gateway184), as well as one or more isolatedinternal networks186 not connected togateway184. Gateway184 (e.g., executing as a virtual machine) is configured to provideVMs172 and other components incloud computing environment170 with connectivity to an external network140 (e.g., Internet).Gateway184 manages external public IP addresses forvirtual data center180 and one or more private internalnetworks interconnecting VMs172.Gateway184 is a WAN facing device providing services such as intelligent routing, traffic steering, WAN optimization, encryption, etc.Gateway184 may be configured to provide virtual private network (VPN) connectivity over anetwork140 with another VPN endpoint, such as agateway124 withinvirtualized computing system102. In other embodiments,gateway184 may be configured to communicate withvirtualized computing system102 using a high-throughput, dedicated link betweenvirtualized computing system102 andcloud computing system150.Layer 2 concentrators (L2C)125 and185 are parallel togateways124 and184 and configured to provide a “stretched” L2 network that spansvirtualized computing system102 andcloud computing system150. Stretched network145 may be separate fromnetwork140 used bygateways124 and184 so that, e.g., VM migration traffic overnetwork140 does not create latency in stretched network145. Examples of how to create stretched networks are described in detail in the patent application entitled “CreatingLayer 2 Extension Networks in a Hybrid Cloud Computing System” (Attorney Docket No. D222.02).
• In one embodiment, eachvirtual data center180 includes a “hybridity” director module (depicted as hybridity director174) configured to communicate with the correspondinghybrid cloud manager132 invirtualized computing system102 to enable a common virtualized computing platform betweenvirtualized computing system102 andcloud computing system150. Hybridity directory (e.g., executing as a virtual machine) may communicate withhybrid cloud manager132 using Internet-based traffic via a VPN tunnel established betweengateways124 and184, or alternatively, usingdirect connect142.
Inefficient Traffic Routing in Stretched Networks
• FIG. 2A illustrates an example of inefficient traffic routing in stretched networks. As shown,networks201 and202 are stretched networks that spanvirtualized computing system102 andcloud computing system150, whilenetworks203 and204 are networks withinvirtualized computing system102 andcloud computing system150, respectively, andnetwork205 is an external network outsidevirtualized computing system102 andcloud computing system150.Extended networks201 and202 may be created in any feasible fashion, such as creating a new network incloud computing system150 corresponding to an existingvirtualized computing system102 network (with the new network having the same gateway address asrouter230 in virtualized computing and disconnected from local cloud router240), plugging a port of a L2 concentrator into the new network on thecloud computing system150 side and a port of another L2 concentrator into the existing network on thevirtualized computing system102 side, bridging the L2 concentrator pair. A more detailed description of how to create stretched networks is provided in the patent application entitled “CreatingLayer 2 Extension Networks in a Hybrid Cloud Computing System” (Attorney Docket No. D222.02). Illustratively, each of the L2 concentrators may be configured to bridge (e.g., bridges created by L2 concentrators211-214) each connected local area network (LAN) link(s) to a wide area network (WAN) link. The cloud-side L2 concentrator bridge may also be configured to drop packets with source MAC address ofrouter240. It should be understood that multiple LANs may be stretched or a single LAN stretched per L2 concentrator based on, e.g., a clustering scheme.
• Without any routing optimization, extended networks such asnetworks201 and202 need to be routed back tovirtualized computing system102 for layer 3 (L3) access, as the default gateway that forwards packets from VMs onextended networks201 and202 to other networks isrouter230, which is invirtualized computing system102. For example,router230 is the default gateway ofVM222 onnetwork201 and, as a result,traffic225 betweenVM222 andVM223 onnetwork204 needs to be routed back tovirtualized computing system102 and throughrouter230 before thetraffic225 is directed back tocloud computing system150 throughrouter240 and then toVM223. This type of tromboning is an inefficient traffic pattern, astraffic225 travels back and forth betweenvirtualized computing system102 andcloud computing system150 even though bothVMs222 and223 are on the cloud side. That is, it would be more efficient to route traffic fromVM222 toVM223 entirely withincloud computing system150, without having to go through the physically separatevirtualized computing system102.
• FIG. 2B shows a naive egress optimization approach that attempts to solve the tromboning problem discussed above but results in asymmetric routing. As shown,router240 is set as the default gateway for VMs incloud computing system150 that are on extended networks. For example,VM222 may be configured to userouter240 as its default gateway such thattraffic206 betweenVM222 andVM223 is routed locally viarouter240 without leavingcloud computing system150. However, the use of different gateways on the same extended network can produce asymmetric routing, in which traffic in one direction from a source to a destination does not follow the same path as traffic back in the opposite direction. Illustratively,router230 is the default gateway ofVM221, so traffic fromVM221 toVM223 followspath207. However, return traffic fromVM223 toVM221 follows adifferent path208. Such asymmetric routing breaks stateful flow inspection middleboxes, such as firewalls (e.g., firewall250), which will only see one side of a flow (e.g., the TCP SYN) but not the other side (e.g., the TCP SYN_ACK) that returns via a different flow along the other path. Most stateful firewalls require seeing both sides of a flow to distinguish between good and malicious traffic and will not allow asymmetric flows, such as flows alongpaths207 and208, to pass. Asrouters230 and240 are on the edge facing the WAN (e.g., the Internet), firewalls need to be employed to prevent unauthorized network access. As a result, the naive optimization approach of settingrouter240 as the default gateway for VMs incloud computing system150 that are on extended networks will not work with network topologies such as the topology shown inFIG. 2B.
• AsFIGS. 2A and 2B show, without routing optimization, routing of traffic between VMs that are migrated tocloud computing system102 and attached to extended networks (e.g.,networks201 and202) and VMs on other networks in the cloud may be inefficient, as such traffic is first routed back to a default gateway invirtualized computing system102. However, applying naive routing optimization may result in, e.g., asymmetric routing of traffic between cloud-side VMs on an extended network and VMs on stub networks on thevirtualized computing system102 side.
Routing Optimization Using Specific Prefix Routes
• FIG. 3 illustrates an approach for routing optimization in a hybrid cloud computing system using specific prefix routes, according to an embodiment. This approach is one way of resolving the tromboning and asymmetric routing problems discussed above. It is assumed for simplicity that extended networks without routing optimization have already been created, but a similar approach may be employed for routing optimization during the creation of an extended network, as described in greater detail below.
• In one embodiment,hybridity director174router240 optimizes routing to eliminate inefficient tromboning by making configuration changes toL2 concentrators212 and214 androuter240; injecting specific prefix routes, such as /32 routes, intorouter240; and propagating those specific prefix routes back torouter230. In particular,hybridity director174 configuresL2 concentrators212 or214 used byextended networks201 and202 to block packets with the cloud computing system router's (e.g., router240) source MAC address and to block ARP requests for a gateway IP address from/to the cloud networks that are parts of theextended networks201 and202. By blocking the ARP requests for the gateway IP address from/to the cloud networks in particular, such ARP requests from VMs222-223 will not reach thedefault gateway router230 and will not receiverouter230's MAC address in response.Hybridity director174 further configuresrouter240 to have the same gateway IP address as that of thedefault gateway router230. As a result,router240 will respond to VMs222-223's ARP requests for a gateway IP address with its own MAC address, and cloud VMs222-223 onextended networks201 and202 may then believe they are communicating with theirdefault gateway router230 when they are actually communicating withrouter240. Further,router240 may maintain a list of local VMs obtained by, e.g., havingrouter240 update its VM list whenever it receives ARP requests on extended networks. It should be understood that these configurations may eliminates inefficient traffic flows internal tovirtualized computing system102 andcloud computing system150 such as thetromboning path225 shown inFIG. 2. For example, packets sent fromVM222 toVM223 will not take thetromboning path225 viarouter230 invirtualized computing system102 but will instead be forwarded, based on the MAC address ofrouter240 obtained through the ARP requests described above, torouter240 and take theefficient path310.
• In addition to making configuration changes toL2 concentrators212 and214 and torouter240 to eliminate internal traffic flows that are inefficient,hybridity director174 also eliminates external (tovirtualized computing system102 and cloud computing system150) traffic flows that are inefficient by injecting specific prefix routes, such as /32 routes, for VMs on route-optimized networks intorouter240 and propagating those routes, as well as routes for networks that are wholly in cloud computing system (e.g., network204), back torouter230. As shown,hybridity director174 renumbersrouter240 with a /32 IP address, and injects /32 routes intorouter240 to indicate ownership of theVM222/32 andVM224/32 addresses ofVMs222 and224, respectively, byrouter240.Router240's routing table350 may then include connected routes for theVM222/32 andVM224/32 addresses, as well as fornetwork204 that is wholly incloud computing system150.Router240 further propagates theVM222/32 andVM224/32 routes and thenetwork204 route torouter230 invirtualized computing system102. As a result,router230's routing table340 may be updated to indicate thatVM222/32 andVM224/32 are reachable viarouter240 rather than connect routes, as well as thatnetwork204 is reachable viarouter240. Then, whenrouter230 communicates paths to peer routers via BGP,router230 will announce out that theVM222/32 andVM224/32 addresses andnetwork204 are reachable viarouter240, and remote clients such as VM330 will then communicate with VMs222-224 incloud computing system150 viarouter240 rather than via a tromboning path throughrouter230. For example,router230 may initially announce a block of /24 addresses, such as 192.168.1.0/24 which includes 256 IP addresses, that are owned by network201 (and similarly for network202), and then a VM on network201 (or on network202) may move fromvirtualized computing system102 tocloud computing system150. In order to make that VM owned byrouter240 incloud computing system150, a more specific route (the /32 route) may be injected that takes precedence over the previous /24 route. That is, the /24 route published byrouter230 is less specific than the injected /32 route, so there is a more specific route torouter240 over L3 compared to the initial connected route viarouter230. In particular,router240 will fully own the path for other networks to access egress optimization enabled VMs (e.g.,VMs222 and224), so there will not be multiple paths to create asymmetric routing.
• FIG. 4 illustrates amethod400 of optimizing traffic flows in a hybrid cloud computing system using specific prefix routes, according to an embodiment. Similar to the discussion above,method400 assumes that an extended network without routing optimization has already been created, but steps similar to those ofmethod400 may also be employed for routing optimization during the creation of the extended network.
• As shown,method400 begins atstep410, wherehybridity director174 configures a router (e.g., router240) incloud computing system150 to have the same gateway IP address as that of a default gateway router (e.g., router230) of the extended network (e.g.,networks201 or202), where the default gateway router is invirtualized computing system102. As described, VMs incloud computing system150 that are on the extended networks may then believe they are communicating with the default gateway invirtualized computing system102 when actually communicating with the router incloud computing system150.
• Atstep420,hybridity director174 renumbers the cloud computing system router (e.g., router240) to have a /32 IP address, while preventing the /32 IP address from being announced to peer routers.
• Atstep430,hybridity director174 adds /32 routes to the cloud computing system router (e.g., router240) for VMs on the network incloud computing system102. Then, atstep440,hybridity director174 propagates the /32 routes back to a router (e.g., router230) invirtualized computing system102. As described, /32 routes are propagated back torouter230 to indicate ownership of theVM222/32 andVM224/32 addresses byrouter240, as there is a more specific /32 route torouter240 over L3 compared to, e.g., a previous /24 route torouter230. As a result ofsteps430 and440, the cloud router (e.g., router240) will fully own the path for other networks to access VMs on the network (e.g.,network201 or202) incloud computing system102. For example, as a result of /32 routes being propagated back torouter230, whenrouter230 communicates paths to peer routers via BGP,router230 will announce out that the /32 IP addresses of VMs incloud computing system150 are reachable viarouter240. Accordingly, remote clients such as VM330 will communicate with VMs incloud computing system150 viarouter240 rather than via a tromboning path throughrouter230.
• It should be understood that if other VMs (e.g., VM221) on the extended network migrate fromvirtualized computing system102 to cloud computing system,hybridity director174 may also inject /32 routes for the migrated VMs into the cloud router and those routes may also be propagated back to the virtualized computing system router. For example, an inventory service inhybridity director174 may be responsible for detecting new VMs being attached or existing VM being re-attached to route-optimized networks incloud computing system150 through ports on virtual switch(es).Hybridity director174 may also be responsible for migrating VMs fromvirtualized computing system102 tocloud computing system150, as described in detail in the patent application entitled “Live Migration of Virtual Computing Instance Between Data Centers” (Attorney Docket No. D217), and hybridity director would then be aware that such migrated VMs are local tocloud computing system150. In addition, information on other VMs, networks, etc. that are on route-optimized networks tocloud computing system150 may be obtained from, e.g., an inventory maintained by a virtualization management application. Conversely,hybridity director174 may also detect VMs incloud computing system150 being detached from route optimized networks and remove inject /32 routes that are associated with those VMs.
• Atstep450,hybridity director174 configures a cloud L2 concentrator (e.g., L2 concentrator212 or214) used by the existing extended network to block packets with the cloud computing system router's (e.g., router240) source MAC address and to block ARP requests for the gateway IP address from/to the cloud network that is part of the existing extended network.
Routing Optimization Using Packet Introspection
• FIG. 5 illustrates an approach for routing optimization in a hybrid cloud computing system based on packet introspection, according to an embodiment. As shown, packet introspection is used to determine whether traffic from VMs (e.g.,VMs222,223, and224) incloud computing system150 has a local destination that is also incloud computing system150 or a remote destination. To optimize traffic flows, hypervisor filtering modules are configured to modify the destination MAC addresses of packets having local destinations so as to divert such packets to a local router.
• In one embodiment, the hypervisor filtering modules may be software components that interpose on input output (I/O) paths between virtual network interface controllers (VNICs) of VMs and ports of virtual switch(es). Commercially available products, such as NSX traffic filter or DVFilter from VMware, Inc. of Palo Alto, Calif., permit traffic filtering rules to be injected at the hypervisor level to modify packets.FIG. 6 illustrates an example of ahost computer system600 in which a hypervisor filtering module may be implemented, according to an embodiment.Host computer600 may be, e.g., one of the virtual machine hosts incloud computing system150. As shown,host computer600 includes ahypervisor630 that serves as an interface between VMs610_1-3andphysical resources650 available onhost computer600. Each virtual machine610_1-3includes a respective VNIC615_1-3which is responsible for exchanging packets between that virtual machine610_1-3andhypervisor630. VNICs615_1-3may be software abstractions of physical network interface cards. Each of VM610_1-3is connected through its VNIC615_1-3to a respective virtual port provided by avirtual switch630. As described, virtual switches are software-based switches acting as L2 forwarding engines and may be capable of performing VLAN tagging, stripping, filtering, L2 security, checksum, segmentation offload units, and other tasks typically performed by physical switches. In addition to being connected to VNICs615_1-3,virtual switch630 is connected to physical network interface cards (PNICs)640_1-2to allow network traffic to be exchanged betweenvirtual machines130 executing onhost machine100 and destinations on an externalphysical network660. As shown, filter modules620_1-3are associated with respective VNICs615_1-3, and may perform, e.g., filtering and encapsulation of packets originating in VNICs615_1-3, before the packets reachvirtual switch630. In one embodiment, filtering modules620_1-3are specifically configured to introspect packets sent via VNICs615_1-3and modify packets having local destinations to have the destination MAC address of a local router. In turn,virtual switch630 may forward such modified packets to the local router rather than to a default gateway that is not local, as described in greater detail below. Although described herein primarily with respect to hypervisor filters such as filtering modules620_1-3that are between VNICs and virtual switches, it should be understood that hypervisor filters may also be implemented as (or considered as) components of the virtual switches themselves within the hypervisors.
• Returning to theFIG. 5, an unoptimized traffic flow fromVM222 toVM223 incloud computing system150 may follow the tromboning path shown inFIG. 2A to adefault gateway router230 invirtualized computing system102. To eliminate such tromboning, packet introspection may be employed to identify the local destinations of such packets and divert those packets along thepath410 shown inFIG. 5 tolocal router240, as thenetwork204 to which thedestination VM223 is attached is entirely local tocloud computing system150. In one embodiment, a virtual switch may be configured so thatVM222,223, and224 learn the MAC of R1 and already treat that as default router. Only when the virtual switch sees a packet which has to be redirected to R2, is the MAC changed on the fly in the filter.
• In one embodiment,hybridity director174 incloud computing system150 maintains an inventory of what sits in the cloud, such as extended networks (e.g.,networks201 and202), local networks (e.g., network204), and VMs (e.g.,VMs222,223,224) in the cloud. As described,hybridity director174 may be responsible for migrating VMs fromvirtualized computing system102 tocloud computing system150, and hybridity director would then be aware that such migrated VMs are local tocloud computing system150.Hybridity director174 may obtain information on other VMs, networks, etc. that are in the cloud from elsewhere, such as from an inventory maintained by a virtualization management application which may be, e.g., a vCenter® Server available from VMware, Inc. To optimize traffic flows,hybridity director174 configures hypervisor filtering modules incloud computing system150 with information specifying destinations that are local to cloud computing system. In addition, the hypervisor filtering modules are configured byhybridity director174 with traffic filtering rules that divert traffic having local destinations tolocal router240.
• In one embodiment, hypervisor filtering modules incloud computing system150 may be configured with filtering rules to introspect IP packets leaving the VNICs ofVMs222,223, and224. Packets sent byVMs222,223, and224 to destinations that are not on the local networks of may each have the destination MAC address(es) of the VMs'222,223, and224 default gateway(s). The hypervisor filtering modules may, according to the configured filtering rules, rewrite the destination MAC addresses of packets destined to local destinations such as VMs on network204 (e.g., VM223), which is a local network wholly owned byrouter240, to the MAC address ofrouter240. Doing so eliminates the tromboning of traffic fromVM222 toVM223 discussed above with respect toFIG. 2A, astraffic having VM223 as its destination will be diverted torouter240 and followpath410 rather than tromboning to thedefault gateway router230. In other cases when the destination of a packet is not local (e.g., the destination IP address is that ofVM220 onnetwork203 in virtualized computing system102) or when the destination is on the same local subnet, the hypervisor filtering module that inspects the packet will not recognize the destination as a local address that needs to be diverted and will allow the packet to be routed according to its default path.
• It should be understood that the packet introspection technique discussed with respect toFIG. 5 is purely delegated and may be offered as a service. In particular, the packet introspection technique does not require coordination betweenvirtualized computing system102 andcloud computing system150 to work.
• FIG. 7 illustrates amethod700 of optimizing traffic flows in a hybrid cloud computing system using packet introspection, according to an embodiment. As shown,method700 begins atstep710, wherehybridity director174 configures a hypervisor filtering module (e.g., one of filtering modules620_1-3) with a traffic filtering rule that diverts traffic having local destinations tolocal cloud router240. Althoughmethod700 is described with respect to one hypervisor filtering module for simplicity, it should be understood thathybridity director174 may generally configure multiple hypervisor filtering modules incloud computing system150. In one embodiment, the hypervisor filtering modules may be configured with filtering rules to introspect IP packets and to rewrite the destination MAC addresses of VMs on local networks to the MAC address ofrouter240. Doing so ensures that packets with local destinations are forwarded to thelocal router240 incloud computer system150, rather than to, e.g., a default gateway router that located invirtualized computing system102.
• Atstep720,hybridity manager174 detects a new VM that is local tocloud computing system150. As described, an inventory service inhybridity director174 may be responsible for detecting new VMs being attached or existing VM being re-attached to networks incloud computing system150 through ports on virtual switch(es). Such VMs may be detected as new local VMs atstep720. As described,hybridity director174 may also participate in migrating VMs tocloud computing system150 and thus be aware that such migrated VMs are local tocloud computing system150. In addition, information on other VMs, networks, etc. that are local to the cloud may be obtained from, e.g., an inventory maintained by a virtualization management application.
• Atstep730,hybridity director174 configures the hypervisor filtering module with information indicating that the detected VM is local tocloud computing system150. Then, atstep740, the hypervisor filtering module introspects a packet sent via an associated VNIC of a VM. The introspection of one packet atstep740 is described for simplicity, but it should be understood that the hypervisor filtering module may generally introspect all packets leaving the VNIC to which it is associated. In one embodiment, such introspection may include determining the destination IP addresses of packets leaving the associated VNIC.
• Atstep740, the hypervisor filtering module determines whether the destination MAC address of the packet should be rewritten because the destination IP address of the VM to which the packet is being sent is local tocloud computing system150 and the VM sending the packet is on an optimized network. As described, the hypervisor filtering module is configured atstep710 with a traffic filtering rule that diverts packets having local destinations tolocal cloud router240, by modifying the destination MAC addresses of those packets.
• If the hypervisor filtering module determines that the destination MAC address should be rewritten atstep740, then atstep750, the hypervisor filtering module rewrites the destination MAC address of the packet to the MAC address oflocal cloud router240. Such a modified packet may then be forwarded by a virtual switch torouter740 based on the re-written destination MAC address.
• If, however, it is determined atstep740 that the destination MAC address should not be rewritten, then atstep760, the hypervisor filtering module allows the packet to be routed according to its default path.
• In addition to the packet introspection approach discussed above, specific prefix routes, such as /32 routes, for VMs on route optimized networks incloud computing system150 may be injected intocloud router240 and propagating torouter230 in the virtualized computing system. Doing so may eliminate inefficient routing of external traffic, as described above with respect to routing optimization using specific prefix routes.
Temporarily Tromboning Traffic of Live-Migrated Virtual Machines
• FIG. 8 illustrates an approach for tromboning traffic of live-migrated virtual machines to preserve active sessions, according to an embodiment. Live migration of VMs fromvirtualized computing system102 tocloud computing system150 differs from disk-replication based VM migration in that the VM is powered off during DR-based migration and all connections (active sessions) are terminated, whereas active sessions are carried over tocloud computing system150 when a VM is live migrated fromvirtualized computing system102. It should be understood that, to keep the active sessions after a live VM migration, tromboning of traffic may be desirable. In particular,firewall250 invirtualized computing system102 may continue to tracking symmetric flows of the active sessions, so immediately optimizing routing as described above to divert traffic from the migrated VM torouter240 may break the symmetry expected byfirewall250.
• As shown, to maintain the active sessions of a live-migrated VM (e.g., VM222), the live-migrated VM is first attached to an unoptimized network withrouter230 as the default gateway so that traffic tromboning occurs. In one embodiment, two stub networks, one of which is route optimized and another of which is not optimized, are created incloud computing system150 for each of extendednetworks201 and202. Illustratively, hybridity director174 (or another cloud management application) creates an optimizednetwork201aand an unoptimized network201bforextended network201 and an optimized network202aand anunoptimized network202bforextended network202. The routing of traffic on optimizednetworks201aand202amay be optimized according to the specific prefix technique discussed above with respect toFIG. 3-4 or the packet introspection techniques discussed above with respect toFIGS. 5-6, while VMs onunoptimized networks201band202bcontinue usingrouter230 as their default gateway, which may result in tromboning traffic. Live migrated VMs are first attached tounoptimized network201bor202bto continue behaving as if they were invirtualized computing system102, with some traffic tromboning back tovirtualized computing system102. By contrast, new VMs created incloud computing system102 and VMs migrated tocloud computing system102 using DR-based migration may be directly attached to optimizednetworks201aor202a, as such VMs do not have active sessions that should be preserved.
• The live-migrated VMs that are attached tounoptimized networks201bor202bmay later be re-attached to optimizednetworks201aor202aat a suitable time. In one embodiment, such VMs may be reattached to optimizednetworks201aor202awhen the VMs are powered off, in a maintenance mode, or in a quiet period. It should be understood that, when the VM is powered off or in the maintenance mode or quiet period, active sessions may be terminated. As a result, the VM may be shifted during those times from an unoptimized network to an optimized network, without downtime or affecting active session.
• In one embodiment, to create an extended network such asnetwork201 or202 with route optimized and unoptimized stub networks,hybridity director174 first (1) creates a routed network incloud computing system150 with a router (e.g., router240) having the same gateway IP address as that of a default gateway of an existing network invirtualized computing system102 that is being stretched, (2) renumbers the router with a /32 gateway IP address, (3) filters the /32 gateway IP address from being announced to peer routers, and (4) creates a virtual extensible LAN (VXLAN) network for an isolated stretched network. It should be understood that, similar to the discussion above with respect to proximity routing using specific prefix routes, the routed network in this case will become a route optimized stub network. By contrast, the isolated VXLAN network will become an unoptimized stub network.Hybridity director174 then connects the newly-created routed and VXLAN networks incloud computing system150 with the existing network invirtualized computing system102, thereby creating the extended network, by (5) configuring an L2 concentrator (e.g., L2 concentrator212 or214) incloud computing system150 to drop packets having the source MAC address ofrouter240, (6) configuring the same L2 concentrator to drop ARP requests for the gateway IP address from/to the routed network, and (7) bridging the existing network invirtualized computing system102, the routed network incloud computing system150, and the isolated VXLAN network incloud computing system150 via an L2 concentrator pair in the virtual102 and cloud computing systems150 (e.g.,L2 concentrator pair211 and212, or213 and214). Thereafter, when a VM is migrated tocloud computing system150 via DR-based migration,hybridity director174 connects the migrated VM to the routed network, which is the route optimized stub of the extended network, and adds a /32 link route to the cloud router (e.g., router240) for the IP address of the migrated VM that is also propagated back to a gateway router (e.g., router230) invirtualized computing system102, as described above. By contrast, when a VM is live-migrated tocloud computing system150,hybridity director174 connects such a migrated VM to the isolated VXLAN network, which is the unoptimized stub of the extended network, with VXLAN properties of the virtual switch port that the live-migrated VM is connected to being overridden with those of the isolated VXLAN network.
• It should be understood that, rather than having to create both stub networks incloud computing system150 as new networks, one or more such networks may already exist in some cases. For example, a stub network may exist incloud computing system150 with VMs on it, and an extended network with the existing stub network as a route optimized network and another stub network as an unoptimized network may be created as follows: (1) renumber the existing stub network's router (e.g., router240) with a /32 gateway IP address that is not announced to peer routers, (2) add /32 routes for VMs on the existing stub network, (3) create an isolated network to be the unoptimized stub network, and (4) bridge the two stub networks incloud computing system150 and a network invirtualized computing system102 using an L2 concentrator pair. As another example, an extended network may already exist without routing optimization, and such an extended network may be transformed into an extended network with two stub networks, one of which is route optimized, as follows: (1) create an isolated stub network to be an unoptimized stub network and add it to the L2 concentrator bridge used by the existing extended network, (2) renumber the cloud computing system router (e.g., router240) to have a /32 IP address while preventing the /32 IP address from being announced to peer routers, (3) add /32 routes for VMs on the network incloud computing system102 that is part of the existing extended network, (4) configure the cloud L2 concentrator used by the existing extended network to block packets with the cloud computing system router's (e.g., router240) source MAC address, (5) configure the cloud L2 concentrator to block ARP requests for the gateway IP address from/to the cloud network that is part of the existing extended network, (6) connect cloud computing system router (e.g., router240) to the network in the cloud that is part of the existing extended network.
• FIG. 9 illustrates amethod900 of temporarily tromboning traffic of live-migrated virtual machines, according to an embodiment. As shown,method900 begins atstep910, wherehybridity manager174 determines whether a VM is being migrated fromvirtualized computing system102 tocloud computing system150 is live-migrated or not. As described, live migration is treated differently from DR-based migration, as a VM is powered off during DR-based migration, with all connections (active sessions) being terminated, whereas active sessions may be carried over tocloud computing system150 when a VM is live migrated fromvirtualized computing system102. In one embodiment,hybridity manager174 may be responsible for migrating VMs, as noted above, and able to distinguish between live and DR-based migrations. However, it should be understood thathybridity manager174 may also determine whether the VM is being live migrated or not in any other feasible manner.
• If the VM is not being live migrated, then atstep920,hybridity manager174 attaches the migrated VM directly to a route optimized stub network of the extended network, such as the optimizednetwork201aor202aofextended network201 or202, respectively.
• If on the other hand the VM is being live migrated, then atstep930hybridity manager174 attaches the live-migrated VM to an unoptimized stub network of the extended network. As described, doing so permits active sessions that are carried over tocloud computing system150 during the live migration to be preserved, as traffic on the unoptimized network can trombone back tovirtualized computing system102.
• Atstep940,hybridity manager174 determines whether the VM attached to the unoptimized stub network can be shifted to the route optimized stub network of the extended network. In one embodiment,hybridity manager174 may monitor for the VM being powered off or rebooted, or placed in a maintenance mode or quiet period. As described, active sessions may be terminated when the VM is powered off or in the maintenance mode or quiet period, and the VM may then be shifted during those times to an optimized network without downtime or affecting active sessions.
• Ifhybridity manager174 determines that the VM attached to the unoptimized stub network can be shifted to the route optimized stub network, thenmethod900 returns to step940, wherehybridity manager174 may, e.g., continue monitoring for the VM being powered off or rebooted, or placed in a maintenance mode or quiet period.
• On the other hand, ifhybridity manager174 determines that the VM attached to the unoptimized stub network can be shifted to the route optimized stub network, then atstep950, hybridity manager detaches the VM from the unoptimized network and re-attaches the VM to the optimized network. In one embodiment,hybridity manager174 may reconfigure properties of a virtual port on the unoptimized stub network (e.g.,unoptimized network201bor202b) to which the VM is connected to instead point to a virtual port on the optimized stub network (e.g.,network201aor202a). In a particular embodiment,hybridity manager174 may disconnect the VNIC of the VM from the virtual port on the unoptimized stub network, modify the VXLAN ID and multicast IP address for of the virtual port through APIs provided by the a virtual switch, and reconnecting the VNIC to the virtual port. For example, suppose the optimized network has VXLAN ID 10000 and the unoptimized network has VXLAN ID 10001, then the port properties may be modified to change what the virtual switch encapsulates the VM's traffic to from VXLAN ID 10001 to VXLAN ID 10000. As a result, the VM will have a port on VXLAN with ID 10000, whose traffic is route optimized.
• It should be understood that if, e.g., all VMs onextended networks201 or202 are moved to thecloud computing system150 side and (either initially or eventually) transferred to optimizednetwork201aor202a, then the L2 stretching andunoptimized networks201bor202bmay be torn down and the L2 concentrator bridges removed so all that remains are optimizednetworks201aor202awithincloud computing system150.Unoptimized networks201bor202bmay also be removed or not used at all if, for example, there are no live VM migrations for which active sessions need to be preserved.
• Although discussed herein primarily with respect to tromboning in which traffic with both source and destination within the cloud is routed from the cloud to the physically-separate data center and then back to the cloud, it should be understood that the opposite type of tromboning involving traffic with both source and destination within the data center being routed to the cloud may be remedied using techniques similar to those disclosed herein.
• Advantageously, techniques disclosed herein provide stateful connection optimization in a hybrid cloud computing system. Such optimization may eliminate inefficient traffic patterns such as tromboning, while ensuring that traffic flows are symmetric. In addition, live-migrated VMs may be first attached to unoptimized networks in the cloud to force traffic tromboning and preserve active sessions, with such live-migrated VMs being later shifted to optimized networks when the VMs are powered off, or placed in a maintenance mode, or enter a quiet period, and the active sessions are terminated.
• The various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities usually, though not necessarily, these quantities may take the form of electrical or magnetic signals where they, or representations of them, are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the invention may be useful machine operations. In addition, one or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for specific required purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the required operations.
• The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.
• One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer. Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs), CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The computer readable medium can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
• Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation, unless explicitly stated in the claims.
• In addition, while described virtualization methods have generally assumed that virtual machines present interfaces consistent with a particular hardware system, persons of ordinary skill in the art will recognize that the methods described may be used in conjunction with virtualizations that do not correspond directly to any particular hardware system. Virtualization systems in accordance with the various embodiments, implemented as hosted embodiments, non-hosted embodiments, or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
• Certain embodiments as described above involve a hardware abstraction layer on top of a host computer. The hardware abstraction layer allows multiple contexts or virtual computing instances to share the hardware resource. In one embodiment, these virtual computing instances are isolated from each other, each having at least a user application running therein. The hardware abstraction layer thus provides benefits of resource isolation and allocation among the virtual computing instances. In the foregoing embodiments, virtual machines are used as an example for the virtual computing instances and hypervisors as an example for the hardware abstraction layer. As described above, each virtual machine includes a guest operating system in which at least one application runs. It should be noted that these embodiments may also apply to other examples of virtual computing instances, such as containers not including a guest operation system, referred to herein as “OS-less containers” (see, e.g., www.docker.com). OS-less containers implement operating system—level virtualization, wherein an abstraction layer is provided on top of the kernel of an operating system on a host computer. The abstraction layer supports multiple OS-less containers each including an application and its dependencies. Each OS-less container runs as an isolated process in userspace on the host operating system and shares the kernel with other containers. The OS-less container relies on the kernel's functionality to make use of resource isolation (CPU, memory, block I/O, network, etc.) and separate namespaces and to completely isolate the application's view of the operating environments. By using OS-less containers, resources can be isolated, services restricted, and processes provisioned to have a private view of the operating system with their own process ID space, file system structure, and network interfaces. Multiple containers can share the same kernel, but each container can be constrained to only use a defined amount of resources such as CPU, memory and I/O.
• Many variations, modifications, additions, and improvements are possible, regardless the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions. Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the appended claims(s).

Claims (20)

What is claimed is:

1. A computer-implemented method of optimizing connections over an extended network created by bridging a first network in a first computing system, a second network in a second computing system, and a wide area network using a first bridging appliance in the first computing system and a second bridging appliance in the second computing system, comprising:

configuring the first bridging appliance to:

block packets with a source media access control (MAC) address belonging to a first router in the first computing system, and

block address resolution protocol (ARP) requests for a gateway Internet Protocol (IP) address from and to the first network;

configuring the first router in the first computing system to have a same gateway IP address as a second router in the second computing system that is a default gateway of the extended network;

injecting, into the first router, one or more specific prefix routes for one or more respective virtual computing instances on the first network; and

propogating the one or more specific prefix routes to the second router.

2. The method ofclaim 1, further comprising, propogating a route to a third network that is local to the first computing system to the second router.

3. The method ofclaim 1, wherein:

the one or more specific routes are /32 routes; and

the first router is configured to have a /32 IP address which is not announced to peer routers.

4. The method ofclaim 1, further comprising, responsive to determining a first virtual computing instance has attached to the first network in a first computing system:

injecting, into the first router, an additional specific prefix route for the first virtual computing instance; and

propogating the additional specific prefix route to the second router.

5. The method ofclaim 4, wherein the first computing instance has attached to the first network in a first computing system after migrating from the second computing system to the first computing system.

6. The method ofclaim 1, further comprising, responsive to determining a first virtual computing instance has detached from the first network in a first computing system:

removing, into the first router, an injected specific prefix route associated with the first virtual computing instance.

7. The method ofclaim 1, wherein the first bridging appliance and the second bridging appliance are layer 2 (L2) concentrator appliances.

8. The method ofclaim 1, wherein the first computing system is a cloud computing system and the second computing system is an on-premise virtualized computing system.

9. The method ofclaim 1, wherein the virtual computing instances on the first network are virtual machines.

10. A non-transitory computer-readable storage medium containing a program which, when executed by one or more processors, performs operations for optimizing connections over an extended network created by bridging a first network in a first computing system, a second network in a second computing system, and a wide area network using a first bridging appliance in the first computing system and a second bridging appliance in the second computing system, the operations comprising:

configuring the first bridging appliance to:

block packets with a source media access control (MAC) address belonging to a first router in the first computing system, and

block address resolution protocol (ARP) requests for a gateway Internet Protocol (IP) address from and to the first network;

configuring the first router in the first computing system to have a same gateway IP address as a second router in the second computing system that is a default gateway of the extended network;

injecting, into the first router, one or more specific prefix routes for one or more respective virtual computing instances on the first network; and

propogating the one or more specific prefix routes to the second router.

11. The computer-readable storage medium ofclaim 10, the operations further comprising, propogating a route to a third network that is local to the first computing system to the second router.

12. The computer-readable storage medium ofclaim 10, wherein:

the one or more specific routes are /32 routes; and

the first router is configured to have a /32 IP address which is not announced to peer routers.

13. The computer-readable storage medium ofclaim 10, the operations further comprising, responsive to determining a first virtual computing instance has attached to the first network in a first computing system:

injecting, into the first router, an additional specific prefix route for the first virtual computing instance; and

propogating the additional specific prefix route to the second router.

14. The computer-readable storage medium ofclaim 13, wherein the first computing instance has attached to the first network in a first computing system after migrating from the second computing system to the first computing system.

15. The computer-readable storage medium ofclaim 10, the operations further comprising, responsive to determining a first virtual computing instance has detached from the first network in a first computing system:

removing, into the first router, an injected specific prefix route associated with the first virtual computing instance.

16. The computer-readable storage medium ofclaim 10, wherein the first bridging appliance and the second bridging appliance are layer 2 (L2) concentrator appliances.

17. The computer-readable storage medium ofclaim 10, wherein the first computing system is a cloud computing system and the second computing system is an on-premise virtualized computing system.

18. The computer-readable storage medium ofclaim 10, wherein the virtual computing instances on the first network are virtual machines.

19. A system, comprising:

a processor; and

a memory, wherein the memory includes a program executable in the processor to perform operations for optimizing connections over an extended network created by bridging a first network in a first computing system, a second network in a second computing system, and a wide area network using a first bridging appliance in the first computing system and a second bridging appliance in the second computing system, the operations comprising:

configuring the first bridging appliance to:

block packets with a source media access control (MAC) address belonging to a first router in the first computing system; and

block address resolution protocol (ARP) requests for a gateway Internet Protocol (IP) address from and to the first network,

configuring the first router in the first computing system to have a same gateway IP address as a second router in the second computing system that is a default gateway of the extended network,

injecting, into the first router, one or more specific prefix routes for one or more respective virtual computing instances on the first network, and

propogating the one or more specific prefix routes to the second router.

20. The system ofclaim 19, the operations further comprising, propogating a route to a third network that is local to the first computing system to the second router.

Images