Network Working Group L. DunbarInternet Draft J. GuichardIntended status: Informational FutureweiExpires: July 22, 2021 Ali Sajassi Cisco J. Drake Juniper B. Najem Bell Canada Ayan Barnerjee D. Carrel IPsec Research January 22, 2021BGP Usage for SDWAN Overlay Networksdraft-ietf-bess-bgp-sdwan-usage-02Abstract The document discusses the usage and applicability of BGP as control plane for multiple SDWAN scenarios. The goal of the document is to demonstrate how BGP-based control plane is used for large scale SDWAN overlay networks with little manual intervention. SDWAN edge nodes are commonly interconnected by multiple underlay networks which can be owned and managed by different network providers.Status of this Memo This Internet-Draft is submitted in full conformance with the provisions ofBCP 78 andBCP 79. This Internet-Draft is submitted in full conformance with the provisions ofBCP 78 andBCP 79. This document may not be modified, and derivative works of it may not be created, except to publish it as an RFC and to translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note thatxxx, et al. Expires July 22, 2021 [Page 1]
Internet-Draft BGP Usage for SDWAN January 22, 2021 other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed athttp://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed athttp://www.ietf.org/shadow.html This Internet-Draft will expire on July 22, 2021.Copyright Notice Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject toBCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of theTrust Legal Provisions and are provided without warranty as described in the Simplified BSD License.Table of Contents1. Introduction...................................................32. Conventions used in this document..............................43. Use Case Scenario Description and Requirements.................53.1. Requirements..............................................63.1.1. Supporting Multiple SDWAN Segmentations..............63.1.2. Client Service Requirement...........................63.1.3. Application Flow Based Segmentation..................73.1.4. Zero Touch Provisioning..............................83.1.5. Constrained Propagation of SDWAN Edge Properties.....9Dunbar, et al. Expires July 22, 2021 [Page 2]
Internet-Draft BGP Usage for SDWAN January 22, 20213.2. Scenario #1: Homogeneous WAN.............................103.3. Scenario #2: Hybrid WAN Underlay.........................113.4. Scenario #3: Private VPN PE based SDWAN..................134. BGP Walk Through..............................................144.1. BGP Walk Through for Homogeneous SDWAN...................144.2. BGP Walk Through for Hybrid WAN Underlay.................16 4.3. BGP Walk Through for Application Flow Based Segmentation.174.4. Client Service Provisioning Model........................184.5. Benefit of Using Recursive Next Hop Resolution...........194.6. Why BGP as Control Plane for SDWAN?......................195. SDWAN Traffic Forwarding Walk Through.........................205.1. Packet Walk-Through for Scenario #1......................205.2. Packet Walk-Through for Scenario #2......................215.3. Packet Walk-Through for Scenario #3......................226. Manageability Considerations..................................227. Security Considerations.......................................238. IANA Considerations...........................................239. References....................................................239.1. Normative References.....................................239.2. Informative References...................................2410. Acknowledgments..............................................251. Introduction Here are some key characteristics of "SDWAN" networks: - Augment of transport, which refers to utilizing overlay paths over different underlay networks. Very often there are multiple parallel overlay paths between any two SDWAN edges, some of them are private networks over which traffic can traverse with or without encryption, others require encryption, e.g. over untrusted public networks. - Enable direct Internet access from remote sites, instead hauling all traffic to Corporate HQ for centralized policy control. - Some traffic flows can be forwarded based on their application identifiers instead of based on destination IP addresses, by the edge nodes placing the traffic flows onto specific overlay paths based on their application requirement. - The traffic flows forwarding can also be based on specific performance criteria (e.g. packets delay, packet loos, jitter) to provide better application performance by choosing the right underlay that meets or exceeds the specified criteria.Dunbar, et al. Expires July 22, 2021 [Page 3]
Internet-Draft BGP Usage for SDWAN January 22, 2021 [Net2Cloud-Problem] describes the network related problems that enterprises face to connect enterprises' branch offices to dynamic workloads in different Cloud DCs, including using SDWAN to aggregate multiple paths provided by different service providers to achieve better performance and to accomplish application ID based forwarding. Even though SDWAN has been positioned as a flexible way to reach dynamic workloads in third party Cloud data centers over different underlay networks, scaling becomes a major issue when there are hundreds or thousands of nodes to be interconnected by an SDWAN overlay networks. This document describes using BGP as control plane to scale large SDWAN overlay networks.2. Conventions used in this document Cloud DC: Third party data centers that host applications and workloads owned by different organizations or tenants. Controller: Used interchangeably with SDWAN controller to manage SDWAN overlay path creation/deletion and monitor the path conditions between sites. CPE: Customer Premise Equipment CPE-Based VPN: Virtual Private Secure network formed among CPEs. This is to differentiate from more commonly used PE- based VPNs [RFC 4364]. Homogeneous SDWAN: A type of SDWAN network in which all traffic to/from the SDWAN edge nodes has to be encrypted regardless of underlay networks. For lack of better terminology, we call this Homogeneous SDWAN throughout this document. ISP: Internet Service ProviderDunbar, et al. Expires July 22, 2021 [Page 4]
Internet-Draft BGP Usage for SDWAN January 22, 2021 NSP: Network Service Provider. NSP usually provides more advanced network services, such as MPLS VPN, private leased lines, or managed Secure WAN connections, many times within a private trusted domain, whereas an ISP usually provides plain internet services over public untrusted domains. PE: Provider Edge SDWAN Edge Node: an edge node, which can be physical or virtual, maps the attached clients' traffic to the wide area network (WAN) overlay tunnels. SDWAN: Software Defined Wide Area Network. A connectivity service, offered by a Service Provider, that optimizes transport of IP Packets over multiple underlay connectivity services by recognizing applications at Ingress and determining forwarding behavior by applying policies to them. SDWAN IPsec SA: IPsec Security Association between two SDWAN ports or nodes. SDWAN over Hybrid Networks: SDWAN over Hybrid Networks typically have edge nodes utilizing bandwidth resources from different types of underlay networks, some being private networks and others being public Internet. WAN Port: A Port or Interface facing an ISP or Network Service Provider (NSP), with address allocated by the ISP or the NSP. C-PE: SDWAN Edge node, which can be CPE for customer managed SDWAN, or PE for provider managed SDWAN services. ZTP: Zero Touch Provisioning3. Use Case Scenario Description and Requirements SDWAN networks can have different topologies and have different traffic patterns. To make it easier for the focused discussion inDunbar, et al. Expires July 22, 2021 [Page 5]
Internet-Draft BGP Usage for SDWAN January 22, 2021 subsequent drafts on SDWAN control plane and data plane, this section describes several SDWAN scenarios that may have different impact on their corresponding control planes & data planes.3.1. Requirements3.1.1. Supporting Multiple SDWAN Segmentations The term "network segmentation", a.k.a. SDWAN instances, is referring to the process of dividing the network into logical sub- networks using isolation techniques on a forwarding device such as a switch, router, or firewall. For a homogeneous network, such as MPLS VPN or Layer 2 network, VRF or VLAN are used to achieve the network segmentation. As SDWAN is an overlay network arching over multiple types of networks, MPLS L2VPN/L3VPN or pure L2 underlay can continue using the VRF, VN-ID or VLAN to differentiate SDWAN network segmentations. For public internet, the IPsec inner encapsulation header can carry the SDWAN Instance Identifier to differentiate the packets belonging to different SDWAN instances. BGP already has the capability to differentiate control packets for different network instances. When using BGP for SDWAN, the SDWAN segmentations can be differentiated by the SDWAN Target ID in the BGP Extended Community in the same way as VPN instances being represented by the Route Target. Even though the SDWAN Target ID is for the same purpose as the VPN ID in Route Target, it is better to use a different name to differentiate from VPN if a CPE supports traditional VPN with multiple VRFs and supports multiple SDWAN Segmentations (instances). The actual SDWAN Target ID encoding is specified by [SDWAN-EDGE-Discovery].3.1.2. Client Service Requirement Client interface of SDWAN nodes can be IP or Ethernet based. For Ethernet based client interfaces, SDWAN edge should support VLAN-based service interfaces (EVI100), VLAN bundle service interfaces (EVI200), or VLAN-Aware bundling service interfaces. EVPN service requirements are applicable to the Client traffic, as described in theSection 3.1 of RFC8388. For IP based client interfaces, L3VPN service requirements are applicable.Dunbar, et al. Expires July 22, 2021 [Page 6]
Internet-Draft BGP Usage for SDWAN January 22, 20213.1.3. Application Flow Based Segmentation Application Flow based Segmentation, also known as SDWAN Traffic Segmentation, enables the separation of the traffic based on the business and the security needs for different users' groups and/or application requirements. Each user group and/or applications may need different isolated topology and/or policies to fulfill the business requirements. The Application Flow based Segmentation concept is analogous to VLAN (in L2 network) and VRF (in L3 network). One can think about the Application Flow based Segmentation as a feature that can be provided or enabled on a single SDWAN service (or domain) to a single Subscriber. Each SDWAN Service can have one or more overlay segments to support the business requirement; each segment has its own policy, topology and application/user groups. Applications/users' group can belong to more than one segment. For example, a retail business requires the point-of-sales (PoS) application in all stores to be isolated from other applications AND routed only to the payment processing entity at a hub site (i.e. hub and spoke); however, the same retail business allows other applications to be routed to all sites (i.e. multipoint-to multipoint) AND isolated from the PoS application. In the figure below, the traffic from the PoS application follows a Tree topology, whereas other traffic can be multipoint-to-multipoint topology.Dunbar, et al. Expires July 22, 2021 [Page 7]
Internet-Draft BGP Usage for SDWAN January 22, 2021 +--------+ Payment traffic |Payment | +------+----+-+gateway +------+----+-----+ / / | +----+---+ | \ \ / / | | | \ \ +-+--+ +-+--+ +-+--+ | +-+--+ +-+--+ +-+--+ |Site| |Site| |Site| | |Site| |Site| |Site| | 1 | | 2 | | 3 | | |4 | | 5 | | 6 | +--+-+ +--+-+ +--|-+ | +--|-+ +--|-+ +--|-+ | | | | | | | ==+=======+=======+====+======+=======+=======+=== Multi-point connection for Other traffic Another example is an enterprise who wants to isolate the traffic for each department with each department has its own topology and policy; the HR department may need to access certain applications that are NOT accessible by the engineering department. In addition, the contractors may have a limited access to the enterprise resources.3.1.4. Zero Touch Provisioning Unlike traditional EVPN or L3VPN whose PEs are deployed for long term, SDWAN edge nodes (virtual or physical) deployment at a specific location can be ephemeral. Therefore, Zero Touch Provisioning (ZTP), or Plug and Play, is a common requirement for SDWAN. When an SDWAN edge is physically installed at a location or instantiated on a VM in a Cloud DC, ZTP automates follow-up steps, including updates to the OS, software version, and configuration prior to connection. From network control perspective, ZTP includes the following: - Upon power up, an SDWAN node can establish transport layer secure connection (such as TLS, SSL, etc.) to its controller whose address can be burned or preconfigured on the device. - The SDWAN Controller can designate a Local Network Controller in the proximity of the SDWAN node; the Local Network Controller manages and monitor the communication policies of the edge node.Dunbar, et al. Expires July 22, 2021 [Page 8]
Internet-Draft BGP Usage for SDWAN January 22, 20213.1.5. Constrained Propagation of SDWAN Edge Properties One SDWAN edge node may only be authorized to communicate with a small number of other SDWAN edge nodes. Under this circumstance, the property of the SDWAN edge node cannot be propagated to other nodes that are not authorized to communicate. But a remote SDWAN edge node upon powering up might not have the proper policies to know who the authorized peers are. Therefore, it is very essential for SDWAN deployment have a central point to distribute the properties of an SDWAN edge node to its authorized peers. BGP is well suited for this purpose.RFC 4684 has specified the procedure to constrain the distribution of BGP UPDATE to only a subset of nodes. Basically, each edge node informs the Route Reflector (RR) [RFC4456] on its interested SDWAN instances. The RR only propagates the BGP UPDATE for the relevant SDWAN instances to the edge. The connection between a SDWAN edge node and its RR can be over insecure network. Therefore, upon power up, a SDWAN node needs to establish a secure transport layer connection (TLS, SSL, etc.) to its designated RR. The BGP UPDATE messages need to be sent over the secure channel (TLS, SSL, etc.) to the RR. +---+ Peer Group 1 |RR | Peer Group 2 +======+====+=+ +======+====+=====+ / / | +---+ | \ \ / / | | \ \ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ +-+--+ |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| |C-PE| | 1 | | 2 | | 3 | |4 | | 5 | | 6 | +----+ +----+ +----+ +----+ +----+ +----+ Tenant 1 Tenant 2 Figure 1: Peer Groups managed by RR Tenant separation is achieved by the SDWAN instance identification represented in control plane and data plane, respectively.Dunbar, et al. Expires July 22, 2021 [Page 9]
Internet-Draft BGP Usage for SDWAN January 22, 20213.2. Scenario #1: Homogeneous WAN This is referring to a type of SDWAN network with edge nodes encrypting all traffic over WAN to other edge nodes, regardless of whether the underlay is private or public. For lack of better terminology, we call this Homogeneous SDWAN throughout this document. Some typical scenarios for the use of a Homogeneous SDWAN network are as follows: - A small branch office connecting to its HQ offices via the Internet. All sensitive traffic to/from this small branch office has to be encrypted, which is usually achieved using IPsec SAs. - A store in a shopping mall may need to securely connect to its applications in one or more Cloud DCs via the Internet. A common way of achieving this is to establish IPsec SAs to the Cloud DC gateway to carry the sensitive data to/from the store. As described in [SECURE-EVPN], the granularity of the IPsec SAs for Homogeneous SDWAN can be per site, per subnet, per tenant, or per address. Once the IPsec SA is established for a specific subnet/tenant/site, all traffic to/from the subnets/tenants/site are encrypted. +---+ +--------------|RR |------------+ / Untrusted +-+-+ \ / \ / \ +----+ +---------+ +------+ +----+ | CN3|--| P1-----+ -------------+------ P1 |--| CN1| +----+ | C-PE1 P2-----+ | | C-PE2| +----+ +----+ | P3-----+ Wide +------ P2 | +----+ | CN2|--| | | area +------ P3 |--| CN3| +----+ +---------+ | network | +------+ +----+ | | +----+ +---------+ | all packets | +------+ +----+ | CN1|--| P1-----+ encrypted +------ P1 |--| CN1| +----+ | C-PE3 P2-----+ by | | C-PE4| +----+ +----+ | P3-----+ IPsec SAs +------ P2 | +----+ | CN2|--| P4-----+--------------+ | |--| CN2| +----+ +---------+ +------+ +----+ CN: Client Networks, which is same as Tenant Networks used by NVo3 Figure 2: Homogeneous SDWANDunbar, et al. Expires July 22, 2021 [Page 10]
Internet-Draft BGP Usage for SDWAN January 22, 2021 One of the key properties of homogeneous SDWAN is that the SDWAN Local Network Controller (RR)is connected to C-PEs via untrusted public network, therefore, requiring secure connection between RR and C-PEs (TLS, DTLS, etc.). Homogeneous SDWAN has some similarity to commonly deployed IPsec VPN, albeit the IPsec VPN is usually point-to-point among a small number of nodes and with heavy manual configuration for IPsec between nodes, whereas an SDWAN network can have a large number of edge nodes and require zero touch provisioning upon powering up. Existing Private VPNs (e.g. MPLS based) can use homogeneous SDWAN to extend over public network to remote sites to which the VPN operator does not own or lease infrastructural connectivity, as described in [SECURE-EVPN] and [SECURE-L3VPN]3.3. Scenario #2: Hybrid WAN Underlay In this scenario, SDWAN edge nodes (a.k.a. C-PEs) have some WAN ports to private networks such as L3VPN and some WAN ports to the public Internet. Since IPsec requires additional processing power and encrypted traffic over Internet usually does not have the premium SLA commonly offered by Private VPNs, especially over long distance, this scenario is referring to a SDWAN network in which traffic over IP VPN are forwarded natively without IPsec protection. Only the traffic sent over the public Internet are protected by IPsec SAs. One C-PE might have Internet facing WAN ports managed by different ISPs/NSPs and the WAN ports' addresses being assigned by the corresponding ISPs/NSPs. Clients might have policies to specify: 1) some flows only traversing private VPNs, 2) some can traverse either if their packets are encrypted when over the public Internet, and 3) Some flows, especially internet bound browsing ones can be handed off to the internet without any encryption. If a flow traversing multiple segments, such as A<->B<->C<->D, has the Policy 2) above, the flow can traverse different underlays in different segments, such as over Private network underlay between A<->B without encryption, or over the public internet between B<->C protected by an IPsec SA. As shown in the figure below, C-PE-1 has two different types of interfaces (A1 to Internet and A2 & A3 to VPN). The C-PEs' loopback addresses and addresses attached to C-PEs may or may not be visibleDunbar, et al. Expires July 22, 2021 [Page 11]
Internet-Draft BGP Usage for SDWAN January 22, 2021 to the ISPs/NSPs. The addresses for the WAN ports can have addresses allocated by service providers or dynamically assigned (e.g. by DHCP). +---+ +--------------|RR |----------+ / Untrusted +-+-+ \ / \ / \ +----+ +---------+ packets encrypted over +------+ +----+ | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| +----+ | C-PE1 A2-\ | C-PE2| +----+ +----+ | A3--+--+ +---+---B2 | +----+ | CN2|--| | |PE+--------------+PE |---B3 |--| CN3| +----+ +---------+ +--+ trusted +---+ +------+ +----+ | WAN | +----+ +---------+ +--+ packets +---+ +------+ +----+ | CN1|--| C1--|PE| go natively |PE |-- D1 |--| CN1| +----+ | C-PE3 C2--+--+ without encry+---+ | C-PE4| +----+ | | +--------------+ | | | | | | +----+ | | without encrypt over | | +----+ | CN2|--| C3--+---- Untrusted --+------D2 |--| CN2| +----+ +---------+ +------+ +----+ CN: Client Network Figure 3: Hybrid SDWAN In addition, the connection between C-PEs and their Controller (RR) might be via the untrusted public network. It is necessary to encrypt the communication between RR and C-PEs, by TLS, DTLS, etc.. There could be multiple SDWAN edges (C-PEs) sharing a common property, such as a geographic location. Some applications over SDWAN may need to traverse specific geographic locations for various reasons, such as to comply with regulatory rules, to utilize specific value-added services, or others. Services may not be congruent, i.e. the packets from A-> B may traverse one underlay network, and the packets from B -> A may traverse a different underlay.Dunbar, et al. Expires July 22, 2021 [Page 12]
Internet-Draft BGP Usage for SDWAN January 22, 20213.4. Scenario #3: Private VPN PE based SDWAN This scenario refers to existing VPN (e.g. MPLS based VPN, such as EVPN or IPVPN) adding extra ports facing untrusted public networks to allow PEs to offload some low priority traffic to public networks when the VPN MPLS paths are congested. Throughout this document, this scenario is also called Internet Offload for Private VPN, or PE based SDWAN. Here are some differences from the Scenario #2 (Section 3.3): - The packets offloaded to untrusted public network are still part of the VPN traffic, therefore, must be encrypted. - For MPLS based VPN, PEs would have MPLS as payload encapsulated within the IPsec tunnel egressing to the Internet WAN ports, MPLS-in-IP-in-IPsec. - The BGP RR is connected to PEs in the same way as VPN, i.e. via the trusted network. PE based SDWAN can be used by VPN service providers to temporarily increase bandwidth between sites when not sure if the demand will sustain for long period of time or as a temporary solution before the permanent infrastructure is built or leased. +---+ +======>|PE2| // +---+ // ^ // || VPN // VPN v ++--+ ++-+ +---+ |PE1| <====> |RR| <===> |PE3| +-+-+ +--+ +-+-+ | | +--- Public Internet -- + Offload Figure 4: Additional Internet paths added to the VPNDunbar, et al. Expires July 22, 2021 [Page 13]
Internet-Draft BGP Usage for SDWAN January 22, 20214. BGP Walk Through4.1. BGP Walk Through for Homogeneous SDWAN In the figure below, packets destined towards multiple routes attached to the C-PE2 can be carried by one IPsec tunnel. Then one BGP UPDATE can be announced by C-PE2 to its RR. +---+ +---------|RR |----------+ / Untrusted+---+ \ / \ C-PE1/ \ +---------+ +------+ --+---+---------------------------------> |-10.1.x.x/16 | / | |C-PE2 |- VLAN = 15 | / | +-----> | --|/1.1.1.1 | | | |-12.1.1.x/24 +---------+ | +------+ | 2.2.2.2 | C-PE3 | +---------+ | --|---+---------------------------+ | / | | / | --|/3.3.3.3 | +---------+ Figure 5: Homogeneous SDWAN The BGP UPDATE Message from C-PE2 to RR should have the client routes encoded in the MP-NLRI Path Attribute and the IPsec Tunnel associated information encoded in the Tunnel-Encap Path Attributes as described in the [SECURE-EVPN]. Alternatively, two separate BGP UPDATEs can be used to optimize the BGP UPDATE packet size, as described bySection 4 and 8 of [Tunnel- encap]. UPDATE U1 has its Nexthop to the node loopback address and is reclusively resolved to the IPsec tunnel detailed attributes advertised by the UPDATE U2 for the Node Loopback address: Suppose that: - a given packet P destined towards the client addresses attached to C-PE2 (e.g. prefix 10.1.x.x/16) can be carried by any IPsec tunnels terminated at C-PE2;Dunbar, et al. Expires July 22, 2021 [Page 14]
Internet-Draft BGP Usage for SDWAN January 22, 2021 - The path along which P is to be forwarded is determined by BGP UPDATE U1; - UPDATE U1 does not have a Tunnel Encapsulation attribute; - UPDATE U1 can include Encapsulation Extended Community and/or Color Extended Community; - The address of the next hop of UPDATE U1 is router C-PE2; - The best route to router C-PE2 is a BGP route that was advertised in UPDATE U2; - UPDATE U2 has a Tunnel Encapsulation attribute to further describe the IPsec detailed attributes. UPDATE U1: - MP-NLRI Path Attribute: 10.1.x.x/16 12.1.1.x/24 - Nexthop: 2.2.2.2 (C-PE2) - Encapsulation Extended Community: Type = IPsec UPDATE U2: - MP-NLRI Path Attribute: 2.2.2.2 (C-PE2) - Tunnel Encapsulation Path Attributes (as described in the [SECURE-EVPN]) If different client routes attached to C-PE2 needs to be reached by separate IPsec tunnels, then The Color-Extended-Community [Tunnel- encap] is used to associate routes with the tunnels. SeeSection 8 of [Tunnel-encap]. If C-PE2 doesn't have the policy on authorized peers for the specific client routes, RR needs to check the client routes policies to propagate the BGP UPDATE messages to the remote authorized edge nodes.Dunbar, et al. Expires July 22, 2021 [Page 15]
Internet-Draft BGP Usage for SDWAN January 22, 20214.2. BGP Walk Through for Hybrid WAN Underlay In this scenario, some client routes can be forwarded by any tunnels terminated at the edge node and some client routes can be forwarded by some specific tunnels (such as only MPLS VPN). Color Extended Community (Section 4 & 8 of [Tunnel-Encap]) can be used to represent specific tunnels for the client routes. For example, in the Figure 5 above, suppose that Route 10.1.x.x/16 can be carried by either MPLS or IPsec, and Route 12.1.1.x/24 can only be carried by MPLS, the following UDPATE messages can be used: UPDATE #1a for Route Route 10.1.x.x/16: - MP-NLRI Path Attribute: 10.1.x.x/16 Nexthop: 2.2.2.2 (C-PE2) - Encapsulation Extended Community: Type = SDWAN-Hybrid - Color Extended Community: RED UPDATE #1b for Route Route 12.1.1.x/24: - MP-NLRI Path Attribute: 12.1.1.x/24 Nexthop: 2.2.2.2 (C-PE2) - Encapsulation Extended Community: Type= SDWAN-Hybrid - Color Extended Community: YELLOW UPDATE #2a: for IPsec tunnels terminated at the node: - MP-NLRI Path Attribute: 2.2.2.2 (C-PE2) - Tunnel Encapsulation Path Attributes: TYPE=SDWAN-Hybrid - Color Extended Community: RED UPDATE #2b: for MPLS-in-GRE terminated at the node: - MP-NLRI Path Attribute: 2.2.2.2 (C-PE2) - Tunnel Encapsulation Path Attributes: TYPE=SDWAN-Hybrid - Color Extended Community: YELLOW IANA Action: require a new value for SDWAN-Hybrid Tunnel Type.Dunbar, et al. Expires July 22, 2021 [Page 16]
Internet-Draft BGP Usage for SDWAN January 22, 20214.3. BGP Walk Through for Application Flow Based Segmentation If the applications are assigned with unique IP addresses, the Application Flow based Segmentation described inSection 3.1.2 can be achieved by advertising different BGP UPDATE messages to different nodes. In the Figure below, the following BGP Updates can be advertised to ensure that Payment Application only communicates with the Payment Gateway: BGP UPDATE #1a from C-PE2 to RR for the P2P topology that is only propagated to Payment GW node: UPDATE #1a (only to the Payment GW node): - MP-NLRI Path Attribute: - 30.1.1.x/24 - Nexthop: 2.2.2.2 - Encapsulation extended community: Tunneltype=IPSEC - Color Extended Community: BLUE BGP UPDATE #1b from C-PE2 to RR for the routes to be reached by C- PE1 and C-PE2: - MP-NLRI Path Attribute: - 10.1.x.x - 12.4.x.x - Nexthop:2.2.2.2 - Encapsulation extended community: Tunnel-type=IPSEC - Color Extended Community: RED BGP UPDATE #2 describes the IPsec detailed attributes for IPsec tunnels terminated at C-PE2 2.2.2.2. UPDATE #2a: for all IPsec SAs terminated at the node: - MP-NLRI Path Attribute: 2.2.2.2 (C-PE2) - Tunnel Encapsulation Path Attributes: TYPE=IPsec (for all IPsec SAs) - Color Extended Community: REDDunbar, et al. Expires July 22, 2021 [Page 17]
Internet-Draft BGP Usage for SDWAN January 22, 2021 UPDATE #2b: for the IPsec SA to the Payment Gateway: - MP-NLRI Path Attribute: 2.2.2.2 (C-PE2) - Tunnel Encapsulation Path Attributes: TYPE=IPsec (for the IPsec SA to Payment GW). - Color Extended Community: Blue +-------+ |Payment| +-------->| GW |<-------+ /Hub-spoke +-------+ \ /for Payment App \ C-PE1 / \ C-PE2 +------/--+ +----\-+ --|-----/ | | -|- 30.1.1.x/24 + ---------------------------------------> |-10.1.x.x/16 | | | |- | | +------------> |- 12.1.1.x/24 --|---------------------------+ | | +---------+ +------> |- VLAN=25; / +------+ 22.1.1.x/24 +---------+ / --| -----------------------------+ | C-PE3 | / | | / --| --------------------------+ +---------+ Figure 6: Application Based SDWAN Segmentation4.4. Client Service Provisioning Model The provisioning tasks described inSection 4 of RFC8388 are the same for the SDWAN client traffic. When client traffic is multi- homed to two (or more) C-PEs, the Non-Service-Specific parameters need to be provisioned per theSection 4.1.1 of RFC8388. Since some SDWAN nodes are ephemeral and have small number of IP subnets or VLANs attached to their client ports, it is recommended to have default and simplified Service-specific parameters for each client port, remotely managed by the SDWAN Network Controller via the secure channel (TLS/DTLS) between the controller and the C-PEs.Dunbar, et al. Expires July 22, 2021 [Page 18]
Internet-Draft BGP Usage for SDWAN January 22, 20214.5. Benefit of Using Recursive Next Hop Resolution Using the Recursive Next Hop Resolution described in the Section 8 of [Tunnel-Encap], not only the clients' routes UPDATE messages become very compact, but also they are not impacted by the underlay network tunnels attributes changes. This method is especially useful when the underlay tunnels are IPsec based which requires periodic messages updates for the pairwise re-keying process.4.6. Why BGP as Control Plane for SDWAN? For a small sized SDWAN network, traditional hub & spoke model using NHRP or DSVPN/DMVPN with a hub node (or controller) managing SDWAN node WAN ports mapping (e.g. local & public addresses and tunnel identifiers mapping) can work reasonably well. However, for a large SDWAN network, say more than 100 nodes with different types of topologies, the traditional approach becomes very messy, complex and error prone. Here are some of the compelling reasons of using BGP instead of extending NHRP/DSVPN/DMVPN. (Same as the reasons quoted by LSVR on why using BGP): - BGP has the built-in capability to constrain the propagation of SDWAN edge node properties to a small number of edge nodes [RFC4684]. - RR already has the capability to apply policies to communications among peers. - BGP is widely deployed as sole protocol (seeRFC 7938) - Robust and simple implementation - Wide acceptance - minimal learning - Reliable transport - Guaranteed in-order delivery - Incremental updates - Incremental updates upon session restart - No flooding and selective filteringDunbar, et al. Expires July 22, 2021 [Page 19]
Internet-Draft BGP Usage for SDWAN January 22, 20215. SDWAN Traffic Forwarding Walk Through BGP based EVPN control plane are still applicable to routes attached to the client ports of SDWAN nodes.Section 5 of RFC8388 describes the BGP EVPN NLRI Usage for various routes of client traffic. The procedures described in theSection 6 of RFC8388 are same for the SDWAN client traffic. The only additional consideration for SDWAN is to control how traffic egress the SDWAN edge node to various WAN ports.5.1. Packet Walk-Through for Scenario #1 A single IPsec security association (SA) protects data in one direction. Under the Scenario #1, two SAs must be present to secure traffic in both directions between any two end points, which can be two C-PE nodes, two client ports, or two prefixes. Upon power up, a SDWAN node can learn client routes from the Client facing ports in the same way as EVPN described inRFC8388. Controller, i.e. BGP-RR, facilitates the IPsec SA establishment and rekey management as described in [SECURE-EVPN]. Controller manages how client's routes are associated with individual IPSec SA. Using Figure 2 of theSection 3.2 as an example. Let's assume that the IPsec SAs terminate at the C-PE nodes. To enable full mesh communication within client CN2 that are attached to C-PE1, C-PE3, and C-PE4, six one directional IPsec SAs must be established: C-PE1 <-> C-PE3; C-PE1 <-> C-PE4; C-PE3 <-> C-PE4. The C-PE node address (or loopback address) acts as the Next Hop address for the prefixes attached to the C-PE node. When a C-PE receives a packet from its client port, the C-PE uses the IPsec SA whose destination address matches the Next Hop address of the packet's destination to encrypt the packet and forward the encrypted packet to the target C-PE via one of the C-PE's WAN ports. When a C-PE receives an encrypted packet from its WAN port, it decrypted the packet and forward the inner packet to the client port based on the inner packet's destination address.Dunbar, et al. Expires July 22, 2021 [Page 20]
Internet-Draft BGP Usage for SDWAN January 22, 20215.2. Packet Walk-Through for Scenario #2 In this scenario as shown in the Figure 3 ofSection 3.3, there are trusted VPN path and IPsec SAs over Public Internet between two C- PEs. There could be user specified policy governing that some flows can only be forwarded to the trusted VPN. Upon receiving a packet from a client port, if the packet belongs to a flow that can only be forwarded to the trusted VPN, the forwarding processing is same as the VPN. If the packet belongs to a flow that can be forwarded to either VPN or IPsec SA, the C-PE node can make the local decision in choosing the least congested path to forward the packet. Since it is not necessary to encrypt the packets forwarded to the trusted VPN, it is more efficient for the IPsec SAs to be terminated at the Internet facing WAN ports of the C-PE nodes. For a C-PE with multiple WAN ports provided by different ISPs, the C-PE can have multiple IPsec SAs terminated at one WAN port of a remote C-PE. If the IPsec SA is chosen, the packet is encrypted and encapsulated in the inner payload of the IPsec SA and egress out the corresponding WAN port of the IPsec SA. Upon receiving a packet from a WAN port, especially from the Internet facing WAN port, additional anti-DDoS mechanism has to be enabled to prevent potential attacks from the Internet facing port. Control Plane should not learn routes from the Internet facing WAN ports. +---+ +--------------|RR |----------+ / Untrusted +-+-+ \ / Network \ / \ +----+ +---------+ packets encrypted over +--------+ +----+ | CN3|--| A1-----+ Untrusted +------ B1 |--| CN1| +----+ | C-PE-a A2-----+ +-------B2 C-PE-b| +----+ |10.1.1.1 | |10.1.2.1| +----+ | | +--+ +---+ | | +----+ | CN2|--| A3 |PE+--------------+PE |---B3 |--| CN3| +----+ +---------+ +--+ trusted +---+ +--------+ +----+ | VPN | +--------------+ Figure 8: SDWAN Scenario #2Dunbar, et al. Expires July 22, 2021 [Page 21]
Internet-Draft BGP Usage for SDWAN January 22, 20215.3. Packet Walk-Through for Scenario #3 In this scenario all PEs have secure interfaces facing clients and facing MPLS backbone with some PEs having additional connection by unsecure public Internet. For the MPLS based PEs offloading some MPLS packets to the internet path, the MPLS data frames need to be encapsulated in IP [RFC4023] and secured by IPsec SA. The IP header of the MPLS-in-IP packet becomes the outer IP header of the resulting packet when IPsec transport mode is used to secure the MPLS packet. This is followed by an IPsec header, followed by the MPLS label stack. The IPsec header must set the payload type to MPLS by using the IP protocol number specified in thesection 3 of RFC4023. If IPsec transport mode is applied on a MPLS-in-GRE packet, the GRE header follows the IPsec header. The end points of an IPsec SA between two PEs can be PEs' Internet facing WAN ports addresses or the PEs' loopback addresses. The IPsec SA end points should not be the client addresses unless the traffic to/from those client addresses always go through the IPsec SA even when the MPLS backbone has enough capacity to transport the traffic. When the PEs' Internet facing ports are behind the NAT [RFC3715], an outer UDP field can be added outside the encrypted payload [RFC3948]. Three UDP ports must be open on the PEs: UDP port 4500 (used for NAT traversal), UDP port 500 (used for IKE) and IP protocol 50 (ESP). IPsec IKE (Internet Key Exchange) between the two PEs would be over the NAT [RFC3947] as well. Upon receiving a packet from a client port, the forwarding and MPLS processing is same as the MPLS VPN. If the MPLS backbone path to the packets next hop is congested, the IPsec SA towards the target PEs are used to encrypt the MPLS packet. Upon receiving a packet from the Internet facing WAN port, the packet is decrypted and the inner MPLS payload is extracted to be sent to the MPLS VPN engine. Same as the Scenario #2, additional anti-DDoS mechanism must be enabled to prevent potential attacks from the Internet facing port. Control Plane should not learn routes from the Internet facing WAN ports.6. Manageability Considerations SDWAN overlay networks utilize the SDWAN controller to facilitate route distribution, central configurations, and others. SDWAN EdgeDunbar, et al. Expires July 22, 2021 [Page 22]
Internet-Draft BGP Usage for SDWAN January 22, 2021 nodes need to advertise the attached routes to their controller (i.e. RR in BGP case).7. Security Considerations Having WAN ports facing the public Internet introduces the following security risks: 1) Potential DDoS attack to the C-PEs with ports facing internet. 2) Potential risk of provider VPN network being injected with illegal traffic coming from the public Internet WAN ports on the C- PEs.8. IANA Considerations Need a new tunnel type: SDWAN-Hybrid9. References9.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels",BCP 14,RFC 2119, March 1997. [RFC4364] E. rosen, Y. Rekhter, "BGP/MPLS IP Virtual Private networks (VPNs)", Feb 2006. [RFC7296] C. Kaufman, et al, "Internet Key Exchange Protocol Version 2 (IKEv2)", Oct 2014. [RFC7432] A. Sajassi, et al, "BGP MPLS-Based Ethernet VPN", Feb 2015. [RFC8365] A. Sajassi, et al, "A network Virtualization Overlay Solution Using Ethernet VPN (EVPN)", March 2018.Dunbar, et al. Expires July 22, 2021 [Page 23]
Internet-Draft BGP Usage for SDWAN January 22, 20219.2. Informative References [RFC8192] S. Hares, et al, "Interface to Network Security Functions (I2NSF) Problem Statement and Use Cases", July 2017 [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent Address Family Identifier (SAFI) and the BGP Tunnel Encapsulation Attribute", April 2009. [RFC8388] J. Rabadan, et al, "Usage and Applicability of BGP MPLS- Based Ethernet VPN", May 2018. [Net2Cloud-Gap] L. Dunbar, A. Malis, C. Jacquenet, "Gap Analysis of Interconnecting Underlay with Cloud Overlay",draft-dm-net2cloud-gap-analysis-02, work in progress, Oct. 2018. [SDWAN-EDGE-Discovery] L. Dunbar, S. Hares, R. Raszuk, K. Majumdar, "BGP UPDATE for SDWAN Edge Discovery",draft-dunbar-idr-sdwan-edge-discovery-01, work-in-progress, Nov 2020. [VPN-over-Internet] E. Rosen, "Provide Secure Layer L3VPNs over Public Infrastructure",draft-rosen-bess-secure-l3vpn-00, work-in-progress, July 2018 [DMVPN] Dynamic Multi-point VPN:https://www.cisco.com/c/en/us/products/security/dynamic-multipoint-vpn-dmvpn/index.html [DSVPN] Dynamic Smart VPN:http://forum.huawei.com/enterprise/en/thread-390771-1-1.html [SECURE-EVPN] A. Sajassi, et al, "Secure EVPN",draft-sajassi-bess-secure-evpn-01, Work-in-progress, March 2019. [SECURE-L3VPN] E. Rosen, R. Bonica, "Secure Layer L3VPN over Public Infrastructure",draft-rosen-bess-secure-l3vpn-00, Work- in-progress, June 2018. [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation, storage, distribution and enforcement of policies for network security", Nov 2007.Dunbar, et al. Expires July 22, 2021 [Page 24]
Internet-Draft BGP Usage for SDWAN January 22, 2021 [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect Underlay to Cloud Overlay Problem Statement",draft-dm-net2cloud-problem-statement-02, June 2018 [Net2Cloud-gap] L. Dunbar, A. Malis, and C. Jacquenet, "Gap Analysis of Interconnecting Underlay with Cloud Overlay",draft-dm-net2cloud-gap-analysis-02, work-in-progress, Aug 2018. [Tunnel-Encap] E. Rosen, et al "The BGP Tunnel Encapsulation Attribute",draft-ietf-idr-tunnel-encaps-10, Aug 2018.10. Acknowledgments Acknowledgements to Adrian Farrel, Joel Halpern, John Scudder, Darren Dukes, Andy Malis and Donald Eastlake for their review and contributions. This document was prepared using 2-Word-v2.0.template.dot.Dunbar, et al. Expires July 22, 2021 [Page 25]
Internet-Draft BGP Usage for SDWAN January 22, 2021Authors' Addresses Linda Dunbar Futurewei Email: ldunbar@futurewei.com James Guichard Futurewei Email: james.n.guichard@futurewei.com Ali Sajassi Cisco Email: sajassi@cisco.com John Drake Juniper Email: jdrake@juniper.net Basil Najem Bell Canada Email: basil.najem@bell.ca David Carrel IPsec Research Email: carrel@ipsec.org Ayan Banerjee Cisco Email: ayabaner@cisco.comDunbar, et al. Expires July 22, 2021 [Page 26]Datatracker
draft-ietf-bess-bgp-sdwan-usage-02
This is an older version of an Internet-Draft whose latest revision state is "Active".
| Document | Document type | This is an older version of an Internet-Draft whose latest revision state is "Active". | |
|---|---|---|---|
| Select version | |||
| Compare versions | |||
| Authors | Linda Dunbar,Jim Guichard,Ali Sajassi,John Drake,Basil Najem,David Carrel | ||
| Replaces | draft-dunbar-bess-bgp-sdwan-usage | ||
| RFC stream |   | ||
| Other formats | |||
| Additional resources | Mailing list discussion |