技术领域technical field
本发明涉及网络技术领域，尤其涉及一种虚拟专用局域网业务(Virtual PrivateLAN Service，VPLS)报文处理方法及设备。The present invention relates to the field of network technologies, and in particular, to a method and device for processing a virtual private local area network service (Virtual Private LAN Service, VPLS) message.
背景技术Background technique
VPLS在传统多协议标签交换(Multi-Protocol Label Switch，MPLS)L2VPN(Layer2Virtual Private Network，二层虚拟专用网络)方案的基础上发展而成，其核心思想是利用信令协议在VPLS实例中的服务提供商边缘(Provider Edge，PE)设备之间建立及维护伪线(Pseudo Wire，PW)，将二层协议帧封装后在PW上传输、交换，使广域范围内多个局域网(Local Area Network，LAN)在数据链路层面被整合为一张网络，向用户提供虚拟的以太网(Ethernet)服务。VPLS is developed on the basis of the traditional Multi-Protocol Label Switch (MPLS) L2VPN (Layer2Virtual Private Network, Layer 2 Virtual Private Network) scheme. Its core idea is to use signaling protocols to serve in VPLS instances. Establish and maintain a pseudo wire (Pseudo Wire, PW) between Provider Edge (PE) devices, encapsulate Layer 2 protocol frames and transmit and exchange them on the PW, so that multiple local area networks (Local Area Networks) within a wide area can be transmitted and exchanged. , LAN) is integrated into a network at the data link level to provide users with virtual Ethernet (Ethernet) services.
图1为现有VPLS组网示意图，如图1所示，VPLS网络可以包含：多个PE设备和用户网络边缘(Customer Edge，CE)设备，每个PE设备可以连接一个或多个CE设备，每个CE设备和其下挂的大量的个人电脑(Private Computer，PC)和服务器等设备通过各自的PE设备接收同一个虚拟专用网络(Virtual Private Network，VPN)的报文，这个VPN称为一个VPLS实例，每个PE设备上可以配置多个虚拟转发实例(Virtual Switching Instance，VSI)，任意两个PE设备上处于同一VPLS网络的VSI间建立有PW，PE设备可以接收CE设备发送的以太报文，并将收到的以太报文通过PW进行VPLS报文转发。Figure 1 is a schematic diagram of an existing VPLS network. As shown in Figure 1, a VPLS network may include: multiple PE devices and customer edge (Customer Edge, CE) devices, each PE device can be connected to one or more CE devices, Each CE device and a large number of personal computers (PCs) and servers attached to it receive packets from the same virtual private network (VPN) through their respective PE devices. This VPN is called a VPN. VPLS instance, each PE device can be configured with multiple virtual switching instances (Virtual Switching Instance, VSI), and a PW is established between the VSIs on the same VPLS network on any two PE devices, and the PE device can receive the Ethernet packets sent by the CE device. and forwards the received Ethernet packets through the PW through VPLS packets.
其中，为了增强VPLS网络的转发性能，现有技术人员通常借助虚拟网络功能(Virtual Network Function，VNF)技术，将其中一个PE设备作为一台VNF服务器，在该服务器中采用分布式系统VNF，虚拟出多个虚拟机(Virtual Machine，VM)(如：控制面VM和转发面VM)，每个转发面VM可以连接多个用户，多个转发面VM可共用同一个VSI，且该VSI可与其他PE设备的VSI建立PW，各转发面VM可以通过建立起来的PW与其他PE设备进行VPLS报文转发。Among them, in order to enhance the forwarding performance of the VPLS network, the existing technical personnel usually use a virtual network function (Virtual Network Function, VNF) technology, one of the PE devices is used as a VNF server, and a distributed system VNF is used in the server. Create multiple virtual machines (Virtual Machines, VMs) (for example: control plane VM and forwarding plane VM), each forwarding plane VM can connect to multiple users, and multiple forwarding plane VMs can share the same VSI, and the VSI can be connected with VSIs of other PE devices establish PWs, and VMs on each forwarding plane can forward VPLS packets with other PE devices through the established PWs.
例如，图2为采用VNF技术的VPLS组网示意图，如图2所示，在PE1中采用分布式系统VNF，虚拟出控制面VM和两个转发面VM：VM1和VM2，且VM1连接家庭1的用户设备，VM2连接家庭2的用户设备，VM1和VM2共用VSI，该VSI可与PE2设备的VSI建立伪线PW1，此时，从家庭2到PE1的转发路径可以有两条：家庭2-VM2-VM1-PW1-PE2、家庭2-VM2-PW1-PE2。For example, Figure 2 is a schematic diagram of a VPLS network using VNF technology. As shown in Figure 2, a distributed system VNF is used in PE1 to virtualize a control plane VM and two forwarding plane VMs: VM1 and VM2, and VM1 is connected to Home 1 VM2 is connected to the user equipment of home 2, VM1 and VM2 share VSI, and this VSI can establish a pseudo-wire PW1 with the VSI of PE2 device. At this time, there can be two forwarding paths from home 2 to PE1: home 2- VM2-VM1-PW1-PE2, Home2-VM2-PW1-PE2.
由上可知，在现有采用VNF的VPLS组网中，VNF系统整个看作一个VSI，多个VM通过该VSI接入到同一个VPLS广播域，因此，这些VM与远端PE只能建立一条PW，VM接入的流量可能会通过VNF系统内部的其他VM转发至远端PE，导致VPLS转发流量的迂回，例如，图2中家庭2到PE1的转发路径可能为：家庭2-VM2-VM1-PW1-PE2。由于VNF系统中不同VM间管理接口的带宽承载能力有限，当大量的数据流量通过管理接口从其他VM转发至远端PE时，很容易出现丢包现象，进而导致流量转发性能低下。As can be seen from the above, in the existing VPLS networking using VNF, the entire VNF system is regarded as a VSI, and multiple VMs access the same VPLS broadcast domain through the VSI. Therefore, these VMs and remote PEs can only establish one channel. The traffic accessed by the PW and VM may be forwarded to the remote PE through other VMs in the VNF system, resulting in the detour of VPLS traffic forwarding. For example, the forwarding path from home 2 to PE1 in Figure 2 may be: home 2-VM2-VM1 -PW1-PE2. Due to the limited bandwidth carrying capacity of management interfaces between different VMs in a VNF system, when a large amount of data traffic is forwarded from other VMs to remote PEs through the management interface, packet loss is likely to occur, resulting in poor traffic forwarding performance.
发明内容SUMMARY OF THE INVENTION
本发明的实施例提供一种VPLS报文处理方法及设备，以解决现有采用VNF技术的VPLS组网中，多个VM共用一个VSI，导致的VPLS转发流量迂回，流量转发性能低下的问题。Embodiments of the present invention provide a VPLS packet processing method and device to solve the problems of detours in VPLS forwarding traffic and low traffic forwarding performance caused by multiple VMs sharing one VSI in the existing VPLS networking using VNF technology.
为达到上述目的，本发明的实施例采用如下技术方案：To achieve the above object, the embodiments of the present invention adopt the following technical solutions:
第一方面，本发明实施例提供一种VPLS报文处理方法，应用于一服务器，该服务器可以包括：至少一个转发面虚拟机VM，每个转发面VM连接至少一个用户，每个转发面VM上设置一个虚拟转发实例VSI，所述至少一个转发面VM上的VSI属于同一VPLS网络，所述报文处理方法可以包括：In a first aspect, an embodiment of the present invention provides a VPLS packet processing method, which is applied to a server. The server may include: at least one forwarding plane virtual machine VM, each forwarding plane VM is connected to at least one user, and each forwarding plane VM A virtual forwarding instance VSI is set on the above, and the VSI on the at least one forwarding plane VM belongs to the same VPLS network, and the packet processing method may include:
第一转发面VM接收第一用户发送的包含：所述第一用户的媒体接入层MAC地址、第二用户的MAC地址、所述第一用户所在的第一虚拟局域网vlan标识的报文，所述第一转发面VM为所述服务器中的任一转发面VM，所述第一用户与所述第一转发面VM连接，所述第二用户与第一对端设备连接，所述第一转发面VM的VSI与所述第一对端设备的VSI属于所述VPLS网络；The first forwarding plane VM receives a packet sent by the first user and includes: the MAC address of the media access layer of the first user, the MAC address of the second user, and the identity of the first virtual local area network (vlan) where the first user is located, The first forwarding plane VM is any forwarding plane VM in the server, the first user is connected to the first forwarding plane VM, the second user is connected to the first peer device, and the first user is connected to the first peer device. The VSI of a forwarding plane VM and the VSI of the first peer device belong to the VPLS network;
查询与所述VSI对应的媒体接入控制MAC转发表，获取所述MAC转发表中与所述第二用户的MAC地址对应的包含：所述第二用户的MAC地址、与所述第二用户的MAC地址对应的所述第一转发面VM上的第一接口的MAC转发表项；Querying the MAC forwarding table corresponding to the VSI, and obtaining the MAC forwarding table corresponding to the MAC address of the second user includes: the MAC address of the second user, and the MAC address of the second user. The MAC forwarding entry of the first interface on the first forwarding plane VM corresponding to the MAC address;
将所述报文封装上所述第一转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的第一私网标签和第一公网隧道标签；encapsulating the message on the first private network label and the first public network tunnel label of the pseudowire PW established between the VSI of the first forwarding plane VM and the VSI of the first peer device;
将封装标签后的报文封装上所述接口的MAC地址、所述第一对端设备的MAC地址，形成第一VPLS报文，将所述第一VPLS报文通过所述第一接口发送出去。Encapsulate the packet after the encapsulation label with the MAC address of the interface and the MAC address of the first peer device to form a first VPLS packet, and send the first VPLS packet through the first interface .
如此，在采用VNF技术的服务器中每个VM上均配置VSI，使每个VM均可以通过自身的VSI与其他设备(如远端服务器或其他VM)直接进行报文转发，避免了转发流量的内部迂回，提高了流量转发的性能，同时，不再占用VM间管理接口带宽，提高了采用NFV技术的设备的健壮性。In this way, a VSI is configured on each VM in a server using VNF technology, so that each VM can directly forward packets with other devices (such as remote servers or other VMs) through its own VSI, avoiding traffic forwarding. The internal detour improves the performance of traffic forwarding, and at the same time, it no longer occupies the bandwidth of the management interface between VMs, which improves the robustness of devices using NFV technology.
可选的，在第一方面的一种可实现方式中，所述第一对端设备可以为PE设备，也可以为所述服务器中除所述第一转发面VM之外的VM，即对端设备可以为服务器自身，还可以为除该服务器之外的其他服务器上的任一转发面VM。Optionally, in an implementation manner of the first aspect, the first peer device may be a PE device, or may be a VM in the server other than the first forwarding plane VM, that is, the peer device. The end device may be the server itself, or may be any forwarding plane VM on a server other than the server.
如此，使转发面VM不仅可以与远端设备间收发VPLS报文，还可以与本地的转发面VM间收发VPLS报文。In this way, the forwarding plane VM can not only send and receive VPLS packets with the remote device, but also send and receive VPLS packets with the local forwarding plane VM.
可选的，在第一方面的又一种可实现方式中，在第一转发面VM接收第一用户发送的报文之前，方法还可以包括：Optionally, in another implementation manner of the first aspect, before the first forwarding plane VM receives the message sent by the first user, the method may further include:
第一转发面VM与第一对端设备间建立标签分发协议会话ldp session，通过ldpsession，获取第一转发面VM的VSI与第一对端设备的VSI间建立的PW的私网标签和公网隧道标签。Establish a label distribution protocol session ldp session between the VM of the first forwarding plane and the first peer device, and obtain the private network label and public network of the PW established between the VSI of the VM of the first forwarding plane and the VSI of the first peer device through the ldp session Tunnel label.
具体的，当第一对端设备为第二转发面VM时，第一转发面VM与第二转发面VM间建立标签分发协议会话ldp session可以包括：Specifically, when the first peer device is the second forwarding plane VM, establishing a label distribution protocol session ldp session between the first forwarding plane VM and the second forwarding plane VM may include:
在第一转发面VM和第二转发面VM上分别运行标签分发协议LDP进程，通过内部网关协议路由互相学习对方的回环网络协议IP地址、内联口IP地址、以及外联口IP地址，将第一转发面VM的内联口IP地址和第二转发面VM的内联口IP地址作为传输transport地址建立ldp session。Run the label distribution protocol LDP process on the first forwarding plane VM and the second forwarding plane VM respectively, and learn each other's loopback network protocol IP address, inline port IP address, and external port IP address through interior gateway protocol routing. The IP address of the inline port of the VM of the first forwarding plane and the IP address of the inline port of the VM of the second forwarding plane are used as transport transport addresses to establish an ldp session.
当第一对端设备为第二转发面VM时，服务器还可以包括：主控VM，第一转发面VM与第二转发面VM间建立标签分发协议会话ldp session可以包括：When the first peer device is the second forwarding plane VM, the server may further include: a master control VM, and establishing a label distribution protocol session ldp session between the first forwarding plane VM and the second forwarding plane VM may include:
在主控VM上运行LDP进程，通过内部网关协议IGP路由服务器中每个转发面VM的回环IP地址、内联口IP地址以及外联口IP地址，主控VM将第一转发面VM的内联口IP地址和第二转发面VM的内联口IP地址作为传输transport地址建立ldp session。Run the LDP process on the master VM, and use the loopback IP address, inline port IP address, and external port IP address of each forwarding plane VM in the IGP routing server. The IP address of the inline port and the IP address of the inline port of the VM on the second forwarding plane are used as the transport address to establish an ldp session.
作为发送VPLS报文的逆过程，第一转发面VM还可以接收VPLS报文，具体实现如下：As an inverse process of sending VPLS packets, the VM on the first forwarding plane can also receive VPLS packets. The specific implementation is as follows:
第一转发面VM接收第三用户发送的包含：目的接口的MAC地址、第二对端设备的MAC地址、第二公网隧道标签、第二私网标签、目的用户的MAC地址、第三用户的MAC地址、第三用户所在虚拟局域网的第二vlan标识以及负载的第二VPLS报文，第三用户与第二对端设备连接，第二对端设备的VSI和第一转发面VM的VSI属于VPLS网络，第二私网标签和第二公网隧道标签为：第一转发面VM的VSI与第二对端设备的VSI间建立的PW的私网标签和公网隧道标签；The first forwarding plane VM receives the data sent by the third user and includes: the MAC address of the destination interface, the MAC address of the second peer device, the second public network tunnel label, the second private network label, the destination user's MAC address, and the third user's MAC address. MAC address, the second vlan identifier of the virtual local area network where the third user is located, and the second VPLS packet loaded, the third user is connected to the second peer device, the VSI of the second peer device and the VSI of the first forwarding plane VM Belonging to the VPLS network, the second private network label and the second public network tunnel label are: the private network label and public network tunnel label of the PW established between the VSI of the first forwarding plane VM and the VSI of the second peer device;
查询MAC地址表，若所述MAC地址表中目的接口的MAC地址与所述第一转发面VM的接口的MAC地址相同，则确定所述第二VPLS报文为发送给自身的VPLS报文；query the MAC address table, if the MAC address of the destination interface in the MAC address table is the same as the MAC address of the interface of the first forwarding plane VM, then determine that the second VPLS packet is a VPLS packet sent to itself;
解封装第二VPLS报文，若发现所述第二VPLS报文为MPLS报文，则查询入标签映射ilm表，找到与第二公网隧道标签对应的第一ilm表项，若第一ilm表项中的动作为下一跳pop，则依次弹出第二VPLS报文中的第二公网隧道标签、以及第二私网标签，在ilm表中查询与第二私网标签对应的第二ilm表项，获取第二ilm表项中与第二私网标签相对应的第一转发面VM的VSI；Decapsulate the second VPLS packet, and if the second VPLS packet is found to be an MPLS packet, query the label mapping ilm table to find the first ilm entry corresponding to the second public network tunnel label, if the first ilm The action in the entry is the next hop pop, then the second public network tunnel label and the second private network label in the second VPLS packet are popped in sequence, and the second private network label corresponding to the second private network label is queried in the ilm table. ilm entry, to obtain the VSI of the first forwarding plane VM corresponding to the second private network label in the second ilm entry;
查询与VSI对应的MAC转发表，获取与目的用户的MAC地址对应的MAC转发表项，与目的用户的MAC地址对应的MAC转发表项包含：目的用户的MAC地址、与目的用户的MAC地址对应的第一转发面VM上的第二接口、目的用户所在的虚拟局域网的第三vlan标识；Query the MAC forwarding table corresponding to the VSI, and obtain the MAC forwarding table entry corresponding to the MAC address of the destination user. The MAC forwarding table entry corresponding to the MAC address of the destination user includes: the MAC address of the destination user and the MAC address corresponding to the destination user. The second interface on the first forwarding plane VM, the third vlan identifier of the virtual local area network where the destination user is located;
将弹出第二私网标签后的第二VPLS报文中的第二vlan标识替换为第三vlan标识，将替换后的报文通过第二接口发往目的用户。The second vlan identifier in the second VPLS packet after the second private network label is popped is replaced with the third vlan identifier, and the replaced packet is sent to the destination user through the second interface.
如此，通过自身VSI与对端设备VSI间建立的通道，实现VPLS报文，并对接收到的VPLS报文进行处理。In this way, the VPLS packet is implemented through the channel established between the VSI of the self and the VSI of the peer device, and the received VPLS packet is processed.
第二方面，本发明实施例还提供一种服务器，该服务器可以包括：至少一个转发面虚拟机VM，每个转发面VM连接至少一个用户，每个转发面VM上设置一个虚拟转发实例VSI，所述至少一个转发面VM上的VSI属于同一虚拟专用局域网业务VPLS网络，所述至少一个转发面VM中的任一转发面VM包括：In a second aspect, an embodiment of the present invention further provides a server, where the server may include: at least one forwarding plane virtual machine VM, each forwarding plane VM is connected to at least one user, and each forwarding plane VM is set with a virtual forwarding instance VSI, The VSI on the at least one forwarding plane VM belongs to the same virtual private local area network service VPLS network, and any forwarding plane VM in the at least one forwarding plane VM includes:
接收单元，用于接收第一用户发送的包含：所述第一用户的媒体接入层MAC地址、第二用户的MAC地址、所述第一用户所在的第一虚拟局域网vlan标识的报文，所述第一用户与所述转发面VM连接，所述第二用户与第一对端设备连接，所述转发面VM的VSI与所述第一对端设备的VSI属于所述VPLS网络；a receiving unit, configured to receive a packet sent by the first user and including: the MAC address of the media access layer of the first user, the MAC address of the second user, and the identity of the first virtual local area network vlan where the first user is located, The first user is connected to the forwarding plane VM, the second user is connected to a first peer device, and the VSI of the forwarding plane VM and the VSI of the first peer device belong to the VPLS network;
查询单元，用于查询与所述VSI对应的媒体接入控制MAC转发表，获取所述MAC转发表中与所述第二用户的MAC地址对应的MAC转发表项，所述与所述第二用户的MAC地址对应的MAC转发表项包含：所述第二用户的MAC地址、与所述第二用户的MAC地址对应的所述转发面VM上的第一接口；a query unit, configured to query the MAC forwarding table corresponding to the VSI, and obtain the MAC forwarding table entry corresponding to the MAC address of the second user in the MAC forwarding table, and the MAC forwarding table entry corresponding to the MAC address of the second user is obtained. The MAC forwarding table entry corresponding to the MAC address of the user includes: the MAC address of the second user, and the first interface on the forwarding plane VM corresponding to the MAC address of the second user;
封装单元，用于将所述接收单元接收到的报文封装上第一私网标签和第一公网隧道标签，所述第一私网标签为：所述转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的私网标签，所述第一公网隧道标签为：所述转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的公网隧道标签；an encapsulation unit, configured to encapsulate a first private network label and a first public network tunnel label on the message received by the receiving unit, where the first private network label is: the VSI of the forwarding plane VM and the first private network label The private network label of the pseudowire PW established between the VSIs of the pair of end devices, and the first public network tunnel label is: the pseudowire PW established between the VSI of the forwarding plane VM and the VSI of the first opposite end device public network tunnel label;
以及，将封装标签后的报文封装上所述接口的MAC地址、所述第一对端设备的MAC地址，形成第一VPLS报文；and, encapsulating the message after the encapsulation label with the MAC address of the interface and the MAC address of the first peer device to form a first VPLS message;
发送单元，用于将封装单元封装后的第一VPLS报文通过所述第一接口发送出去。The sending unit is configured to send the first VPLS packet encapsulated by the encapsulating unit through the first interface.
具体的，上述接收单元、查询单元、封装单元以及发送单元的执行过程可以参照第一方面所述方法中的执行过程，在此不再详细赘述。Specifically, for the execution process of the receiving unit, the query unit, the encapsulation unit, and the sending unit, reference may be made to the execution process in the method described in the first aspect, and details are not described herein again.
需要说明的是，第二方面中接收单元、发送单元可以集成为服务器的通信单元，第二方面中的查询单元、封装单元可以为单独设立的处理器，也可以集成在处理设备的某一个处理器中实现，此外，也可以以程序代码的形式存储于处理设备的存储器中，由处理设备的某一个处理器调用并执行以上查询单元、封装单元的功能。这里所述的处理器可以是一个中央处理器(Central Processing Unit，CPU)，或者是特定集成电路(ApplicationSpecific Integrated Circuit，ASIC)，或者是被配置成实施本发明实施例的一个或多个集成电路。It should be noted that the receiving unit and the sending unit in the second aspect can be integrated as the communication unit of the server, and the query unit and the encapsulating unit in the second aspect can be a processor established separately, or can be integrated in a certain processing device of the processing device. In addition, it can also be stored in the memory of the processing device in the form of program codes, and a certain processor of the processing device calls and executes the functions of the above query unit and encapsulation unit. The processor described here may be a central processing unit (Central Processing Unit, CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits configured to implement embodiments of the present invention .
由上可知，本发明实施例提供一种VPLS报文处理方法及设备，第一转发面VM接收第一用户发送的包含第一用户的MAC地址、第二用户的MAC地址的报文，查询MAC转发表，获取所述MAC转发表中与第二用户的MAC地址对应的MAC转发表项，将所述报文封装上与私网标签和公网隧道标签，将封装后的报文封装上所述第一接口的MAC地址、第一对端设备的接口的MAC地址，形成VPLS报文，将封装后的VPLS报文通过所述第一接口发送出去。如此，可以通过在每个VM中均配置VSI，使每个VM均可以通过自身的VSI与其他设备(如远端PE设备或其他VM)直接进行报文转发，避免了转发流量的内部迂回，提高了流量转发的性能，同时，不再占用VM间管理接口带宽，提高了采用NFV技术的设备的健壮性。It can be seen from the above that the embodiments of the present invention provide a method and device for processing a VPLS packet. The first forwarding plane VM receives the packet sent by the first user and contains the MAC address of the first user and the MAC address of the second user, and queries the MAC address. A forwarding table, obtains a MAC forwarding table entry corresponding to the MAC address of the second user in the MAC forwarding table, encapsulates the message with a private network label and a public network tunnel label, and encapsulates the encapsulated message on the The MAC address of the first interface and the MAC address of the interface of the first peer device are formed to form a VPLS packet, and the encapsulated VPLS packet is sent out through the first interface. In this way, by configuring a VSI in each VM, each VM can directly forward packets with other devices (such as remote PE devices or other VMs) through its own VSI, avoiding the internal detour of forwarding traffic. The performance of traffic forwarding is improved, and at the same time, the bandwidth of the management interface between VMs is no longer occupied, and the robustness of devices using NFV technology is improved.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.
图1为现有VPLS组网示意图；Figure 1 is a schematic diagram of an existing VPLS network;
图2为现有采用VNF技术的VPLS组网示意图；FIG. 2 is a schematic diagram of an existing VPLS network using VNF technology;
图3为本发明实施例提供的采用VNF技术的VPLS组网示意图；3 is a schematic diagram of a VPLS networking using VNF technology provided by an embodiment of the present invention;
图4为本发明实施例提供的一种服务器的结构图；4 is a structural diagram of a server according to an embodiment of the present invention;
图5为本发明实施例提供的一种VPLS报文处理方法的流程图；5 is a flowchart of a method for processing a VPLS packet according to an embodiment of the present invention;
图6为本发明实施例提供的采用VNF技术的VPLS组网示意图；6 is a schematic diagram of a VPLS networking using VNF technology provided by an embodiment of the present invention;
图7为本发明实施例提供的一种VPLS报文处理方法的流程图；7 is a flowchart of a method for processing a VPLS packet according to an embodiment of the present invention;
图8为本发明实施例提供的一种服务器的结构图。FIG. 8 is a structural diagram of a server according to an embodiment of the present invention.
具体实施方式Detailed ways
本发明的核心思想是：在采用VNF技术的设备中将原有的一个VSI进行VSI多实例化，在每个VM中均设置有VSI，通过本VM的VSI与远端PE的VSI建立PW，在建立的PW上传输VPLS报文，不再通过其他VM来转发本VM上的报文，避免了流量转发的迂回。The core idea of the present invention is as follows: in a device using the VNF technology, an original VSI is multi-instantiated by VSI, a VSI is set in each VM, and a PW is established through the VSI of the VM and the VSI of the remote PE, VPLS packets are transmitted on the established PW, and packets on this VM are no longer forwarded through other VMs, which avoids the detour of traffic forwarding.
下面结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
需要理解的是，在本发明的描述中，术语“第一”、“第二”、“另一”等指示的系统或元件为基于实施例描述的具有一定功能的系统或元件，仅是为了便于描述本发明和简化描述，而不是指示或暗示所指的系统或元件必须有此命名，因此不能理解为对本发明的限制。It should be understood that, in the description of the present invention, the system or element indicated by the terms "first", "second", "another", etc. is a system or element with a certain function described based on the embodiments, only for the purpose of It is for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have this name, and therefore should not be construed as a limitation of the present invention.
为了便于实施例的描述，下面先简单介绍一下本发明实施例的应用场景。参见图3，为本发明实施例提供的采用VNF技术的VPLS组网示意图，如图3所示，该VPLS组网可以包含：采用VNF技术的服务器10、PE设备20和CE设备30，每个服务器10、PE设备20可以连接一个或多个CE设备30，每个CE设备30和其下挂的大量的个人电脑(Private Computer，PC)、用户终端和服务器等设备可以通过各自连接的设备接收来自同一个虚拟专用网络(VirtualPrivate Network，VPN)的报文，这个VPN称为一个VPLS网络。In order to facilitate the description of the embodiments, the following briefly introduces application scenarios of the embodiments of the present invention. Referring to FIG. 3, it is a schematic diagram of a VPLS networking using VNF technology provided by an embodiment of the present invention. As shown in FIG. 3, the VPLS networking may include: a server 10 using VNF technology, a PE device 20, and a CE device 30, each The server 10 and the PE device 20 can be connected to one or more CE devices 30, and each CE device 30 and a large number of personal computers (Private Computers, PCs), user terminals, servers and other devices attached to it can be received by the respective connected devices Packets from the same virtual private network (Virtual Private Network, VPN), this VPN is called a VPLS network.
在如3所示的VPLS组网中，服务器10可以为PE设备，譬如交换机等，它可以借助VNF技术，采用大容量服务器(X86服务器)来实现，在该服务器10上可以配置多个虚拟机VM101，如：控制面VM102和多个转发面VM101，每个转发面VM101可以通过CE设备30与多个用户连接，每个转发面VM101上可以配置一个VSI，与转发面VM101连接的用户可以与转发面VM101上的VSI建立相对应的绑定关系，VM101上的VSI可以与远端的任一PE设备20上的VSI建立伪线PW，用户可以通过该PW与远端PE设备20进行VPLS报文转发。需要说明的是，在图3所示的VPLS组网中，当服务器10与远端PE设备20间的距离比较远时，可以在二者之间设置多个中心节点P(如交换设备)来将VPLS报文转发至对端设备。In the VPLS networking shown in Figure 3, the server 10 can be a PE device, such as a switch, etc., which can be implemented by using a large-capacity server (X86 server) with the help of VNF technology, and multiple virtual machines can be configured on the server 10. VM101, such as: control plane VM102 and multiple forwarding plane VM101, each forwarding plane VM101 can be connected to multiple users through CE device 30, one VSI can be configured on each forwarding plane VM101, and users connected to the forwarding plane VM101 can communicate with The VSI on the forwarding plane VM101 establishes a corresponding binding relationship. The VSI on VM101 can establish a pseudowire PW with the VSI on any remote PE device 20, and the user can perform VPLS reporting with the remote PE device 20 through the PW. Text forwarding. It should be noted that, in the VPLS networking shown in FIG. 3, when the distance between the server 10 and the remote PE device 20 is relatively long, multiple central nodes P (such as switching devices) can be set between them to Forward VPLS packets to the peer device.
其中，在VPLS网络中，PE设备20可以表示服务提供商边缘设备，譬如交换机等，使用VSI进行VPLS转发。Wherein, in the VPLS network, the PE device 20 may represent an edge device of a service provider, such as a switch, etc., which uses VSI to perform VPLS forwarding.
CE设备20可以表示客户网络边缘设备，其可以通过接入电路(Access Circuit，AC)与服务器10、PE设备20等设备连接，在CE设备30下可以下挂属于同一VPN网络的多个用户设备，如：个人电脑、移动终端等设备。The CE device 20 may represent a customer network edge device, which can be connected to the server 10, the PE device 20 and other devices through an access circuit (AC), and multiple user devices belonging to the same VPN network can be attached to the CE device 30 , such as: personal computers, mobile terminals and other equipment.
VSI表示虚链路(即伪线PW)，是为一对PE设备之间、VM与PE设备之间每一个提供的服务所建立起来的点到点连接，多条虚链路可以承载在一对PE设备之间、VM与PE设备之间的一个公共网络隧道内，VPLS网络中任意两个PE、服务器之间都存在这样的隧道，是一个全连接网络。VSI stands for Virtual Link (Pseudo-Wire PW), which is a point-to-point connection established for each service provided between a pair of PE devices and between VM and PE devices. Multiple virtual links can be carried on one For a public network tunnel between PE devices and between VM and PE devices, such a tunnel exists between any two PEs and servers in the VPLS network, which is a fully connected network.
虚拟机VM101可以是通过专用的虚拟化软件管理，如VMware，在一台物理服务器上虚拟出的具有自己的操作系统、应用程序、虚拟的硬件环境(包括：虚拟中央处理器(Central Processing Unit，简称为CPU)、内存、输入输出(Input Output，IO)设备、虚拟交换机等)的服务器，可以为物理服务器中的主控板或者线路板。其中，控制面VM102主要控制转发面VM101的流量转发、参数设置等工作，转发面VM101可以作为一个虚拟路由器，实现VPLS报文的转发。The virtual machine VM101 can be managed by a dedicated virtualization software, such as VMware, and virtualized on a physical server with its own operating system, application programs, and a virtual hardware environment (including: a virtual central processing unit (Central Processing Unit, A server that is referred to as CPU), memory, input and output (Input Output, IO) device, virtual switch, etc.) may be a main control board or a circuit board in a physical server. The control plane VM102 mainly controls the traffic forwarding and parameter setting of the forwarding plane VM101, and the forwarding plane VM101 can be used as a virtual router to realize the forwarding of VPLS packets.
具体的，如图4所示，本发明实施例提供的采用VNF技术的服务器10可以包括：硬件层、运行在该硬件层之上的控制面VM102、以及至少一个转发面VM101，该硬件层可以包括：处理器1011、通信接口1012、存储器1013、至少一个通信总线1014，通信总线1014用于实现这些装置之间的连接和相互通信；控制面VM102作为管理层，用以完成硬件资源的管理、分配；为转发面VM101呈现虚拟硬件平台，实现转发面VM101的调度和隔离。Specifically, as shown in FIG. 4 , the server 10 using the VNF technology provided by the embodiment of the present invention may include: a hardware layer, a control plane VM102 running on the hardware layer, and at least one forwarding plane VM101, and the hardware layer may It includes: a processor 1011, a communication interface 1012, a memory 1013, and at least one communication bus 1014. The communication bus 1014 is used to realize the connection and mutual communication between these devices; the control plane VM102 is used as a management layer to complete the management of hardware resources, Allocate; present a virtual hardware platform for the forwarding plane VM101 to implement scheduling and isolation of the forwarding plane VM101.
处理器1011可能是一个中央处理器(Central Processing Unit，简称为CPU)，也可以是特定集成电路(Application Specific Integrated Circuit，ASIC)，或者是被配置成实施本发明实施例的一个或多个集成电路，例如：一个或多个微处理器(Digital SignalProcessor，DSP)，或，一个或者多个现场可编程门阵列(Field Programmable Gate Array，FPGA)。The processor 1011 may be a central processing unit (Central Processing Unit, CPU for short), a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or is configured to implement one or more integrated circuits in the embodiments of the present invention Circuit, for example: one or more microprocessors (Digital SignalProcessor, DSP), or, one or more Field Programmable Gate Array (Field Programmable Gate Array, FPGA).
通信接口1012可能是远程直接数据存取(Remote Direct Memory Access，RDMA)网卡，可用于与外部网元之间进行数据交互。The communication interface 1012 may be a remote direct data access (Remote Direct Memory Access, RDMA) network card, which can be used for data interaction with external network elements.
存储器1013，可以是易失性存储器(Volatile Memory)，例如随机存取存储器(Random-Access Memory，RAM)；或者非易失性存储器(Non-Volatile Memory)，例如只读存储器(Read-Only Memory，ROM)，快闪存储器(Flash Memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的存储器的组合。The memory 1013 may be a volatile memory (Volatile Memory), such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory) , ROM), flash memory (Flash Memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); or a combination of the above types of memory.
通信总线1014可以分为地址总线、数据总线、控制总线等，可以是工业标准体系结构(Industry Standard Architecture，ISA)总线、外部设备互连(Peripheral ComponentInterconnect，PCI)总线或扩展工业标准体系结构(Extended Industry StandardArchitecture，EISA)总线等。为便于表示，图4中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。The communication bus 1014 can be divided into an address bus, a data bus, a control bus, etc., and can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (Extended Industry Standard Architecture) bus. Industry Standard Architecture, EISA) bus, etc. For ease of presentation, only one thick line is used in FIG. 4, but it does not mean that there is only one bus or one type of bus.
可选的，处理器1011可以通过专用的虚拟化软件管理，在服务器10上虚拟出多个转发面VM101，并为每个转发面VM101配置至少一个VSI，其中，转发面VM101可用于将用户的报文通过自身的VSI转发至与PE设备20连接的其他用户，或者，接收其他用户的报文，将接收到的报文转发至与自身连接的用户，具体实现如下：Optionally, the processor 1011 can be managed by dedicated virtualization software, virtualize multiple forwarding plane VMs 101 on the server 10, and configure at least one VSI for each forwarding plane VM101, wherein the forwarding plane VM101 can be used to The packet is forwarded to other users connected to the PE device 20 through its own VSI, or, it receives packets from other users, and forwards the received packets to the user connected to itself. The specific implementation is as follows:
转发面VM101接收第一用户发送的包含：第一用户的MAC地址、第二用户的MAC地址、第一虚拟局域网(virtual local network，vlan)标识的报文后，可以查询与VSI对应的媒体接入控制MAC转发表，获取MAC转发表中与第二用户的MAC地址对应的MAC转发表项，该MAC转发表项可以包含与第二用户的MAC地址对应的第一接口；After the forwarding plane VM101 receives the packet sent by the first user and includes: the MAC address of the first user, the MAC address of the second user, and the identifier of the first virtual local network (vlan), it can query the media interface corresponding to the VSI. Controlling the MAC forwarding table, and obtaining the MAC forwarding table entry corresponding to the MAC address of the second user in the MAC forwarding table, where the MAC forwarding table entry may include the first interface corresponding to the MAC address of the second user;
转发面VM101将接收到的报文先封装上与VSI对应的第一私网标签和第一公网隧道标签，然后再将封装后的报文封装上第一接口的MAC地址、对端设备的接口的MAC地址，形成第一VPLS报文，将封装后的第一VPLS报文通过第一接口发送出去。The forwarding plane VM101 first encapsulates the received packet with the first private network label and the first public network tunnel label corresponding to the VSI, and then encapsulates the encapsulated packet with the MAC address of the first interface and the address of the peer device. The MAC address of the interface forms the first VPLS packet, and the encapsulated first VPLS packet is sent out through the first interface.
其中，第一私网标签和第一公网隧道标签为：转发面VM101的VSI与对端设备的VSI间建立的伪线PW的私网标签和公网隧道标签，且转发面VM101的VSI与对端设备的VSI属于同一个VPLS网络。The first private network label and the first public network tunnel label are: the private network label and public network tunnel label of the pseudowire PW established between the VSI of the forwarding plane VM101 and the VSI of the peer device, and the VSI of the forwarding plane VM101 is the same as the The VSI of the peer device belongs to the same VPLS network.
需要说明的是，上述对端设备可以为不同与该转发面VM101所在的服务器的远端的PE设备，也可以为与该转发面VM101同在一个服务器的其他转发面VM或者不同于该服务器的其他服务器上的转发面VM；当对端设备为PE设备时，可以通过转发面VM101上的外联口向PE设备发送报文，当对端设备为其他转发面VM时，可以通过该转发面VM101的内联口向其他转发面VM发送报文。It should be noted that the above-mentioned peer device may be a remote PE device that is different from the server where the forwarding plane VM101 is located, or may be other forwarding plane VMs on the same server as the forwarding plane VM101 or different from the server. The forwarding plane VM on other servers; when the peer device is a PE device, it can send packets to the PE device through the external interface on the VM101 of the forwarding plane; when the peer device is another forwarding plane VM, it can send packets through the forwarding plane The inline port of VM101 sends packets to other forwarding plane VMs.
作为报文发送过程的逆过程，当转发面VM101接收到第三用户发送的包含：目的接口的MAC地址、所述第二对端设备的MAC地址、第二公网隧道标签、第二私网标签、目的用户的MAC地址、所述第三用户的MAC地址、第二虚拟局域网标识以及负载的第二VPLS报文时，转发面VM101可以解封装该第二VPLS报文；所述第三用户与第二对端设备连接，所述第二对端设备的VSI和转发面VM101的VSI属于同一VPLS网络，所述第二私网标签为：转发面VM101的VSI与所述第二对端设备的VSI间建立的PW的私网标签，所述第二公网隧道标签为：转发面VM101的VSI与所述第二对端设备的VSI间建立的PW的公网隧道标签；As the inverse process of the message sending process, when the forwarding plane VM101 receives the message sent by the third user including: the MAC address of the destination interface, the MAC address of the second peer device, the second public network tunnel label, the second private network When the label, the MAC address of the destination user, the MAC address of the third user, the second virtual local area network identifier, and the second VPLS packet loaded, the forwarding plane VM101 can decapsulate the second VPLS packet; the third user Connected to the second peer device, the VSI of the second peer device and the VSI of the forwarding plane VM101 belong to the same VPLS network, and the second private network label is: the VSI of the forwarding plane VM101 and the second peer device The private network label of the PW established between the VSIs, the second public network tunnel label is: the public network tunnel label of the PW established between the VSI of the forwarding plane VM101 and the VSI of the second peer device;
查询MAC地址表，若发现目的接口的MAC地址与转发面VM101的接口的MAC地址相同，则确定所述第二VPLS报文为发送给自身的VPLS报文；Query the MAC address table, if it is found that the MAC address of the destination interface is the same as the MAC address of the interface of the forwarding plane VM101, then determine that the second VPLS packet is a VPLS packet sent to itself;
解封装所述第二VPLS报文，若发现所述第二VPLS报文为MPLS报文，则查询入标签映射ilm表，找到与第二公网隧道标签对应的第一ilm表项，若ilm表项中的动作为下一跳pop，则依次弹出该VPLS报文中的第二公网隧道标签、以及第二私网标签；Decapsulate the second VPLS packet, and if the second VPLS packet is found to be an MPLS packet, query the ilm table for label mapping to find the first ilm entry corresponding to the second public network tunnel label, if ilm The action in the entry is the next hop pop, then pop the second public network tunnel label and the second private network label in the VPLS packet in sequence;
在ilm表中查询与第二私网标签对应的第二ilm表项，获取第二ilm表项中与第二私网标签相对应的VSI，查询与所述VSI对应的MAC转发表，获取与目的用户的MAC地址对应的包含：与目的用户的MAC地址对应的第二接口，与目的用户的MAC地址对应的第三虚拟局域网标识的MAC转发表项，将弹出第二私网标签后的第二VPLS报文中的第二虚拟局域网标识替换为第三虚拟局域网标识，将替换后的报文通过第二接口发送出去。Query the second ilm entry corresponding to the second private network label in the ilm table, obtain the VSI corresponding to the second private network label in the second ilm entry, query the MAC forwarding table corresponding to the VSI, and obtain the The MAC address corresponding to the destination user includes: the second interface corresponding to the MAC address of the destination user, the MAC forwarding entry of the third virtual local area network identifier corresponding to the MAC address of the destination user, and the second interface after the second private network label will pop up. The second virtual local area network identifier in the second VPLS packet is replaced with the third virtual local area network identifier, and the replaced packet is sent out through the second interface.
如此，在采用VNF技术的服务器中每个VM上均配置VSI，使每个VM均可以通过自身的VSI与其他设备(如远端服务器或其他VM)直接进行报文转发，避免了转发流量的内部迂回，提高了流量转发的性能，同时，不再占用VM间管理接口带宽，提高了采用NFV技术的设备的健壮性。In this way, a VSI is configured on each VM in a server using VNF technology, so that each VM can directly forward packets with other devices (such as remote servers or other VMs) through its own VSI, avoiding traffic forwarding. The internal detour improves the performance of traffic forwarding, and at the same time, it no longer occupies the bandwidth of the management interface between VMs, which improves the robustness of devices using NFV technology.
为了便于描述，以下实施例一以步骤的形式示出并详细描述了本发明提供的VPLS报文处理过程，其中，示出的步骤也可以在VM中执行，也可以在一组可执行指令的计算机系统中执行。此外，虽然在图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。For the convenience of description, the following embodiment 1 shows and describes in detail the VPLS packet processing process provided by the present invention in the form of steps, wherein the shown steps can also be executed in the VM, or can be executed in a set of executable instructions. executed in the computer system. Furthermore, although a logical order is shown in the figures, in some cases, steps shown or described may be performed in an order different from that herein.
实施例一Example 1
图5为本发明实施例提供的一种报文处理方法，应用于服务器，该服务器可以为图3中采用VNF技术的设备，该服务器可以包括：至少一个转发面VM，每个转发面VM可预先配置有回环IP地址、内联口IP地址以及外联口IP地址，内联口为服务器中转发面VM之间传输报文的接口，外联口可以为服务器中的转发面VM与远端设备(PE设备)之间传输报文的接口，每个转发面VM可以连接至少一个用户，且这些用户可以如图3所示直接下挂在CE设备下面，由CE设备通过AC链路与该转发面VM连接，每个转发面VM上可设置一个VSI，且所有转发面VM上的VSI可以属于同一个VPLS网络；如图5所示，所述报文处理方法可以包括以下步骤：FIG. 5 is a packet processing method according to an embodiment of the present invention, which is applied to a server. The server may be the device using the VNF technology in FIG. 3 . The server may include: at least one forwarding plane VM, and each forwarding plane VM may be The loopback IP address, the IP address of the inline port, and the IP address of the external port are pre-configured. The inline port is the interface for transmitting packets between the forwarding plane VMs in the server, and the external port can be the forwarding plane VM in the server and the remote port. An interface for transmitting packets between devices (PE devices). Each forwarding plane VM can be connected to at least one user, and these users can be directly attached to the CE device as shown in Figure 3. The CE device communicates with the device through the AC link. The forwarding plane VMs are connected. One VSI can be set on each forwarding plane VM, and the VSIs on all the forwarding plane VMs can belong to the same VPLS network; as shown in FIG. 5 , the packet processing method can include the following steps:
S101：第一转发面VM接收第一用户发送的报文，所述报文包含：第一用户的MAC地址、第二用户的MAC地址、第一vlan标识。S101: The first forwarding plane VM receives a packet sent by the first user, where the packet includes: the MAC address of the first user, the MAC address of the second user, and the first vlan identifier.
其中，第一转发面VM可以为服务器中的任一转发面VM。The first forwarding plane VM may be any forwarding plane VM in the server.
第一用户可以为通过CE设备与第一转发面VM连接的任一用户，为发送报文的源端，且该第一用户与第一转发面VM的VSI相绑定，即第一用户的报文可以通过与第一转发面VM的VSI对应的PW发送出去。The first user can be any user connected to the VM on the first forwarding plane through the CE device, and is the source end of sending packets, and the first user is bound to the VSI of the VM on the first forwarding plane, that is, the first user's The packet can be sent out through the PW corresponding to the VSI of the first forwarding plane VM.
第一vlan标识为第一用户所处虚拟局域网的标识，用于确定第一用户与第一转发面VM之间连接的接口。The first vlan identifier is an identifier of the virtual local area network where the first user is located, and is used to determine an interface connected between the first user and the first forwarding plane VM.
第二用户可以为通过CE设备与第一对端设备连接的任一用户，为接收报文的目的端，所述第二用户与第一对端设备的VSI关联，即第二用户的报文可以通过与第一对端设备的VSI对应的PW收发VPLS报文。The second user can be any user connected to the first peer device through the CE device, and is the destination of receiving packets. The second user is associated with the VSI of the first peer device, that is, the second user's message VPLS packets can be sent and received through the PW corresponding to the VSI of the first peer device.
可选的，第一对端设备可以为与不同于服务器的远端设备(如PE设备)，可以为服务器中不同于第一转发面VM的第二转发面VM，即第一转发面VM可以与服务器内部的其他转发面VM之间收发VPLS报文，也可以与远端PE设备之间收发VPLS报文。需要说明的是，第一转发面VM的VSI和第一对端设备的VSI属于同一VPLS网络。Optionally, the first peer device may be a remote device (such as a PE device) that is different from the server, and may be a second forwarding plane VM in the server that is different from the first forwarding plane VM, that is, the first forwarding plane VM may be Send and receive VPLS packets with other forwarding plane VMs inside the server, and also send and receive VPLS packets with remote PE devices. It should be noted that the VSI of the first forwarding plane VM and the VSI of the first peer device belong to the same VPLS network.
可理解的是，本发明实施例所述的用户可以是指该用户手持的用户终端，是一个设备，为了便于描述，本发明将该用户终端简称为用户。It is understandable that the user in this embodiment of the present invention may refer to a user terminal held by the user, which is a device. For convenience of description, the present invention will simply refer to the user terminal as a user.
S102：第一转发面VM查询与所述VSI对应的MAC转发表，获取与第二用户的MAC地址对应的MAC转发表项，所述MAC转发表项包含：第二用户的MAC地址、与第二用户的MAC地址对应的第一接口。S102: The first forwarding plane VM queries the MAC forwarding table corresponding to the VSI, and obtains a MAC forwarding table entry corresponding to the MAC address of the second user, where the MAC forwarding table entry includes: the MAC address of the second user, and the MAC address of the second user. The first interface corresponding to the MAC address of the second user.
其中，在VPLS网络中，转发面VM或PE设备可以接收VPLS广播域中的广播报文，学习接收到的广播报文中所有本地、远端连接的用户的MAC地址，建立与该转发面VM或PE设备中的VSI对应的MAC转发表。Among them, in the VPLS network, the forwarding plane VM or PE equipment can receive broadcast packets in the VPLS broadcast domain, learn the MAC addresses of all local and remote users in the received broadcast packets, and establish a connection with the forwarding plane VM. Or the MAC forwarding table corresponding to the VSI in the PE device.
MAC转发表可以包含多个MAC转发表项，每个MAC转发表项可以包含：MAC地址、与该MAC地址对应的接口(即传输接口)，当然，在实际应用中，MAC转发表项包括但不限于与MAC地址对应的接口，如果MAC地址是从某个vsi(广播域)学到的，则MAC地址转发表项还可以包含与MAC地址对应的vsi；如果MAC地址是从PW或者从lsp(可以是ldp、te隧道)进来的，则MAC地址转发表项中包含令牌(token)；如果MAC地址是从带虚拟局域网(vlan)的接口进来的，则MAC地址转发表项还可以包含虚拟局域网标识(vlan id)，即MAC地址转发表项除与MAC地址对应的接口之外，还可以根据实际情况包含：与MAC地址对应的VSI标识(vid)、vlan id、token等多个信息。The MAC forwarding table may contain multiple MAC forwarding entries, and each MAC forwarding entry may include: the MAC address and the interface corresponding to the MAC address (ie, the transmission interface). Of course, in practical applications, the MAC forwarding entry includes but Not limited to the interface corresponding to the MAC address, if the MAC address is learned from a certain vsi (broadcast domain), the MAC address forwarding entry can also include the vsi corresponding to the MAC address; if the MAC address is learned from the PW or from the lsp (It can be ldp, te tunnel), the MAC address forwarding entry contains a token (token); if the MAC address comes from an interface with a virtual local area network (vlan), the MAC address forwarding entry can also contain The virtual local area network identifier (vlan id), that is, the MAC address forwarding entry, in addition to the interface corresponding to the MAC address, can also include, according to the actual situation: VSI identifier (vid), vlan id, token and other information corresponding to the MAC address .
例如，下表1为MAC地址转发表，如图1所示，MAC地址111-1所在的MAC地址转发表项可以包含：MAC地址111-1、广播域vsi、虚拟局域网标识10、以及接口G1/0/0，不包含token信息，这表示MAC地址111-1是从广播域vsi学习到的，且该MAC地址是从虚拟局域网10的接口G1/0/0进来的，不是从PW或者lsp链路上进来的。For example, the following table 1 is the MAC address forwarding table. As shown in Figure 1, the MAC address forwarding table entry where the MAC address 111-1 is located may include: the MAC address 111-1, the broadcast domain vsi, the virtual local area network identifier 10, and the interface G1 /0/0, does not contain token information, which means that the MAC address 111-1 is learned from the broadcast domain vsi, and the MAC address comes from the interface G1/0/0 of the virtual local area network 10, not from the PW or lsp incoming on the link.
表1Table 1
需要说明的是，若在MAC地址表中未找到与报文中的目的MAC地址对应的MAC地址，则表示该报文可能是未知单播地址、组播地址或广播地址的以太报文，则可以采用洪泛方式，将该以太报文复制多份，分别从与本地VSI关联的所有接口和PW广播出去。It should be noted that if the MAC address corresponding to the destination MAC address in the packet is not found in the MAC address table, it means that the packet may be an Ethernet packet with an unknown unicast address, multicast address or broadcast address. You can use flooding to copy the Ethernet packet in multiple copies and broadcast it from all interfaces and PWs associated with the local VSI.
S103：第一转发面VM将报文封装上第一私网标签和第一公网隧道标签。S103: The first forwarding plane VM encapsulates the message with a first private network label and a first public network tunnel label.
其中，所述第一私网标签为：所述第一转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的私网标签，所述第一公网隧道标签为：所述第一转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的公网隧道标签。The first private network label is: the private network label of the pseudowire PW established between the VSI of the first forwarding plane VM and the VSI of the first peer device, and the first public network tunnel label is : the public network tunnel label of the pseudowire PW established between the VSI of the first forwarding plane VM and the VSI of the first peer device.
可选的，第一转发面VM可以在第一转发面VM接收第一用户发送的报文之前，与第一对端设备建立标签分发协议会话ldp session；Optionally, the first forwarding plane VM may establish a label distribution protocol session ldp session with the first peer device before the first forwarding plane VM receives the message sent by the first user;
通过建立起来的ldp session，学习第一转发面VM的VSI与第一对端设备的VSI间建立的PW的私网标签和公网隧道标签。Through the established ldp session, learn the private network label and public network tunnel label of the PW established between the VSI of the VM on the first forwarding plane and the VSI of the first peer device.
具体的，在创建VSI、ldp session之后，本端可以给对端分配ldp标签(即私网标签)，同时发送携带有VSI标识的消息给对端，对端接收到该消息后，发现自身VSI的标识与该消息中的标识相同，则与本端之间建立起PW，对建立起来的PW分配公网隧道标签。Specifically, after creating a VSI and ldp session, the local end can assign an ldp label (ie, a private network label) to the opposite end, and at the same time send a message carrying the VSI identifier to the opposite end. After receiving the message, the opposite end finds its own VSI If the identifier is the same as the identifier in the message, a PW is established with the local end, and a public network tunnel label is allocated to the established PW.
可理解的是，当第一对端设备为不同于第一转发面VM所在的服务器的PE设备时，第一转发面VM与PE设备建立的ldp session可以为远端标签分发协议会话remote-ldpsession；It is understandable that when the first peer device is a PE device different from the server where the first forwarding plane VM is located, the ldp session established between the first forwarding plane VM and the PE device may be the remote-ldpsession remote label distribution protocol session. ;
此时，可以在第一转发面VM和PE设备中分别运行LDP进程，通过内部网关协议路由双方互相学习对方的回环IP地址、内联口IP地址、外联口IP地址，由回环IP地址较大的一端将第一转发面VM的外联口IP地址和第二PE的外联口IP地址作为传输transport地址，主动呼叫对端，建立起该remote-ldp session，根据建立起来的remote-ldp session为第一转发面VM的VSI与PE设备的VSI间的PW分配私网标签和公网隧道标签。At this time, the LDP process can be run on the VM and PE devices of the first forwarding plane respectively, and the two sides can learn each other's loopback IP address, inline port IP address, and external port IP address through the internal gateway protocol routing. The larger end uses the IP address of the external interface of the VM on the first forwarding plane and the IP address of the external interface of the second PE as the transport address, calls the opposite end actively, and establishes the remote-ldp session. According to the established remote-ldp The session allocates a private network label and a public network tunnel label to the PW between the VSI of the VM on the first forwarding plane and the VSI of the PE device.
需要说明的是，在本发明实例中，VPLS网络中的转发面VM具有各自的回环(loopback)地址、内联口IP地址、外联口IP地址，在建立ldp-session之前，转发面VM可以将自身的内联口IP地址、外联口IP地址均发布到共网(Internal Gateway Protocol，IGP)路由上，由IGP路由学习每个转发面VM的内联口IP地址、外联口IP地址。It should be noted that, in the example of the present invention, the VM on the forwarding plane in the VPLS network has its own loopback address, IP address of the inline port, and IP address of the external port. Before establishing the ldp-session, the VM on the forwarding plane can Advertise the IP address of the internal port and the IP address of the external port to the internal gateway protocol (IGP) router, and the IGP router learns the IP address of the internal port and the IP address of the external port of each forwarding plane VM. .
当所述第一对端设备为第二转发面VM时，第一转发面VM与第二转发面VM建立的ldp session为本地标签分发协议会话(local-ldp session)，其中，第一转发面VM与第二转发面VM建立local-ldp session可以包括下述两种方式：When the first peer device is the second forwarding plane VM, the ldp session established by the first forwarding plane VM and the second forwarding plane VM is a local label distribution protocol session (local-ldp session), wherein the first forwarding plane VM The VM and the VM on the second forwarding plane can establish a local-ldp session in the following two ways:
一、在第一转发面VM和第二转发面VM上分别运行标签分发协议LDP进程，通过IGP路由互相学习对方的回环网络协议IP地址、内联口IP地址、以及外联口IP地址；1. Run the label distribution protocol LDP process on the first forwarding plane VM and the second forwarding plane VM respectively, and learn each other's loopback network protocol IP address, inline port IP address, and external port IP address through IGP routing;
由回环IP地址较大的一端将所述第一转发面VM的内联口IP地址和所述第二转发面VM的内联口IP地址作为传输transport地址，主动呼叫对端，建立local-ldp session。The end with the larger loopback IP address uses the IP address of the inline port of the VM of the first forwarding plane and the IP address of the inline port of the VM of the second forwarding plane as the transport address, calls the opposite end actively, and establishes local-ldp session.
具体的，第一转发面VM和第二转发面VM需要协商出主从关系，决定谁来发起TCP链接，使用要建立TCP连接的源地址和目的地址比较大小来决定主从，地址大的为主，源地址大于目的地址，则本地为主；源地址小于目的地址，则本地为从。由主设备来发起TCP连接，从设备永远不会主动发起TCP连接。Specifically, the first forwarding plane VM and the second forwarding plane VM need to negotiate a master-slave relationship to decide who initiates the TCP connection. The master-slave is determined by comparing the source address and destination address of the TCP connection to be established. The larger address is Master, if the source address is greater than the destination address, the local is the master; if the source address is less than the destination address, the local is the slave. The master device initiates the TCP connection, and the slave device never initiates the TCP connection actively.
第一转发面VM和第二转发面VM之间建立TCP连接之后，主LSR发起initialization消息，开始协商ldp参数(准备建立ldp session)，协商的参数包括ldp协议版本、标签分发方式、keepalive时间、VPI/VCI/DLCI标签空间范围、是否启用环路避免等参数，协商成功后，就可以建立ldp session。After a TCP connection is established between the first forwarding plane VM and the second forwarding plane VM, the main LSR initiates an initialization message and starts negotiating ldp parameters (preparing to establish an ldp session). The negotiated parameters include the ldp protocol version, label distribution method, keepalive time, VPI/VCI/DLCI label space range, whether to enable loop avoidance and other parameters, after successful negotiation, the ldp session can be established.
二、服务器还可以包括：主控VM；2. The server may also include: a master VM;
在所述主控VM上运行LDP进程，通过IGP路由学习服务器上所有转发面VM的回环IP地址、内联口IP地址以及外联口IP地址；Run the LDP process on the master control VM, and learn the loopback IP addresses, inline port IP addresses and external port IP addresses of all forwarding plane VMs on the server through IGP routing;
主控VM将所述第一转发面VM的内联口IP地址和所述第二转发面VM的内联口IP地址作为传输transport地址建立该local-ldp session。The master control VM establishes the local-ldp session by using the inline port IP address of the first forwarding plane VM and the inline port IP address of the second forwarding plane VM as transport addresses.
S104：第一转发面VM将封装后的报文封装上第一接口的MAC地址、第一对端设备的接口的MAC地址，形成第一VPLS报文，将第一VPLS报文通过第一接口发送出去。S104: The first forwarding plane VM encapsulates the encapsulated packet with the MAC address of the first interface and the MAC address of the interface of the first peer device to form a first VPLS packet, and transmits the first VPLS packet through the first interface send out.
其中，第一对端设备的接口的MAC地址可以从下一跳标记转发条目(nhlfe)表中查找到，所述nhlfe表可以包含至少一个nhlfe表项，每个nhlfe表项可以包含：token、公网隧道标签、私网隧道标签以及目的MAC地址。Wherein, the MAC address of the interface of the first peer device can be found from the next hop mark forwarding entry (nhlfe) table, the nhlfe table can include at least one nhlfe entry, and each nhlfe entry can include: token, Public network tunnel label, private network tunnel label, and destination MAC address.
具体的，可以查找在MAC地址转发表项中查看与第二用户的MAC地址相对应的token，然后根据查到的token值，在nhlfe表中查找与该token值对应的nhlfe表项，在该nhlfe表项中找到目的MAC地址作为第一对端设备的接口的MAC地址。Specifically, the token corresponding to the MAC address of the second user can be searched in the MAC address forwarding entry, and then the nhlfe entry corresponding to the token value is searched in the nhlfe table according to the found token value. The destination MAC address is found in the nhlfe entry as the MAC address of the interface of the first peer device.
进一步可选的，作为发送VPLS报文的逆过程，当第一转发面VM接收到VPLS报文时，所述方法还可以包括：Further optionally, as the inverse process of sending the VPLS packet, when the first forwarding plane VM receives the VPLS packet, the method may further include:
第一转发面VM接收第三用户发送的第二VPLS报文，所述第三用户与第二对端设备连接，所述第二对端设备的VSI和所述第一转发面VM的VSI属于所述VPLS网络，所述第二VPLS报文包含：目的接口的MAC地址、所述第二对端设备的MAC地址、第二公网隧道标签、第二私网标签、目的用户的MAC地址、所述第三用户的MAC地址、所述第三用户所在虚拟局域网的第二vlan标识以及负载，所述第二私网标签为：所述第一转发面VM的VSI与所述第二对端设备的VSI间建立的PW的私网标签，所述第二公网隧道标签为：所述第一转发面VM的VSI与所述第二对端设备的VSI间建立的PW的公网隧道标签；The first forwarding plane VM receives the second VPLS packet sent by a third user, the third user is connected to the second peer device, and the VSI of the second peer device and the VSI of the first forwarding plane VM belong to In the VPLS network, the second VPLS packet includes: the MAC address of the destination interface, the MAC address of the second peer device, the second public network tunnel label, the second private network label, the MAC address of the destination user, The MAC address of the third user, the second vlan identifier and load of the virtual local area network where the third user is located, and the second private network label is: the VSI of the first forwarding plane VM and the second peer The private network label of the PW established between the VSIs of the device, and the second public network tunnel label is: the public network tunnel label of the PW established between the VSI of the first forwarding plane VM and the VSI of the second peer device ;
所述第一转发面VM查询MAC地址表，若所述目的接口的MAC地址与所述第一转发面VM的接口的MAC地址相同，则确定所述第二VPLS报文为发送给自身的VPLS报文；The first forwarding plane VM queries the MAC address table, and if the MAC address of the destination interface is the same as the MAC address of the interface of the first forwarding plane VM, it is determined that the second VPLS packet is the VPLS sent to itself message;
解封装所述第二VPLS报文，若发现所述第二VPLS报文为MPLS报文，则查询入标签映射ilm表，找到与所述第二公网隧道标签对应的第一ilm表项，若所述第一ilm表项中的动作为下一跳pop，则依次弹出所述第二VPLS报文中的第二公网隧道标签、以及第二私网标签，在所述ilm表中查询与所述第二私网标签对应的第二ilm表项，获取所述第二ilm表项中与所述第二私网标签相对应的所述第一转发面VM的VSI；Decapsulate the second VPLS packet, and if it is found that the second VPLS packet is an MPLS packet, query the label mapping ilm table to find the first ilm entry corresponding to the second public network tunnel label, If the action in the first ilm entry is the next hop pop, pop the second public network tunnel label and the second private network label in the second VPLS packet in sequence, and query the ilm table the second ilm entry corresponding to the second private network label, obtain the VSI of the first forwarding plane VM corresponding to the second private network label in the second ilm entry;
查询与所述VSI对应的MAC转发表，获取与所述目的用户的MAC地址对应的MAC转发表项，所述与所述目的用户的MAC地址对应的MAC转发表项包含：所述目的用户的MAC地址、与所述目的用户的MAC地址对应的所述第一转发面VM上的第二接口、所述目的用户所在的虚拟局域网的第三vlan标识；Query the MAC forwarding table corresponding to the VSI, and obtain the MAC forwarding table entry corresponding to the MAC address of the target user, where the MAC forwarding table entry corresponding to the MAC address of the target user includes: MAC address, the second interface on the first forwarding plane VM corresponding to the MAC address of the destination user, and the third vlan identifier of the virtual local area network where the destination user is located;
将弹出所述第二私网标签后的第二VPLS报文中的第二vlan标识替换为所述第三vlan标识，将替换后的报文通过所述第二接口发往所述目的用户。The second vlan identifier in the second VPLS packet after the second private network label is popped is replaced with the third vlan identifier, and the replaced packet is sent to the destination user through the second interface.
其中，所述ilm表为：入标签映射(incoming label map)表，在创建ldp session时生成，ilm表可以包含多个ilm表项，每个ilm表项可以包含：标签(公网隧道标签或私网标签)、VSI标识(vid)、token、以及动作(action)。The ilm table is: an incoming label map (incoming label map) table, which is generated when an ldp session is created. The ilm table can contain multiple ilm entries, and each ilm entry can contain: a label (public network tunnel label or private network label), VSI identification (vid), token, and action (action).
下面以图6所示的采用VNF技术的VPLS组网场景，对上述报文方法进行说明：The above packet method is described below with the VPLS networking scenario using the VNF technology shown in Figure 6:
如图6所示，该VPLS组网中采用VNF技术的服务器被虚拟化出一个控制VM和两个转发面VM：VM1，VM2，其与远端的PE设备处于同一个VPLS广播域，转发面VM1、转发面VM2和远端PE设备建立全连接的VPLS PW。其中，在转发面VM1、转发面VM2和远端PE设备之间部署有交换机P，每个转发VM有各自的loopback地址，VM的内联接口ip地址，外联接口ip地址均需要发布到公网IGP路由中，VM的内、外联接口需要使能mpls ldp。As shown in Figure 6, the server using VNF technology in the VPLS network is virtualized into a control VM and two forwarding plane VMs: VM1 and VM2, which are in the same VPLS broadcast domain as the remote PE device, and the forwarding plane is in the same VPLS broadcast domain. VM1, VM2 on the forwarding plane and the remote PE device establish a fully connected VPLS PW. Among them, switch P is deployed between the forwarding plane VM1, the forwarding plane VM2 and the remote PE device, each forwarding VM has its own loopback address, the VM's inline interface ip address, and the external interface ip address need to be published to the public In the network IGP routing, mpls ldp needs to be enabled on the internal and external interfaces of the VM.
转发面VM1的loopback地址1.1.1.1和远端的PE的loopback地址2.2.2.2建立remote-ldp session，转发面VM2的loopback地址3.3.3.3和远端的PE的loopback地址2.2.2.2建立remote-ldp session，VPLS根据不同的remote-ldp seesion分配vpls PW的label；转发VM1，VM2以内联接口作为transport地址建立TCP连接，它们之间通过互相发送hello消息触发local-ldp session的建立。VPLS根据local-ldp sesison分配VM之间的vpls PW的标签(label)。The loopback address 1.1.1.1 of VM1 on the forwarding plane and the loopback address 2.2.2.2 of the remote PE establish a remote-ldp session, and the loopback address 3.3.3.3 of VM2 on the forwarding plane and the loopback address 2.2.2.2 of the remote PE establish a remote-ldp session. session, VPLS assigns the label of the vpls PW according to different remote-ldp sessions; forwards VM1 and VM2 with the inline interface as the transport address to establish a TCP connection, and sends hello messages to each other to trigger the establishment of the local-ldp session. VPLS assigns labels of vpls PWs between VMs according to local-ldp sesison.
分配的标签如下表2所示，PE设备分配给VM1的vpls PW标签为1，分配给VM1的公网隧道标签为4，PE设备分配给VM2的vpls PW标签为2，分配给VM2的公网隧道标签为5，VM1分配给VM2的vpls PW标签为1，分配给VM2的公网隧道标签为4，VM1分配给PE设备的vpls PW标签为1，分配给PE设备的公网隧道标签为4，VM2分配给VM1的vpls PW标签为2，分配给VM1的公网隧道标签为5，VM2分配给PE设备的vpls PW标签为2，分配给PE设备的公网隧道标签为5：The assigned labels are shown in Table 2 below. The vpls PW label assigned to VM1 by the PE device is 1, the public network tunnel label assigned to VM1 is 4, the vpls PW label assigned to VM2 by the PE device is 2, and the public network label assigned to VM2 is 2. The tunnel label is 5, the vpls PW label assigned by VM1 to VM2 is 1, the public network tunnel label assigned to VM2 is 4, the vpls PW label assigned by VM1 to the PE device is 1, and the public network tunnel label assigned to the PE device is 4 , the vpls PW label assigned to VM1 by VM2 is 2, the public network tunnel label assigned to VM1 is 5, the vpls PW label assigned to the PE device by VM2 is 2, and the public network tunnel label assigned to the PE device is 5:
表2Table 2
如图7所示，当VM1和PE之间报文交互(如：CE1->CE方向)时，在VM1，对接收到vlan为10、源MAC地址为111-1、目的MAC地址为111-2的报文，查MAC地址表，找到与CE的MAC地址111-2对应的MAC地址表项，获得接口为G2/0/0，公网走MPLS，压上私网标签1，公网隧道标签4，然后根据token100，查nhlfe表，获得目的MAC地址为222-2，封装上目的MAC地址222-2、源MAC地址222-1，将封装后的报文从G2/0/0发送出去。As shown in Figure 7, when packets are exchanged between VM1 and PE (for example: CE1->CE direction), in VM1, the received vlan is 10, the source MAC address is 111-1, and the destination MAC address is 111- 2, check the MAC address table, find the MAC address entry corresponding to the CE's MAC address 111-2, obtain the interface as G2/0/0, use MPLS on the public network, add the private network label 1, and the public network tunnel Label 4, and then check the nhlfe table according to token100, obtain the destination MAC address as 222-2, encapsulate the destination MAC address 222-2, source MAC address 222-1, and send the encapsulated packet from G2/0/0 .
在PE设备侧，接收到报文发现是发给本设备，解封装，发现是MPLS报文，用标签4查ilm表，发现该标签4对应的ilm表项中的动作为pop，则继续用私网标签1查ilm表，发现该私网标签1对应的广播域为vsi，则查看与该vsi对应的MAC地址转发表，找到CE设备的目的地址111-2所在的MAC地址转发表项，确定与111-2对应的接口为：G2/0/0、虚拟局域网标识为：40，则将报文中的vlan标识替换为40，通过G2/0/0发送至用户CE。On the PE device side, after receiving the packet, it is found that it is sent to the device, decapsulates it, and finds that it is an MPLS packet. It checks the ilm table with label 4, and finds that the action in the ilm entry corresponding to the label 4 is pop, then continue to use The private network label 1 checks the ilm table and finds that the broadcast domain corresponding to the private network label 1 is vsi, then checks the MAC address forwarding table corresponding to the vsi, and finds the MAC address forwarding table entry where the destination address 111-2 of the CE device is located. It is determined that the interface corresponding to 111-2 is: G2/0/0, and the VLAN ID is: 40, then replace the vlan ID in the packet with 40, and send it to the user CE through G2/0/0.
CE用户收到报文，发现是发给本机的以太报文，接收上送本机处理。The CE user receives the message, finds that it is an Ethernet message sent to the local machine, and sends it to the local machine for processing.
其中，CE->CE1方向的转发流程也和上述流程类似，在此不再赘述。The forwarding process in the CE->CE1 direction is also similar to the above process, and will not be repeated here.
由上可知，本发明实施例提供一种VPLS报文处理方法，第一转发面VM接收第一用户发送的包含第一用户的MAC地址、第二用户的MAC地址的报文，查询MAC转发表，获取所述MAC转发表中与第二用户的MAC地址对应的MAC转发表项，将所述报文封装上与私网标签和公网隧道标签，将封装后的报文封装上所述第一接口的MAC地址、第一对端设备的接口的MAC地址，形成VPLS报文，将封装后的VPLS报文通过所述第一接口发送出去。如此，可以通过在每个VM中均配置VSI，使每个VM均可以通过自身的VSI与其他设备(如远端PE设备或其他VM)直接进行报文转发，避免了转发流量的内部迂回，提高了流量转发的性能，同时，不再占用VM间管理接口带宽，提高了采用NFV技术的设备的健壮性。As can be seen from the above, an embodiment of the present invention provides a method for processing a VPLS packet. The first forwarding plane VM receives a packet sent by a first user and includes the MAC address of the first user and the MAC address of the second user, and queries the MAC forwarding table. , obtain the MAC forwarding table entry corresponding to the MAC address of the second user in the MAC forwarding table, encapsulate the message with a private network label and a public network tunnel label, and encapsulate the encapsulated message on the first The MAC address of an interface and the MAC address of the interface of the first peer device form a VPLS packet, and the encapsulated VPLS packet is sent out through the first interface. In this way, by configuring a VSI in each VM, each VM can directly forward packets with other devices (such as remote PE devices or other VMs) through its own VSI, avoiding the internal detour of forwarding traffic. The performance of traffic forwarding is improved, and at the same time, the bandwidth of the management interface between VMs is no longer occupied, and the robustness of devices using NFV technology is improved.
根据本发明实施例，本发明下述实施例还提供了一种服务器10，优选地用于实现上述方法实施例中的方法。According to the embodiments of the present invention, the following embodiments of the present invention further provide a server 10, which is preferably used to implement the methods in the above method embodiments.
实施例二Embodiment 2
图8为本发明实施例提供的服务器10的结构图，如图8所示，所述服务器10可以包括：至少一个转发面虚拟机VM301，每个转发面VM301可连接至少一个用户，每个转发面VM301上可设置一个虚拟转发实例VSI，所述至少一个转发面VM301上的VSI属于同一虚拟专用局域网业务VPLS网络，所述至少一个转发面VM中的任一转发面VM301可以包括：8 is a structural diagram of a server 10 according to an embodiment of the present invention. As shown in FIG. 8 , the server 10 may include: at least one forwarding plane virtual machine VM301, each forwarding plane VM301 can be connected to at least one user, and each forwarding plane VM301 can be connected to at least one user. A virtual forwarding instance VSI can be set on the plane VM301, the VSI on the at least one forwarding plane VM301 belongs to the same virtual private local area network service VPLS network, and any forwarding plane VM301 in the at least one forwarding plane VM can include:
接收单元3011，用于接收第一用户发送的报文，所述报文包含：所述第一用户的媒体接入层MAC地址、第二用户的MAC地址、所述第一用户所在的第一虚拟局域网vlan标识，所述第二用户与第一对端设备连接，所述转发面VM的VSI与所述第一对端设备的VSI属于所述VPLS网络。A receiving unit 3011, configured to receive a message sent by a first user, where the message includes: the MAC address of the media access layer of the first user, the MAC address of the second user, the first user's first user The virtual local area network vlan identifier, the second user is connected to the first peer device, and the VSI of the forwarding plane VM and the VSI of the first peer device belong to the VPLS network.
其中，所述第一对端设备可以为PE设备，或者，第二转发面VM，所述第二转发面VM为所述至少一个转发面VM中除所述转发面VM之外的任一转发面VM。The first peer device may be a PE device, or a second forwarding plane VM, and the second forwarding plane VM is any one of the at least one forwarding plane VM except the forwarding plane VM. face VM.
查询单元3012，用于查询与所述VSI对应的媒体接入控制MAC转发表，获取所述MAC转发表中与所述第二用户的MAC地址对应的MAC转发表项，所述与所述第二用户的MAC地址对应的MAC转发表项包含：所述第二用户的MAC地址、与所述第二用户的MAC地址对应的所述转发面VM上的第一接口。The query unit 3012 is configured to query the media access control MAC forwarding table corresponding to the VSI, and obtain the MAC forwarding table entry corresponding to the MAC address of the second user in the MAC forwarding table. The MAC forwarding entry corresponding to the MAC addresses of the two users includes: the MAC address of the second user, and the first interface on the forwarding plane VM corresponding to the MAC address of the second user.
封装单元3013，用于将所述接收单元接收到的报文封装上第一私网标签和第一公网隧道标签；an encapsulation unit 3013, configured to encapsulate the first private network label and the first public network tunnel label on the message received by the receiving unit;
以及，将封装标签后的报文封装上所述接口的MAC地址、所述第一对端设备的MAC地址，形成第一VPLS报文。And, encapsulating the packet with the label encapsulated on the MAC address of the interface and the MAC address of the first peer device to form a first VPLS packet.
其中，所述第一私网标签为：所述转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的私网标签，所述第一公网隧道标签为：所述转发面VM的VSI与所述第一对端设备的VSI间建立的伪线PW的公网隧道标签。The first private network label is: the private network label of the pseudowire PW established between the VSI of the forwarding plane VM and the VSI of the first peer device, and the first public network tunnel label is: all The public network tunnel label of the pseudowire PW established between the VSI of the forwarding plane VM and the VSI of the first peer device.
发送单元3014，用于将封装单元3013封装后的第一VPLS报文通过所述第一接口发送出去。The sending unit 3014 is configured to send the first VPLS packet encapsulated by the encapsulating unit 3013 through the first interface.
可选的，如图8所示，所述转发面VM301还可以包括：Optionally, as shown in FIG. 8 , the forwarding plane VM301 may further include:
会话建立单元3015，用于在接收单元3011接收第一用户发送的报文之前，与所述第一对端设备建立标签分发协议会话ldp session，通过ldp session建立与第一对端设备间的lsp隧道，获取转发面301的VSI与第一对端设备VSI之间的第一公网隧道标签，然后，再通过lsp隧道建立VPLS的PW，获取转发面301的VSI与第一对端设备VSI之间的第一私网标签。The session establishing unit 3015 is configured to establish a label distribution protocol session ldp session with the first peer device before the receiving unit 3011 receives the message sent by the first user, and establish an lsp with the first peer device through the ldp session tunnel, obtain the first public network tunnel label between the VSI of the forwarding plane 301 and the VSI of the first peer device, and then establish a VPLS PW through the lsp tunnel, and obtain the difference between the VSI of the forwarding plane 301 and the VSI of the first peer device The first private network label in between.
当所述第一对端设备为第二转发面VM时，所述会话建立单元3015具体可以用于：When the first peer device is the second forwarding plane VM, the session establishment unit 3015 can be specifically configured to:
在第一转发面VM和所述第二转发面VM上分别运行标签分发协议LDP进程，通过IGP路由互相学习对方的回环网络协议IP地址、内联口IP地址、以及外联口IP地址，将所述第一转发面VM的内联口IP地址和所述第二转发面VM的内联口IP地址作为传输transport地址建立所述ldp session；Run the label distribution protocol LDP process on the first forwarding plane VM and the second forwarding plane VM respectively, and learn each other's loopback network protocol IP address, inline port IP address, and external port IP address through IGP routing. The inline port IP address of the first forwarding plane VM and the inline port IP address of the second forwarding plane VM are used as transport transport addresses to establish the ldp session;
或者，所述服务器10还可以包括：主控VM302，所述主控VM302用于：在所述主控VM302上运行LDP进程，通过IGP路由学习所述服务器10中每个转发面VM的回环IP地址、内联口IP地址以及外联口IP地址，将所述第一转发面VM的内联口IP地址和所述第二转发面VM的内联口IP地址作为传输transport地址建立所述ldp session。Alternatively, the server 10 may further include: a master VM 302, where the master VM 302 is configured to: run an LDP process on the master VM 302, and learn the loopback IP of each forwarding plane VM in the server 10 through IGP routing address, the IP address of the inline port, and the IP address of the external port, and the IP address of the inline port of the VM of the first forwarding plane and the IP address of the inline port of the VM of the second forwarding plane are used as transport addresses to establish the ldp session.
进一步可选的，作为发送报文的逆过程，所述接收单元3011，还可以用于：Further optionally, as an inverse process of sending a message, the receiving unit 3011 can also be used to:
接收第三用户发送的第二VPLS报文，所述第三用户与第二对端设备连接，所述第二对端设备的VSI和所述转发面VM的VSI属于所述VPLS网络，所述第二VPLS报文包含：目的接口的MAC地址、所述第二对端设备的MAC地址、第二公网隧道标签、第二私网标签、目的用户的MAC地址、所述第三用户的MAC地址、所述第三用户所在虚拟局域网的第二vlan标识以及负载，所述第二私网标签为：所述转发面VM的VSI与所述第二对端设备的VSI间建立的PW的私网标签，所述第二公网隧道标签为：所述转发面VM的VSI与所述第二对端设备的VSI间建立的PW的公网隧道标签；receiving a second VPLS packet sent by a third user, the third user is connected to a second peer device, the VSI of the second peer device and the VSI of the forwarding plane VM belong to the VPLS network, and the The second VPLS packet includes: the MAC address of the destination interface, the MAC address of the second peer device, the second public network tunnel label, the second private network label, the MAC address of the destination user, and the MAC address of the third user. address, the second vlan identifier of the virtual local area network where the third user is located, and the load, and the second private network label is: the private network of the PW established between the VSI of the forwarding plane VM and the VSI of the second peer device network label, the second public network tunnel label is: the public network tunnel label of the PW established between the VSI of the VM on the forwarding plane and the VSI of the second peer device;
所述查询单元3012，还用于查询MAC地址表，若所述MAC地址表中目的接口的MAC地址与所述第一转发面VM的接口的MAC地址相同，则确定所述第二VPLS报文为发送给自身的VPLS报文；The query unit 3012 is further configured to query the MAC address table, and if the MAC address of the destination interface in the MAC address table is the same as the MAC address of the interface of the first forwarding plane VM, determine the second VPLS packet It is a VPLS packet sent to itself;
如图8所示，所述转发面VM301还可以包括：As shown in FIG. 8 , the forwarding plane VM301 may further include:
解封装单元3016，用于解封装所述第二VPLS报文；a decapsulation unit 3016, configured to decapsulate the second VPLS packet;
所述查询单元3012，还用于若所述解封装单元3016发现所述第二VPLS报文为MPLS报文，则查询入标签映射ilm表，找到与所述第二公网隧道标签对应的第一ilm表项；The querying unit 3012 is further configured to query the inbound label mapping ilm table if the decapsulation unit 3016 finds that the second VPLS packet is an MPLS packet, and find the first corresponding to the second public network tunnel label. an ilm entry;
所述解封装单元3016，还用于若所述查询单元3012查询到的第一ilm表项中的动作为下一跳pop，则依次弹出所述第二VPLS报文中的第二公网隧道标签、以及第二私网标签；The decapsulation unit 3016 is further configured to pop up the second public network tunnel in the second VPLS packet in turn if the action in the first ilm entry queried by the query unit 3012 is the next hop pop label, and the second private network label;
所述查询单元3012，还用于在所述ilm表中查询与所述第二私网标签对应的第二ilm表项，获取所述第二ilm表项中与所述第二私网标签相对应的所述转发面VM的VSI；The query unit 3012 is further configured to query the second ilm entry corresponding to the second private network label in the ilm table, and obtain the second ilm entry corresponding to the second private network label. the corresponding VSI of the forwarding plane VM;
查询与所述VSI对应的MAC转发表，获取与所述目的用户的MAC地址对应的MAC转发表项，所述与所述目的用户的MAC地址对应的MAC转发表项包含：所述目的用户的MAC地址、与所述目的用户的MAC地址对应的所述转发面VM上的第二接口、所述目的用户所在的虚拟局域网的第三vlan标识；Query the MAC forwarding table corresponding to the VSI, and obtain the MAC forwarding table entry corresponding to the MAC address of the target user, where the MAC forwarding table entry corresponding to the MAC address of the target user includes: MAC address, the second interface on the forwarding plane VM corresponding to the MAC address of the destination user, and the third vlan identifier of the virtual local area network where the destination user is located;
所述发送单元3014，还用于将弹出所述第二私网标签后的第二VPLS报文中的第二vlan标识替换为所述第三vlan标识，将替换后的报文通过所述第二接口发往所述目的用户。The sending unit 3014 is further configured to replace the second vlan identifier in the second VPLS packet after the second private network label is popped with the third vlan identifier, and pass the replaced packet through the third vlan identifier. The second interface is sent to the destination user.
由上可知，本发明实施例提供一种服务器，该服务器可以包含至少一个转发面VM，且每个转发面VM上设置有一个VSI，至少一个转发面VM中的任一转发面VM接收到第一用户发送的包含第一用户的MAC地址、第二用户的MAC地址的报文后，查询MAC转发表，获取所述MAC转发表中与第二用户的MAC地址对应的MAC转发表项，将所述报文封装上第一私网标签和第一公网隧道标签，将封装后的报文封装上第一接口的MAC地址、所述第一对端设备的接口的MAC地址，形成VPLS报文，将封装后的VPLS报文通过第一接口发送出去。如此，可以通过在每个VM中均配置VSI，使每个VM均可以通过自身的VSI与其他设备(如远端服务器或其他VM)直接进行报文转发，避免了转发流量的内部迂回，提高了流量转发的性能，同时，不再占用VM间管理接口带宽，提高了采用NFV技术的设备的健壮性。As can be seen from the above, an embodiment of the present invention provides a server, where the server may include at least one forwarding plane VM, and each forwarding plane VM is provided with a VSI, and any forwarding plane VM in the at least one forwarding plane VM receives the first After a user sends a packet containing the MAC address of the first user and the MAC address of the second user, the user queries the MAC forwarding table, obtains the MAC forwarding table entry corresponding to the second user's MAC address in the MAC forwarding table, and stores the The packet is encapsulated with a first private network label and a first public network tunnel label, and the encapsulated packet is encapsulated with the MAC address of the first interface and the MAC address of the interface of the first peer device to form a VPLS packet. message, and send the encapsulated VPLS message through the first interface. In this way, VSI can be configured in each VM, so that each VM can directly forward packets with other devices (such as remote servers or other VMs) through its own VSI, avoiding the internal detour of forwarding traffic and improving The performance of traffic forwarding is improved, and at the same time, the bandwidth of the management interface between VMs is no longer occupied, which improves the robustness of devices using NFV technology.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的单元和系统的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the units and systems described above, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，设备和方法，可以通过其它的方式实现。例如，以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically separate, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.
上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，简称ROM)、随机存取存储器(Random Access Memory，简称RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The above-mentioned integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM for short), Random Access Memory (RAM for short), magnetic disk or CD, etc. that can store program codes medium.
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件(例如处理器)来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器、随机存储器、磁盘或光盘等。Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware (such as a processor) through a program, and the program can be stored in a computer-readable storage medium, The storage medium may include: read-only memory, random access memory, magnetic disk or optical disk, and the like.
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610348464.0ACN106101023B (en) | 2016-05-24 | 2016-05-24 | A kind of VPLS message processing method and equipment |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610348464.0ACN106101023B (en) | 2016-05-24 | 2016-05-24 | A kind of VPLS message processing method and equipment |
| Publication Number | Publication Date |
|---|---|
| CN106101023A CN106101023A (en) | 2016-11-09 |
| CN106101023Btrue CN106101023B (en) | 2019-06-28 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610348464.0AActiveCN106101023B (en) | 2016-05-24 | 2016-05-24 | A kind of VPLS message processing method and equipment |
| Country | Link |
|---|---|
| CN (1) | CN106101023B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107547387B (en)* | 2017-08-22 | 2019-11-05 | 新华三技术有限公司 | Session establishing method and device |
| CN109981308B (en)* | 2017-12-27 | 2022-05-24 | 北京华为数字技术有限公司 | Message transmission method and device |
| CN110417658B (en) | 2018-04-28 | 2022-08-12 | 北京京东尚科信息技术有限公司 | Network access method and device for edge router |
| CN111835635B (en)* | 2019-04-18 | 2021-12-14 | 华为技术有限公司 | Method, equipment and system for publishing route in BGP network |
| CN112751766B (en)* | 2019-10-30 | 2023-07-11 | 华为技术有限公司 | Message forwarding method and system, related equipment and chip |
| CN111193653B (en) | 2019-12-31 | 2021-08-06 | 腾讯科技（深圳）有限公司 | Data transmission method, device, equipment and storage medium |
| CN113329454B (en)* | 2020-02-29 | 2023-01-06 | 华为技术有限公司 | Method, network element, system and equipment for publishing routes |
| WO2021227905A1 (en)* | 2020-05-11 | 2021-11-18 | PrimeWan Limited | Virtual network |
| CN113114553B (en)* | 2021-04-23 | 2022-04-01 | 新华三信息安全技术有限公司 | Method and device for realizing butt joint of different networks under EVPN |
| CN113438162B (en)* | 2021-05-21 | 2022-11-04 | 翱捷科技股份有限公司 | Method and device for realizing two-layer forwarding |
| CN113542096B (en)* | 2021-06-24 | 2022-11-18 | 新华三云计算技术有限公司 | Virtual channel negotiation method and device |
| CN113923162B (en)* | 2021-10-09 | 2023-04-18 | 新华三信息安全技术有限公司 | Message forwarding method, device, equipment and storage medium |
| CN113949662B (en)* | 2021-11-18 | 2023-04-21 | 新华三大数据技术有限公司 | Message forwarding method, device, network equipment and storage medium |
| CN116366546B (en)* | 2023-04-20 | 2024-11-29 | 苏州盛科通信股份有限公司 | Message forwarding method and related device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2190148A1 (en)* | 2007-09-12 | 2010-05-26 | Huawei Technologies Co., Ltd. | Method, system and device for removing media access control address |
| CN102325073A (en)* | 2011-07-06 | 2012-01-18 | 杭州华三通信技术有限公司 | VPLS (Virtual Private Local Area Network Service)-based message processing method and device thereof |
| CN102932499A (en)* | 2012-11-01 | 2013-02-13 | 杭州华三通信技术有限公司 | Method and device for learning media access control (MAC) addresses in virtual private lan service (VPLS) networks |
| CN103685022A (en)* | 2012-09-17 | 2014-03-26 | 杭州华三通信技术有限公司 | Message forwarding method and service provider network edge equipment |
| CN103795630A (en)* | 2014-01-15 | 2014-05-14 | 杭州华三通信技术有限公司 | Message transmitting method and device of label switching network |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2190148A1 (en)* | 2007-09-12 | 2010-05-26 | Huawei Technologies Co., Ltd. | Method, system and device for removing media access control address |
| CN102325073A (en)* | 2011-07-06 | 2012-01-18 | 杭州华三通信技术有限公司 | VPLS (Virtual Private Local Area Network Service)-based message processing method and device thereof |
| CN103685022A (en)* | 2012-09-17 | 2014-03-26 | 杭州华三通信技术有限公司 | Message forwarding method and service provider network edge equipment |
| CN102932499A (en)* | 2012-11-01 | 2013-02-13 | 杭州华三通信技术有限公司 | Method and device for learning media access control (MAC) addresses in virtual private lan service (VPLS) networks |
| CN103795630A (en)* | 2014-01-15 | 2014-05-14 | 杭州华三通信技术有限公司 | Message transmitting method and device of label switching network |
| Publication number | Publication date |
|---|---|
| CN106101023A (en) | 2016-11-09 |
| Publication | Publication Date | Title |
|---|---|---|
| CN106101023B (en) | A kind of VPLS message processing method and equipment | |
| CN104335532B (en) | The method and apparatus for routing the packet to the far-end address of Virtual Switch Instance | |
| CN108574616B (en) | Method, equipment and system for processing route | |
| CN103200069B (en) | A kind of method and apparatus of Message processing | |
| EP2853066B1 (en) | Layer-3 overlay gateways | |
| AU2013303200B2 (en) | Method and system for virtual and physical network integration | |
| CN105591916B (en) | A kind of message transmitting method and device | |
| WO2016173271A1 (en) | Message processing method, device and system | |
| GB2497202A (en) | Transmitting frames between, possibly different, local VLANs by encapsulating frames for global VLAN tunnel | |
| WO2015043168A1 (en) | Method, apparatus and system for implementing communication of virtual network | |
| CN109691026B (en) | Method and device for updating multiple multi-protocol label switching bidirectional forwarding detection sessions | |
| EP3456020B1 (en) | Mechanism for inline packet response generation in software defined networks | |
| WO2020212998A1 (en) | Network address allocation in a virtual layer 2 domain spanning across multiple container clusters | |
| CN110474829A (en) | The method and apparatus of transmitting message | |
| CN112769584B (en) | Method, device and storage medium for sharing upper link by network slice | |
| US20180270084A1 (en) | Technique for exchanging datagrams between application modules | |
| CN115442184A (en) | Access system and method, access server, system and storage medium | |
| WO2022053007A1 (en) | Network reachability verification method and apparatus, and computer storage medium | |
| WO2015081534A1 (en) | Data packet transmission system, transmission method and device thereof | |
| WO2017036384A1 (en) | Provider edge device and data forwarding method | |
| WO2020152691A1 (en) | Multi-network internet protocol version 6 (ipv6) duplicate address detection using ethernet virtual private network (evpn) | |
| CN113647065B (en) | virtual network topology | |
| CN115460138A (en) | Network interworking method and device | |
| CN115460140A (en) | Network intercommunication method and device | |
| CN115460141A (en) | Network intercommunication method and device |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |