Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Kubernetes (K8s) is an exemplary open source container cluster management system for automatically deploying, extending, and managing containerized applications. In the kubernets cluster, Pod is the basis for all traffic types, also the minimum unit level managed by K8s, which is a combination of one or more containers. One or more containers may run within a Pod, with the containers sharing the same Pod's network environment. The K8s performs scheduling, expanding, resource sharing and life cycle management by taking Pod as a minimum unit.
At present, the scheme that kubernets introduce north-south traffic into a cluster mainly includes Ingress, nodoport Service, and loadbalancerservice, where traffic from a client to a server is the north-south traffic, that is, traffic between client-servers. Wherein Ingress is a seven-layer virtual service for introducing network traffic into the Kubernetes cluster network according to Host and Path. NodePort Service is a four-layer virtual Service for introducing network traffic into a Kubernet cluster network according to Port. The loadbalance Service is a four-layer virtual Service based on a NodePort Service and an External hardware load balancer and is used for introducing network traffic into a Kubernets cluster network according to an External IP.
The virtual service is an abstraction of the application service, and high availability and performance horizontal extension of the back-end service are realized by providing a uniform portal Internet Protocol (IP), Port, Host and Path for the application server cluster and dispatching a request from a client to a proper back-end server according to a load balancing algorithm. The virtual services are generally divided into four-layer virtual services and seven-layer virtual services, where the four-layer virtual services are scheduled according to Open System Interconnection (OSI) network packet information of four layers and less, and the seven-layer virtual services are scheduled according to OSI network packet information of seven layers and less.
The disadvantages of NodePort Service: the method needs to occupy precious host port resources, the configurable Service quantity is limited by the port quantity, and multi-layer forwarding is usually needed (for example, the Service is hit in Node a and is scheduled to Node B), the performance is poor, and the troubleshooting is difficult (the too long flow path not only reduces the network performance, but also increases the network complexity, which causes the troubleshooting difficulty to rise). Secondly, for services such as classic HTTP/HTTPs/FTP, the ports used are usually fixed 80, 443, etc., but the Service issued by the nodoport Service needs to avoid such common ports, for example, the port range used by the nodoport Service is 30000-.
The loadbalance Service has the following defects: the loadbalance Service is realized based on the nodoport Service, and has the same port number limitation and performance problems.
Disadvantages of Ingress: the traditional Ingress is realized based on the Nginx, the Nginx is a reverse proxy server with an open source code, the Nginx is deployed in a kubernets cluster in a Pod form, and generally, flow is led into the Nginx Pod through a nodoport or a loadbalancerservice, so that a destination IP of a network data packet reaching the ngx Pod is definitely the ngx Pod IP, and therefore the Nginx cannot judge a Service to which the ngx belongs according to the destination IP of the data packet.
It can be seen that, in the traditional Ingress, traffic is introduced into a cluster through a node port or a Loadbalancer Service, and then node scheduling is performed according to application layer information such as Host and Path, and this kind of drainage mode cannot determine the Service to which the Ingress belongs according to the destination IP of a data packet, which makes it difficult for the traditional Service that does not support domain names to be deployed into kubernets. It is also affected by the port management difficulty of nodecort and loadbalancerervice, and performance issues.
In view of the above-mentioned drawbacks of the related art, embodiments of the present invention provide a data processing method, which can receive request data packets of different destination IPs and distinguish the services to which the request data packets belong according to the destination IPs. In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Fig. 1 is a schematic diagram of a container cluster according to an embodiment of the present invention, where a Node in fig. 1 is a Node in the container cluster, and the Node includes a container group (Pod). In the related art, a Pod only includes an eth0 network Interface, an eth0 is a Veth Interface created by a kubernets CNI (connectivity network Interface) plug-in for the Pod, and the Veth is a virtualized network device.
As shown in fig. 1, the embodiment of the present invention creates one data plane (dataplane), one eth1 (first network interface) and one eth2 (second network interface) in one Pod of one node of the container cluster. The data plane is deployed in a container cluster in the form of a container, namely, the first container in the embodiment of the present invention, eth1 and eth2 are taken over by the data plane process.
In practical application, the DataPlane is a software load data plane supporting the Intel DPDK technology, the DPDK is called a DataPlane development kit as a whole and is a data plane development tool set provided by Intel, and the DPDK application program runs on a user space and utilizes a data plane library provided by the DPDK application program itself to send and receive data packets.
eth1 may be a MacVTap interface operating in VEPA mode, created based on a certain portal (Nic) in the default network namespace, and adds eth1 to the network namespace where the data plane process is located. Nic is a portal in the DefaultNetworkNamespace of Kubernets WorkerNode. MacVTap is a Linux kernel device driver aimed at simplifying virtualized bridged networks, essentially a combination of a Macvlan driver and a Tap device, and is typically used to replace Linux bridge + Tap in a virtualized environment to provide virtual network devices for virtual machines.
eth2 may be a MacVTap interface operating in Passthru mode, and eth2 may be created based on eth0, in the same network namespace as the data plane process.
An externally reachable IP address is configured on eth 1: ip1 for receiving external network traffic. And moving the own Pod IP address (IP2) on the eth0 to the eth2 for accessing the Pod network. If the network hybrid mode is deployed in the virtualization environment, the underlying virtualization platform is required to start the network hybrid mode, so as to ensure that a data packet with a destination media access control address (MAC) of eth1 is not discarded by the virtualization platform.
Fig. 2 is a schematic diagram of an implementation flow of a data processing method according to an embodiment of the present invention, where the data processing method is applied to a first container in a first container group of a container cluster, that is, a data plane in fig. 1, a physical execution subject of the data processing method is a node in the container cluster, where the first container group is located in the node, and the node may be an electronic device such as a desktop computer and a notebook computer. Referring to fig. 2, the data processing method includes:
s201, receiving a request data packet based on a first network interface.
Here, the first container corresponds to the data plane in fig. 1, the first container group further includes a first network interface and a second network interface, the first network interface corresponds to eth1 in fig. 1, and the second network interface corresponds to eth2 in fig. 1.
In an embodiment, the first network interface and the second network interface are hosted by the first container; wherein the first network interface is created based on a portal of a node where the first container group is located; the second network interface is created based on a third network interface of the first container group; the third network interface characterizes a network interface created by a network plug-in of the container cluster for the first container group; the IP address of the second network interface is characterized as the IP address of the third network interface.
The first container receives an external network request through a first network interface, which is created based on a certain portal (Nic) in a default network namespace and adds eth1 to the network namespace where the data plane process is located. The first network interface is configured with an externally reachable IP address for receiving external network traffic.
The third network interface corresponds to eth0 in fig. 1, and moves the Pod IP address carried by the eth0 to the second network interface for accessing the Pod network.
In practical applications, the first network interface and the second network interface may be MacVTap interfaces, which are Linux kernel device drivers aimed at simplifying virtualized bridge networks, and are essentially a combination of Macvlan drivers and Tap devices, and are generally used to provide virtual network devices for virtual machines in a virtualized environment instead of Linux bridge + Tap. The first container is a software load data plane supporting the Intel DPDK technology, and two MacVTap high-performance virtual network ports are taken over through the Intel DPDK technology. The performance influence and port management difficulty of the NodePort/LoadbalancerService can be avoided through the MacVTap high-performance virtual network port, the route of the south-north flow entering the cluster is shortened, and the difficulty of troubleshooting is reduced while the performance is improved.
In one embodiment, before receiving the request packet based on the first network interface, the method further comprises:
s301, receives a setup broadcast request.
Here, the setting broadcast request is transmitted by a client of an upstream network device, that is, an external device of the container cluster. The broadcast request is configured to obtain a network address of the first network interface.
Specifically, the upstream network device sends an Address Resolution Protocol (ARP) broadcast request, which is used to obtain the MAC Address of the first network interface. ARP is used to implement the mapping from IP address to MAC address, i.e. to ask for the MAC address corresponding to the target IP.
S302, sending the network address of the first network interface to an opposite device based on the set broadcast request, so that the opposite device sends a request packet to the first network interface based on the network address; the peer device characterizes a sender of the set broadcast request.
Here, the peer device is an upstream network device, and the first container provides the network address of the first network interface (MAC address of eth 1) to the upstream network device in response to the set broadcast request. Thus, the upstream network device may send network traffic to the first network interface through the two-layer network.
In the embodiment of the present invention, the request packet may be a packet from an IP with different destination, and may be a request packet such as HTTP/HTTPs/FTP. The embodiment of the invention supports pure IP service, namely the traditional service which does not support domain names.
S202, matching a back-end server based on the request data packet; the back-end server characterizes nodes in the container cluster that provide application services.
And after receiving the request data packet through the first network interface, the first container is matched with the back-end server according to the request data packet.
Referring to fig. 4, in an embodiment, the matching the backend server based on the request packet includes:
s401, matching virtual service based on request parameters in the request data packet.
S402, selecting a corresponding back-end server based on the matched virtual service.
Here, the virtual service refers to an application service provided by a container cluster, the request parameter includes information such as a destination IP, a destination port, a Host, and a Path in the request packet, and the virtual service is matched according to the information, one virtual service may correspond to a plurality of backend servers, and an appropriate backend server is selected according to a scheduling algorithm. For example, a backend server with a lower load may be selected according to the load of the backend server.
S203, modifying the network address in the request data packet based on the matched back-end server and the second network interface.
In an embodiment, the modifying the network address in the request packet based on the matched backend server and the second network interface includes:
and modifying the source IP address in the request data packet into the IP address of the second network interface.
And modifying the destination IP address of the request data packet into the matched IP address of the back-end server.
Specifically, the source IP of the request packet is modified to the IP address of eth2, and the destination IP is modified to the corresponding IP address of the backend server, and then the request packet is sent out via the eth2 through the Pod network.
S204, the modified request data packet is sent to the back-end server based on the second network interface.
The second network interface is in butt joint with the Pod network, so that the modified request data packet is sent to the corresponding back-end server through the second network interface.
The embodiment of the invention is applied to a first container in a first container group of a container cluster, the first container group further comprises a first network interface and a second network interface, a request data packet is received based on the first network interface, a back-end server is matched based on the request data packet, a network address in the request data packet is modified based on the matched back-end server and the second network interface, and the modified request data packet is sent to the corresponding back-end server through the second network interface, wherein the back-end server represents a node providing application service in the container cluster. In the embodiment of the invention, the first container can receive the data packets of different destination IPs through the first network interface, can distinguish the service to which the first container belongs according to the destination IP of the request data packet, and then sends the data packets to the back-end server corresponding to the service through the second network interface. The embodiment of the invention does not need to distinguish services according to ports, and better conforms to the service scenes of common fixed ports such as HTTP, HTTPS and the like.
Referring to fig. 5, in an embodiment, after sending the modified request packet to the backend server based on the second network interface, the method further includes:
s501, receiving a response data packet of the back-end server based on the second network interface.
And the back-end server processes the request data packet after receiving the request data packet sent by the upstream network equipment, and sends the response data packet to the second network interface through the Pod network after the processing is finished.
S502, modifying the response data packet, and sending out the modified response data packet based on the first network interface.
And the first container receives a response data packet sent by the back-end server through the second network interface, matches the connection tracking information, and returns the response data packet to the upstream network equipment through the first network interface.
In one embodiment, the modifying the response packet includes:
modifying the target IP address of the response data packet into the source IP address of the request data packet;
and modifying the source IP address of the response data packet into the destination IP address of the request data packet.
After matching connection tracking, the destination IP address and the source IP address of the response data packet are modified into the source IP address and the destination IP address of the corresponding request data packet, so that the response data packet can be ensured to return to the client correctly.
Referring to fig. 6, fig. 6 is a schematic diagram of a data path of north-south traffic according to an embodiment of the present invention. The upstream network device sends an ARP broadcast request for resolving the MAC address corresponding toip 1. The data plane process receives the ARP broadcast request through the eth1, and provides the MAC address of the eth1 to the upstream network device in response to the ARP request.
The upstream network device sends the network request to eth1 through the two-layer network, the data plane process receives the network request through eth1, matches the virtual service according to the information of destination IP, destination port, Host, Path and the like in the request data packet, and selects a proper back-end server according to the scheduling algorithm. And modifying the source IP of the request data packet into IP2, modifying the destination IP into a corresponding back-end server IP, and sending the request data packet out through the eth2 through the Pod network.
And after receiving the request data packet, the back-end server processes the request data packet, and after the processing is finished, the back-end server sends a response data packet to eth2 through the Pod network, and after receiving the response data packet through eth2, the data plane process matches the connection tracking information and returns the response data packet to the upstream network equipment througheth 1. After matching connection tracking, the destination IP and the source IP of the response packet are modified into the source IP and the destination IP of the corresponding request packet, so as to ensure that the response packet can return to the client correctly.
The embodiment of the invention can distinguish different service flows according to the destination IP of the request data packet, for example, suppose that two virtual services exist: VS1 and VS2, representing two IP services, respectively, with VIP 192.168.0.1 and 192.168.0.2, respectively. The upstream network device assigns VS1 an externally reachable floating IP: EIP1, assigning VS2 an externally reachable floating IP:EIP 2.
The client accesses the EIP1, the upstream network device maps the EIP1 to 192.168.0.1 (the destination IP of the packet is changed to 192.168.0.1 through DNAT), and sends the packet to IP1 (which may be implemented by configuring a static route). The data plane process receives the request packet, successfully matches to VS1 according to the destination IP (192.168.0.1), and forwards the data packet to the corresponding back-end server.
The client accesses the EIP2, and the upstream network device maps EIP2 to 192.168.0.2 (the destination IP of the packet is changed to 192.168.0.2 by DNAT), and sends the packet to IP1 (which may be implemented by configuring a static route). The data plane process receives the request packet, successfully matches to the VS2 according to the destination IP (192.168.0.2), and forwards the data packet to the corresponding back-end server.
The embodiment of the invention can release pure IP Service in a Kubernetes environment, can receive data packets of different target IPs, distinguish the Service to which the data packets belong according to the target IPs of the data packets, is not influenced by the port management difficulty and the performance problem of NodePort/Loadbaseband Service, does not need to distinguish the Service through the port, and better accords with the Service use scenes of HTTP/HTTPS and the like which usually use fixed ports.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The technical means described in the embodiments of the present invention may be arbitrarily combined without conflict.
In addition, in the embodiments of the present invention, "first", "second", and the like are used for distinguishing similar objects, and are not necessarily used for describing a specific order or a sequential order.
Referring to fig. 7, fig. 7 is a schematic diagram of a first container according to an embodiment of the present invention, and as shown in fig. 7, the first container includes a receiving module, a matching module, a modifying module, and a sending module.
A receiving module, configured to receive a request packet based on a first network interface;
a matching module for matching a backend server based on the request data packet; the back-end server represents the nodes providing the application service in the container cluster;
a modification module, configured to modify a network address in the request packet based on the matched backend server and the second network interface;
and the sending module is used for sending the modified request data packet to the back-end server based on the second network interface.
In one embodiment, the modifying module modifies the network address in the request packet based on the matched backend server and the second network interface, including:
modifying the source IP address in the request data packet into the IP address of a second network interface;
and modifying the destination IP address of the request data packet into the matched IP address of the back-end server.
In one embodiment, the apparatus further comprises:
a second receiving module, configured to receive a response packet of the backend server based on the second network interface;
and the modification module is also used for modifying the response data packet and sending out the modified response data packet based on the first network interface.
In one embodiment, the modifying module modifies the response packet, including:
modifying the target IP address of the response data packet into the source IP address of the request data packet;
and modifying the source IP address of the response data packet into the destination IP address of the request data packet.
In an embodiment, the receiving module is further configured to: receiving a set broadcast request;
a second sending module, configured to send the network address of the first network interface to an opposite device based on the set broadcast request, so that the opposite device sends a request packet to the first network interface based on the network address; the peer device characterizes a sender of the set broadcast request.
In one embodiment, the matching module matches a backend server based on the request packet, including:
matching virtual services based on request parameters in the request data packet;
and selecting a corresponding back-end server based on the matched virtual service.
In an embodiment, the first network interface and the second network interface are hosted by the first container; wherein the first network interface is created based on a portal of a node where the first container group is located; the second network interface is created based on a third network interface of the first container group; the third network interface characterizes a network interface that a network plugin of the container cluster creates for the first container group; the IP address of the second network interface is characterized as the IP address of the third network interface.
In practical applications, the receiving module, the matching module, the modifying module and the sending module may be implemented by a Processor in an electronic device, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), a Micro Control Unit (MCU), or a Programmable gate array (FPGA).
It should be noted that: in the above embodiment, when performing data processing, the first container is only illustrated by dividing the modules, and in practical applications, the processing may be distributed to different modules according to needs, that is, the internal structure of the first container is divided into different modules to complete all or part of the processing described above. In addition, the first container and the data processing method provided in the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments, and are not described herein again.
Based on the hardware implementation of the program module, in order to implement the method of the embodiment of the present application, an embodiment of the present application further provides an electronic device. Fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application, and as shown in fig. 8, the electronic device includes:
the communication interface can carry out information interaction with other equipment such as network equipment and the like;
and the processor is connected with the communication interface to realize information interaction with other equipment, and is used for executing the method provided by one or more technical schemes of the electronic equipment side when running a computer program. And the computer program is stored on the memory.
Of course, in practice, the various components in an electronic device are coupled together by a bus system. It will be appreciated that a bus system is used to enable communications among the components. The bus system includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as a bus system in fig. 8.
The memory in the embodiments of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program for operating on an electronic device.
It will be appreciated that the memory can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memories described in the embodiments of the present application are intended to comprise, without being limited to, these and any other suitable types of memory.
The method disclosed in the embodiments of the present application may be applied to a processor, or may be implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The processor described above may be a general purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in a memory where a processor reads the programs in the memory and in combination with its hardware performs the steps of the method as previously described.
Optionally, when the processor executes the program, the corresponding process implemented by the electronic device in each method of the embodiment of the present application is implemented, and for brevity, no further description is given here.
In an exemplary embodiment, the present application further provides a storage medium, specifically a computer storage medium, for example, a first memory storing a computer program, where the computer program is executable by a processor of an electronic device to perform the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disk, or CD-ROM.
In the several embodiments provided in the present application, it should be understood that the disclosed first container, electronic device and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The technical means described in the embodiments of the present application may be arbitrarily combined without conflict.
In addition, in the examples of the present application, "first", "second", and the like are used for distinguishing similar objects, and are not necessarily used for describing a specific order or a sequential order.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.