CN110198246B - Method and system for monitoring flow - Google Patents

Abstract

The invention relates to a method and a system for monitoring flow, which bind flow labels to containers/virtual machines in advance, classify and filter the data messages with the same label according to the labels of the data messages of each container/virtual machine, perform cluster analysis on the data messages with the same label so as to obtain a flow data statistical result and further obtain flow monitoring information of each container/virtual machine, and finally monitor the flow of each container/virtual machine according to the flow monitoring information, thereby solving the problem of difficult flow monitoring of containers/virtual machines without independent network cards.

Description

Method and system for monitoring flow

Technical Field

The invention relates to the technical field of flow monitoring, in particular to a method and a system for monitoring flow of a virtual machine or a container.

Background

With the popularization of cloud computing, more and more customers choose to deploy their own applications onto virtual machines of a cloud platform, which puts higher requirements on the security performance of the cloud platform. In order to ensure the information security of the virtual machine, the cloud platform needs to monitor the network traffic of the virtual machine. The currently common monitoring method is to monitor the overall flow rate of the network card of the virtual machine, which is relatively simple to implement, but when the number of the virtual machines is large and IP addresses are not sufficiently allocated, the existing method for monitoring the flow rate through the network card of the virtual machine cannot solve the problem of monitoring the flow rate of each virtual machine.

In addition, many open-source application container engines are currently available, such as Docker, which enables developers to package their applications and dependencies into a portable container and then distribute them to any popular Linux machine, and also enables virtualization. The container completely uses a sandbox mechanism, does not depend on any language, has no interface with each other and almost no performance overhead, and can be easily operated in a data center.

Docker becomes an ideal container technology choice for implementing applications running across multiple different environments, through which we can very conveniently manage services and dependencies between services. After Docker has been produced, most enterprises begin to build services using containers to facilitate better resource savings.

The flow monitoring of the Docker container is realized by collecting the flow of the network card in and out based on the kernel, but the independent IP cannot be distributed to the container due to the limited IP number under the computer room network architecture, so that the independent network card cannot be configured to the container, and the mother computer network must be shared, therefore, the traditional method for flow monitoring based on the network card cannot be realized.

In addition, based on the packet capturing and flow counting mode of the process ports in the container, the realization principle is as follows: (1) when the container is scheduled, recording the port to be monitored; (2) when the container is started, starting tcpdump to carry out packet capturing statistics, and writing the tcpdump into a file; (3) and analyzing the packet capturing file by the monitoring script or the program, and reporting at intervals. Thus, there are problems at least including: (1) the tcpdump packet capturing has large system overhead and large intrusion of monitoring on services; (2) inaccurate, only the flow of the monitoring port can be monitored and distinguished, and the flow monitoring cannot be realized for the client end service; (3) the service monitoring port needs to be registered in advance, the port is accurately managed in a life cycle, and smooth flow monitoring is difficult to realize in a port changing scene. Therefore, the packet capturing and flow counting mode based on the process port in the container has the problems of insufficient accuracy, high system overhead and the like, and is unacceptable for internet online services.

It can be seen that, in the prior art, there is no solution for monitoring traffic of a virtual machine to which an independent IP is not allocated, nor a solution for monitoring traffic of a Docker container to which an independent IP is not allocated.

Disclosure of Invention

The invention provides a method and a system for monitoring flow, which realize marking, collecting, counting and monitoring of the flow of each container/virtual machine from a kernel level, solve the problem of difficult flow monitoring of the container/virtual machine without an independent network card, simultaneously realize accurate monitoring of the flow, greatly reduce the system overhead, meet the vital flow monitoring requirement of online service, ensure the delivery quality of the online service, facilitate the rapid discovery and positioning of the problems and improve the safety of the container/virtual machine. The invention is realized by the following technical scheme:

first, the present invention provides a method for monitoring traffic of a virtual machine, where the virtual machine does not allocate an independent IP, including:

acquiring data messages of each virtual machine;

filtering the data messages according to the labels of the data messages of the virtual machines;

performing cluster analysis on the data messages with the labels to obtain a flow data statistical result;

acquiring flow monitoring information of each virtual machine according to the flow data statistical result;

and monitoring the flow of each virtual machine according to the flow monitoring information.

Further, the filtering the data packet according to the label of the data packet includes: and allocating unique labels to the virtual machines in advance, wherein the labels of the data messages of the virtual machines are the same as the unique labels allocated to the virtual machines in advance.

Further, the tags include normal tags and suspected malicious tags.

Further, the filtering the data packet according to the label of the data packet includes: determining the data message with the normal label as a normal message, and carrying out flow data statistics; and determining the data message with the malicious label as the data message generated by the suspected malicious virtual machine, and sending a reminding signal to the traffic monitoring management platform.

Further, the reminding signal comprises: a text reminding signal, a sound reminding signal and a vibration reminding signal.

Further, the traffic monitoring information includes: upload/download speed, number of uploads/downloads, total traffic.

Further, the filtering the data packet according to the label of the data packet includes: filtering is performed using a netfilter/iptables IP packet filtering system.

Second, the present invention provides a method for monitoring traffic of a container, wherein the container is not allocated with a separate IP, and the method comprises:

acquiring data messages of each container;

filtering the data messages according to the labels of the data messages of the containers;

performing cluster analysis on the data messages with the labels to obtain a flow data statistical result;

acquiring flow monitoring information of each container according to the flow data statistical result;

and monitoring the flow of each container according to the flow monitoring information.

Thirdly, the present invention provides a system for monitoring traffic of a virtual machine, where the virtual machine is not allocated with an independent IP, and the system includes:

the system comprises a flow monitoring management platform, host machine nodes and virtual machines;

the flow monitoring management platform comprises a starting command generation module, a statistical result receiving module, a flow monitoring information acquisition module and a flow monitoring module;

the starting command generating module is used for generating a command for controlling the virtual machine to start and sending the command to the host machine node;

the statistical result receiving module is used for receiving the statistical result of the flow data reported by the host machine;

the traffic monitoring information obtaining module is used for obtaining traffic monitoring information of each virtual machine according to the traffic data statistical result;

the flow monitoring module is used for monitoring the flow of each virtual machine according to the flow monitoring information;

the host machine node comprises a trigger starting module, a data message acquiring module, a filtering module, a flow data statistical result acquiring module and a flow data statistical result reporting module;

the triggering and starting module is used for triggering the virtual machine to start according to the starting command sent by the flow monitoring and management platform;

the data message acquisition module is used for acquiring data messages of the virtual machines;

the filtering module is used for filtering the data messages according to the labels of the data messages of the virtual machines;

the flow data statistical result obtaining module is used for carrying out cluster analysis on the data messages with the labels to obtain flow data statistical results;

and the traffic data statistical result reporting module is used for reporting the traffic data statistical result to a traffic monitoring management platform.

Further, the host node further includes:

and the label distribution module is used for distributing unique labels to the virtual machines in advance, and the labels of the data messages of the virtual machines are the same as the unique labels pre-distributed to the virtual machines.

Further, the tags include normal tags and suspected malicious tags.

Further, the filtering module further comprises:

the normal message determining module is used for determining the data message with the normal label as a normal message and carrying out flow data statistics;

and the suspected malicious message determining module is used for determining the data message with the malicious label as the data message generated by the suspected malicious virtual machine and sending a reminding signal to the flow monitoring management platform.

Further, the reminding signal comprises: a text reminding signal, a sound reminding signal and a vibration reminding signal.

Further, the traffic monitoring information includes: upload/download speed, number of uploads/downloads, total traffic.

Further, the filtering the data packet according to the label of the data packet includes: filtering is performed using a netfilter/iptables IP packet filtering system.

Fourthly, the invention provides a system for monitoring the flow of a container, wherein the container is not allocated with an independent IP, and the system is characterized by comprising a flow monitoring management platform, a host machine node and a container;

the flow monitoring management platform comprises a starting command generation module, a statistical result receiving module, a flow monitoring information acquisition module and a flow monitoring module;

the starting command generating module is used for generating a command for controlling the starting of the container and sending the command to the host machine node;

the statistical result receiving module is used for receiving the statistical result of the flow data reported by the host machine;

the flow monitoring information obtaining module is used for obtaining the flow monitoring information of each container according to the flow data statistical result;

the flow monitoring module is used for monitoring the flow of each container according to the flow monitoring information;

the host machine node comprises a trigger starting module, a data message acquiring module, a filtering module, a flow data statistical result acquiring module and a flow data statistical result reporting module;

the triggering and starting module is used for triggering the container to start according to the starting command sent by the flow monitoring and management platform;

the data message acquisition module is used for acquiring the data messages of all the containers;

the filtering module is used for filtering the data messages according to the labels of the data messages of the containers;

the flow data statistical result obtaining module is used for carrying out cluster analysis on the data messages with the labels to obtain flow data statistical results;

and the traffic data statistical result reporting module is used for reporting the traffic data statistical result to a traffic monitoring management platform.

Further, the host node further includes:

and the label distribution module is used for distributing unique labels to the containers in advance, and the labels of the data messages of the containers are the same as the unique labels pre-distributed to the containers.

Further, the tags include normal tags and suspected malicious tags.

Further, the filtering module further comprises:

the normal message determining module is used for determining the data message with the normal label as a normal message and carrying out flow data statistics;

and the suspected malicious message determining module is used for determining the data message with the malicious label as the data message generated by the suspected malicious container and sending a reminding signal to the flow monitoring management platform.

Further, the reminding signal comprises: a text reminding signal, a sound reminding signal and a vibration reminding signal.

Further, the traffic monitoring information includes: upload/download speed, number of uploads/downloads, total traffic.

Further, the filtering the data packet according to the label of the data packet includes: filtering is performed using a netfilter/iptables IP packet filtering system.

Further, the container is specifically a Docker container.

The invention has the beneficial effects that:

the invention provides a method and a system for monitoring the flow of a container/virtual machine, which are characterized in that flow labels are bound to the container/virtual machine in advance, then classification and filtration are carried out according to the labels of data messages of each container/virtual machine, and the data messages with the same label are subjected to cluster analysis, so that a flow data statistical result is obtained, flow monitoring information of each container/virtual machine is further obtained, finally, the flow of each container/virtual machine is monitored according to the flow monitoring information, and the problem of difficulty in flow monitoring of the container/virtual machine without an independent network card is solved. In addition, the data messages filtered according to the labels are determined to be the data messages generated by the suspected malicious container/virtual machine, and the reminding signals are sent out, so that the risk of data tampering is reduced, the safety of the container/virtual machine is improved, and the safety degree of data assets is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic view of an application scenario of a method for monitoring traffic of a virtual machine according to an embodiment of the present invention;

fig. 2 is a first flowchart of a method for monitoring traffic of a virtual machine according to an embodiment of the present invention;

fig. 3 is a second flowchart of a method for monitoring traffic of a virtual machine according to an embodiment of the present invention;

fig. 4 is a schematic application scenario diagram of a method for monitoring traffic of a container according to an embodiment of the present invention;

FIG. 5 is a first flowchart of a method for flow monitoring of a container, according to an embodiment of the present invention;

FIG. 6 is a second flowchart of a method for flow monitoring a container provided by an embodiment of the present invention;

fig. 7 is a timing diagram of flow monitoring performed by the docker management platform according to the embodiment of the present invention;

fig. 8 is a schematic diagram of statistical results based on nfacct according to an embodiment of the present invention;

fig. 9 is a schematic diagram of a flow monitoring result of an actual docker management platform according to an embodiment of the present invention;

fig. 10 is a schematic structural diagram of a system for monitoring traffic of a container/virtual machine according to an embodiment of the present invention;

fig. 11 is a schematic structural diagram of a flow monitoring management platform in the system according to the embodiment of the present invention;

fig. 12 is a schematic structural diagram of a host node in the system according to the embodiment of the present invention.

Detailed description of the invention

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Fig. 1 shows an application scenario of the method and system for monitoring traffic of a virtual machine according to the embodiment of the present invention. The virtual machine flow monitoring management platform comprises monitoring equipment and a server, wherein the server sends flow data of each virtual machine on the server, which is acquired by each host machine, to the monitoring equipment, and monitoring personnel for monitoring the flow of the virtual machine monitor the flow of the virtual machine through the monitoring equipment.

Referring to fig. 2, it is a first flowchart of a method for monitoring traffic of a virtual machine, where the virtual machine is not allocated with an independent IP, according to an embodiment of the present invention, the method includes:

and S101, acquiring data messages of each virtual machine.

And S102, filtering the data messages according to the labels of the data messages of the virtual machines.

Step S103, carrying out cluster analysis on the data messages with the labels to obtain a flow data statistical result.

And step S104, obtaining the flow monitoring information of each virtual machine according to the flow data statistical result.

And step S105, monitoring the flow of each virtual machine according to the flow monitoring information.

Referring to fig. 3, which is a second flowchart of a method for monitoring traffic of a virtual machine, where the virtual machine is not allocated with a separate IP according to an embodiment of the present invention, the method adds a specific implementation manner of filtering according to a label to the method of fig. 1, and includes:

and step S200, respectively allocating unique labels to the virtual machines in advance.

In the step, the unique labels are respectively distributed when the virtual machines are configured, and the data messages entering and exiting from the virtual machines can be uniquely determined by distributing the unique labels to the virtual machines in advance.

The tags in this step include a normal tag and a suspected malicious tag, which may specifically be ID numbers, for example, the ID tags are: 1. 2, 3 … …. In addition, for the incoming and outgoing messages in the suspected malicious virtual machine, a suspected malicious tag may be set, for example, the ID number of the message is set to 0, or may also be set to NULL. The specific configuration may be set according to actual needs, and is not particularly limited herein.

Step S201, obtaining data messages of each virtual machine.

Specifically, all data packets transmitted by all virtual machines in the network may be acquired through the hosts of the virtual machines, and specifically, the embodiment of the present invention does not specifically limit the execution subject of the specific action. For example, in a feasible implementation manner, the data packet information of each virtual machine may be acquired by a third party application program in the host of the virtual machine or other devices in communication with the host, and the acquired data packet information of each virtual machine is reported to the virtual machine traffic monitoring management platform by the third party application program or other devices in communication with the host, so that the data packet of each virtual machine can be acquired in the process.

The acquisition operation can be real-time or non-real-time, and a user can set the acquisition operation according to actual needs.

Step S202, filtering the data message according to the label of the data message of each virtual machine.

In this step, since the unique tag has been previously assigned to each virtual machine in step S200, the tags of the data packets entering and exiting from each virtual machine are also uniquely determined, and the tags of the data packets are the same as the unique tags previously assigned to each virtual machine.

Because the data messages entering and exiting each virtual machine obtain the unique label, the data messages with the same label can be classified according to the label of the data message.

The filtering in this step is performed by using a netfilter/iptables IP packet filtering system.

Step S203, carrying out cluster analysis on the data message with the label to obtain a flow data statistical result.

Specifically, the data packets with the same label are respectively counted to obtain the traffic data statistical result of the corresponding virtual machine with the same label, and the traffic data statistical result is reported to the virtual machine traffic monitoring management platform. The traffic data statistics result may be, for example, transmission time, data packet size, uplink and downlink types, etc. of data packets with the same label. The flow data statistics are shown in the following table:

further, step S203 further includes: step S2030, judging whether the label is a normal label, if so, executing step S2031, otherwise, executing step S2032; step S2031, determining the data message with the normal label as a normal message, and carrying out traffic data statistics; step S2032, determining the data message with the malicious label as the data message generated by the suspected malicious virtual machine, and sending a reminding signal to the flow monitoring management platform. Wherein, the reminder signal includes: a text reminding signal, a sound reminding signal, a vibration reminding signal and the like. Specifically, the corresponding malicious tag data packet reminding threshold may be set according to an actual requirement for a security level, where the security level may be, for example, a high security level, a medium security level, or a low security level, and the corresponding malicious tag data packet reminding threshold is set to 1, 5, or 10 suspected malicious data packets, which is only an example and is not limited herein, and may also be other setting manners that may occur to those skilled in the art.

And step S204, obtaining the flow monitoring information of each virtual machine according to the flow data statistical result.

And the virtual machine flow monitoring management platform calculates and obtains the flow monitoring information of each virtual machine according to the obtained flow data statistical result of each virtual machine. The traffic monitoring information includes: upload/download speed, upload/download traffic per unit time, total traffic, etc.

And S205, monitoring the flow of each virtual machine according to the flow monitoring information.

The method and the system for monitoring the flow of the virtual machines distribute labels to the virtual machines in advance, classify and filter the data messages with the same label according to the labels of the data messages entering and exiting the virtual machines, perform cluster analysis on the data messages with the same label to obtain a flow data statistical result, further obtain the flow monitoring information of each virtual machine, and finally monitor the flow of each virtual machine according to the flow monitoring information, so that the problem of difficulty in monitoring the flow of the virtual machines without independent network cards is solved. In addition, the data messages filtered according to the labels are determined to be the data messages generated by the suspected malicious virtual machine, and the reminding signals are sent out, so that the risk of data tampering is reduced, the safety of the virtual machine is improved, and the safety degree of data assets is improved.

Fig. 4 shows an application scenario of the method and system for monitoring the flow of a container according to the embodiment of the present invention. The container flow monitoring management platform comprises monitoring equipment and a server, wherein the server sends flow data of each container acquired by each host machine to the monitoring equipment, and monitoring personnel for monitoring the flow of the container monitor the flow of the container through the monitoring equipment.

Referring to fig. 5, a first flowchart of a method for monitoring traffic of a container to which an independent IP is not allocated according to an embodiment of the present invention is shown, the method including:

step S301, obtaining data messages of each container.

Step S302, filtering the data message according to the label of the data message of each container.

Step S303, carrying out cluster analysis on the data message with the label to obtain a flow data statistical result.

And step S304, obtaining flow monitoring information of each container according to the flow data statistical result.

And S305, monitoring the flow of each container according to the flow monitoring information.

Please refer to fig. 6, which is a second flowchart of a method for monitoring traffic of a container, where the container is not allocated with an independent IP, and the container is specifically a docker container, compared with the method in fig. 3, the method adds a specific implementation manner of filtering according to a label, and specifically, nfacct implements container-level traffic monitoring by means of iptables and a cgroups-based network label; namely, completing container message filtering through a cgroups network label, and then carrying out flow information statistics on the filtered message; the method comprises the following steps:

and step S400, respectively allocating unique labels to the containers in advance.

In the step, the unique labels are respectively distributed when the containers are configured, and the unique labels are distributed to the containers in advance, so that the data messages entering and exiting from the containers can be uniquely determined.

The tags in this step include a normal tag and a suspected malicious tag, which may specifically be ID numbers, for example, the ID tags are: 1. 2, 3 … …. In addition, for the incoming and outgoing messages in the suspected malicious container, the suspected malicious tag may be set, for example, the ID number of the message is set to 0, or may also be set to NULL. The specific configuration may be set according to actual needs, and is not particularly limited herein.

Step S401, obtaining the data message of each container.

Specifically, all data packets transmitted by all containers in the network may be acquired by the hosts of the containers, and specifically, the embodiment of the present invention does not specifically limit the execution subject of the specific action. For example, in a feasible implementation manner, the data packet information of each container may be collected by a third party application program in the host of the container or other devices in communication with the host, and the collected data packet information of each container is reported to the container traffic monitoring management platform by the third party application program or other devices in communication with the host, so that the data packet of each container can be acquired in the process.

The acquisition operation can be real-time or non-real-time, and a user can set the acquisition operation according to actual needs.

Each host may include 0-16 containers.

And S402, filtering the data messages according to the labels of the data messages of the containers.

In this step, since a unique label has been previously assigned to each container in step S400, the labels of the data packets that have entered and exited from each container are also uniquely determined, and the label of each data packet is the same as the unique label previously assigned to each container.

Because the data messages entering and exiting each container obtain the unique label, the data messages with the same label can be classified according to the label of the data message.

The filtering in this step is performed by using a netfilter/iptables IP packet filtering system.

Step S403, performing cluster analysis on the data messages with the labels to obtain a flow data statistical result.

Specifically, the data packets with the same label are respectively counted to obtain the traffic data statistical result of the corresponding container with the same label, and the traffic data statistical result is reported to the container traffic monitoring management platform. The traffic data statistics result may be, for example, transmission time, data packet size, uplink and downlink types, etc. of data packets with the same label. The flow data statistics are shown in the following table:

further, step S403 further includes: step 4030, determine whether the label is a normal label, if it is a normal label, executestep 4031, otherwise, executestep 4032; step S4031, confirm the data message with normal label as the normal message, and carry on the data statistics of the flowrate; step S4032, the data packet with the malicious label is determined as a data packet generated by a suspected malicious container, and a warning signal is sent to the traffic monitoring and managing platform. Wherein, the reminder signal includes: a text reminding signal, a sound reminding signal, a vibration reminding signal and the like. Specifically, the corresponding malicious tag data packet reminding threshold may be set according to an actual requirement for a security level, where the security level may be, for example, a high security level, a medium security level, or a low security level, and the corresponding malicious tag data packet reminding threshold is set to 1, 5, or 10 suspected malicious data packets, which is only an example and is not limited herein, and may also be other setting manners that may occur to those skilled in the art.

And S404, acquiring flow monitoring information of each container according to the flow data statistical result.

And the container flow monitoring management platform calculates and obtains the flow monitoring information of each container according to the obtained flow data statistical result of each container. The traffic monitoring information includes: upload/download speed, upload/download traffic per unit time, total traffic, etc.

And S405, monitoring the flow of each container according to the flow monitoring information.

The method and the system for monitoring the flow of the container distribute labels to the containers in advance, classify and filter the data messages according to the labels of the data messages entering and exiting the containers, perform cluster analysis on the data messages with the same label to obtain a flow data statistical result, further obtain flow monitoring information of each container, and finally monitor the flow of each container according to the flow monitoring information, so that the problem of difficulty in monitoring the flow of the container without an independent network card is solved. In addition, the data messages filtered according to the labels are determined to be the data messages generated by the suspected malicious container, and a reminding signal is sent out, so that the risk of data tampering is reduced, the safety of the container is improved, and the safety degree of data assets is improved.

When the method and the system for monitoring the container flow are implemented specifically, certain basic environment guarantee is needed, the method and the system mainly comprise three links for guaranteeing normal acquisition of data, the three links are respectively the basic environment guarantee of the system, and the container flow monitoring configuration is associated with the platform system and when a flow acquisition rule is set. Realizing flow counting, namely, depending on support of iptables for cgroups net _ cls configuration match rule and loading nfacct and xt _ cgroup kernel modules; loading an xt _ cgroup module of a corresponding version according to the linux kernel version; nfacct belongs to a transplanting module, and not only module loading is completed during processing, but also so files such as netlink and the like need to be updated; in view of the above logic, the related information and the logic flow are packaged and packaged, and the nodes are deployed uniformly.

When the method and the system for monitoring the container flow are implemented specifically, the filtering is carried out by utilizing a netfilter/iptables IP information packet filtering system, and the method and the system are realized based on a netfilter-iptables system.

netfilter is a series of call entries embedded in the kernel IP protocol stack, and is set on the path of message processing, which corresponds to a series of hook functions. And iptables is a rule defining how a hook function on netfilter works; when a message enters a call entry on the netfilter, the message information automatically flows into an NF _ HOOK function, and the function filters out a corresponding NF _ HOOK _ OPTS _ LIST, namely an NF _ HOOKS chain, based on the protocol type in the message information and the current call entry. The NF _ hoks chain can provide extra-core logic registration, so that the extra-core logic can be associated with a hook of the netfilter, based on this point, the iptables becomes an extra-core configuration tool of the netfilter, and the NF _ hoks becomes a relation link message of the netfilter and the iptables, enters a data structure of the iptables after passing through NF _ HOOKS, and passes through the chain, the rules and the matches/target in sequence at this time, and the processes belong to a well-regulated data structure inside the iptables and are regarded as address information in a unified manner. After addressing, the specific kernel processing logic is positioned, the message enters a kernel processing module, and after the message is processed, the message returns to a calling inlet. Therefore, the netfilter-iptables expansion can be realized in multiple links, namely parallel expansion can be realized from nf _ hooks registration to matches, but at present, the parallel expansion mainly surrounds matches/target, and target can only appear once for a rule, and the realization process of the method needs two times of processing, so that the method is selected to expand matches.

The following provides a complete application flow of iptables.

The iptables is divided into a core part and an out-of-core part, and the user is set as the out-of-core part. The extra-core part firstly carries out user setting on the iptables, analyzes matches, judges whether match-name exists or not, judges whether match self-defined check parameters are legal or not if the match-name exists, merges data structures if the parameters pass verification, then realizes communication through socket and completes the process of writing in the core function netfilter from a user layer to form a part of an iptables core table, realizes processing logic hooks, associates new table chain information with table chain information of service network messages, and then enters the matches for processing.

The following is a specific example of the method for monitoring the flow of the container, and specifically, the flow of the docker container is monitored by using an iptables + xt _ cgroup + nfacct flow monitoring scheme.

In a common application scene, a docker management platform is built for starting and managing containers, and monitoring operation indexes of the containers, such as cpu, memory, flow and the like, so as to monitor operation conditions of services in real time. The docker management platform can quickly and accurately realize flow monitoring by combining the flow monitoring scheme, and a timing chart of the implementation of the scheme is shown in fig. 7.

The flow monitoring implementation process of the docker management platform is as follows:

1, the Docker management platform sends a container starting command to the host, the Docker process of the host creates a container and creates a cgroup configuration corresponding to the container

2. The host machine writes the unique id corresponding to the container into the class of the net _ class subsystem of the cgroup, and the class is used for labeling the incoming traffic packet

3. Host machine uses nfacct program to create nfacct statistical object corresponding to container

4. The host adds a filtering rule through an iptable command, and binds the nfacct object and a traffic label (classid). Thus, the iptable monitors the traffic packet of the corresponding label of the container and uses nfacct process traffic statistics.

5. The host machine obtains the traffic statistical data of the nfacct object corresponding to the container through the nfacct command and reports the traffic statistical data to the management platform

6. The management platform receives the flow data reported by the host machine, displays and monitors the alarm, and the service operation and maintenance personnel can control the flow condition of the container in real time.

Fig. 8 is a flow monitoring diagram of an actual docker management platform according to the above scheme. Based on the statistical effect of nfacct, we adopt a virtual-to-one mode to collect and report data, compare the data with OMG-ITIL single-machine data display, and effectively verify the statistical reliability, please refer to FIG. 9 specifically.

Please refer to fig. 10, which is a schematic structural diagram of a system for performing traffic monitoring on a container/virtual machine according to an embodiment of the present invention, where the system includes a trafficmonitoring management platform 10, ahost node 11, and a container/virtual machine 12. Referring to fig. 11, the trafficmonitoring management platform 10 includes a startcommand generating module 20, a statisticalresult receiving module 21, a traffic monitoringinformation obtaining module 22, and atraffic monitoring module 23.

The startingcommand generating module 20 is configured to generate a command for controlling the virtual machine to start, and send the command to the host node;

the statisticalresult receiving module 21 is configured to receive a statistical result of the flow data reported by the host;

the traffic monitoringinformation obtaining module 22 is configured to obtain traffic monitoring information of each virtual machine according to the traffic data statistical result;

theflow monitoring module 23 is configured to monitor the flow of each virtual machine according to the flow monitoring information;

referring to fig. 11, thehost node 11 includes atrigger start module 31, a datapacket obtaining module 32, afiltering module 33, a traffic data statisticalresult obtaining module 34, and a traffic data statistical result reporting module 35;

the triggering and startingmodule 31 is configured to trigger a virtual machine to start according to the starting command sent by the traffic monitoring and management platform;

the datapacket obtaining module 32 is configured to obtain data packets of each virtual machine;

thefiltering module 33 is configured to filter the data packet according to the label of the data packet of each virtual machine;

the traffic data statisticalresult obtaining module 34 is configured to perform cluster analysis on the data packets with the tags to obtain a traffic data statistical result;

and the traffic data statistical result reporting module 35 is configured to report the traffic data statistical result to a traffic monitoring management platform.

The host node further comprises: thelabel distribution module 30 is configured to distribute unique labels to the virtual machines in advance, where the labels of the data packets of the virtual machines are the same as the unique labels pre-distributed to the virtual machines.

Thefiltering module 33 specifically includes: the normal message determining module is used for determining the data message with the normal label as a normal message and carrying out flow data statistics; and the suspected malicious message determining module is used for determining the data message with the malicious label as the data message generated by the suspected malicious virtual machine and sending a reminding signal to the flow monitoring management platform.

The tags include normal tags and suspected malicious tags.

The reminding signal comprises: a text reminding signal, a sound reminding signal and a vibration reminding signal.

The traffic monitoring information includes: upload/download speed, number of uploads/downloads, total traffic.

The filtering the data message according to the label of the data message includes: filtering is performed using a netfilter/iptables IP packet filtering system.

The container is specifically a Docker container.

The system for monitoring the flow of the container/virtual machine distributes labels to the container/virtual machine in advance, performs classification filtering according to the labels of data messages entering and exiting each container/virtual machine, performs cluster analysis on the data messages with the same label to obtain a flow data statistical result, further obtains the flow monitoring information of each container/virtual machine, and finally monitors the flow of each container/virtual machine according to the flow monitoring information, thereby solving the problem of difficult flow monitoring of the container/virtual machine without an independent network card. In addition, the data messages filtered according to the labels are determined to be the data messages generated by the suspected malicious container, and a reminding signal is sent out, so that the risk of data tampering is reduced, the safety of the container/virtual machine is improved, and the safety degree of data assets is improved.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the above-described apparatus embodiments are merely illustrative, for example, the division of the modules is only one logical function division, and there may be other division manners in actual implementation, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. Some or all of the modules/units can be selected according to actual needs to achieve the purpose of implementing the scheme of the invention.

In addition, each module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (10)

1. A method for monitoring the flow of a virtual machine, wherein the virtual machine is not allocated with an independent IP and is not configured with an independent network card to share a mother machine network, is characterized by comprising the following steps:

acquiring data messages of each virtual machine, specifically including acquiring all data packets transmitted by all virtual machines in a network through host machines of the virtual machines;

filtering the data messages according to the labels of the data messages of the virtual machines, wherein the labels of the data messages of the virtual machines are respectively the same as the unique labels pre-allocated to the virtual machines;

performing cluster analysis on the data messages with the labels to obtain a flow data statistical result;

acquiring flow monitoring information of each virtual machine according to the flow data statistical result;

and monitoring the flow of each virtual machine according to the flow monitoring information.

2. A method for monitoring the flow of a container, wherein the container is not allocated with an independent IP and is not configured with an independent network card sharing mother machine network, the method comprises the following steps:

acquiring data messages of each container, specifically including acquiring all data packets transmitted by all virtual machines in a network through host machines of the virtual machines;

filtering the data messages according to the labels of the data messages of the containers, wherein the labels of the data messages of the containers are respectively the same as the unique labels pre-allocated to the containers;

performing cluster analysis on the data messages with the labels to obtain a flow data statistical result;

acquiring flow monitoring information of each container according to the flow data statistical result;

and monitoring the flow of each container according to the flow monitoring information.

3. The method of claim 2, wherein the tags comprise normal tags and suspected malicious tags.

4. The method according to claim 2, wherein the filtering the data packet according to the label of the data packet comprises:

determining the data message with the normal label as a normal message, and carrying out flow data statistics;

and determining the data message with the malicious label as the data message generated by the suspected malicious virtual machine, and sending a reminding signal to the traffic monitoring management platform.

5. The method of claim 4, wherein the alert signal comprises: a text reminding signal, a sound reminding signal and a vibration reminding signal.

6. A method of flow monitoring a vessel according to any of claims 2 to 5, wherein the flow monitoring information comprises: upload/download speed, number of uploads/downloads, total traffic.

7. The method according to any of claims 2 to 5, wherein the filtering the data packets according to their labels comprises: filtering is performed using a netfilter/iptables IP packet filtering system.

8. Method for flow monitoring of a vessel according to any of claims 2-5, characterised in that the vessel is embodied as a Docker vessel.

9. A system for monitoring the flow of a virtual machine, wherein the virtual machine is not allocated with an independent IP and is not configured with an independent network card to share a mother machine network, and is characterized by comprising a flow monitoring management platform, a host machine node and the virtual machine;

the flow monitoring management platform comprises a starting command generation module, a statistical result receiving module, a flow monitoring information acquisition module and a flow monitoring module;

the starting command generating module is used for generating a command for controlling the virtual machine to start and sending the command to the host machine node;

the statistical result receiving module is used for receiving the statistical result of the flow data reported by the host machine;

the traffic monitoring information obtaining module is used for obtaining traffic monitoring information of each virtual machine according to the traffic data statistical result;

the flow monitoring module is used for monitoring the flow of each virtual machine according to the flow monitoring information;

the host machine node comprises a trigger starting module, a data message acquiring module, a filtering module, a flow data statistical result acquiring module and a flow data statistical result reporting module;

the triggering and starting module is used for triggering the virtual machine to start according to the starting command sent by the flow monitoring and management platform;

the data message acquisition module is used for acquiring data messages of all the virtual machines, and specifically comprises the steps of acquiring all data packets transmitted by all the virtual machines in a network through host machines of the virtual machines;

the filtering module is used for filtering the data messages according to the labels of the data messages of the virtual machines, wherein the labels of the data messages of the virtual machines are respectively the same as the unique labels pre-allocated to the virtual machines;

the flow data statistical result obtaining module is used for carrying out cluster analysis on the data messages with the labels to obtain flow data statistical results;

and the traffic data statistical result reporting module is used for reporting the traffic data statistical result to a traffic monitoring management platform.

10. A system for monitoring the flow of a container, the container is not allocated with an independent IP and is not configured with an independent network card to share a mother machine network, and the system is characterized by comprising a flow monitoring management platform, a host machine node and a container;

the flow monitoring management platform comprises a starting command generation module, a statistical result receiving module, a flow monitoring information acquisition module and a flow monitoring module;

the starting command generating module is used for generating a command for controlling the starting of the container and sending the command to the host machine node;

the statistical result receiving module is used for receiving the statistical result of the flow data reported by the host machine;

the flow monitoring information obtaining module is used for obtaining the flow monitoring information of each container according to the flow data statistical result;

the flow monitoring module is used for monitoring the flow of each container according to the flow monitoring information;

the host machine node comprises a trigger starting module, a data message acquiring module, a filtering module, a flow data statistical result acquiring module and a flow data statistical result reporting module;

the triggering and starting module is used for triggering the container to start according to the starting command sent by the flow monitoring and management platform;

the data message acquisition module is used for acquiring data messages of each container, and specifically comprises the steps of acquiring all data packets transmitted by all virtual machines in a network through host machines of the virtual machines;

the filtering module is used for filtering the data messages according to the labels of the data messages of the containers, wherein the labels of the data messages of the containers are respectively the same as the unique labels pre-allocated to the containers;

the flow data statistical result obtaining module is used for carrying out cluster analysis on the data messages with the labels to obtain flow data statistical results;

and the traffic data statistical result reporting module is used for reporting the traffic data statistical result to a traffic monitoring management platform.

Images