CN110995855B - Microservice cluster scheduling method, scheduling device, and computer-readable storage medium - Google Patents
Microservice cluster scheduling method, scheduling device, and computer-readable storage mediumDownload PDFInfo
- Publication number
- CN110995855B CN110995855BCN201911280416.2ACN201911280416ACN110995855BCN 110995855 BCN110995855 BCN 110995855BCN 201911280416 ACN201911280416 ACN 201911280416ACN 110995855 BCN110995855 BCN 110995855B
- Authority
- CN
- China
- Prior art keywords
- control module
- microservice
- node
- physical machine
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034methodMethods0.000titleclaimsabstractdescription49
- 238000010586diagramMethods0.000claimsabstractdescription19
- 238000004590computer programMethods0.000claimsdescription6
- 238000010276constructionMethods0.000claimsdescription3
- 230000009286beneficial effectEffects0.000abstractdescription3
- 238000005516engineering processMethods0.000description18
- 230000008569processEffects0.000description6
- 238000004891communicationMethods0.000description4
- 238000012545processingMethods0.000description3
- 230000008901benefitEffects0.000description2
- 230000015556catabolic processEffects0.000description1
- 230000008878couplingEffects0.000description1
- 238000010168coupling processMethods0.000description1
- 238000005859coupling reactionMethods0.000description1
- 230000007423decreaseEffects0.000description1
- 238000006731degradation reactionMethods0.000description1
- 238000011161developmentMethods0.000description1
- 230000006870functionEffects0.000description1
- 230000003993interactionEffects0.000description1
- 238000002955isolationMethods0.000description1
- 238000012986modificationMethods0.000description1
- 230000004048modificationEffects0.000description1
- 238000009662stress testingMethods0.000description1
- 238000012360testing methodMethods0.000description1
Images
Classifications
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及信息技术领域，具体涉及一种微服务集群调度方法、调度装置及计算机可读存储介质。The present invention relates to the field of information technology, in particular to a microservice cluster scheduling method, a scheduling device and a computer-readable storage medium.
背景技术Background technique
云计算作为一种新型服务提供方式，获得了产业界和学术界的青睐。云计算的关键技术就是虚拟化技术，通过将各类资源进行虚拟化，云计算服务提供商能够很便捷的将各类资源进行定制交付给用户使用，众多应用也逐渐开始迁移到云计算集群内。As a new service provision method, cloud computing has been favored by industry and academia. The key technology of cloud computing is virtualization technology. By virtualizing various resources, cloud computing service providers can easily customize and deliver various resources to users, and many applications are gradually migrating to cloud computing clusters. .
传统的虚拟化技术包括KVM和Xen等。但是传统的虚拟化技术由于过于笨重，对于应用集群内某个组件进行创建，修改以及迁移操作都十分的复杂，因此云计算服务提供商需要更加轻量级的虚拟化技术。容器技术则是一种轻量级的操作系统级的虚拟化技术。相较于传统的虚拟化技术对于硬件层的虚拟化，容器的虚拟化停留在操作系统层，使其无论是创建，修改还是迁移都十分的便捷。Traditional virtualization technologies include KVM and Xen. However, the traditional virtualization technology is too cumbersome to create, modify and migrate a component in the application cluster. Therefore, cloud computing service providers need more lightweight virtualization technology. Container technology is a lightweight operating system-level virtualization technology. Compared with the traditional virtualization technology for hardware layer virtualization, container virtualization stays at the operating system layer, making it very convenient to create, modify or migrate.
随着容器技术的发展成熟，基于容器技术的云计算系统(以下简称容器云)已经开始逐步的取代传统的基于虚拟机的云计算系统。由于容器具有轻量化的特点，容器的部署更加便捷。轻量化的容器使得一个服务能够运行在单个容器中，从而降低了服务之间的耦合性，提高了服务更新迭代的效率。利用容器的这种特性，越来越多的开发者选择利用容器部署在线服务应用。这些基于容器技术的在线服务应用被称为微服务系统。With the development and maturity of container technology, cloud computing systems based on container technology (hereinafter referred to as container clouds) have begun to gradually replace traditional cloud computing systems based on virtual machines. Due to the lightweight characteristics of containers, the deployment of containers is more convenient. Lightweight containers enable a service to run in a single container, thereby reducing the coupling between services and improving the efficiency of service update iterations. Taking advantage of this feature of containers, more and more developers choose to use containers to deploy online service applications. These online service applications based on container technology are called microservice systems.
容器技术很快的被各类云计算服务提供商使用。由于容器的这些特点，用户在部署其应用的时候往往会将每个组件运行在独立的容器内，以便方便快捷的对应用进行维护。这造成了容器云复杂的内部结构。同时容器的弱隔离性的特点也导致了容器之间相互干扰较为严重。当多个容器部署在一台物理机上时，不同容器之间的会产生相互竞争。这虽然保证了物理机资源的高效利用，但是却造成了服务质量的下降，如连接中断重连，数据包丢失重传等。虽然基于容器技术的微服务系统能够实现便捷部署和快速迭代更新，但是也是由于容器的轻量化特性，云集群内每台物理机上往往会运行几十甚至上百个容器。这些容器相互竞争，造成了虽然系统资源利用率高，但是每个微服务的服务质量较差，从而造成了整体服务质量无法保障，严重影响了用户体验。Container technology is quickly being used by various cloud computing service providers. Due to these characteristics of containers, users often run each component in a separate container when deploying their applications, so that applications can be maintained conveniently and quickly. This results in the complex internal structure of container clouds. At the same time, the weak isolation of containers also leads to serious mutual interference between containers. When multiple containers are deployed on a physical machine, different containers will compete with each other. Although this ensures efficient utilization of physical machine resources, it causes service quality degradation, such as connection interruption and reconnection, and data packet loss and retransmission. Although the microservice system based on container technology can realize convenient deployment and rapid iterative update, due to the lightweight characteristics of containers, dozens or even hundreds of containers are often running on each physical machine in a cloud cluster. These containers compete with each other, resulting in high system resource utilization, but the service quality of each microservice is poor, resulting in the inability to guarantee the overall service quality and seriously affecting the user experience.
因此，需要一种针对基于容器技术的微服务系统的调度系统以提高微服务系统的服务质量。基于容器技术的云计算平台通常由上千台物理机组成，每台物理机上通常运行数十个容器，每个容器内均运行一个服务。现有的集群任务调度系统针对传统的虚拟机平台的应用实现了较高的集群资源利用率。然而在对基于容器技术的微服务系统内的任务进行调度的时候，现有的调度系统往往只能单纯的提高集群的资源利用率，而缺少对于服务质量的考虑。即传统面向虚拟机的调度算法在应用于微服务系统时往往只考虑优化资源利用率而忽略了不同服务之间的相互干扰，最终造成整体服务质量的下降。Therefore, a scheduling system for a microservice system based on container technology is needed to improve the service quality of the microservice system. A cloud computing platform based on container technology usually consists of thousands of physical machines, each of which usually runs dozens of containers, and each container runs a service. The existing cluster task scheduling system achieves high cluster resource utilization for the application of the traditional virtual machine platform. However, when scheduling tasks in a microservice system based on container technology, the existing scheduling systems often can only simply improve the resource utilization of the cluster, but lack consideration of service quality. That is, when traditional virtual machine-oriented scheduling algorithms are applied to micro-service systems, they often only consider optimizing resource utilization and ignore the mutual interference between different services, which ultimately leads to a decline in overall service quality.
综上所述，急需针对现有的集群任务调度系统的缺点，提出一种针对微服务系统的基于最小花费最大流的调度方法和调度系统。To sum up, it is urgent to propose a scheduling method and scheduling system based on the minimum cost and maximum flow for the microservice system, aiming at the shortcomings of the existing cluster task scheduling system.
发明内容SUMMARY OF THE INVENTION
有鉴于此，为了解决现有的集群任务调度系统不能同时兼顾资源利用率和服务质量的问题。本发明提供一种微服务集群调度方法，其包括如下步骤：In view of this, in order to solve the problem that the existing cluster task scheduling system cannot take into account both resource utilization and service quality. The present invention provides a microservice cluster scheduling method, which includes the following steps:
步骤S1，控制模块获取微服务集群内物理机可用资源，并计算物理机可用容量以及花费；Step S1, the control module obtains the available resources of the physical machine in the microservice cluster, and calculates the available capacity and cost of the physical machine;
步骤S2，控制模块利用最小花费最大流算法计算微服务系统架构图中每一层的服务容器分配的地址；Step S2, the control module calculates the address allocated by the service container of each layer in the microservice system architecture diagram by using the minimum cost maximum flow algorithm;
步骤S3，控制模块将任务调度方案交由调度器执行调度。Step S3, the control module submits the task scheduling scheme to the scheduler for scheduling.
较佳地，步骤S1的具体步骤包括：Preferably, the specific steps of step S1 include:
步骤S11，控制模块确定源节点、终点与中间节点；Step S11, the control module determines a source node, an end point and an intermediate node;
步骤S12，控制模块计算开销流量网络每个物理机的可用容量以及每个服务节点的流量大小；Step S12, the control module calculates the available capacity of each physical machine of the overhead traffic network and the traffic size of each service node;
步骤S13，控制模块计算开销流量网络每个服务节点分配到不同的物理机上的开销；Step S13, the control module calculates the overhead that each service node of the overhead traffic network allocates to different physical machines;
步骤S14，控制模块调用最小开销最大流算法计算最大流以及其对应的分配方法；Step S14, the control module invokes the minimum overhead maximum flow algorithm to calculate the maximum flow and its corresponding allocation method;
步骤S15，控制模块更新物理机的剩余可用资源量。Step S15, the control module updates the remaining available resources of the physical machine.
较佳地，所述步骤S11中，所述源节点包括微服务集群的服务节点以及虚拟占用节点，所述虚拟占用节点用于将已经占用的资源拟作为虚拟的服务节点所提交的请求。Preferably, in the step S11, the source node includes a service node of the micro-service cluster and a virtual occupied node, and the virtual occupied node is used to use the occupied resource as a request submitted by the virtual service node.
较佳地，所述中间节点是物理机节点，所述物理机节点包含物理机的位置标记即物理机所属的机架、机柜和机房信息。Preferably, the intermediate node is a physical machine node, and the physical machine node includes a location mark of the physical machine, that is, information about a rack, a cabinet, and a computer room to which the physical machine belongs.
较佳地，控制模块判断某个服务组件所需要的资源是否大于某台物理机的剩余可用资源，若大于，则该服务组件将强制不允许向这台物理机上分配，其余所有节点能够进行任意分配。Preferably, the control module determines whether the resources required by a certain service component are greater than the remaining available resources of a certain physical machine. If it is larger than the remaining available resources of a certain physical machine, the service component will be forcibly not allowed to be allocated to this physical machine, and all other nodes can perform arbitrary operations. distribute.
较佳地，步骤S2的具体步骤包括：Preferably, the specific steps of step S2 include:
步骤S21，控制模块接收微服务架构图，并计算初始系统容量；Step S21, the control module receives the microservice architecture diagram, and calculates the initial system capacity;
步骤S22，控制模块分割微服务集群的架构图形成微服务层级图；Step S22, the control module divides the architecture diagram of the microservice cluster to form a microservice hierarchy diagram;
步骤S23，控制模块取微服务层级图的第i层的节点，按照费用网络与容量网络的构造方法重新计算费用与容量；Step S23, the control module takes the node of the i-th layer of the microservice hierarchy graph, and recalculates the cost and capacity according to the construction method of the cost network and the capacity network;
步骤S24，控制模块调用微服务集群最小开销最大流方法计算第i层节点的分配策略；Step S24, the control module invokes the microservice cluster minimum overhead maximum flow method to calculate the allocation strategy of the i-th layer node;
步骤S25，控制模块判断是否所有的节点都已分配完成，分配完成则继续执行步骤S26，若未分配完成，则取微服务层级图下一层，重复执行步骤S23；Step S25, the control module determines whether all the nodes have been allocated, and if the allocation is completed, continue to perform step S26, if not, take the next layer of the microservice hierarchy diagram, and repeat step S23;
步骤S26，输出每个节点的分配结果。Step S26, output the assignment result of each node.
本发明还提供一种微服务集群调度装置，该装置包括控制模块，所述控制模块执行程序时能够实现上述的微服务集群调度方法。The present invention also provides a micro-service cluster scheduling device, which includes a control module, which can implement the above-mentioned micro-service cluster scheduling method when executing a program.
本发明还一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被执行时能够实现上述的微服务集群调度方法。The present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and characterized in that, when the computer program is executed, the above-mentioned microservice cluster scheduling method can be implemented.
与现有技术比较本发明的有益效果在于：Compared with the prior art, the beneficial effects of the present invention are:
本发明提供了一种微服务集群方法，该方法兼顾了物理机集群的资源使用率以及微服务集群的服务质量，使用该调度方法，既保证了物理机集群的高资源使用率，又提高了微服务集群的服务质量。The invention provides a micro-service cluster method, which takes into account the resource utilization rate of the physical machine cluster and the service quality of the micro-service cluster. Using the scheduling method not only ensures the high resource utilization rate of the physical machine cluster, but also improves the Quality of service for microservice clusters.
附图说明Description of drawings
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.
图1为本发明中微服务集群调度方法的流程示意图；1 is a schematic flowchart of a microservice cluster scheduling method in the present invention;
图2为本发明中微服务集群调度方法步骤S1的具体步骤流程示意图；FIG. 2 is a schematic flowchart of the specific steps of step S1 of the microservice cluster scheduling method in the present invention;
图3为本发明中微服务集群调度方法步骤S2的具体步骤流程示意图。FIG. 3 is a schematic flowchart of the specific steps of step S2 of the microservice cluster scheduling method in the present invention.
具体实施方式Detailed ways
以下结合附图，对本发明上述的和另外的技术特征和优点作更详细的说明。The above and other technical features and advantages of the present invention will be described in more detail below with reference to the accompanying drawings.
在本发明的描述中，需要理解的是，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本发明的描述中，“多个”的含义是至少两个，除非另有明确具体的限定。In the description of the present invention, it should be understood that the terms "first" and "second" are only used for description purposes, and cannot be interpreted as indicating or implying relative importance or the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means at least two unless expressly and specifically defined otherwise.
在本发明中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”、“固定”等术语应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或成一体；可以是机械连接，也可以是电连接或彼此可通讯；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通或两个元件的相互作用关系，除非另有明确的限定。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本发明中的具体含义。In the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrated; it can be a mechanical connection or an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two components or the interaction relationship between the two components, unless otherwise expressly qualified. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention.
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本发明实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本发明。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本发明的描述。In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.
为了说明本发明所述的技术方案，下面通过具体实施例来进行说明。In order to illustrate the technical solutions of the present invention, the following specific embodiments are used for description.
本发明提供一种微服务集群调度方法，其包括如下步骤：The present invention provides a microservice cluster scheduling method, which includes the following steps:
步骤S1，控制模块获取微服务集群内物理机可用资源，并计算物理机可用容量以及花费；Step S1, the control module obtains the available resources of the physical machine in the microservice cluster, and calculates the available capacity and cost of the physical machine;
步骤S2，控制模块利用最小花费最大流算法计算微服务系统架构图中每一层的服务容器分配的地址；Step S2, the control module calculates the address allocated by the service container of each layer in the microservice system architecture diagram by using the minimum cost maximum flow algorithm;
步骤S3，控制模块将任务调度方案交由调度器执行调度。Step S3, the control module submits the task scheduling scheme to the scheduler for scheduling.
执行步骤S1的具体步骤包括：The specific steps of executing step S1 include:
步骤S11，控制模块确定源节点、终点与中间节点。Step S11, the control module determines a source node, an end point and an intermediate node.
源节点包括微服务集群的服务节点以及虚拟占用节点。其中虚拟占用节点的作用是抽象物理机节点上已经占用的资源，将这些已经占用的资源看当作是一个虚拟的服务节点提交的请求。将已占用的物理资源抽象成一个虚拟节点的有益效果在于减少了服务节点调度过程中对可用容量重新计算的复杂度。Source nodes include service nodes and virtual occupied nodes of the microservice cluster. The function of the virtual occupied node is to abstract the resources already occupied on the physical machine node, and regard these occupied resources as a request submitted by a virtual service node. The beneficial effect of abstracting the occupied physical resource into a virtual node is to reduce the complexity of recalculating the available capacity during the scheduling process of the service node.
中间节点是物理机节点，物理机节点包含物理机的位置标记即物理机所属的机架、机柜和机房等信息。终点是虚拟的节点，并没有实际节点与之对应，该终点用于计算最小开销最大流。The intermediate node is a physical machine node, and the physical machine node contains the location mark of the physical machine, that is, information such as the rack, cabinet, and equipment room to which the physical machine belongs. The end point is a virtual node, and there is no actual node corresponding to it, and the end point is used to calculate the minimum cost maximum flow.
步骤S12，控制模块计算开销流量网络每个物理机的可用容量以及每个服务节点的流量大小。Step S12, the control module calculates the available capacity of each physical machine of the overhead traffic network and the traffic size of each service node.
在该方法中，重新抽象了微服务集群服务节点调度的过程，将微服务集群节点调度抽象为开销流量网络，降低了服务节点调度的时间复杂度。In this method, the process of microservice cluster service node scheduling is re-abstracted, and the microservice cluster node scheduling is abstracted into an overhead traffic network, which reduces the time complexity of service node scheduling.
为了提高微服务集群的服务质量，将流量定义为了每秒钟物理机能处理的数据包的数量。因此每条边的最大流量为物理机能够处理的数据包上限，该上限fdmaxi定义为：To improve the quality of service of a microservice cluster, traffic is defined as the number of packets per second processed by a physical machine. Therefore, the maximum flow of each edge is the upper limit of the data packets that the physical machine can handle. The upper limit fdmaxi is defined as:
fdmaxi＝min(k1Ci,k2Mi,k3Di,k4Ni)fdmaxi = min(k1 Ci , k2 Mi , k3 Di , k4 Ni )
其中k1表示第i台物理机可处理的数据包的数量与CPU数量的比例，k2表示第i台物理机可处理的数据包的数量与内存大小的比例，k3表示第i台物理机可处理的数据包的数量与磁盘IO速率的比例，k4表示第i台物理机可处理的数据包的数量与网络速率之间的比例，这些比例可以通过对服务器进行压力测试得到，是一个相对固定的值。where k1 represents the ratio of the number of data packets that can be processed by the i-th physical machine to the number of CPUs, k2 represents the ratio of the number of data packets that can be processed by the i-th physical machine to the memory size, and k3 represents the i-th physical machine. The ratio of the number of data packets that the machine can process to the disk IO rate, k4 represents the ratio between the number of data packets that the i-th physical machine can process and the network speed, these ratios can be obtained by stress testing the server, and are a relatively fixed value.
将物理机可处理的数据包上限作为流量，保证了物理机在处理流量数据时不会出现资源不足的情况，提高了服务质量。The upper limit of the data packets that can be processed by the physical machine is used as the flow, which ensures that the physical machine does not have insufficient resources when processing the flow data, and improves the service quality.
确定了最大容量后，根据此定义本发明中当前边的容量可以定义为：After the maximum capacity is determined, according to this definition, the capacity of the current edge in the present invention can be defined as:
fdpi＝min(k1Cpi,k2Mpi,k3Dpi,k4Npi)fdpi =min(k1 Cpi ,k2 Mpi ,k3 Dpi ,k4 Npi )
其中Cpi表示第i台主机CPU的剩余可用资源，Mpi表示第i台主机内存的剩余可用资源，Dpi表示第i台主机磁盘的剩余可用资源，Npi分别表示第i台主机网络的剩余可用资源。Among them, Cpi represents the remaining available resources of the i-th host CPU, Mpi represents the remaining available resources of the i-th host’s memory, Dpi represents the remaining available resources of the i-th host’s disks, and Npi respectively represents the i-th host’s network resources. Remaining available resources.
第i个节点的流量定义为max(k1Cri,k2Mri,k3Dri,k4Nri)，其中，Cri表示第i个节点CPU的需求，Mri表示第i个节点内存的需求，Dri表示第i个节点磁盘的需求，Nri表示第i个节点网络的需求。The traffic of the ith node is defined as max(k1 Cri , k2 Mri , k3 Dri , k4 Nri ), where Cri represents the CPU demand of the ith node, and Mri represents the ith node The memory requirements of the nodes, Dri represents the disk requirements of the i-th node, and Nri represents the network requirements of the i-th node.
除了流量之外，本发明中的每个物理机节点运行的所有容器组件的资源之和要小于物理机的最大剩余可用资源，因而定义rrpi＝(Cpi,Mpi,Dpi,Npi)，如果某个服务组件所需要的资源大于某台物理机的剩余可用资源，则该服务组件将强制不允许向这台物理机上分配，其余所有节点都可以任意分配到任意的机器上。In addition to traffic, the sum of the resources of all container components running on each physical machine node in the present invention is smaller than the maximum remaining available resources of the physical machine, so rrpi =(Cpi ,Mpi ,Dpi ,Npi ), if the resources required by a service component are greater than the remaining available resources of a physical machine, the service component will forcibly not allow allocation to this physical machine, and all other nodes can be arbitrarily allocated to any machine.
步骤S13，控制模块计算开销流量网络每个服务节点分配到不同的物理机上的开销。Step S13, the control module calculates the overhead allocated to different physical machines by each service node of the overhead traffic network.
每条边的开销是结构图当前层级与上一层级之间节点的连接开销。根据上一层级的节点的位置，当前层级每一个服务节点的开销分为三类。令当前节点为vp，与有连接的上层节点为vpp。如果vpp在待分配的物理机上，则vp分配到该机器上的开销是a1，如果vpp位于待分配的物理机的相同机架上，则vp分配到该机器上的开销是a2，vpp位于待分配的物理机的相同机柜上，则vp分配到该机器上的开销是a3，如果vpp位于待分配的物理机的相同机房内，则vp分配到该机器上的开销是a4，如果vpp位于待分配的物理机的不同机房内，则vp分配到该机器上的开销是a5。a1、a2、a3、a4和a5根据不同的物理机集群测试得出。The cost of each edge is the connection cost of the node between the current level of the structure graph and the previous level. According to the position of the node in the upper level, the cost of each service node in the current level is divided into three categories. Let the current node be vp , and the upper node connected to it be vpp . If vpp is on the physical machine to be allocated, the cost of vp being allocated to that machine is a1 , and if vpp is on the same rack as the physical machine to be allocated, the cost of vp being allocated to that machine is a2 , vpp is located on the same cabinet of the physical machine to be allocated, then the cost of vp being allocated to this machine is a3 , if vpp is located in the same room of the physical machine to be allocated, then vp is allocated to this machine The overhead on the machine is a4 , and if vpp is located in a different room of the physical machine to be allocated, the cost of vp being allocated to that machine is a5 . a1 , a2 , a3 , a4 and a5 are obtained from different physical machine cluster tests.
步骤S14，控制模块调用最小开销最大流算法计算最大流以及其对应的分配方法；Step S14, the control module invokes the minimum overhead maximum flow algorithm to calculate the maximum flow and its corresponding allocation method;
步骤S15，控制模块更新物理机的剩余可用资源量。Step S15, the control module updates the remaining available resources of the physical machine.
执行步骤S2的具体步骤包括：The specific steps of executing step S2 include:
步骤S21，控制模块接收微服务架构图，并计算初始系统容量；Step S21, the control module receives the microservice architecture diagram, and calculates the initial system capacity;
步骤S22，控制模块分割微服务集群的架构图形成微服务层级图；控制模块按照微服务集群的架构图分割应用层级，从而构成微服务层级图。Step S22, the control module divides the architecture diagram of the microservice cluster to form a microservice hierarchy diagram; the control module divides the application hierarchy according to the architecture diagram of the microservice cluster, thereby forming a microservice hierarchy diagram.
令微服务集群中负责接收用户请求的微服务为根节点vr，每个微服务vi所属的层次为max(dist(vr,vi))，其中dist(vr,vi)表示vi与vr距离，记录下该节点所连接的节点的编号。Let the microservice responsible for receiving user requests in the microservice cluster be the root node vr , and the level to which each micro servicevi belongs is max(dist(vr ,vi )), where dist(vr ,vi ) represents The distance between vi and vr , record the number of the node connected to this node.
步骤S23，控制模块取微服务层级图的第i层的节点，按照费用网络与容量网络的构造方法重新计算费用与容量；在该方法中，将不同层级节点之间的连接作为构造开销网络的依据，保证了服务节点与服务节点之间的通信质量。Step S23, the control module takes the node of the i-th layer of the microservice hierarchy graph, and recalculates the cost and capacity according to the construction method of the cost network and the capacity network; According to this, the communication quality between the service node and the service node is guaranteed.
步骤S24，控制模块调用微服务集群最小开销最大流方法计算第i层节点的分配策略；在该方法中，将不同位置的物理机之间的开销设置为不同的权值，尽可能的保证了微服务结构图中的相邻节点之间的通信质量。Step S24, the control module invokes the microservice cluster minimum overhead maximum flow method to calculate the allocation strategy of the i-th layer nodes; in this method, the overheads between physical machines in different locations are set to different weights to ensure as much as possible. Communication quality between adjacent nodes in a microservices structure graph.
步骤S25，控制模块判断是否所有的节点都已分配完成，分配完成则继续执行步骤S26，若未分配完成，则取微服务层级图下一层，重复执行步骤S23；Step S25, the control module determines whether all the nodes have been allocated, and if the allocation is completed, continue to perform step S26, if not, take the next layer of the microservice hierarchy diagram, and repeat step S23;
步骤S26，输出每个节点的分配结果。Step S26, output the assignment result of each node.
本发明提供的微服务集群调度方法是一种基于微服务架构最小开销最大流的调度方法，该方法利用重新构建的流量网络保证微服务集群的资源利用率，主要体现在以下几方面：The microservice cluster scheduling method provided by the present invention is a scheduling method based on the minimum overhead and maximum flow of the microservice architecture. The method utilizes the reconstructed traffic network to ensure the resource utilization rate of the microservice cluster, which is mainly reflected in the following aspects:
1、在该方法中，重新抽象了微服务集群服务节点调度的过程，将微服务集群节点调度抽象为开销流量网络，降低了服务节点调度的时间复杂度。1. In this method, the process of microservice cluster service node scheduling is re-abstracted, and the microservice cluster node scheduling is abstracted into an overhead traffic network, which reduces the time complexity of service node scheduling.
2、在该方法中，将已占用的物理资源抽象成一个虚拟节点，减少了服务节点调度过程中对可用容量重新计算的复杂度。2. In this method, the occupied physical resource is abstracted into a virtual node, which reduces the complexity of recalculating the available capacity during the scheduling process of the service node.
3、在该方法中，将物理机可处理的数据包上限作为流量，保证了物理机在处理流量数据时不会出现资源不足的情况，提高了服务质量。3. In this method, the upper limit of the data packets that can be processed by the physical machine is taken as the flow, which ensures that the physical machine does not have insufficient resources when processing the flow data, and improves the service quality.
进一步，该方法还利用重新构建的开销网络提高微服务集群的服务质量。Further, the method also utilizes the reconstructed overhead network to improve the service quality of the microservice cluster.
主要体现在以下几方面：Mainly reflected in the following aspects:
1、在该方法中，将不同层级节点之间的连接作为构造开销网络的依据，保证了服务节点与服务节点之间的通信质量。1. In this method, the connection between nodes at different levels is used as the basis for constructing an overhead network, which ensures the communication quality between the service node and the service node.
2、在该方法中，将不同位置的物理机之间的开销设置为不同的权值，尽可能的保证了微服务结构图中的相邻节点之间的通信质量。2. In this method, the overheads between physical machines in different locations are set to different weights, so as to ensure the communication quality between adjacent nodes in the microservice structure diagram as much as possible.
现有技术在调度微服务集群任务的时候，采用的是基于可用资源的调度方案，缺少对于微服务关键指标服务质量的考虑，容易造成虽然整个物理机集群资源利用率高但是微服务集群的服务质量不高的情况。When scheduling microservice cluster tasks in the prior art, a scheduling scheme based on available resources is adopted, which lacks consideration of the service quality of key indicators of microservices, which is likely to cause high resource utilization of the entire physical machine cluster, but the services of the microservice cluster A case of poor quality.
本发明综合考虑了物理机集群的资源使用率以及微服务集群的服务质量，提出了一种基于最小开销最大流的调度方法，既保证了物理机集群的高资源使用率，又提高了微服务集群的服务质量。The invention comprehensively considers the resource utilization rate of the physical machine cluster and the service quality of the micro-service cluster, and proposes a scheduling method based on the minimum overhead and maximum flow, which not only ensures the high resource utilization rate of the physical machine cluster, but also improves the micro-service. The quality of service of the cluster.
本发明还提供一种微服务集群调度装置，该装置包括控制模块，所述控制模块执行程序时能够实现本发明中提出的步骤S1至步骤S3的微服务集群调度方法。The present invention also provides a micro-service cluster scheduling apparatus, which includes a control module, which can implement the micro-service cluster scheduling method from steps S1 to S3 proposed in the present invention when the control module executes the program.
本发明还提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现本发明中提出的步骤S1至步骤S3的微服务集群调度方法。The present invention further provides a computer-readable storage medium, which stores a computer program, characterized in that, when the computer program is executed by a processor, the micro-processing of steps S1 to S3 proposed in the present invention is implemented. Service cluster scheduling method.
以上所述实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围，均应包含在本发明的保护范围之内。The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the foregoing implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the within the protection scope of the present invention.
Claims (6)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911280416.2ACN110995855B (en) | 2019-12-13 | 2019-12-13 | Microservice cluster scheduling method, scheduling device, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911280416.2ACN110995855B (en) | 2019-12-13 | 2019-12-13 | Microservice cluster scheduling method, scheduling device, and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110995855A CN110995855A (en) | 2020-04-10 |
| CN110995855Btrue CN110995855B (en) | 2022-06-21 |
Family
ID=70093255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911280416.2AActiveCN110995855B (en) | 2019-12-13 | 2019-12-13 | Microservice cluster scheduling method, scheduling device, and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110995855B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112148484B (en)* | 2020-09-14 | 2024-02-23 | 北京大学 | Coupling degree-based micro-service online distribution method and system |
| CN112463168B (en)* | 2020-12-01 | 2023-01-10 | 苏州浪潮智能科技有限公司 | vPP service containerization deployment method, device, terminal and storage medium |
| CN114489946A (en)* | 2022-01-26 | 2022-05-13 | 南京南瑞信息通信科技有限公司 | A cluster software-defined method and device with adaptive granularity |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6542511B1 (en)* | 1998-04-30 | 2003-04-01 | Nortel Networks Limited | Programmable transport and network architecture |
| CN106708625A (en)* | 2016-12-08 | 2017-05-24 | 中国科学院软件研究所 | Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method |
| CN108804664A (en)* | 2018-06-08 | 2018-11-13 | 北京搜狗科技发展有限公司 | A kind of problem distribution method and relevant apparatus |
| CN109117164A (en)* | 2018-06-22 | 2019-01-01 | 北京大学 | Micro services update method and system based on key element difference analysis |
| CN109582448A (en)* | 2018-10-17 | 2019-04-05 | 中国电子科技集团公司第二十八研究所 | A kind of edge calculations method for scheduling task towards criticality and timeliness |
| CN110515716A (en)* | 2019-08-29 | 2019-11-29 | 中国科学院软件研究所 | A cloud optimization scheduling method and system supporting priority and anti-affinity |
| CN110532078A (en)* | 2019-08-29 | 2019-12-03 | 中国科学院软件研究所 | A kind of edge calculations method for optimizing scheduling and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11184382B2 (en)* | 2017-10-17 | 2021-11-23 | Twistlock, Ltd. | Techniques for protecting applications from unsecure network exposure |
| US10541893B2 (en)* | 2017-10-25 | 2020-01-21 | Cisco Technology, Inc. | System and method for obtaining micro-service telemetry data |
- 2019
- 2019-12-13CNCN201911280416.2Apatent/CN110995855B/enactiveActive
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6542511B1 (en)* | 1998-04-30 | 2003-04-01 | Nortel Networks Limited | Programmable transport and network architecture |
| CN106708625A (en)* | 2016-12-08 | 2017-05-24 | 中国科学院软件研究所 | Minimum-cost maximum-flow based large-scale resource scheduling system and minimum-cost maximum-flow based large-scale resource scheduling method |
| CN108804664A (en)* | 2018-06-08 | 2018-11-13 | 北京搜狗科技发展有限公司 | A kind of problem distribution method and relevant apparatus |
| CN109117164A (en)* | 2018-06-22 | 2019-01-01 | 北京大学 | Micro services update method and system based on key element difference analysis |
| CN109582448A (en)* | 2018-10-17 | 2019-04-05 | 中国电子科技集团公司第二十八研究所 | A kind of edge calculations method for scheduling task towards criticality and timeliness |
| CN110515716A (en)* | 2019-08-29 | 2019-11-29 | 中国科学院软件研究所 | A cloud optimization scheduling method and system supporting priority and anti-affinity |
| CN110532078A (en)* | 2019-08-29 | 2019-12-03 | 中国科学院软件研究所 | A kind of edge calculations method for optimizing scheduling and system |
Non-Patent Citations (4)
| Title |
|---|
| "Max-Flow Min-Cost Routing in a Future-Internet with Improved QoS Guarantees";Ted H. Szymanski;《IEEE Transactions on Communications 》;20130211;全文* |
| "Performance Tuning and Modeling for Big Data Applications in Docker Containers";Kejiang Ye etal;《2017 International Conference on Networking, Architecture, and Storage (NAS)》;20170907;全文* |
| "基于容器云平台的网络资源管理与配置系统设计与实现";陈星宇;《万方数据》;20160809;全文* |
| "虚拟计算系统云计算数据中心性能调优能耗管理方法";叶可江;《万方数据》;20141103;全文* |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110995855A (en) | 2020-04-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11714671B2 (en) | Creating virtual machine groups based on request | |
| CN111344688B (en) | Method and system for providing resources in cloud computing | |
| EP3281359B1 (en) | Application driven and adaptive unified resource management for data centers with multi-resource schedulable unit (mrsu) | |
| CN103414761B (en) | Mobile terminal cloud resource scheduling method based on Hadoop framework | |
| US20150277955A1 (en) | System and method for controlling virtual-machine migrations based on processor usage rates and traffic amounts | |
| CN106133693B (en) | Virtual machine migration method, device and equipment | |
| WO2015196931A1 (en) | Disk io-based virtual resource allocation method and device | |
| CN110995855B (en) | Microservice cluster scheduling method, scheduling device, and computer-readable storage medium | |
| KR20140027518A (en) | Method and apparatus for assignment of virtual resources within a cloud environment | |
| WO2018006676A1 (en) | Acceleration resource processing method and apparatus and network function virtualization system | |
| WO2018000991A1 (en) | Data balancing method and device | |
| US10592451B2 (en) | Memory access optimization for an I/O adapter in a processor complex | |
| CN104714847A (en) | Dynamically Change Cloud Environment Configurations Based on Moving Workloads | |
| TW201336264A (en) | Method for increasing virtual machines | |
| CN107346264A (en) | A kind of method, apparatus and server apparatus of virtual machine load balance scheduling | |
| CN106445677A (en) | Load balancing method and device | |
| CN104331332B (en) | Virtual resource pre-allocation method based on SLA | |
| CN108132827B (en) | Network slice resource mapping method, related equipment and system | |
| WO2015149514A1 (en) | Virtual machine deploying method and apparatus | |
| CN103561055A (en) | Web application automatic elastic extension method under cloud computing environment based on sessions | |
| WO2013082742A1 (en) | Resource scheduling method, device and system | |
| CN103838520A (en) | Input/output traffic control method and equipment | |
| WO2016155291A1 (en) | Method and apparatus for managing virtualized network function scaling | |
| CN107547658A (en) | A method for automatically allocating and connecting cloud disks on a centralized cloud storage platform | |
| CN104866375B (en) | A kind of method and device for migrating virtual machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |