CN109831321A - RDMA application transport parameter adaptive selection method in a kind of data center - Google Patents

Abstract

The invention discloses RDMA application transport parameter adaptive selection methods in a kind of data center, method includes the following steps: step (1) analysis application characteristic relevant with acquisition RDMA application transport parameter selection and server resource state；Step (2) building combines trade-off decision tree-model based on the RDMA configured transmission of application characteristic and server resource status, adaptively selects optimal RDMA configured transmission to combine to different application；Step (3) by the parameter selection process of decision-tree model and RDMA Communication Programming that step (2) obtains be packaged into general API give using.This method is combined by theory analysis and experimental configuration using available RDMA configured transmission, RDMA configured transmission combination trade-off decision tree-model is established according to application characteristic and server resource status, optimal RDMA configured transmission selection is provided for application, to optimize the performance of RDMA application, RDMA application and development difficulty and development cycle are reduced simultaneously, data center's novel delay sensitive application of energizing innovation.

Description

Translated fromChinese

一种数据中心中RDMA应用传输参数自适应选择方法A method for adaptive selection of transmission parameters for RDMA applications in data centers

技术领域technical field

本发明属于数据中心网络和RDMA领域，尤其涉及一种数据中心中RDMA应用传输参数自适应选择方法。The invention belongs to the field of data center networks and RDMA, and in particular relates to an adaptive selection method for RDMA application transmission parameters in a data center.

背景技术Background technique

数据中心是云计算和大数据应用的基础设施，目前数据中心网络基本都是采用TCP/IP协议并基于以太网技术搭建。随着以太网的提速，TCP/IP协议及其实现方式已经无法适应高速数据传输的需求，成为网络性能进一步增长的瓶颈。并且，随着动态随机存储器价格的下降，数据中心中基于内存的分布式应用程序在硬件上得到了保障，将此类应用所用到的工作数据集放置在内存，能够移除磁盘或闪存的开销，使得应用程序高效地执行随机数据存取。然而，网络通信却成为制约该类型应用发展的瓶颈，RDMA技术应运而生。RDMA即远程直接内存存取技术是数据中心服务器间数据交换的一种方式。RDMA将协议栈下放到硬件中，实现了kernel bypass(内核旁路)技术，避免了传统网络中数据从用户态到内核态的拷贝开销，同时省去了内核处理的开销，为数据中心应用提供了低延时和高带宽的性能优势，从而缓解了基于内存的分布式应用的网络瓶颈。The data center is the infrastructure for cloud computing and big data applications. At present, the data center network basically adopts the TCP/IP protocol and is built based on the Ethernet technology. With the acceleration of Ethernet, the TCP/IP protocol and its implementation have been unable to meet the needs of high-speed data transmission, and become a bottleneck for the further growth of network performance. And, as the price of dynamic random access memory has fallen, memory-based distributed applications in the data center are guaranteed in hardware, and placing the working data sets used by such applications in memory can remove the overhead of disk or flash memory , allowing the application to perform random data access efficiently. However, network communication has become a bottleneck restricting the development of this type of application, and RDMA technology has emerged as the times require. RDMA, or Remote Direct Memory Access technology, is a way of data exchange between servers in a data center. RDMA decentralizes the protocol stack to hardware and implements the kernel bypass technology, which avoids the cost of copying data from user mode to kernel mode in traditional networks, and saves the overhead of kernel processing, providing data center applications. It provides the performance advantages of low latency and high bandwidth, thereby alleviating the network bottleneck of memory-based distributed applications.

当前流行的RDMA实现主要有3种：InfiniBand、RoCE(RDMA Over ConvergedEthernet)和iWARP(Internet Wide Area RDMA Protocol)。InfiniBand是针对RDMA特性设计的，较早出现在高性能计算领域，并且近几年开始在数据中心得到普及。RoCE和iWARP可以在传统以太网环境中实现RDMA的特性，因此也得到广泛的应用。There are currently three popular RDMA implementations: InfiniBand, RoCE (RDMA Over ConvergedEthernet) and iWARP (Internet Wide Area RDMA Protocol). InfiniBand is designed for RDMA characteristics, which appeared earlier in the field of high-performance computing, and has been popularized in data centers in recent years. RoCE and iWARP can realize the characteristics of RDMA in the traditional Ethernet environment, so they are also widely used.

随着RDMA网卡的价格降低和RoCE以及iWARP的出现，越来越多的数据中心将其引入，用于解决分布式应用面临的网络瓶颈。其中包括但不限于，传统的分布式应用，如分布式事务处理、MPI消息通信；新兴的分布式应用，基于内存的键值对存储、大规模图计算系统、分布式共享内存。此外，随着大数据与数据挖掘的兴起，相关的处理大数据的框架也在急需I/O方面的提升。许多的相关工作将RDMA引入这一类现有的分布式系统中，如对大量数据进行分布式处理的Hadoop、以内存计算为基础的Spark、通用深度学习框架TensorFlow。With the price reduction of RDMA network cards and the emergence of RoCE and iWARP, more and more data centers have introduced them to solve network bottlenecks faced by distributed applications. These include, but are not limited to, traditional distributed applications, such as distributed transaction processing and MPI message communication; emerging distributed applications, memory-based key-value storage, large-scale graph computing systems, and distributed shared memory. In addition, with the rise of big data and data mining, related frameworks for processing big data are also in urgent need of I/O improvement. Many related works introduce RDMA into this type of existing distributed systems, such as Hadoop for distributed processing of large amounts of data, Spark based on in-memory computing, and TensorFlow, a general-purpose deep learning framework.

但是，数据中心这些分布式应用要想充分利用RDMA的优势，传统的基于TCP/IP的通信框架将不再适用，取而代之的是RDMA的通信框架，而RDMA的通信过程和传统网络有很大的区别，涉及很多参数的选择，具体通信过程如图1所示，该过程以send/recv为例，首先需要创建QP(Queue Pair)以及CQ(Completion Queue)，QP由发送队列和接收队列组成，在功能上类似于TCP/IP通信过程中的socket，用于通信双方建立连接，连接的类型包括Reliable Connection(RC)、Unreliable Connection(UC)以及Unreliable Datagram(UD)，CQ用来通知数据传输的完成；在QP和CQ创建完成，通信双方建立好连接后，接收端给接收队列发出接收的请求，该请求包括接收的数据包所存的地址以及长度；接着发送端给发送队列发出发送的请求，该请求不仅包括数据包的地址和长度信息，还包括是否选择inline模式传输、是否选择signal模式传输以及具体的verbs等参数信息，其中，inline模式指的是对于数据包大小满足特定范围的数据可以将数据包和请求一起发送到发送队列，signal模式指的是产生数据传输完成的通知发送给完成队列，verbs包括WRITE、READ、SEND/RECV；最后在数据传输完成后，产生的完成通知发送到CQ中，如何取出这个完成的通知又包括event-triggered polling和busy polling两种。因此，RDMA这种复杂的传输过程给RDMA应用带来了一些新的挑战：However, in order to make full use of the advantages of RDMA in distributed applications such as data centers, the traditional TCP/IP-based communication framework will no longer be applicable, and the RDMA communication framework will be replaced, and the communication process of RDMA has great differences with traditional networks. The difference involves the selection of many parameters. The specific communication process is shown in Figure 1. This process takes send/recv as an example. First, QP (Queue Pair) and CQ (Completion Queue) need to be created. QP consists of a sending queue and a receiving queue. Similar in function to the socket in the TCP/IP communication process, it is used to establish a connection between the two parties. The types of connections include Reliable Connection (RC), Unreliable Connection (UC) and Unreliable Datagram (UD). CQ is used to notify data transmission. Completed; after the QP and CQ are created and the connection between the two communicating parties is established, the receiving end sends a receiving request to the receiving queue, which includes the address and length of the received data packet; then the sending end sends a request to the sending queue, The request includes not only the address and length information of the data packet, but also whether to select inline mode transmission, whether to select signal mode transmission, and specific verbs and other parameter information. Send the data packet together with the request to the send queue. The signal mode refers to generating a notification of data transmission completion and sending it to the completion queue. Verbs include WRITE, READ, SEND/RECV; finally, after the data transmission is completed, the generated completion notification is sent to In CQ, how to take out the completed notification includes event-triggered polling and busy polling. Therefore, the complex transmission process of RDMA brings some new challenges to RDMA applications:

(1)RDMA原生的编程框架以及API非常复杂，从创建QP建立连接、注册内存、发送以及接受数据到监听数据包发送完成，一个简单的数据包收发过程需要上百行的代码，而一个复杂的分布式应用涉及更多的通信过程，对于开发人员来说，需要更多的时间和精力，开发周期过长。(1) The native programming framework and API of RDMA are very complex. From creating a QP to establish a connection, registering memory, sending and receiving data, to monitoring the completion of data packet transmission, a simple data packet sending and receiving process requires hundreds of lines of code, while a complex The distributed application involves more communication processes, which requires more time and energy for developers, and the development cycle is too long.

(2)由于RDMA通信过程涉及大量的传输参数选择，不同的参数选择会对吞吐量、延时、CPU使用等产生不同程度的影响，因此，对于不同的应用来说，如何根据应用的特性来决策最佳的参数组合选择以最大化利用RDMA的优势来优化RDMA应用性能就成了目前的一大挑战。(2) Since the RDMA communication process involves a large number of transmission parameter selections, different parameter selections will have different effects on throughput, delay, CPU usage, etc. Therefore, for different applications, how to choose according to the characteristics of the application Determining the best parameter combination selection to maximize the advantages of RDMA to optimize the performance of RDMA applications has become a major challenge at present.

综上所述，复杂的RDMA传输过程给RDMA和数据中心应用的结合带来了极大的开发难度和开发周期，并且，RDMA大量的参数选择给优化RDMA应用性能造成了极大的挑战。而目前大部分研究旨在针对数据中心某一种应用利用RDMA进行加速，所涉及的参数选择太少，没有充分利用RDMA的优势来优化RDMA应用的性能。因此，为了应对RDMA与数据中心应用结合带来的挑战，本发明针对数据中心多种应用设计了一种数据中心中RDMA应用传输参数自适应选择方法，在降低RDMA应用开发难度和周期的同时，提升RDMA应用的性能。To sum up, the complex RDMA transmission process brings great development difficulty and development cycle to the combination of RDMA and data center applications, and a large number of RDMA parameter selections pose a great challenge to optimizing the performance of RDMA applications. However, most of the current research is aimed at using RDMA to accelerate a certain application in the data center. The parameters involved are too few, and the advantages of RDMA are not fully utilized to optimize the performance of RDMA applications. Therefore, in order to cope with the challenges brought by the combination of RDMA and data center applications, the present invention designs an adaptive selection method for RDMA application transmission parameters in the data center for various applications in the data center, while reducing the difficulty and cycle of RDMA application development, Improve the performance of RDMA applications.

根据现有利用RDMA加速某种应用的研究可知，在开发过程中，最主要的是如何根据应用的特性以及服务器的资源状态去选择相应的RDMA传输参数来提升应用的性能。而应用特性的不同以及服务器资源状态的变化，必然会导致RDMA传输参数选择的不同，因此，为了提供一个通用的优化RDMA应用性能的编程库，本发明具体分析了RDMA从建立连接、数据传输到数据传输完成各个阶段所涉及的所有参数，并结合实验验证各个参数的不同选择对吞吐量、延时、CPU等性能的影响，在实验过程中再分析参数的选择对应的应用特性以及服务器资源状态并结合现有的研究中提到的应用特性，总结出RDMA传输参数选择过程中涉及的应用特性和服务器资源状态。基于此，构建从应用特性以及服务器资源状态的RDMA传输参数组合选择的决策树模型，自适应地给不同的应用选择最佳的RDMA传输参数组合以优化RDMA应用性能。最后，将上述得到的决策树模型以及RDMA通信编程的参数选择过程封装成通用的API给应用使用，降低RDMA应用开发难度，减少开发周期。According to the existing research on using RDMA to accelerate a certain application, in the development process, the most important thing is how to select the corresponding RDMA transmission parameters according to the characteristics of the application and the resource status of the server to improve the performance of the application. The difference of application characteristics and the change of server resource status will inevitably lead to different selection of RDMA transmission parameters. Therefore, in order to provide a general programming library for optimizing RDMA application performance, the present invention specifically analyzes RDMA from establishing connection, data transmission to Data transmission completes all parameters involved in each stage, and combines experiments to verify the impact of different parameters on throughput, delay, CPU and other performance, and analyzes the application characteristics and server resource status corresponding to the selection of parameters during the experiment. Combined with the application characteristics mentioned in the existing research, the application characteristics and server resource status involved in the RDMA transmission parameter selection process are summarized. Based on this, a decision tree model is constructed, which is selected from the combination of RDMA transmission parameters of application characteristics and server resource status, and adaptively selects the best combination of RDMA transmission parameters for different applications to optimize the performance of RDMA applications. Finally, the decision tree model obtained above and the parameter selection process of RDMA communication programming are encapsulated into a common API for application use, which reduces the difficulty of RDMA application development and shortens the development cycle.

发明内容SUMMARY OF THE INVENTION

发明目的：针对以上问题，本发明提供一种数据中心中RDMA应用传输参数自适应选择方法，构造基于应用特性组合以及服务器资源状态的RDMA传输参数组合选择的决策树模型，为应用提供最佳的RDMA传输参数选择，从而优化RDMA应用的性能，同时降低RDMA应用开发难度，减少开发周期。Purpose of the invention: In view of the above problems, the present invention provides an adaptive selection method for RDMA application transmission parameters in a data center, and constructs a decision tree model based on the combination of application characteristics and server resource status. RDMA transmission parameter selection, thereby optimizing the performance of RDMA applications, reducing the difficulty of RDMA application development, and shortening the development cycle.

技术方案：为实现本发明的目的，本发明所采用的技术方案是：一种数据中心中RDMA应用传输参数自适应选择方法，该方法包括以下步骤：Technical scheme: In order to realize the purpose of the present invention, the technical scheme adopted in the present invention is: a method for adaptive selection of RDMA application transmission parameters in a data center, the method comprises the following steps:

(1)分析和采集RDMA传输参数选择过程中涉及的应用特性以及服务器资源状态；(1) Analyze and collect the application characteristics and server resource status involved in the RDMA transmission parameter selection process;

(2)构建基于应用特性和服务器资源状态的RDMA传输参数组合选择决策树模型；(2) Construct a decision tree model for RDMA transmission parameter combination selection based on application characteristics and server resource status;

(3)将决策树模型封装在通用的API中，RDMA应用在使用所述API传输时，根据获取RDMA应用自身携带的应用特性和服务器资源状态搜索决策树模型，选择最佳的RDMA传输参数组合。(3) The decision tree model is encapsulated in a general API. When the RDMA application uses the API for transmission, the decision tree model is searched according to the application characteristics and server resource status carried by the RDMA application itself, and the best combination of RDMA transmission parameters is selected. .

进一步的，步骤(1)中，分析和采集RDMA传输参数选择过程中涉及的应用特性以及服务器资源状态，方法如下：Further, in step (1), the application characteristics and server resource status involved in the RDMA transmission parameter selection process are analyzed and collected, and the method is as follows:

(101)RDMA传输参数选择具体包括：(1)RDMA建立连接时的QP类型Ctype：ReliableConnection，记为RC、Unreliable Connection，记为UC以及Unreliable Datagram记为UD；(2)verbs的选择Verbs：WRITE、READ、SEND/RECV；(3)RDMA数据传输结束时CQE的poll策略Polltype：event-triggered polling，记为epoll、busy polling记为busy；(4)inline模式传输IsInline：inline、no inline；(5)signal模式传输IsSignal：signal、unsignal；(101) The selection of RDMA transmission parameters specifically includes: (1) QP type Ctype when RDMA establishes a connection: ReliableConnection, denoted as RC, Unreliable Connection, denoted as UC and Unreliable Datagram as UD; (2) Verbs selection Verbs: WRITE , READ, SEND/RECV; (3) CQE poll policy Polltype at the end of RDMA data transmission: event-triggered polling, recorded as epoll, busy polling as busy; (4) inline mode transmission IsInline: inline, no inline; ( 5) Signal mode transmission IsSignal: signal, unsignal;

(102)RDMA传输参数选择过程中涉及的应用特性包括：(1)应用通信框架CF：一对一通信，记为1→1、一对多通信，记为1→n；(2)应用数据包大小Msize：以B为单位；(3)应用数据包类型Mtype：控制信息，记为control；数据信息，记为data；(102) The application characteristics involved in the RDMA transmission parameter selection process include: (1) application communication framework CF: one-to-one communication, denoted as 1→1, one-to-many communication, denoted as 1→n; (2) application data Packet size Msize: in B; (3) Application data packet type Mtype: control information, denoted as control; data information, denoted as data;

(103)RDMA传输参数选择过程中涉及的服务器资源状态包括：(1)当前应用所在发送端服务器CPU资源使用情况CPU_l，以百分比为单位；(2)当前应用所在接受端服务器CPU资源使用情况CPU_r，以百分比为单位。(103) The server resource status involved in the RDMA transmission parameter selection process includes: (1) CPU resource usage of the sending end server where the current application is located, CPU₁ , in percentage units; (2) CPU resource usage of the receiving end server where the current application is located CPU_r , in percent.

进一步的，步骤(2)中，基于应用特性以及服务器资源状态的RDMA传输参数组合如下：(201)分析RDMA传输参数性质和特点，确定有效传输参数组合，通过分析和测试RDMA硬件限制和组合的兼容性，得到有效传输参数组合21种，包括：(1)RC+write+busy+inline+unsignal、(2)RC+write+busy+inline+signal、(3)RC+write+busy+no inline+unsignal、(4)RC+write+busy+no inline+signal、(5)RC+write+epoll+no inline+unsignal、(6)RC+write+epoll+no inline+signal、(7)RC+write+epoll+inline+unsignal、(8)RC+read+epoll+no inline+signal、(9)RC+read+epoll+no inline+unsignal、(10)RC+send/recv+busy+inline+signal、(11)RC+send/recv+busy+no inline+signal、(12)RC+send/recv+epoll+inline+signal、(13)RC+send/recv+epoll+no inline+signal、(14)UD+send/recv+busy+inline+unsignal、(15)UD+send/recv+busy+inline+signal、(16)UD+send/recv+busy+no inline+unsignal、(17)UD+send/recv+busy+no inline+signal、(18)UD+send/recv+epoll+inline+unsignal、(19)UD+send/recv+epoll+inline+signal、(20)UD+send/recv+epoll+no inline+unsignal、(21)UD+send/recv+epoll+no inline+signal。Further, in step (2), the combination of RDMA transmission parameters based on application characteristics and server resource status is as follows: (201) analyze the properties and characteristics of RDMA transmission parameters, determine effective transmission parameter combinations, and analyze and test RDMA hardware restrictions and combinations. Compatibility, get 21 valid transmission parameter combinations, including: (1) RC+write+busy+inline+unsignal, (2) RC+write+busy+inline+signal, (3) RC+write+busy+no inline +unsignal, (4)RC+write+busy+no inline+signal, (5)RC+write+epoll+no inline+unsignal, (6)RC+write+epoll+no inline+signal, (7)RC+ write+epoll+inline+unsignal, (8) RC+read+epoll+no inline+signal, (9) RC+read+epoll+no inline+unsignal, (10) RC+send/recv+busy+inline+signal , (11) RC+send/recv+busy+no inline+signal, (12) RC+send/recv+epoll+inline+signal, (13) RC+send/recv+epoll+no inline+signal, (14) )UD+send/recv+busy+inline+unsignal, (15)UD+send/recv+busy+inline+signal, (16)UD+send/recv+busy+no inline+unsignal, (17)UD+send /recv+busy+no inline+signal, (18)UD+send/recv+epoll+inline+unsignal, (19)UD+send/recv+epoll+inline+signal, (20)UD+send/recv+epoll +no inline+unsignal, (21)UD+send/recv+epoll+no inline+signal.

进一步的，步骤(2)中，决策树模型构建如下：Further, in step (2), the decision tree model is constructed as follows:

(20201)设RDMA通信过程中所涉及的与应用性能相关的数有N_p个，分别记为P₁，P₂，...，P_Np，各个参数提供的选择数量分别为N_P1，N_P2，...，N_P3，根据步骤(101)，设置：(20201) Suppose there are N_p numbers of application performance-related numbers involved in the RDMA communication process, which are respectively denoted as P₁ , P₂ , ..., P_Np , and the number of options provided by each parameter is respectively N_P1 , N_P2 , ..., N_P3 , according to step (101), set:

N_p＝5N_p = 5

P₁：Ctype，P₂：Verbs，P₃：Polltype，P₄：IsInline，P₅：IsSignalP1:_Ctype , P2: Verbs,_P3 :_Polltype , P4:_IsInline ,_P5 : IsSignal

P₁＝{RC，UD}，P₂＝{write，read，send/recv}P₁ = {RC, UD}, P₂ = {write, read, send/recv}

P₃＝{busy，epoll}，P₄＝{inline，noninline}，P₅＝{signal，unsignal}_P3 ={busy, epoll}, P4={inline,_noninline },_P5 ={signal, unsignal}

根据步骤(201)，得到P₁，P₂，P₃，P₄，P₅的有效排列组合21组，记为V_pⁱ，i＝1，2，...，21；According to step (201), 21 groups of effective permutations and combinations of P₁ , P₂ , P₃ , P₄ , and P₅ are obtained, denoted as V_pⁱ , i=1, 2, . . . , 21;

(20202)根据步骤(102)和步骤(103)的分析结果，应用特性和服务器资源状态一共有5个，分别记为F₁，F₂，...，F₅，其中，CPU_l＞50％为enough，否则为not enough，CPU_r＞60％为enough，否则为not enough，设置：(20202) According to the analysis results of steps (102) and (103), there are 5 application characteristics and server resource states in total, denoted as F₁ , F₂ , . . . , F₅ , where CPU₁ >50 % is enough, otherwise it is not enough, CPU_r > 60% is enough, otherwise it is not enough, set:

F₁：CF，F₂：Msize，F₃：Mtype，F₄：CPU_l，F₅：CPU_rF1_: CF, F2:_Msize , F3:_Mtype ,_F4 : CPU_l ,_F5 : CPU_r

F₁＝{1→1，1→n}，F₂＝(0，+∞)，F₃＝{control，data}F₁ ={1→1, 1→n}, F₂ =(0,+∞), F₃ ={control,data}

F₄＝{enough，not enough}，F₅＝{enough，not enough}_F4 ={enough, not enough},_F5 ={enough, not enough}

(20203)构建从应用特性以及服务器资源状态组合V_f到RDMA传输参数组合V_p选择的决策树模型DT(V_f)，自适应地给不同的应用选择最佳的RDMA传输参数组合，其中：(20203) Build a decision tree model DT(V_f ) selected from the combination of application characteristics and server resource status V_f to the combination of RDMA transmission parameters V_p , and adaptively select the best combination of RDMA transmission parameters for different applications, where:

V_f＝(F₁，F₂，F₃，F₄，F₅)V_f = (F₁ , F₂ , F₃ , F₄ , F₅ )

V_p＝DT(V_f)V_p =DT(V_f )

测试不同V_p中的21种取值和其所对应的V_f，组合构建样本S，综合样本信息熵E(S)与判断属性值的获取时间T(F_j)构建决策树，在快速分类的同时减少决策树判断过程中属性值的获取开销，E(S)与T(F_j)的具体定义如下：Test 21 values of different V_p and their corresponding V_f , construct a sample S in combination, and synthesize the sample information entropy E(S) and the acquisition time T(F_j ) of the judgment attribute value to construct a decision tree. While reducing the cost of acquiring attribute values in the decision tree judgment process, the specific definitions of E(S) and T(F_j ) are as follows:

设样本集S中，在所有样本集S中出现的频率为其中，则信息熵E(S)为：Assuming that in the sample set S, The frequency of occurrence in all sample sets S is in, Then the information entropy E(S) is:

根据步骤(20202)中的F₁，F₂，...，F₅，设F_j有k个不同的取值，j＝1，2，3，4，5，不不同的F_j取值如下：According to F₁ , F₂ ,..., F₅ in step (20202), let F_j have k different values, j=1, 2, 3, 4, 5, different F_j values as follows:

F₁＝{1→1，1→n}，F₂＝(0，+∞)，F₃＝{control，data}F₁ ={1→1, 1→n}, F₂ =(0,+∞), F₃ ={control,data}

F₄＝{enough，not enough}，F₅＝{enough，not enough}_F4 ={enough, not enough},_F5 ={enough, not enough}

上述每个F_j的k个取值将S划分为k个样本子集其中，表示的是F_j的第1个取值所对应的参数组合的集合，1N取值范围[1，k]；The k values of each F_j above divide S into k sample subsets in, Indicates the parameter combination corresponding to the first value of F_j The set of , 1N value range [1, k];

那么按判断属性F_j划分S后的信息熵定义为：Then the information entropy after dividing S according to the judgment attribute F_j is defined as:

其中，|S|表示样本集S中的样本个数，表示样本集中的样本个数；where |S| represents the number of samples in the sample set S, Represents a sample set The number of samples in;

设置判断属性值的获取开销T(F_j)取值如下：The value of the acquisition cost T(F_j ) for setting the judgment attribute value is as follows:

(a)在构建决策树时，首先从V_f中选择一个状态属性F_j作为根节点，其中，然后将F_j的k个不同的取值作为根节点的边，每条边将样本集S划分为不同子集将F_j从V_f中移除，1的取值范围[1，k]。(a) When building a decision tree, first select a state attribute F_j from V_f as the root node, where, Then the k different values of F_j are used as the edges of the root node, and each edge divides the sample set S into different subsets Remove F_j from V_f , the value range of 1 is [1, k].

(b)采用上述同样的方式进行迭代选择决策树中每条边对应的下一层节点F_j，此时的表示的是V_f中的F_j的k个取值划分后的信息熵，计算中的为样本集中的在样本集中出现的频率，本层节点找到后，将F_j从V_f中移除，继续寻找本层节点F_j的k条边所对应的下一层节点，直到V_f为空，此时树的最后一层节点F_j的状态属性将样本集S划分成的子集将仅包含一个元素，将该元素对应的V_p设置为叶子节点；(b) Use the same method as above to iteratively select the next-level node F_j corresponding to each edge in the decision tree. At this time, the Represents the k value division of F_j in V_f After the information entropy, in the calculation for the sample set middle in the sample set the frequency of occurrence in After the node in this layer is found, remove F_j from V_f , and continue to search for the next layer node corresponding to the k edges of node F_j in this layer, until V_f is empty, at this time the last layer of the tree node F The state attribute of_j divides the sample set S into subsets Will only contain one element, set the corresponding V_p of the element as a leaf node;

(c)按照步骤(a)-(b)的方法，从根节点开始，找到每层节点F_j以及最后一层叶子节点为止，即可生成一颗决策树。(c) According to the method of steps (a)-(b), starting from the root node, finding each layer of node F_j and the last layer of leaf nodes, a decision tree can be generated.

进一步的，步骤(3)中，将决策树模型封装在通用的API中，RDMA应用在使用所述API传输时，根据获取RDMA应用自身携带的应用特性和服务器资源状态搜索决策树模型，选择最佳的RDMA传输参数组合，方法如下：Further, in step (3), the decision tree model is encapsulated in a general API, and when the RDMA application is transmitted using the API, the decision tree model is searched according to the application characteristics and server resource status carried by the RDMA application itself, and the most The best combination of RDMA transmission parameters is as follows:

(301)通用的API包括：Rdma_listen(bool Ctype)、Rdma_connect(char*address，char*port，bool Ctype)、client_send(void*buf，int len，bool control，bool n)、client_recv(void*buf，int len，bool control，bool n)、server_send(void*buf，intlen，bool control，bool n)、server_recv(void*buf，int len，bool control，bool n)；其中，Rdma_listen指的是在服务端监听连接，Ctype代表QP连接类型；Rdma_connect指的是在客户端发起连接，address和port参数分别代表服务端的地址和端口，Ctype代表QP连接类型；client_send指的是客户端发送数据，buf和len参数分别代表发送的数据所在的地址和长度；client_recv指的是客户端接收数据，buf和len参数分别代表接收的数据所在的地址和长度；server_send指的是服务端发送数据，buf和len参数分别代表发送的数据所在的地址和长度；server_recv指的是服务端接收数据，buf和len参数分别代表接收的数据所在的地址和长度，bool类型的control和n变量分别表示是否是控制信息、是否是多对多通信；(301) Common APIs include: Rdma_listen(bool Ctype), Rdma_connect(char*address, char*port, bool Ctype), client_send(void*buf, int len, bool control, bool n), client_recv(void*buf, int len, bool control, bool n), server_send(void*buf, intlen, bool control, bool n), server_recv(void*buf, int len, bool control, bool n); where Rdma_listen refers to the server Listening for connections, Ctype represents the QP connection type; Rdma_connect refers to initiating a connection on the client side, the address and port parameters represent the address and port of the server respectively, Ctype represents the QP connection type; client_send refers to the client sending data, buf and len parameters Represents the address and length of the sent data respectively; client_recv refers to the client receiving data, buf and len parameters represent the address and length of the received data respectively; server_send refers to the server sending data, buf and len parameters represent respectively The address and length of the sent data; server_recv refers to the data received by the server, the buf and len parameters represent the address and length of the received data, respectively, and the bool type control and n variables respectively indicate whether it is control information and whether it is multiple to-many communication;

(302)将步骤(2)决策树模型封装在client_send、server_send、client_recv和server_recv中，RDMA应用在使用本发明的API传输数据的过程中，通过RDMA应用自身携带的应用特性和服务器资源状态，通过深度优先的方式搜索决策树模型，从而自适应地选择最佳的有效RDMA传输参数组合。(302) Encapsulate the decision tree model of step (2) in client_send, server_send, client_recv and server_recv, and in the process of using the API of the present invention to transmit data, the RDMA application uses the application characteristics and server resource status carried by the RDMA application itself, The decision tree model is searched in a depth-first manner to adaptively select the best effective RDMA transmission parameter combination.

有益效果：与现有技术相比，本发明的技术方案具有以下有益技术效果：Beneficial effects: compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

1.具体分析了RDMA从建立连接、数据传输到数据传输完成各个阶段所涉及的所有参数，更为全面的优化RDMA应用性能，为利用RDMA加速某种应用的研究提供了参考。1. It analyzes all the parameters involved in each stage of RDMA from connection establishment, data transmission to completion of data transmission, and optimizes the application performance of RDMA more comprehensively, which provides a reference for the research of using RDMA to accelerate a certain application.

2.构造了从应用特性组合以及服务器资源状态组合到RDMA传输参数组合选择的决策树模型，自适应地为应用提供最佳的RDMA传输参数选择，适用于数据中心多种应用。2. Construct a decision tree model from the combination of application characteristics and server resource status to the combination of RDMA transmission parameters, which adaptively provides the best RDMA transmission parameter selection for applications, which is suitable for various applications in data centers.

3.将决策树模型以及RDMA通信编程的参数选择过程封装成通用的API给应用使用，降低RDMA应用开发难度，减少开发周期。3. The decision tree model and the parameter selection process of RDMA communication programming are encapsulated into a common API for application use, which reduces the difficulty of RDMA application development and shortens the development cycle.

附图说明Description of drawings

图1为RDMA以send/recv为例的具体通信流程图；Fig. 1 is the specific communication flow chart of RDMA taking send/recv as an example;

图2为本发明实现的通用编程库在系统中的位置图；Fig. 2 is the location diagram of the general programming library realized by the present invention in the system;

图3为本发明的实现步骤流程图；Fig. 3 is the realization step flow chart of the present invention;

图4为本发明生成的决策树实例图。FIG. 4 is an example diagram of a decision tree generated by the present invention.

具体实施方式Detailed ways

下面结合附图，对本发明的技术方案进行详细的说明。The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings.

如图2所示，在RDMA应用体系结构中，底层是RNIC即RDMA网卡，网卡缓存中保存着QP、CQ等的信息，用户态的RDMA verbs是RDMA原生的编程接口，可以创建QP、CQ，建立连接，发送数据等。在RDMA基本结构的基础上，本发明进一步封装了RDMA verbs，提供给应用更为通用的Common RDMA API，并加入决策树模型为应用提供最佳的RDMA传输参数组合选择，优化RDMA应用性能。本发明的一种数据中心中RDMA应用传输参数自适应选择方法，包括以下步骤：As shown in Figure 2, in the RDMA application architecture, the bottom layer is the RNIC, the RDMA network card. The network card cache stores information such as QP and CQ. The RDMA verbs in user mode are the native programming interface of RDMA, which can create QP, CQ, Establish connections, send data, etc. On the basis of the basic structure of RDMA, the present invention further encapsulates RDMA verbs, provides a more general Common RDMA API for applications, and adds a decision tree model to provide the best combination of RDMA transmission parameters for applications to optimize RDMA application performance. A method for adaptive selection of RDMA application transmission parameters in a data center of the present invention, comprising the following steps:

步骤(1)：分析和采集RDMA传输参数选择过程中涉及的应用特性以及服务器资源状态。Step (1): Analyze and collect application characteristics and server resource status involved in the RDMA transmission parameter selection process.

步骤(101)：RDMA传输参数选择具体包括：(1)RDMA建立连接时的QP类型Ctype：Reliable Connection，记为RC、Unreliable Connection，记为UC以及UnreliableDatagram记为UD；(2)verbs的选择Verbs：WRITE、READ、SEND/RECV；(3)RDMA数据传输结束时CQE的poll策略Polltype：event-triggered polling，记为epoll、busy polling记为busy；(4)inline模式传输IsInline：inline、no inline；(5)signal模式传输IsSignal：signal、unsignal。Step (101): RDMA transmission parameter selection specifically includes: (1) QP type Ctype when RDMA establishes a connection: Reliable Connection, denoted as RC, Unreliable Connection, denoted as UC and UnreliableDatagram as UD; (2) Verbs selection Verbs : WRITE, READ, SEND/RECV; (3) CQE's poll policy at the end of RDMA data transmission Polltype: event-triggered polling, recorded as epoll, busy polling as busy; (4) inline mode transmission IsInline: inline, no inline ; (5) signal mode transmission IsSignal: signal, unsignal.

步骤(102)：RDMA传输参数选择过程中涉及的应用特性包括：(1)应用通信框架CF：一对一通信，记为1→1、一对多通信，记为1→n；(2)应用数据包大小Msize：以B为单位；(3)应用数据包类型Mtype：控制信息，记为control、数据信息，记为data。Step (102): The application characteristics involved in the RDMA transmission parameter selection process include: (1) Application communication framework CF: one-to-one communication, denoted as 1→1, one-to-many communication, denoted as 1→n; (2) Application data packet size Msize: take B as the unit; (3) Application data packet type Mtype: control information, denoted as control, data information, denoted as data.

步骤(103)：RDMA传输参数选择过程中涉及的服务器资源状态包括：(1)当前应用所在发送端服务器CPU资源使用情况CPU_l，以百分比为单位；(2)当前应用所在接受端服务器CPU资源使用情况CPU_r，以百分比为单位。Step (103): The server resource status involved in the RDMA transmission parameter selection process includes: (1) CPU resource usage CPU₁ of the sending end server where the current application is located, in percentage units; (2) The receiving end server CPU resource where the current application is located Usage CPU_r , in percent.

步骤(2)：构建基于应用特性以及服务器资源状态的RDMA传输参数组合选择决策树模型，自适应地给不同的应用选择最佳的RDMA传输参数组合。Step (2): constructing a decision tree model for RDMA transmission parameter combination selection based on application characteristics and server resource status, and adaptively selecting the best RDMA transmission parameter combination for different applications.

步骤(201)：分析RDMA传输参数性质和特点，确定有效传输参数组合。通过分析和测试RDMA硬件限制和组合的兼容性，得到有效传输参数组合21种，包括：(1)RC+write+busy+inline+unsignal、(2)RC+write+busy+inline+signal、(3)RC+write+busy+no inline+unsignal、(4)RC+write+busy+no inline+signal、(5)RC+write+epoll+no inline+unsignal、(6)RC+write+epoll+no inline+signal、(7)RC+write+epoll+inline+unsignal、(8)RC+read+epoll+no inline+signal、(9)RC+read+epoll+no inline+unsignal、(10)RC+send/recv+busy+inline+signal、(11)RC+send/recv+busy+no inline+signal、(12)RC+send/recv+epoll+inline+signal、(13)RC+send/recv+epoll+no inline+signal、(14)UD+send/recv+busy+inline+unsignal、(15)UD+send/recv+busy+inline+signal、(16)UD+send/recv+busy+no inline+unsignal、(17)UD+send/recv+busy+noinline+signal、(18)UD+send/recv+epoll+inline+unsignal、(19)UD+send/recv+epoll+inline+signal、(20)UD+send/recv+epoll+no inline+unsignal、(21)UD+send/recv+epoll+no inline+signal。Step (201): analyze the properties and characteristics of the RDMA transmission parameters, and determine an effective transmission parameter combination. By analyzing and testing the compatibility of RDMA hardware limitations and combinations, 21 valid transmission parameter combinations are obtained, including: (1) RC+write+busy+inline+unsignal, (2) RC+write+busy+inline+signal, ( 3) RC+write+busy+no inline+unsignal, (4)RC+write+busy+no inline+signal, (5)RC+write+epoll+no inline+unsignal, (6)RC+write+epoll+ no inline+signal, (7)RC+write+epoll+inline+unsignal, (8)RC+read+epoll+no inline+signal, (9)RC+read+epoll+no inline+unsignal, (10)RC +send/recv+busy+inline+signal, (11)RC+send/recv+busy+no inline+signal, (12)RC+send/recv+epoll+inline+signal, (13)RC+send/recv +epoll+no inline+signal, (14)UD+send/recv+busy+inline+unsignal, (15)UD+send/recv+busy+inline+signal, (16)UD+send/recv+busy+no inline+unsignal, (17)UD+send/recv+busy+noinline+signal, (18)UD+send/recv+epoll+inline+unsignal, (19)UD+send/recv+epoll+inline+signal, ( 20) UD+send/recv+epoll+no inline+unsignal, (21) UD+send/recv+epoll+no inline+signal.

步骤(202)：构建决策树模型。Step (202): Build a decision tree model.

步骤(20201)：设RDMA通信过程中所涉及的与应用性能相关的数有N_p个，分别记为P₁，P₂，…，P_Np，各个参数提供的选择数量分别为N_P1，N_P2，…，N_P3，根据步骤(101)，设置：Step (₂₀₂₀₁ ): Suppose there are N_p numbers related to the application performance involved in the RDMA communication process, which are respectively denoted as_P1 , P₂ , ._P2 , ..., N_P3 , according to step (101), set:

N_p＝5N_p = 5

P₁：Ctype，P₂：Verbs，P₃：Polltype，P₄：IsInline，P₅：IsSignalP1:_Ctype , P2: Verbs,_P3 :_Polltype , P4:_IsInline ,_P5 : IsSignal

P₁＝{RC，UD}，P₂＝{WRITE，READ，SEND/RECV}P₁ = {RC, UD}, P₂ = {WRITE, READ, SEND/RECV}

P₃＝{busy，epoll}，P₄＝{inline，noninline}，P₅＝{signal，unsignal}_P3 ={busy, epoll}, P4={inline,_noninline },_P5 ={signal, unsignal}

N_P1＝3，N_P2＝3，N_P3＝2，N_P4＝2，N_P5＝2_NP1 =3,_NP2 =3,_NP3 =2,_NP4 =2,_NP5 =2

根据步骤(201)，得到P₁，P₂，P₃，P₄，P₅的有效排列组合21组，记为V_pⁱ，i＝1，2，...，21。According to step (201), 21 groups of effective permutations and combinations of P₁ , P₂ , P₃ , P₄ , and P₅ are obtained, denoted as V_pⁱ , i=1, 2, . . . , 21.

步骤(20202)：根据步骤(102)和步骤(103)的分析结果，应用特性和服务器资源状态一共有5个，分别记为F₁，F₂，...，F₅。其中，CPU_l＞50％为enough，否则为not enough，CPU_r＞60％为enough，否则为not enough，设置：Step (20202): According to the analysis results of step (102) and step (103), there are₅ application characteristics and server resource states in total, which are respectively_denoted as F1, F2, . . . ,_F5 . Among them, CPU_l > 50% is enough, otherwise it is not enough, CPU_r > 60% is enough, otherwise it is not enough, set:

F₁：CF，F₂：Msize，F₃：Mtype，F₄：CPU_l，F₅：CPU_rF1_: CF, F2:_Msize , F3:_Mtype ,_F4 : CPU_l ,_F5 : CPU_r

F₁＝{1→1，1→n}，F₂＝(0，+∞)，F₃＝{control，data}F₁ ={1→1, 1→n}, F₂ =(0,+∞), F₃ ={control,data}

F₄＝{enough，not enough}，F₅＝{enough，not enough}_F4 ={enough, not enough},_F5 ={enough, not enough}

步骤(20203)：构建从应用特性以及服务器资源状态组合V_f到RDMA传输参数组合V_p选择的决策树模型DT(V_f)，自适应地给不同的应用选择最佳的RDMA传输参数组合，其中：Step (20203): construct a decision tree model DT(V_f ) selected from the combination of application characteristics and server resource status V_f to the combination of RDMA transmission parameters V_p , and adaptively select the best combination of RDMA transmission parameters for different applications, in:

V_f＝(F₁，F₂，F₃，F₄，F₅)V_f = (F₁ , F₂ , F₃ , F₄ , F₅ )

V_p＝DT(V_f)V_p =DT(V_f )

测试不同V_p取值和其所对应的V_f，组合构建样本S，综合样本信息熵E(S)与判断属性值的获取时间T(F_j)构建决策树，在快速分类的同时减少决策树判断过程中属性值的获取开销，E(S)与T(F_j)的具体定义如下：Test different V_p values and their corresponding V_f , construct a sample S in combination, and synthesize the sample information entropy E(S) and the acquisition time T(F_j ) of the judgment attribute value to construct a decision tree, which reduces decision-making while classifying quickly. The cost of acquiring attribute values in the tree judgment process, E(S) and T(F_j ) are specifically defined as follows:

设样本集S中，在所有样本中出现的频率为其中则信息熵E(S)为：Assuming that in the sample set S, The frequency of occurrence in all samples is in Then the information entropy E(S) is:

根据步骤(20202)中的F₁，F₂，...，F₅，设F_j有k个不同的取值，j＝1，2，3，4，5，不不同的F_j取值如下：According to F₁ , F₂ ,..., F₅ in step (20202), let F_j have k different values, j=1, 2, 3, 4, 5, different F_j values as follows:

F₁＝{1→1，1→n}，F₂＝(0，+∞)，F₃＝{control，data}F₁ ={1→1, 1→n}, F₂ =(0,+∞), F₃ ={control,data}

F₄＝{enough，not enough}，F₅＝{enough，not enough}_F4 ={enough, not enough},_F5 ={enough, not enough}

例如F₁取值只有两种，即1→1，1→n，F₃取值有两种，即control，data，F₄和F₅取值有2种，即nough，not enough，F₂取值有多种。For example, there are only two values of F₁ , namely 1→1, 1→n, and there are two kinds of values of F₃ , namely control, data, F₄ and F₅ , there are two kinds of values, namely rough, not enough, F₂ There are various values.

上述每个F_j的k个取值将S划分为k个样本子集其中，1的取值范围[1，k]。The k values of each F_j above divide S into k sample subsets Among them, the value range of 1 is [1, k].

那么按判断属性F_j划分S后的信息熵定义为：Then the information entropy after dividing S according to the judgment attribute F_j is defined as:

设置判断属性值的获取开销T(F_j)取值如下：The value of the acquisition cost T(F_j ) for setting the judgment attribute value is as follows:

(a)在构建决策树时，首先从V_f中选择一个状态属性F_j作为根节点，其中，然后将F_j的k个不同的取值作为根节点的边，每条边将样本集S划分为不同子集将F_j从V_f中移除，1的取值范围[1，k]。(a) When building a decision tree, first select a state attribute F_j from V_f as the root node, where, Then the k different values of F_j are used as the edges of the root node, and each edge divides the sample set S into different subsets Remove F_j from V_f , the value range of 1 is [1, k].

(b)采用上述同样的方式进行迭代选择决策树中每条边对应的下一层节点F_j，此时的表示的是V_f中的F_j的k个取值划分后的信息熵，计算中的为样本集中的在样本集中出现的频率，本层节点找到后，将F_j从V_f中移除，继续寻找本层节点F_j的k条边所对应的下一层节点，直到V_f为空，此时树的最后一层节点F_j的状态属性将样本集S划分成的子集将仅包含一个元素，将该元素对应的V_p设置为叶子节点；(b) Use the same method as above to iteratively select the next-level node F_j corresponding to each edge in the decision tree. At this time, the Represents the k value division of F_j in V_f After the information entropy, in the calculation for the sample set middle in the sample set the frequency of occurrence in After the node in this layer is found, remove F_j from V_f , and continue to search for the next layer node corresponding to the k edges of node F_j in this layer, until V_f is empty, at this time the last layer of the tree node F The state attribute of_j divides the sample set S into subsets Will only contain one element, set the corresponding V_p of the element as a leaf node;

(c)按照步骤(a)-(b)的方法，从根节点开始，找到每层节点F_j以及最后一层叶子节点为止，即可生成一颗决策树。(c) According to the method of steps (a)-(b), starting from the root node, finding each layer of node F_j and the last layer of leaf nodes, a decision tree can be generated.

步骤(301)：通用的API包括：Rdma_listen(bool Ctype)、Rdma_connect(char*address，char*port,bool Ctype)、client_send(void*buf,int len,bool control,booln)、client_recv(void*buf,int len,bool control,bool n)、server_send(void*buf,intlen,bool control,bool n)、server_recv(void*buf,int len,bool control,bool n)，其中，Rdma_listen指的是在服务端监听连接，Ctype代表QP连接类型；Rdma_connect指的是在客户端发起连接，address和port参数分别代表服务端的地址和端口，Ctype代表QP连接类型；client_send指的是客户端发送数据，buf和len参数分别代表发送的数据所在的地址和长度；client_recv指的是客户端接收数据，buf和len参数分别代表接收的数据所在的地址和长度；server_send指的是服务端发送数据，buf和len参数分别代表发送的数据所在的地址和长度；server_recv指的是服务端接收数据，buf和len参数分别代表接收的数据所在的地址和长度。bool类型的control和n变量分别表示是否是控制信息、是否是多对多通信。Step (301): Common APIs include: Rdma_listen(bool Ctype), Rdma_connect(char*address, char*port,bool Ctype), client_send(void*buf,int len,bool control,booln), client_recv(void*buf ,int len,bool control,bool n), server_send(void*buf,intlen,bool control,bool n), server_recv(void*buf,int len,bool control,bool n), where Rdma_listen refers to the service The terminal listens for connections, Ctype represents the QP connection type; Rdma_connect refers to the connection initiated on the client side, the address and port parameters represent the address and port of the server respectively, Ctype represents the QP connection type; client_send refers to the client sending data, buf and len The parameters represent the address and length of the data to be sent; client_recv refers to the data received by the client, and the buf and len parameters represent the address and length of the received data respectively; server_send refers to the data sent by the server, and the buf and len parameters are respectively Represents the address and length of the sent data; server_recv refers to the server receiving data, and the buf and len parameters represent the address and length of the received data, respectively. The control and n variables of bool type indicate whether it is control information and whether it is many-to-many communication.

步骤(302)：将步骤(2)决策树模型封装在client_send、server_send、client_recv和server_recv中，RDMA应用在使用本发明的API传输数据的过程中，通过RDMA应用自身携带的应用特性和服务器资源状态，会通过深度优先的方式搜索决策树模型，从而自适应地选择最佳的有效RDMA传输参数组合。如图4所示，决策树每个节点的判断过程在API中都以if/else的形式进行搜索，根据输入的API实参的值，从树的根节点开始判断，最终逐步搜索达到树的叶子节点，得到一个RDMA参数的组合。Step (302): Encapsulate the decision tree model of step (2) in client_send, server_send, client_recv and server_recv, and in the process of using the API of the present invention to transmit data, the RDMA application uses the application characteristics and server resource status carried by itself through the RDMA application , the decision tree model will be searched in a depth-first manner to adaptively select the best effective RDMA transmission parameter combination. As shown in Figure 4, the judgment process of each node of the decision tree is searched in the form of if/else in the API. According to the value of the input API parameters, the judgment starts from the root node of the tree, and finally the search reaches the end of the tree step by step. The leaf node gets a combination of RDMA parameters.

本发明最终实现的Common RDMA API库可以提供给数据中心多种应用使用，降低RDMA应用开发难度，减少开发周期，并利用决策树模型为应用提供最佳的RDMA传输参数组合选择，优化RDMA应用性能。The Common RDMA API library finally realized by the present invention can be used for various applications in the data center, so as to reduce the difficulty of RDMA application development, reduce the development cycle, and use the decision tree model to provide the best RDMA transmission parameter combination selection for the application, so as to optimize the RDMA application performance. .

本发明还可有其他多种实施方式，在不背离本发明精神及其实质的情况下，熟悉本领域的技术人员可根据本发明做出各种相应的改变和变形，这些相应的改变和变形都应属于本发明所附的权利要求的保护范围。The present invention can also have other various embodiments, without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention, and these corresponding changes and deformations All should belong to the protection scope of the appended claims of the present invention.

Claims (5)

1. An adaptive selection method for RDMA application transmission parameters in a data center is characterized by comprising the following steps:

(1) analyzing and collecting application characteristics and server resource states involved in an RDMA transmission parameter selection process;

(2) constructing an RDMA transmission parameter combination selection decision tree model based on application characteristics and server resource states;

(3) and encapsulating the decision tree model in a universal API (application programming interface), and searching the decision tree model according to the acquired application characteristics carried by the RDMA application and the server resource state when the RDMA application uses the API for transmission, and selecting the optimal RDMA transmission parameter combination.

2. The adaptive selection method for RDMA application transmission parameters in a data center according to claim 1, characterized in that in step (1), the application characteristics and server resource status involved in the RDMA transmission parameter selection process are analyzed and collected, and the method is as follows:

(101) the RDMA transfer parameter selection specifically includes: (1) QP type Ctype when RDMA establishes connection: a ReliableConnection, noted as RC, Unreliable Connection, noted as UC, and Unreliable Datagram noted as UD; (2) selection of Verbs: WRITE, READ, SEND/RECV; (3) poll policy Polltype of CQE at the end of RDMA data transfer: event-triggered polling, which is marked as epoll, and busy polling is marked as busy; (4) the inline mode transports IsInline: inline, no inline; (5) signal mode transport IsSignal: signal, unsigned;

(102) application characteristics involved in the RDMA transfer parameter selection process include: (1) the application communication framework CF: one-to-one communication, denoted as 1 → 1, one-to-many communication, denoted as 1 → n; (2) application packet size Msize: taking B as a unit; (3) application packet type Mtype: control information, which is recorded as control; data information marked as data;

(103) the server resource states involved in the RDMA transfer parameter selection process include: (1) CPU resource use condition of current application sending end server CPU_lIn percent; (2) CPU resource use condition of current application receiving end server CPU_rIn units of percentages.

3. The adaptive selection method for RDMA application transfer parameters in a data center according to claim 2, characterized in that in step (2), the RDMA transfer parameters based on application characteristics and server resource status are combined as follows: (201) analyzing the property and the characteristic of RDMA transmission parameters, determining effective transmission parameter combinations, and obtaining 21 effective transmission parameter combinations by analyzing and testing the compatibility of RDMA hardware limitation and combinations, wherein the method comprises the following steps: (1) RC + write + busy + internal + signal, (2) RC + write + busy + internal + signal, (3) RC + write + busy + internal + signal, (4) RC + write + busy + internal + signal, (5) RC + write + internal + signal, (6) RC + write + internal + signal, (7) RC + write + internal + signal, (8) RC + read + internal + signal, (9) RC + read + internal + signal, (10) RC + sense + signal + internal + signal, (11) RC + sense + signal + 12 + internal + signal, (17) RC + sense + signal + 16 + signal + 12 + signal + 16 + signal + 12 + signal + 16 + signal + 12 + signal + 12 + signal + 12 + signal + 12 + signal + 12 + signal + 12, (18) UD + send/recv + epoll + inline + un signal, (19) UD + send/recv + epoll + inline + signal, (20) UD + send/recv + epoll + noninoline + un signal, (21) UD + send/recv + epoll + no inline + signal.

4. The adaptive selection method for RDMA application transport parameters in a data center according to claim 3, wherein in the step (2), the decision tree model is constructed as follows:

(20201) Let N be the number related to application performance involved in RDMA communication_pEach is marked as P₁，P₂，...，P_NpThe selection number provided by each parameter is N_P1，N_P2，...，N_P3According to step (101), setting:

N_p＝5

P₁：Ctype，P₂：Verbs，P₃：Polltype，P₄：IsInline，P5：IsSignal

P₁＝{RC，UD}，P₂＝{write，read，send/recv}

P₃＝{busy，epoll}，P₄＝{inline，noninline}，P₅＝{signal，unsignal}

according to step (201), P is obtained₁，P₂，P₃，P₄，P₅21 effective permutation and combination of (2), denoted as V_pⁱ，i＝1，2，...，21；

(20202) According to the analysis results of the step (102) and the step (103), the number of the application characteristics and the server resource states is 5, and the number is respectively marked as F₁，F₂，...，F₅Wherein, the CPU_lIf more than 50% is equal to enable, otherwise is equal to not enable, CPU_rIf the power is higher than 60%, the power is equal to zero power, and if the power is higher than zero power, the following settings are set:

F₁：CF，F₂：Msize，F₃：Mtype，F₄：CPU_l，F₅：CPU_r

F₁＝{1→1，1→n}，F₂＝(0，+∞)，F₃＝{control，data}

F₄＝{enough，not enough}，F₅＝{enough，not enough}

(20203) Constructing a combination of parameters V from a combination of application characteristics and server resource states Vf to RDMA transfer_pSelected decision tree model DT (V)_f) Adaptively selecting an optimal RDMA transfer parameter combination for different applications, wherein:

V_f＝(F₁，F₂，F₃，F₄，F₅)

V_p＝DT(V_f)

testing of different V_p21 values in (1) and V corresponding to the values_fCombining and constructing a sample S, integrating sample information entropy E (S) and judging acquisition time T (F) of attribute values_j) Constructing a decision tree, reducing the acquisition cost of attribute values in the decision process of the decision tree while quickly classifying, E (S) and T (F)_j) The specific definition of (A) is as follows:

let it be assumed that in the sample set S,the frequency of occurrence in all sample sets S isWherein N is_VpWhen 21, the information entropy e(s) is:

according to F in step (20202)₁，F₂，...，F₅Is provided with F_jThere are k different values, j 1, 2, 3, 4, 5, F which are not different_jThe values are as follows:

F₁＝{1→1，1→n}，F₂＝(0，+∞)，F₃＝{control，data}

F₄＝{enough，not enough}，F₅＝{enough，not enough}

each of the above F_jK values of (a) divide S into k sample subsetsWherein,is represented by F_jThe parameter combination corresponding to the l-th valueSet of (c), value range of l [1, k [ ]]；

Then according to decision attribute F_jThe information entropy after S division is defined as:

wherein | S | represents the number of samples in the sample set S,representing a sample setThe number of samples in (1);

setting acquisition overhead T (F) of judgment attribute value_j) The values are as follows:

(a) when constructing a decision tree, first from V_fSelects one state attribute F_jAs a root node, among other things,then F is mixed_jK different values of (a) as edges of the root node, each edge dividing the sample set S into different subsetsF is to be_jFrom V_fIn the middle, the value range of l [1, k ]]；

(b) Iteratively selecting the next-layer node F corresponding to each edge in the decision tree in the same way_jAt this timeIs shown as V_fF in (1)_jK value divisions ofEntropy of the information after, in the calculationAs a sample setIn (1)In a sample setFrequency of occurrence of, N_Vp21; after the node of the layer is found, F is added_jFrom V_fRemoving the node, and continuously searching the node F of the current layer_jUntil V is reached_fEmpty, the last level node F of the tree_jThe state attribute of (a) divides the sample set S into subsetsWill contain only one element, will this element correspond to V_pSetting as leaf nodes;

(3) according to the method of steps (a) - (b), starting from the root node, finding each layer of nodes F_jAnd generating a decision tree until the last layer of leaf nodes.

5. The RDMA application transmission parameter adaptive selection method in the data center of claim 4, wherein in step (3), the decision tree model is packaged in a general API, and when the RDMA application uses the API for transmission, the RDMA application searches the decision tree model according to the obtained application characteristics carried by the RDMA application and the server resource state, and selects the optimal RDMA transmission parameter combination, and the method comprises the following steps:

(301) the common APIs include: rdma _ list (bone Ctype), Rdma _ connect (bone address, bone port, bone Ctype), client _ send (void buf, int len, bone control, bone n), client _ recv (void buf, int len, bone control, bone n), server _ send (void buf, intlen, bone control, bone n), server _ recv (void buf, int len, bone control, bone n); wherein, Rdma _ list refers to monitoring connection at the server, Ctype represents QP connection type; rdma _ connect refers to initiating connection at a client, address and port parameters respectively represent an address and a port of a server, and Ctype represents a QP connection type; client _ send refers to the client sending data, and buf and len parameters respectively represent the address and length of the sent data; client _ recv means that a client receives data, and buf and len parameters respectively represent the address and the length of the received data; server _ send refers to that the server sends data, and buf and len parameters respectively represent the address and length of the sent data; server _ recv means that a server receives data, buf and len parameters respectively represent the address and the length of the received data, and control and n variables of the pool type respectively represent whether the data is control information or not and whether the data is many-to-many communication or not;

(302) and (3) packaging the decision tree model in the step (2) into client _ send, server _ send, client _ recv and server _ recv, and searching the decision tree model in a depth-first mode through the application characteristics and the server resource state carried by the RDMA application in the process of transmitting data by using the API of the invention so as to adaptively select the optimal effective RDMA transmission parameter combination.