技术领域Technical field
本申请涉及数据处理技术的领域，尤其是涉及一种基于数据中心的数据处理方法及系统。This application relates to the field of data processing technology, and in particular, to a data processing method and system based on a data center.
背景技术Background technique
数据是对事实、概念或指令的一种表达形式，可由人工或自动化装置进行处理，数据经过解释并赋予一定的意义之后，便成为信息，数据处理是对数据的采集、存储、检索、加工、变换和传输。Data is an expression of facts, concepts or instructions, which can be processed by manual or automated devices. After data is interpreted and given a certain meaning, it becomes information. Data processing is the collection, storage, retrieval, processing, and processing of data. transformation and transmission.
相关技术中，在对数据进行存储处理时，一般需要基于数据中心进行实现，数据中心中存在有多个服务器，需要存储的数据通过通信连接的数据链传输至数据中心的服务器中进行存储，当需要使用该数据时又可通过对应的指令于服务器中进行调用。通过服务器中所存储的数据大小情况可将服务器区分为低负载服务器、中负载服务器以及高负载服务器，其中低负载服务器虽然能供数据进行存储，但消耗能源的同时提供不了较好的工作效率，而高负载服务器虽然提供了较好的工作效率但会有可能会存在数据卡顿的情况，因此一般情况下外部所输入的需要进行存储的数据会优先放置于中负载服务器中。In related technologies, when storing and processing data, it generally needs to be implemented based on a data center. There are multiple servers in the data center. The data that needs to be stored is transmitted to the server in the data center for storage through a data link connected by communication. When When the data needs to be used, it can be called in the server through the corresponding instructions. Servers can be divided into low-load servers, medium-load servers and high-load servers according to the size of data stored in the server. Although low-load servers can store data, they consume energy and cannot provide good work efficiency. Although high-load servers provide better work efficiency, there may be data freezes. Therefore, in general, externally input data that needs to be stored will be prioritized in medium-load servers.
针对上述中的相关技术，发明人认为当服务器中的数据被调用后，处于服务器中的数据会消失，当处于非高峰时段而没有新的外部数据进行填补时，会使该服务器长时间处于低负载服务器的情况，此时该服务器不仅消耗能源且提供不了较好的工作效率，造成了能源的浪费，尚有改进空间。Regarding the above-mentioned related technologies, the inventor believes that when the data in the server is called, the data in the server will disappear. When there is no new external data to fill in non-peak hours, the server will be in a low state for a long time. In the case of a loaded server, the server not only consumes energy but cannot provide good work efficiency, resulting in a waste of energy and there is still room for improvement.
发明内容Contents of the invention
为了减少数据存储处理时造成的能源浪费，本申请提供一种基于数据中心的数据处理方法及系统。In order to reduce energy waste caused by data storage and processing, this application provides a data processing method and system based on a data center.
第一方面，本申请提供一种基于数据中心的数据处理方法，采用如下的技术方案：In the first aspect, this application provides a data processing method based on a data center, adopting the following technical solution:
一种基于数据中心的数据处理方法，包括：A data processing method based on a data center, including:
获取当前时间点；Get the current time point;
于当前时间点处于预设的数据非高峰时段内时实时获取各服务器的数据负载数值；Obtain the data load value of each server in real time when the current time point is within the preset non-peak data period;
判断数据负载数值是否小于预设的基准负载数值；Determine whether the data load value is less than the preset base load value;
若数据负载数值小于基准负载数值，则将该服务器定义为低负载服务器，并将该低负载服务器相对应的数据负载数值定义为需求转出数值；If the data load value is less than the baseline load value, the server is defined as a low-load server, and the data load value corresponding to the low-load server is defined as the demand transfer value;
若数据负载数值不小于基准负载数值，则将该服务器定义为正常服务器，并根据正常服务器相对应的数据负载数值以及预设的上限负载数值进行差值计算以确定许可转入数值；If the data load value is not less than the baseline load value, the server is defined as a normal server, and a difference calculation is performed based on the data load value corresponding to the normal server and the preset upper limit load value to determine the allowed transfer value;
根据需求转出数值确定各低负载服务器的转出顺序且根据许可转入数值确定各正常服务器的转入顺序；Determine the transfer-out sequence of each low-load server according to the demand transfer-out value and determine the transfer-in sequence of each normal server according to the permitted transfer-in value;
根据转入顺序以将低负载服务器内的数据根据转出顺序转移至各正常服务器内，并于低负载服务器内的数据全部转移后将该低负载服务器确定为空闲服务器，且对该空闲服务器进行休眠。The data in the low-load server is transferred to each normal server according to the transfer-in order, and after all the data in the low-load server is transferred, the low-load server is determined to be an idle server, and the idle server is hibernate.
通过采用上述技术方案，获取当前时间点以判断当前是否处于非高峰时段，当处于非高峰时段时对每个服务器的负载情况进行确定，以对低负载服务器进行确定，从而能够将低负载服务器中的数据转移至其余的正常服务器中并对不存在数据的服务器进行休眠，从而使得较为散乱的数据能够集中至较少的服务器中，以减少数据存储处理时造成的能源浪费。By adopting the above technical solution, the current time point is obtained to determine whether it is in non-peak period, and when it is in non-peak period, the load condition of each server is determined to determine the low-load server, so that the low-load server can be The data is transferred to the remaining normal servers and the servers without data are put to sleep, so that the more scattered data can be concentrated on fewer servers to reduce the energy waste caused by data storage and processing.
可选的，根据需求转出数值确定各低负载服务器的转出顺序的步骤包括：Optionally, the steps of determining the rollout order of each low-load server based on the demand rollout value include:
于低负载服务器中进行数据组分析以获取独立组分数量以及各数据组的组分负载数值；Perform data set analysis on a low-load server to obtain the number of independent components and the component load value of each data set;
根据预设的排序规则对所有低负载服务器所确定的组分负载数值进行排序，并由大至小对数据组赋予处理难度值；Sort the component load values determined by all low-load servers according to the preset sorting rules, and assign processing difficulty values to the data groups from largest to smallest;
于单个低负载服务器中根据所有的数据组对应的处理难度值进行求和计算以确定综合难度值；Calculate the sum of the processing difficulty values corresponding to all data groups in a single low-load server to determine the comprehensive difficulty value;
根据综合难度值以及相对应的独立组分数量进行均值计算以确定均值难度值；Perform average calculation based on the comprehensive difficulty value and the corresponding number of independent components to determine the average difficulty value;
根据排序规则对均值难度值由小至大进行排序以确定转出顺序。Sort the mean difficulty values from small to large according to the sorting rules to determine the transfer order.
通过采用上述技术方案，对每个服务器中所需转移的数据的难度进行确定，以优先对难度较小的服务器中的数据进行转移。By adopting the above technical solution, the difficulty of the data to be transferred in each server is determined, so that the data in servers with less difficulty is transferred first.
可选的，于均值难度值确定后，转出顺序的确定步骤还包括：Optionally, after the average difficulty value is determined, the steps for determining the transfer order also include:
获取各低负载服务器的连续作业时长；Obtain the continuous operation duration of each low-load server;
根据连续作业时长以当前时间点为后端点以建立检测区间，并于检测区间内生成一可于检测区间内进行移动的检测点，且获取检测点的单点负载数值；Establish a detection interval based on the continuous operation duration with the current time point as the back end point, generate a detection point within the detection interval that can move within the detection interval, and obtain the single-point load value of the detection point;
根据检测点以及检测区间的前端点生成作业区间，并根据作业区间确定检测时长；Generate an operation interval based on the detection point and the front end point of the detection interval, and determine the detection duration based on the operation interval;
于作业区间内根据所有点位的单点负载数值进行求和计算以确定作业负载数值，并根据作业负载数值以及检测时长进行计算以确定检测点的前序影响值；Perform a summation calculation based on the single-point load values of all points within the operating interval to determine the operating load value, and perform calculations based on the operating load value and the detection duration to determine the preceding influence value of the detection point;
根据预设的压力匹配关系以确定前序影响值以及单点负载数值相对应的单点压力值；Determine the previous influence value and the single point pressure value corresponding to the single point load value according to the preset pressure matching relationship;
根据所有的单点压力值进行求和计算以确定综合压力值；Calculate the sum of all single point pressure values to determine the comprehensive pressure value;
根据预设的修正匹配关系以确定综合压力值相对应的修正参数，并根据修正参数对均值难度值进行修正更新。The correction parameters corresponding to the comprehensive pressure value are determined according to the preset correction matching relationship, and the average difficulty value is corrected and updated according to the correction parameters.
通过采用上述技术方案，对各低负载服务器长时间作业所受的压力情况进行确定，以使得压力较大的低负载服务器中的数据能够优先转移，从而使得对应的低负载服务器能够进行休眠以减少压力。By adopting the above technical solution, the pressure of long-term operation of each low-load server is determined, so that the data in the low-load server with greater pressure can be transferred first, so that the corresponding low-load server can sleep to reduce pressure.
可选的，控制数据转移的步骤包括：Optional steps to control data transfer include:
将当前需要进行数据转移的低负载服务器定义为转出服务器，并将当前需要进行数据转入的正常服务器定义为转入服务器，且将转入服务器相对应的许可转入数值定义为剩余负载数值；Define the low-load server that currently needs to transfer data as the transfer-out server, define the normal server that currently needs to transfer data as the transfer-in server, and define the transfer-in value corresponding to the permission of the transfer-in server as the remaining load value. ;
于转出服务器中判断数据组的组分负载数值是否大于剩余负载数值；Determine whether the component load value of the data group is greater than the remaining load value in the transfer server;
若组分负载数值大于剩余负载数值，则将该数据组定义为无效数据组；If the component load value is greater than the remaining load value, the data group is defined as an invalid data group;
若组分负载数值不大于剩余负载数值，则将该数据组定义为有效数据组；If the component load value is not greater than the remaining load value, the data group is defined as a valid data group;
于有效数据组中根据排序规则确定数值最大的组分负载数值，并将该组分负载数值相对应的有效数据组转移至转入服务器内，且于剩余的数据组中继续确定有效数据组以进行数据转移，直至不存在有效数据组时控制转入服务器根据转入顺序变化。Determine the component load value with the largest value in the effective data group according to the sorting rules, transfer the effective data group corresponding to the component load value to the transfer server, and continue to determine the effective data group in the remaining data groups. Data transfer is performed until there is no valid data group and the control transfer server changes according to the transfer order.
通过采用上述技术方案，可对低负载服务器中能够进行转移的数据组由负载大的优先进行转移，便于实现低负载服务器中所有数据组进行转移。By adopting the above technical solution, the data groups that can be transferred in low-load servers can be transferred first in priority with those with large loads, which facilitates the transfer of all data groups in low-load servers.
可选的，还包括：Optional, also includes:
于低负载服务器中将所有的正常服务器均成为过转入服务器后仍留有的数据组定义为疑难数据组，且将疑难数据组所对应的低负载服务器定义为待处理服务器；In the low-load server, the data groups that remain after all normal servers have been transferred to the server are defined as difficult data groups, and the low-load servers corresponding to the difficult data groups are defined as pending servers;
于所有的待处理服务器中根据所有的疑难数据组进行穷举组合以确定数据组组合，并根据数据组组合确定需求占用数量，且根据数据组组合以及需求占用数量确定占用服务器；Exhaustively combine all difficult data groups in all servers to be processed to determine the data group combination, determine the required occupied quantity based on the data group combination, and determine the occupied server based on the data group combination and the required occupied quantity;
根据排序规则确定数值最小的需求占用数量，并判断是否存在至少两个需求占用数量数值最小的情况；Determine the required occupied quantity with the smallest value according to the sorting rules, and determine whether there are at least two situations with the smallest required occupied quantity;
若不存在至少两个需求占用数量数值最小的情况，则根据该数据组组合控制数据组转移至对应的占用服务器中；If there are not at least two situations with the smallest required occupancy quantity, the data group is controlled to be transferred to the corresponding occupancy server according to the data group combination;
若存在至少两个需求占用数量数值最小的情况，则根据各数据组组合以及相对应的占用服务器确定需求转移负载；If there are at least two situations with the smallest required occupancy quantity, the demand transfer load is determined based on each data group combination and the corresponding occupied server;
根据排序规则确定数值最小的需求转移负载，并根据该需求转移负载对应的数据组组合控制数据组进行转移。Determine the demand transfer load with the smallest value according to the sorting rules, and control the data group for transfer according to the data group combination corresponding to the demand transfer load.
通过采用上述技术方案，当出现所有正常服务器均无法容纳对应的数据组时，可对各疑难数据组进行整合归纳，使得部分服务器能够进行休眠作业。By adopting the above technical solution, when all normal servers cannot accommodate the corresponding data group, each difficult data group can be integrated and summarized, so that some servers can perform hibernation operations.
可选的，根据数据组组合以及需求占用数量确定占用服务器的步骤包括：Optionally, the steps to determine the occupied server based on the data group combination and required occupied quantity include:
于待处理服务器中根据需求占用数量进行组合以确定服务器占用组合；Combine the servers to be processed according to the required occupancy quantity to determine the server occupancy combination;
于服务器占用组合中根据各服务器的综合压力值进行均值计算以确定组合压力值；Perform an average calculation based on the comprehensive pressure value of each server in the server occupancy combination to determine the combined pressure value;
于服务器占用组合中根据排序规则确定数值最大的综合压力值，并将该综合压力值定义为极限压力值；Determine the largest comprehensive pressure value among the server occupancy combinations according to the sorting rules, and define the comprehensive pressure value as the ultimate pressure value;
根据数据组组合以及服务器占用组合确定模拟转移负载；Determine simulated transfer load based on data group combination and server occupancy combination;
根据预设的优先级算法结合组合压力值、极限压力值以及模拟转移负载进行计算以确定组合优先参数；Calculation is performed based on the preset priority algorithm combined with the combined pressure value, limit pressure value and simulated transfer load to determine the combined priority parameters;
根据排序规则确定数值最大的组合优先参数，并将该组合优先参数相对应的服务器占用组合中的待处理服务器确定为占用服务器。Determine the combination priority parameter with the largest value according to the sorting rule, and determine the server to be processed in the server occupancy combination corresponding to the combination priority parameter as the occupied server.
通过采用上述技术方案，可根据各服务器的压力情况确定较为合适的占用服务器进行使用。By adopting the above technical solution, a more suitable occupied server can be determined for use according to the pressure of each server.
可选的，于服务器占用组合确定前，基于数据中心的数据处理方法还包括：Optionally, before the server occupancy combination is determined, the data processing method based on the data center also includes:
判断待处理服务器的综合压力值是否大于预设的上限压力值；Determine whether the comprehensive pressure value of the server to be processed is greater than the preset upper limit pressure value;
若待处理服务器的综合压力值不大于上限压力值，则维持待处理服务器定义；If the comprehensive pressure value of the server to be processed is not greater than the upper limit pressure value, the definition of the server to be processed is maintained;
待待处理服务器的综合压力值大于上限压力值，则取消待处理服务器定义。If the comprehensive pressure value of the server to be processed is greater than the upper limit pressure value, the definition of the server to be processed is cancelled.
通过采用上述技术方案，可对部分长时间使用的待处理服务器进行排除，以便于该类待处理服务器进行休眠，减少服务器长时间作业而出现损坏的情况。By adopting the above technical solution, some servers to be processed that have been used for a long time can be eliminated, so that such servers to be processed can be put into sleep mode and the damage caused by long-term operation of the servers can be reduced.
第二方面，本申请提供一种基于数据中心的数据处理系统，采用如下的技术方案：In the second aspect, this application provides a data processing system based on a data center, adopting the following technical solution:
一种基于数据中心的数据处理系统，包括：A data processing system based on a data center, including:
获取模块，用于获取当前时间点；Get module, used to get the current time point;
处理模块，与获取模块和判断模块连接，用于信息的存储和处理；The processing module is connected to the acquisition module and the judgment module and is used for the storage and processing of information;
判断模块，与获取模块和处理模块连接，用于信息的判断；The judgment module is connected with the acquisition module and the processing module and is used for judging information;
于判断模块判断出当前时间点处于预设的数据非高峰时段内时获取模块实时获取各服务器的数据负载数值；When the judgment module determines that the current time point is within the preset data off-peak period, the acquisition module obtains the data load value of each server in real time;
判断模块判断数据负载数值是否小于预设的基准负载数值；The judgment module judges whether the data load value is less than the preset reference load value;
若判断模块判断出数据负载数值小于基准负载数值，则处理模块将该服务器定义为低负载服务器，并将该低负载服务器相对应的数据负载数值定义为需求转出数值；If the judgment module determines that the data load value is less than the baseline load value, the processing module defines the server as a low-load server, and defines the data load value corresponding to the low-load server as the demand transfer value;
若判断模块判断出数据负载数值不小于基准负载数值，则处理模块将该服务器定义为正常服务器，并根据正常服务器相对应的数据负载数值以及预设的上限负载数值进行差值计算以确定许可转入数值；If the judgment module determines that the data load value is not less than the benchmark load value, the processing module defines the server as a normal server, and performs a difference calculation based on the data load value corresponding to the normal server and the preset upper limit load value to determine the permitted transfer. Enter a value;
处理模块根据需求转出数值确定各低负载服务器的转出顺序且根据许可转入数值确定各正常服务器的转入顺序；The processing module determines the transfer-out sequence of each low-load server according to the demand transfer-out value and determines the transfer-in sequence of each normal server according to the permitted transfer-in value;
处理模块根据转入顺序以将低负载服务器内的数据根据转出顺序转移至各正常服务器内，并于低负载服务器内的数据全部转移后将该低负载服务器确定为空闲服务器，且对该空闲服务器进行休眠。The processing module transfers the data in the low-load server to each normal server according to the transfer-in order, and after all the data in the low-load server is transferred, the low-load server is determined to be an idle server, and the idle server is The server goes to sleep.
通过采用上述技术方案，获取模块获取当前时间点以使判断模块判断当前是否处于非高峰时段，当判断模块判断出处于非高峰时段时处理模块对每个服务器的负载情况进行确定，以对低负载服务器进行确定，从而能够将低负载服务器中的数据转移至其余的正常服务器中并对不存在数据的服务器进行休眠，从而使得较为散乱的数据能够集中至较少的服务器中，以减少数据存储处理时造成的能源浪费。By adopting the above technical solution, the acquisition module obtains the current time point so that the judgment module determines whether it is currently in an off-peak period. When the judgment module determines that it is in an off-peak period, the processing module determines the load status of each server to determine whether the load is low. The server determines, so that data in low-load servers can be transferred to other normal servers and servers without data can be hibernated, so that more scattered data can be concentrated on fewer servers to reduce data storage processing. energy wastage caused.
综上所述，本申请包括以下至少一种有益技术效果：To sum up, this application includes at least one of the following beneficial technical effects:
在非高峰时段可对低负载服务器中的数据进行转移以实现数据整合，使得相同量的数据无需启用较多的服务器，从而减少了数据存储处理时造成的能源浪费；Data in low-load servers can be transferred during off-peak hours to achieve data integration, so that the same amount of data does not need to activate more servers, thereby reducing energy waste caused by data storage and processing;
可对数据转移顺序进行确定，以便于对低负载服务器中的数据进行有效转移；The data transfer sequence can be determined to facilitate efficient transfer of data in low-load servers;
可对无法转移的数据组进行整合以减少需要启用的服务器的数量，尽可能减少能源的浪费。Data sets that cannot be transferred can be consolidated to reduce the number of servers that need to be enabled and to minimize wastage of energy.
附图说明Description of drawings
图1是基于数据中心的数据处理方法的流程图。Figure 1 is a flow chart of the data processing method based on the data center.
图2是转出顺序确定方法的流程图。Figure 2 is a flow chart of a method for determining a transfer order.
图3是转出顺序修正方法的流程图。Figure 3 is a flowchart of a rollout sequence correction method.
图4是数据转移控制方法的流程图。Figure 4 is a flow chart of the data transfer control method.
图5是疑难数据组整合处理方法的流程图。Figure 5 is a flow chart of a difficult data group integration processing method.
图6是占用服务器确定方法的流程图。Figure 6 is a flow chart of a method for determining an occupied server.
图7是服务器排除分析方法的流程图。Figure 7 is a flow chart of the server exclusion analysis method.
图8是基于数据中心的数据处理方法的模块流程图。Figure 8 is a module flow chart of the data processing method based on the data center.
具体实施方式Detailed ways
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图1-8及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in conjunction with the accompanying drawings 1-8 and examples. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.
下面结合说明书附图对本申请实施例作进一步详细描述。The embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
本申请实施例公开一种基于数据中心的数据处理方法，在非高峰时段可对每个服务器中的数据存储负载情况进行确定，以对低负载服务器中的数据能进行转移，实现数据的整合，使得所需启用的服务器数量减少，从而减少能源的浪费。The embodiment of the present application discloses a data processing method based on a data center, which can determine the data storage load in each server during off-peak hours, so as to transfer data in low-load servers and achieve data integration. This reduces the number of servers that need to be enabled, thereby reducing energy waste.
参照图1，基于数据中心的数据处理方法的方法流程包括以下步骤：Referring to Figure 1, the method flow of the data center-based data processing method includes the following steps:
步骤S100：获取当前时间点。Step S100: Obtain the current time point.
当前时间点为当前利用数据中心的服务器对数据进行存储处理的时间点。The current point in time is the point in time when the server in the data center is currently used to store and process data.
步骤S101：于当前时间点处于预设的数据非高峰时段内时实时获取各服务器的数据负载数值。Step S101: Obtain the data load value of each server in real time when the current time point is within the preset non-peak data period.
数据非高峰时段为工作人员所设定的认定不会存在数据变动较大的时段，以外卖平台为例，数据非高峰时段可以设定为凌晨2点-5点，具体的可由工作人员根据实际情况进行设定；当当前时间点处于数据非高峰时段内时，此时数据的变动不会较大，需要对当前的低负载服务器进行分析处理；数据负载数值为当前服务器中所存储的数据的量，可通过CPU占用率进行确定。The data non-peak hours are determined by the staff as there will be no periods of large data changes. Taking the takeout platform as an example, the data off-peak hours can be set to 2 a.m. to 5 a.m. The specific data can be determined by the staff according to the actual situation. The situation is set according to the situation; when the current time point is in the non-peak period of data, the change of data will not be large at this time, and the current low-load server needs to be analyzed and processed; the data load value is the proportion of the data stored in the current server. The amount can be determined by CPU usage.
步骤S102：判断数据负载数值是否小于预设的基准负载数值。Step S102: Determine whether the data load value is less than a preset reference load value.
基准负载数值为工作人员所设定的认定该服务器为低负载服务器时的最大数据负载数值，判断的目的是为了得知当前该服务器是否为低负载服务器。The benchmark load value is the maximum data load value set by the staff when the server is considered a low-load server. The purpose of the judgment is to know whether the server is currently a low-load server.
步骤S1021：若数据负载数值小于基准负载数值，则将该服务器定义为低负载服务器，并将该低负载服务器相对应的数据负载数值定义为需求转出数值。Step S1021: If the data load value is less than the baseline load value, define the server as a low-load server, and define the data load value corresponding to the low-load server as the demand transfer value.
当数据负载数值小于基准负载数值时，说明该服务器为低负载服务器，此时对该服务器进行定义以实现对不同服务器的区分；同时定义需求转出数值以对该服务器上需要转出的数据量进行确定，以便于后续分析。When the data load value is less than the baseline load value, it means that the server is a low-load server. At this time, the server is defined to distinguish between different servers; at the same time, the demand transfer value is defined to determine the amount of data that needs to be transferred out on the server. be determined for subsequent analysis.
步骤S1022：若数据负载数值不小于基准负载数值，则将该服务器定义为正常服务器，并根据正常服务器相对应的数据负载数值以及预设的上限负载数值进行差值计算以确定许可转入数值。Step S1022: If the data load value is not less than the baseline load value, the server is defined as a normal server, and a difference calculation is performed based on the data load value corresponding to the normal server and the preset upper limit load value to determine the transfer-in permission value.
当数据负载数值不小于基准负载数值时，说明该服务器不为低负载服务器，此时定义该服务器为正常服务器以进行标识，以实现不同服务器的区分；上限负载数值为服务器所能存储的最大负载数值，许可转入数值为该服务器还可供数据转入的负载数值，由上限负载数值减去数据负载数值进行确定。When the data load value is not less than the baseline load value, it means that the server is not a low-load server. At this time, the server is defined as a normal server for identification to distinguish different servers; the upper limit load value is the maximum load that the server can store. Value, the allowed transfer value is the load value that the server can also transfer data to, and is determined by subtracting the data load value from the upper limit load value.
步骤S103：根据需求转出数值确定各低负载服务器的转出顺序且根据许可转入数值确定各正常服务器的转入顺序。Step S103: Determine the transfer-out sequence of each low-load server according to the demand transfer-out value and determine the transfer-in sequence of each normal server according to the permitted transfer-in value.
转出顺序为各低负载服务器的数据进行转出时的顺序，即哪个低负载服务器的数据先转出哪个低负载服务器的数据后转出，具体确定方法下文进行说明，此处不作赘述；转入顺序为转出的顺序优先转入哪个正常服务器的顺序，该顺序可通过对许可转入数值进行排序，以使得许可转入数值较大的正常服务器顺序靠前，从而使得各正常服务器的负载量能尽可能靠近上限负载数值。The transfer-out order is the order in which the data of each low-load server is transferred out, that is, which low-load server's data is transferred out first and which low-load server's data is transferred out later. The specific determination method is explained below and will not be described here; transfer The incoming order is the order in which the normal server is transferred first in the order of transfer out. This order can be sorted by the allowed transfer value, so that the normal server with a larger allowed transfer value is placed first, thereby reducing the load of each normal server. The amount can be as close as possible to the upper limit load value.
步骤S104：根据转入顺序以将低负载服务器内的数据根据转出顺序转移至各正常服务器内，并于低负载服务器内的数据全部转移后将该低负载服务器确定为空闲服务器，且对该空闲服务器进行休眠。Step S104: Transfer the data in the low-load server to each normal server according to the transfer-in order, and after all the data in the low-load server is transferred, determine the low-load server as an idle server, and Idle servers go to sleep.
控制数据进行转移使得低负载服务器无需继续供数据进行存储，此时将该低负载服务器确定为空闲服务器进行标识，使得能对空闲服务器进行休眠控制，从而实现相同量的数据下所启用的服务器数量减少，进而减少了能源的浪费。The control data is transferred so that the low-load server does not need to continue to store data. At this time, the low-load server is determined as an idle server and identified, so that the idle server can be controlled to sleep, thereby realizing the number of enabled servers with the same amount of data. Reduce, thereby reducing energy waste.
参照图2，根据需求转出数值确定各低负载服务器的转出顺序的步骤包括：Referring to Figure 2, the steps to determine the transfer order of each low-load server based on the demand transfer value include:
步骤S200：于低负载服务器中进行数据组分析以获取独立组分数量以及各数据组的组分负载数值。Step S200: Perform data group analysis on the low-load server to obtain the number of independent components and the component load value of each data group.
数据组为共同进行存储不可进行拆分的数据所形成的数据组合，例如同一时间段下存储进入的一批数据，即使每个数据都是独立的，但是为了方便后续的调用，不可放置于多个服务器中进行存储，这些数据便形成一个数据组；数据组的分析可通过存储时间、数据本身情况等进行分析，独立组分数量为一个低负载服务器中所拥有的数据组的数量，组分负载数值为每个数据组的负载数值。A data group is a data combination formed by jointly storing data that cannot be split. For example, a batch of data is stored in the same time period. Even if each data is independent, in order to facilitate subsequent calls, it cannot be placed in multiple data groups. When stored in a server, these data form a data group; the analysis of the data group can be carried out through storage time, data itself, etc. The number of independent components is the number of data groups owned by a low-load server. The components The load value is the load value for each data group.
步骤S201：根据预设的排序规则对所有低负载服务器所确定的组分负载数值进行排序，并由大至小对数据组赋予处理难度值。Step S201: Sort the component load values determined by all low-load servers according to the preset sorting rules, and assign processing difficulty values to the data groups from large to small.
排序规则为工作人员所设定的能对数值大小进行排序的方法，例如冒泡法，通过排序规则可对每个数据组的组分负载数值进行排序，组分负载数值越低，说明越好对该数据组进行移动，例如负载为1的数据组，在上线负载为100的情况下，负载99的服务器依旧能够对该数据组进行接纳，处理难度值为对数据组进行移动的难度值，负载越大难度越大，通过排序情况可赋予不同的难度值，由大到小可依次进行赋值，例如共有5个数据组，则处理难度值可以由大至小依次为5、4、3、2、1，具体的处理难度值大小由工作人员根据实际情况进行设定。Sorting rules are methods set by staff to sort values, such as the bubbling method. Sorting rules can be used to sort the component load values of each data group. The lower the component load value, the better the explanation. Moving the data group, for example, a data group with a load of 1, when the online load is 100, a server with a load of 99 can still accept the data group, and the processing difficulty value is the difficulty value of moving the data group. The greater the load, the greater the difficulty. Different difficulty values can be assigned according to the sorting situation, and the values can be assigned in order from large to small. For example, there are 5 data groups in total, and the processing difficulty values can be 5, 4, 3, and so on from large to small. 2. 1. The specific processing difficulty value is set by the staff according to the actual situation.
步骤S202：于单个低负载服务器中根据所有的数据组对应的处理难度值进行求和计算以确定综合难度值。Step S202: Perform a summation calculation based on the processing difficulty values corresponding to all data groups in a single low-load server to determine a comprehensive difficulty value.
综合难度值为对该低负载服务器中的数据组进行全部转移的难度值，由该低负载服务器中的所有数据组对应的处理难度值相加获取。The comprehensive difficulty value is the difficulty value of transferring all data groups in the low-load server, and is obtained by adding the processing difficulty values corresponding to all data groups in the low-load server.
步骤S203：根据综合难度值以及相对应的独立组分数量进行均值计算以确定均值难度值。Step S203: Perform average calculation based on the comprehensive difficulty value and the corresponding number of independent components to determine the average difficulty value.
均值难度值为反应处理该低负载服务器中的数据的难易程度值。The mean difficulty value reflects the ease of processing data on that low-load server.
步骤S204：根据排序规则对均值难度值由小至大进行排序以确定转出顺序。Step S204: Sort the average difficulty values from small to large according to the sorting rules to determine the transfer order.
通过排序规则根据均值难度值由小至大确定转出顺序，可对转移难度较小的服务器中的数据进行优先转移，使得正常服务器能尽可能的填满。Through sorting rules, the transfer order is determined from small to large according to the average difficulty value. Data in servers with less difficulty in transferring can be transferred first, so that normal servers can be filled as much as possible.
参照图3，于均值难度值确定后，转出顺序的确定步骤还包括：Referring to Figure 3, after the average difficulty value is determined, the steps for determining the transfer order also include:
步骤S300：获取各低负载服务器的连续作业时长。Step S300: Obtain the continuous operation duration of each low-load server.
连续作业时长为低负载服务器从上次休眠结束后至当前时间点之间的时长。The continuous job duration is the duration between the last hibernation of the low-load server and the current point in time.
步骤S301：根据连续作业时长以当前时间点为后端点以建立检测区间，并于检测区间内生成一可于检测区间内进行移动的检测点，且获取检测点的单点负载数值。Step S301: Establish a detection interval based on the continuous operation duration with the current time point as the back end point, generate a detection point within the detection interval that can move within the detection interval, and obtain the single-point load value of the detection point.
建立检测区间可便于对该低负载服务器于连续作业时长内的作业数据进行获取并分析；生成检测点可对检测区间中每个时间点的数据进行采集，单点负载数值为该低负载服务器于检测点所在位置时所存储的数据负载数值。Establishing a detection interval can facilitate the acquisition and analysis of the operation data of the low-load server within the continuous operation period; generating detection points can collect data at each time point in the detection interval. The single-point load value is the low-load server in The data load value stored when the detection point is located.
步骤S302：根据检测点以及检测区间的前端点生成作业区间，并根据作业区间确定检测时长。Step S302: Generate an operation interval based on the detection point and the front end point of the detection interval, and determine the detection duration based on the operation interval.
建立作业区间可对该低负载服务器于检测点之间的作业情况进行分析；检测时长为作业区间的跨度时长。Establishing an operation interval can analyze the operation of the low-load server between detection points; the detection time is the span of the operation interval.
步骤S303：于作业区间内根据所有点位的单点负载数值进行求和计算以确定作业负载数值，并根据作业负载数值以及检测时长进行计算以确定检测点的前序影响值。Step S303: Perform a summation calculation based on the single-point load values of all points in the operating interval to determine the operating load value, and perform calculations based on the operating load value and the detection duration to determine the preceding influence value of the detection point.
作业负载数值为低负载服务器于作业区间中所承受的所有数据负载，由作业区间中所确定的所有单点负载数值相加获取；不同的作业负载数值说明该服务器于作业区间内的CPU调用率不同，此时服务器作业所产生的热量也不同，因此于检测点时服务器的作业状态也不同，前序影响值即服务器于作业区间内的作业情况对当前检测点作业状态的影响数值，当检测时长越大，作业负载数值越大时，前序影响值也越大，两者的计算公式为ε=αI+βT，其中ε为前序影响值，I为作业负载数值，α为作业负载数值的调整参数，T为检测时长，β为检测时长的调整参数，α以及β均为工作人员设定的定值。The workload value is all the data load borne by the low-load server in the operation interval, and is obtained by adding up all the single-point load values determined in the operation interval; different workload values indicate the CPU utilization rate of the server in the operation interval. Different, the heat generated by the server operation at this time is also different, so the operation status of the server at the detection point is also different. The preorder influence value is the impact value of the operation status of the server in the operation interval on the operation status of the current detection point. When detection The larger the duration and the larger the workload value, the greater the preceding influence value. The calculation formula for both is ε=αI+βT, where ε is the preceding influence value, I is the workload value, and α is the workload value. Adjustment parameters, T is the detection duration, β is the adjustment parameter of the detection duration, α and β are fixed values set by the staff.
步骤S304：根据预设的压力匹配关系以确定前序影响值以及单点负载数值相对应的单点压力值。Step S304: Determine the previous influence value and the single point pressure value corresponding to the single point load value according to the preset pressure matching relationship.
单点压力值为检测点处服务器所受到的存储压力，当前序影响值越大时，说明对当前的存储压力影响较大，当单点负载数值较大时，同样说明对当前的存储压力影响较大，三者之间的压力匹配关系由工作人员事先根据多次试验以确定，不作赘述。The single-point pressure value is the storage pressure experienced by the server at the detection point. The larger the preorder impact value, the greater the impact on the current storage pressure. The greater the single-point load value, the greater the impact on the current storage pressure. The pressure matching relationship between the three is determined by the staff in advance based on multiple tests and will not be described again.
步骤S305：根据所有的单点压力值进行求和计算以确定综合压力值。Step S305: Perform sum calculation based on all single point pressure values to determine the comprehensive pressure value.
综合压力值为当前该低负载服务器进行作业所受到的压力值，该压力值越大说明低负载服务器的存储效果越差，也即需要对该服务器进行休眠处理以减少服务器长时间作业而出现损坏的情况。The comprehensive pressure value is the pressure value currently experienced by the low-load server during operation. The greater the pressure value, the worse the storage effect of the low-load server. That is, the server needs to be hibernated to reduce damage caused by long-term operation of the server. Case.
步骤S306：根据预设的修正匹配关系以确定综合压力值相对应的修正参数，并根据修正参数对均值难度值进行修正更新。Step S306: Determine the correction parameters corresponding to the comprehensive pressure value according to the preset correction matching relationship, and correct and update the average difficulty value according to the correction parameters.
修正参数为对均值难度值进行修正的参数，不同的综合压力值说明服务器需要休息的程度不一样，此时确定不同的修正参数对均值难度值进行修正更新，使得转出顺序排序时可综合服务器使用情况进行确定，提高转出顺序确定的合理性；其中两者之间的修正匹配关系由工作人员事先根据多次试验以确定。The correction parameter is a parameter that corrects the average difficulty value. Different comprehensive pressure values indicate that the server needs to rest to a different extent. At this time, different correction parameters are determined to correct and update the average difficulty value, so that the server can be integrated when the order is transferred out. Determine the usage to improve the rationality of determining the transfer sequence; the corrected matching relationship between the two is determined by the staff in advance based on multiple tests.
参照图4，控制数据转移的步骤包括：Referring to Figure 4, the steps to control data transfer include:
步骤S400：将当前需要进行数据转移的低负载服务器定义为转出服务器，并将当前需要进行数据转入的正常服务器定义为转入服务器，且将转入服务器相对应的许可转入数值定义为剩余负载数值。Step S400: Define the low-load server currently requiring data transfer as the transfer-out server, define the normal server currently requiring data transfer as the transfer-in server, and define the transfer-in permission value corresponding to the transfer-in server as remaining load value.
定义转出服务器以及转入服务器以便于对当前进行数据转移的情况进行分析，同时定义剩余负载数值可对转入服务器中可供数据转入的空间进行确定，便于后续分析。Define the transfer-out server and the transfer-in server to facilitate the analysis of the current data transfer situation. At the same time, define the remaining load value to determine the space available for data transfer in the transfer-in server to facilitate subsequent analysis.
步骤S401：于转出服务器中判断数据组的组分负载数值是否大于剩余负载数值。Step S401: Determine whether the component load value of the data group is greater than the remaining load value in the transfer-out server.
判断的目的是为了得知转出服务器中的该数据组是否能够转移至转入服务器中。The purpose of the judgment is to know whether the data group in the transfer-out server can be transferred to the transfer-in server.
步骤S4011：若组分负载数值大于剩余负载数值，则将该数据组定义为无效数据组。Step S4011: If the component load value is greater than the remaining load value, define the data group as an invalid data group.
当组分负载数值大于剩余负载数值时，说明转入服务器的空间不足以容纳该数据组，此时该数据组无法转移至转入服务器中，从而将该数据组定义为无效数据组以进行标识，以便于后续分析。When the component load value is greater than the remaining load value, it means that the transfer server has insufficient space to accommodate the data group. At this time, the data group cannot be transferred to the transfer server, and the data group is defined as an invalid data group for identification. , to facilitate subsequent analysis.
步骤S4012：若组分负载数值不大于剩余负载数值，则将该数据组定义为有效数据组。Step S4012: If the component load value is not greater than the remaining load value, define the data group as a valid data group.
当组分负载数值不大于剩余负载数值时，说明转入服务器的空间足以容纳该数据组，此时该数据组能转移至转入服务器中，从而将该数据组定义为有效数据组以进行标识，以便于后续分析。When the component load value is not greater than the remaining load value, it means that the space of the transfer server is enough to accommodate the data group. At this time, the data group can be transferred to the transfer server, thereby defining the data group as a valid data group for identification. , to facilitate subsequent analysis.
步骤S402：于有效数据组中根据排序规则确定数值最大的组分负载数值，并将该组分负载数值相对应的有效数据组转移至转入服务器内，且于剩余的数据组中继续确定有效数据组以进行数据转移，直至不存在有效数据组时控制转入服务器根据转入顺序变化。Step S402: Determine the component load value with the largest value in the valid data group according to the sorting rule, transfer the valid data group corresponding to the component load value to the transfer server, and continue to determine the validity in the remaining data groups. The data group is used for data transfer until there is no valid data group. The control transfer server changes according to the transfer order.
通过排序规则将能够转移的有效数据组中负载最大的数据组进行转移，以使得转移难度较大的能优先进行处理，从而使得低负载服务器中的数据能够尽可能的全部进行转移；依次选择不同的转入服务器进行数据转移，使得低负载服务器中的数据能够有效转移。Use sorting rules to transfer the data group with the largest load among the effective data groups that can be transferred, so that the data group that is more difficult to transfer can be processed first, so that all the data in the low-load server can be transferred as much as possible; select different ones in sequence The data is transferred to the server so that the data in the low-load server can be transferred effectively.
参照图5，基于数据中心的数据处理方法还包括：Referring to Figure 5, the data processing method based on the data center also includes:
步骤S500：于低负载服务器中将所有的正常服务器均成为过转入服务器后仍留有的数据组定义为疑难数据组，且将疑难数据组所对应的低负载服务器定义为待处理服务器。Step S500: In the low-load server, the data groups that remain after all normal servers have been transferred to the server are defined as difficult data groups, and the low-load servers corresponding to the difficult data groups are defined as pending servers.
在数据转移过程中，可能出现部分数据组负载较大而不易转移至其余正常服务器的情况，需要进一步分析；此时定义疑难数据组以及待处理服务器以便于后续分析；其中，当有疑难数据组存在时，先不对该疑难数据组进行处理，以根据转出顺序继续对其余低负载服务器进行处理，直至剩余疑难数据组后再对所有的疑难数据组进行统一处理。During the data transfer process, some data groups may have a heavy load and cannot be easily transferred to other normal servers, and further analysis is required. At this time, difficult data groups and servers to be processed are defined to facilitate subsequent analysis; among them, when there are difficult data groups When it exists, the difficult data group will not be processed first, and other low-load servers will continue to be processed according to the transfer order. Until the difficult data groups remain, all difficult data groups will be processed uniformly.
步骤S501：于所有的待处理服务器中根据所有的疑难数据组进行穷举组合以确定数据组组合，并根据数据组组合确定需求占用数量，且根据数据组组合以及需求占用数量确定占用服务器。Step S501: Exhaustively combine all difficult data groups in all servers to be processed to determine the data group combination, determine the required occupancy quantity based on the data group combination, and determine the occupied servers based on the data group combination and the required occupancy quantity.
数据组组合为疑难数据组进行搭配时的组合，例如存在疑难数据组A、B、C，其中数据组组合可以为以下五种情况：①A、B、C；②A、BC；③B，AC；④C、AB；⑤ABC；当五种情况确定后，需要对组合情况进行分析，例如ABC三个疑难数据组的负载相加大于上限负载数值时，则不存在ABC处于同一服务器的情况，此时数据组组合不应存在上述中的第⑤方案；需求占用数量为根据数据组组合对疑难数据组进行组合时所需用到的服务器数量，以上述例子进行说明，其中①方案的需求占用数量为3；占用服务器为数据组组合需要转移至的服务器，该占用服务器于待处理服务器中进行选取，具体确定方法下文进行说明，此处不作赘述。The data group combination is the combination of difficult data groups. For example, there are difficult data groups A, B, and C. The data group combinations can be the following five situations: ①A, B, C; ②A, BC; ③B, AC; ④C , AB; ⑤ABC; When the five situations are determined, the combination situation needs to be analyzed. For example, when the sum of the loads of the three difficult data groups ABC is greater than the upper limit load value, there is no situation where ABC is on the same server. At this time, the data groups The combination should not include the ⑤ option among the above; the required occupancy quantity is the number of servers required to combine difficult data groups according to the data group combination. The above example is used to illustrate, in which the required occupancy quantity of the ① option is 3; The occupied server is the server to which the data group combination needs to be transferred. The occupied server is selected from the servers to be processed. The specific determination method will be described below and will not be described in detail here.
步骤S502：根据排序规则确定数值最小的需求占用数量，并判断是否存在至少两个需求占用数量数值最小的情况。Step S502: Determine the required occupancy quantity with the smallest value according to the sorting rule, and determine whether there are at least two situations with the smallest required occupancy quantity.
通过排序规则可确定出需要服务器最少的方案，判断的目的是为了得知是否有多个使用服务器均最少的方案，以便于判断是否需要于多个方案中继续选取情况，以上述例子进行说明，②③④的需求占用数量均为2，此时需要进一步分析以确定需要选用何种方案。The solution that requires the least servers can be determined through the sorting rules. The purpose of the judgment is to know whether there are multiple solutions that use the least servers, so as to judge whether it is necessary to continue to select among multiple solutions. The above example is used to illustrate. The required occupied quantities of ②③④ are all 2. At this time, further analysis is needed to determine which solution needs to be selected.
步骤S5021：若不存在至少两个需求占用数量数值最小的情况，则根据该数据组组合控制数据组转移至对应的占用服务器中。Step S5021: If there are no at least two situations with the smallest required occupancy quantity, then the control data group is transferred to the corresponding occupancy server according to the data group combination.
当不存在至少两个需求占用数量数值最小的情况时，说明仅有一种方案所需选用的服务器数量最少，此时根据该数据组组合的方案进行数据组转移即可。When there are not at least two situations with the smallest required occupancy quantity, it means that there is only one solution that requires the smallest number of servers. At this time, the data group can be transferred according to the solution of the data group combination.
步骤S5022：若存在至少两个需求占用数量数值最小的情况，则根据各数据组组合以及相对应的占用服务器确定需求转移负载。Step S5022: If there are at least two situations with the smallest required occupied quantity, determine the required load transfer based on each data group combination and the corresponding occupied server.
当存在至少两个需求占用数量数值最小的情况时，说明至少有两个方案所选用的服务器数量同时最少，需要进一步分析；需求转移负载为根据数据组组合以及占用服务器时所需要转移的数据组负载之和。When there are at least two situations with the smallest required occupancy quantity, it means that at least two solutions have the smallest number of servers selected at the same time, which requires further analysis; the demand transfer load is based on the data group combination and the data group that needs to be transferred when the server is occupied. sum of loads.
步骤S503：根据排序规则确定数值最小的需求转移负载，并根据该需求转移负载对应的数据组组合控制数据组进行转移。Step S503: Determine the demand transfer load with the smallest value according to the sorting rule, and control the data group for transfer according to the data group combination corresponding to the demand transfer load.
通过排序规则可确定数值最小的需求转移负载，以根据该需求转移负载对应的数据组组合进行数据转移，从而使得整体所需转移的数量较少，便于实现数据转移。Through the sorting rules, the demand transfer load with the smallest value can be determined, and the data transfer can be carried out according to the data group combination corresponding to the demand transfer load, so that the overall required transfer quantity is smaller and the data transfer is facilitated.
参照图6，根据数据组组合以及需求占用数量确定占用服务器的步骤包括：Referring to Figure 6, the steps to determine the occupied server based on the data group combination and the required occupied quantity include:
步骤S600：于待处理服务器中根据需求占用数量进行组合以确定服务器占用组合。Step S600: Combine the servers to be processed according to the required occupancy quantities to determine the server occupancy combination.
服务器占用组合为所需占用的服务器的组合，例如存在待处理服务器X、Y、Z，需求占用数量为2，则服务器占用组合分别为XY、XZ和YZ。The server occupancy combination is the combination of servers that need to be occupied. For example, there are servers X, Y, and Z to be processed, and the required occupancy quantity is 2, then the server occupancy combinations are XY, XZ, and YZ respectively.
步骤S601：于服务器占用组合中根据各服务器的综合压力值进行均值计算以确定组合压力值。Step S601: Perform average calculation based on the comprehensive pressure value of each server in the server occupancy combination to determine the combined pressure value.
组合压力值为所选用的服务器占用组合中的低负载服务器的综合压力值之和。The combined pressure value is the sum of the comprehensive pressure values of the low-load servers in the selected server occupancy combination.
步骤S602：于服务器占用组合中根据排序规则确定数值最大的综合压力值，并将该综合压力值定义为极限压力值。Step S602: Determine the largest comprehensive pressure value in the server occupancy combination according to the sorting rule, and define the comprehensive pressure value as the limit pressure value.
通过排序规则可确定每个服务器占用组合中最大压力的服务器，并将该服务器的综合压力值定义为极限压力值以进行标识，以便于后续分析。Through sorting rules, the server with the highest pressure in each server occupancy combination can be determined, and the comprehensive pressure value of the server is defined as the ultimate pressure value for identification to facilitate subsequent analysis.
步骤S603：根据数据组组合以及服务器占用组合确定模拟转移负载。Step S603: Determine the simulated transfer load according to the data group combination and server occupancy combination.
模拟转移负载为数据组组合根据服务器占用组合进行转移时所需要进行转移的数据负载，以上述方案②以及XY进行说明，其中A需要占用一个服务器，BC占用一个服务器，此时存在两种情况，即一，A占用X服务器以及BC占用Y服务器；二A占用Y服务器以及BC占用X服务器，此时的模拟转移负载为两种情况下所需转移的数据负载的最小值。The simulated transfer load is the data load that needs to be transferred when the data group combination is transferred according to the server occupancy combination. The above scheme ② and XY are used to illustrate, among which A needs to occupy one server, and BC needs to occupy one server. There are two situations at this time, That is, first, A occupies the X server and BC occupies the Y server; second, A occupies the Y server and BC occupies the X server. The simulated transfer load at this time is the minimum value of the data load required to be transferred in the two situations.
步骤S604：根据预设的优先级算法结合组合压力值、极限压力值以及模拟转移负载进行计算以确定组合优先参数。Step S604: Calculate the combined pressure value, limit pressure value and simulated transfer load according to the preset priority algorithm to determine the combined priority parameter.
组合优先参数为服务器占用组合确定时的优先参数，该数值越大说明该组合优先级越高；优先级算法为，其中ρ为组合优先参数，θ为组合压力值，μ为极限压力值，σ为模拟转移负载。The combination priority parameter is the priority parameter when the server occupancy combination is determined. The larger the value, the higher the priority of the combination; the priority algorithm is , where ρ is the combined priority parameter, θ is the combined pressure value, μ is the ultimate pressure value, and σ is the simulated transfer load.
步骤S605：根据排序规则确定数值最大的组合优先参数，并将该组合优先参数相对应的服务器占用组合中的待处理服务器确定为占用服务器。Step S605: Determine the combination priority parameter with the largest value according to the sorting rule, and determine the server to be processed in the server occupancy combination corresponding to the combination priority parameter as the occupied server.
通过排序规则可确定数值最大的组合优先参数，即该组合优先参数对应的服务器占用组合最为合理，此时根据该服务器占用组合确定占用服务器即可。The sorting rule can be used to determine the combination priority parameter with the largest value, that is, the server occupancy combination corresponding to this combination priority parameter is the most reasonable. At this time, the occupied server can be determined based on the server occupancy combination.
参照图7，于服务器占用组合确定前，基于数据中心的数据处理方法还包括：Referring to Figure 7, before the server occupancy combination is determined, the data processing method based on the data center also includes:
步骤S700：判断待处理服务器的综合压力值是否大于预设的上限压力值。Step S700: Determine whether the comprehensive pressure value of the server to be processed is greater than the preset upper limit pressure value.
上限压力值为工作人员所设定的服务器作业压力较大需要进行休眠状态时的最小综合压力值，判断的目的是为了得知待处理服务器中是否有服务器一定需要进行休眠处理。The upper limit pressure value is the minimum comprehensive pressure value set by the staff when the server operating pressure is high and requires hibernation. The purpose of the judgment is to know whether any of the servers to be processed must be hibernated.
步骤S7001：若待处理服务器的综合压力值不大于上限压力值，则维持待处理服务器定义。Step S7001: If the comprehensive pressure value of the server to be processed is not greater than the upper limit pressure value, maintain the definition of the server to be processed.
当待处理服务器的综合压力值不大于上限压力值时，说明该待处理服务器能够继续进行作业，此时对该待处理服务器的定义进行维持，以便于后续确定占用服务器。When the comprehensive pressure value of the server to be processed is not greater than the upper limit pressure value, it means that the server to be processed can continue to work. At this time, the definition of the server to be processed is maintained to facilitate subsequent determination of occupied servers.
步骤S7002：待待处理服务器的综合压力值大于上限压力值，则取消待处理服务器定义。Step S7002: If the comprehensive pressure value of the server to be processed is greater than the upper limit pressure value, the definition of the server to be processed is cancelled.
当待处理服务器的综合压力值大于上限压力值时，说明该待处理服务器需要进行休眠处理，不可继续进行数据存储，此时对该待处理服务器的定义取消，以减少该待处理服务器被定义为占用服务器的情况发生。When the comprehensive pressure value of the server to be processed is greater than the upper limit pressure value, it means that the server to be processed needs to be dormant and data storage cannot continue. At this time, the definition of the server to be processed is canceled to reduce the number of servers to be processed that are defined. Server occupation occurs.
参照图8，基于同一发明构思，本发明实施例提供一种基于数据中心的数据处理系统，包括：Referring to Figure 8, based on the same inventive concept, an embodiment of the present invention provides a data center-based data processing system, including:
获取模块，用于获取当前时间点；Get module, used to get the current time point;
处理模块，与获取模块和判断模块连接，用于信息的存储和处理；The processing module is connected to the acquisition module and the judgment module and is used for the storage and processing of information;
判断模块，与获取模块和处理模块连接，用于信息的判断；The judgment module is connected with the acquisition module and the processing module and is used for judging information;
于判断模块判断出当前时间点处于预设的数据非高峰时段内时获取模块实时获取各服务器的数据负载数值；When the judgment module determines that the current time point is within the preset data off-peak period, the acquisition module obtains the data load value of each server in real time;
判断模块判断数据负载数值是否小于预设的基准负载数值；The judgment module judges whether the data load value is less than the preset reference load value;
若判断模块判断出数据负载数值小于基准负载数值，则处理模块将该服务器定义为低负载服务器，并将该低负载服务器相对应的数据负载数值定义为需求转出数值；If the judgment module determines that the data load value is less than the baseline load value, the processing module defines the server as a low-load server, and defines the data load value corresponding to the low-load server as the demand transfer value;
若判断模块判断出数据负载数值不小于基准负载数值，则处理模块将该服务器定义为正常服务器，并根据正常服务器相对应的数据负载数值以及预设的上限负载数值进行差值计算以确定许可转入数值；If the judgment module determines that the data load value is not less than the benchmark load value, the processing module defines the server as a normal server, and performs a difference calculation based on the data load value corresponding to the normal server and the preset upper limit load value to determine the permitted transfer. Enter a value;
处理模块根据需求转出数值确定各低负载服务器的转出顺序且根据许可转入数值确定各正常服务器的转入顺序；The processing module determines the transfer-out sequence of each low-load server according to the demand transfer-out value and determines the transfer-in sequence of each normal server according to the permitted transfer-in value;
处理模块根据转入顺序以将低负载服务器内的数据根据转出顺序转移至各正常服务器内，并于低负载服务器内的数据全部转移后将该低负载服务器确定为空闲服务器，且对该空闲服务器进行休眠；The processing module transfers the data in the low-load server to each normal server according to the transfer-in order, and after all the data in the low-load server is transferred, the low-load server is determined to be an idle server, and the idle server is The server goes to sleep;
转出顺序确定模块，用于确定各低负载服务器的转出顺序；The transfer-out sequence determination module is used to determine the transfer-out sequence of each low-load server;
转出顺序修正模块，综合考虑各低负载服务器的作业压力情况以对转出顺序进行修正；The transfer order correction module comprehensively considers the operating pressure of each low-load server to correct the transfer order;
数据转移控制模块，用于控制低负载服务器中的数据进行转移；Data transfer control module, used to control the transfer of data in low-load servers;
疑难数据组整合模块，用于对疑难数据组进行整合以减少需要启用的服务器数量；The difficult data group integration module is used to integrate difficult data groups to reduce the number of servers that need to be enabled;
占用服务器确定模块，用于确定较为合适的占用服务器进行使用；The occupied server determination module is used to determine a more suitable occupied server for use;
服务器排除分析模块，用于对压力较大的待处理服务器进行排除，以减少服务器长时间作业而出现损坏的情况。The server elimination analysis module is used to eliminate servers that are under heavy pressure to be processed, so as to reduce damage caused by long-term operation of the server.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above function allocation can be completed by different functional modules according to needs, that is, The internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working processes of the systems, devices and units described above, reference can be made to the corresponding processes in the foregoing method embodiments, which will not be described again here.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311406046.9ACN117271138B (en) | 2023-10-27 | 2023-10-27 | A data processing method and system based on data center |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311406046.9ACN117271138B (en) | 2023-10-27 | 2023-10-27 | A data processing method and system based on data center |
| Publication Number | Publication Date |
|---|---|
| CN117271138A CN117271138A (en) | 2023-12-22 |
| CN117271138Btrue CN117271138B (en) | 2024-03-01 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311406046.9AActiveCN117271138B (en) | 2023-10-27 | 2023-10-27 | A data processing method and system based on data center |
| Country | Link |
|---|---|
| CN (1) | CN117271138B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118708326B (en)* | 2024-08-30 | 2024-11-22 | 山东辰华科技信息有限公司 | Resource scheduling method and system based on data center |
| CN118779119B (en)* | 2024-09-13 | 2025-01-21 | 山东辰华科技信息有限公司 | A data center-based information storage method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001109642A (en)* | 1999-10-08 | 2001-04-20 | Toshiba Corp | Cluster system and data copying method thereof |
| JP2009116380A (en)* | 2007-11-01 | 2009-05-28 | Nec Corp | Virtual server movement controller, virtual server movement control method and program |
| CN101662370A (en)* | 2008-08-26 | 2010-03-03 | 华为技术有限公司 | Dormancy/wakening method for server and server with dormancy/wakening function |
| CN103188277A (en)* | 2011-12-27 | 2013-07-03 | 中国电信股份有限公司 | Load energy consumption management system, method and server |
| CN106713377A (en)* | 2015-07-29 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Distributed database load leveling method and device |
| CN115840686A (en)* | 2022-11-14 | 2023-03-24 | 苏州浪潮智能科技有限公司 | Server performance test method and device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001109642A (en)* | 1999-10-08 | 2001-04-20 | Toshiba Corp | Cluster system and data copying method thereof |
| JP2009116380A (en)* | 2007-11-01 | 2009-05-28 | Nec Corp | Virtual server movement controller, virtual server movement control method and program |
| CN101662370A (en)* | 2008-08-26 | 2010-03-03 | 华为技术有限公司 | Dormancy/wakening method for server and server with dormancy/wakening function |
| CN103188277A (en)* | 2011-12-27 | 2013-07-03 | 中国电信股份有限公司 | Load energy consumption management system, method and server |
| CN106713377A (en)* | 2015-07-29 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Distributed database load leveling method and device |
| CN115840686A (en)* | 2022-11-14 | 2023-03-24 | 苏州浪潮智能科技有限公司 | Server performance test method and device, electronic equipment and storage medium |
| Title |
|---|
| 史倩倩.基于深度强化学习的数据中心网络能耗管理系统.北京邮电大学硕士学位论文.2021,第1-49页.* |
| Publication number | Publication date |
|---|---|
| CN117271138A (en) | 2023-12-22 |
| Publication | Publication Date | Title |
|---|---|---|
| CN117271138B (en) | A data processing method and system based on data center | |
| CN107851040A (en) | For the system and method using cache requirements monitoring scheduler task in heterogeneous processor cluster framework | |
| CN111782627B (en) | Task and data cooperative scheduling method for wide-area high-performance computing environment | |
| WO2024036927A1 (en) | Consistency algorithm-based cloud-edge collaboration regulation and control method and system for power distribution area group | |
| CN118229039B (en) | Multi-target intelligent optimization APS scheduling method and system | |
| CN111639793A (en) | Boiler group scheduling optimization method and device | |
| CN109144666A (en) | A kind of method for processing resource and system across cloud platform | |
| CN112306696B (en) | Energy-saving and efficient edge computing task unloading method and system | |
| CN118277056A (en) | Task scheduling method and device, terminal equipment and storage medium | |
| CN108429784B (en) | An energy efficiency priority cloud resource allocation and scheduling method | |
| CN110830560A (en) | A Multi-User Mobile Edge Computing Migration Method Based on Reinforcement Learning | |
| CN116360921A (en) | Cloud platform resource optimal scheduling method and system for electric power Internet of things | |
| CN118708326B (en) | Resource scheduling method and system based on data center | |
| CN119010054B (en) | A park energy optimization method, device, electronic equipment and storage medium | |
| Tang et al. | Reliable and adaptive computation offload strategy with load and cost coordination for edge computing | |
| CN118779119B (en) | A data center-based information storage method and system | |
| Xie et al. | A Two‐Workshop Collaborative, Integrated Scheduling Algorithm considering the Prescheduling of the Root‐Subtree Processes | |
| CN118631847A (en) | Distributed dynamic scheduling method, device, equipment and medium for power Internet of Things business | |
| WO2024230640A1 (en) | Multi-energy power system oscillation stability evaluation method and apparatus, device and medium | |
| CN110489221B (en) | Method and system for dynamically adjusting computing resources by automatic task number | |
| CN107870818A (en) | Polycaryon processor interrupts dynamic response method and storage medium | |
| CN117294712A (en) | Dynamic calculation unloading strategy based on task group optimization | |
| CN116643866A (en) | Job scheduling processing method, apparatus, chip, storage medium and program product | |
| CN104702676B (en) | A kind of railway distributive data center resource regulating method | |
| CN115373860A (en) | Scheduling method, device and equipment of GPU (graphics processing Unit) tasks and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address | Address after:252000, 11th Floor, South Fund Building, Lushan Road, East Huaihe Road, Jiuzhou Street, High tech Industrial Development Zone, Liaocheng City, Shandong Province Patentee after:SHANDONG SUNSAM INFORMATION TECHNOLOGY CO.,LTD. Country or region after:China Patentee after:DONGCHANG COLLEGE OF LIAOCHENG University Address before:No. 111, Changjiang Road, Liaocheng hi tech Zone, Shandong, Shandong Patentee before:SHANDONG SUNSAM INFORMATION TECHNOLOGY CO.,LTD. Country or region before:China Patentee before:DONGCHANG COLLEGE OF LIAOCHENG University |