Disclosure of Invention
In order to solve the problems that the prior art mainly depends on static rules and preset scheduling algorithms in resource scheduling, is difficult to cope with high dynamic and high concurrent computing demands, and is difficult to update task and resource states in real time, and the prior art is easy to cause uneven resource allocation, idle computing resources or task delay, for example, when facing large-scale concurrent tasks, the prior scheduling system cannot flexibly adjust the priority of the tasks or adjust the allocation paths of the computing resources, so that certain important tasks cannot be timely executed due to resource competition, even the stability of the whole system is influenced, the prior art is difficult to timely respond and adjust the actual load situation of the resources, and further the optimization of the resource utilization rate is influenced, and the problems are particularly prominent in the scenes of cloud computing, large-scale data processing, high-performance computing and the like, so that the computing resources are wasted and the efficiency of the whole system is reduced. The technical scheme is as follows:
in one aspect, an intelligent scheduling system based on computational power demand prediction is provided, the system comprising:
The task structure analysis module extracts execution time length, priority, dependency strength, CPU occupation, memory demand and bandwidth distribution data according to the task description information, and sorts task dependency paths and resource distribution conditions to obtain a task and resource association network model;
The dependency relationship evaluation module extracts the connection weight between task nodes and identifies the dependency path length based on the task and resource association network model, quantifies the direct and indirect dependency strength, and generalizes the interaction weight to obtain a task and resource interaction strength data set;
The calculation force demand quantification module extracts calculation force demand parameters of the task nodes based on the task and resource interaction intensity data set, and performs hierarchical weighted analysis by combining the availability of the resource nodes and the task priority to obtain a task calculation force demand distribution table;
the resource allocation module is used for identifying the allocation relation between the tasks and the resource nodes according to the task priority and the demand allocation matching value ordering based on the task computing power demand allocation table, adjusting the allocation sequence according to the idle condition of the resource nodes and constructing an initial scheduling resource allocation scheme;
And the scheduling dynamic adjustment module monitors the execution state of the task node and the idle weight of the resource node based on the initial scheduling resource allocation scheme, compares the calculation power requirement of the unassigned task and the resource remaining capacity in real time, fills idle time slots by adjusting the matching sequence of the task and the resource node, and generates a task dynamic scheduling optimization result.
As a further scheme of the invention, the task and resource association network model comprises task nodes, resource nodes, task and resource attribute matching weights, task dependency paths and resource distribution conditions, the task and resource interaction strength data set comprises connection weights among the task nodes, dependency path lengths, direct dependency strengths, indirect dependency strengths and interaction weights among the tasks, the task computing force demand distribution table comprises task node computing force demand parameters, resource node availability, task priority level weighting values, task path matching values and resource load conditions, the initial scheduling resource distribution scheme comprises task and resource node distribution relations, task priority ordering, demand distribution matching values and resource node idle conditions, and the task dynamic scheduling and optimizing results comprise task node execution states, resource node idle weights, unassigned task computing force demands, resource residual capacity and task execution feedback.
As a further aspect of the present invention, the task structure parsing module includes:
the task information extraction sub-module analyzes execution time, priority, dependency strength, CPU occupation, memory demand and bandwidth distribution according to task description information, classifies parameters according to task priority and sorts, marks attribute relations among tasks, and generates a task basic attribute set;
The resource distribution matching submodule matches CPU, memory and bandwidth parameters in the resource nodes based on the task basic attribute set, extracts idle state nodes in the resource distribution, distributes resource node parameters according to task priorities, adjusts distribution sequences, integrates task priorities and resource distribution conditions, and generates a resource matching result;
And the task resource association modeling submodule sorts the dependency paths between the tasks and the resources based on the resource matching result, extracts the associated resource node distribution in the task dependency relationship, distributes weights according to the dependency strength and integrates the weights into model data to obtain a task and resource association network model.
As a further aspect of the present invention, the dependency evaluation module includes:
The weight acquisition submodule extracts the connection weight between task nodes based on the task and resource association network model, invokes weight data between nodes in the network model, analyzes weight distribution in a connection path, classifies the weight data and performs statistics to generate a node connection weight data table;
The path dependence quantization submodule extracts the accumulated weight of each task node in the path based on the node connection weight data table, analyzes the direct connection strength between the nodes, identifies the dependence strength, and counts the weight difference between the nodes to obtain a dependence strength quantization result;
And the interaction influence analysis submodule analyzes priority relation data among task nodes based on the dependency intensity quantization result, calculates the optimization weight of the interaction path, analyzes the interaction weight change among the nodes, adjusts the interaction influence path relation and obtains a task and resource interaction intensity data set.
As a further aspect of the present invention, the optimization weight of the interaction path is calculated according to the formula:
;
Wherein,Representing the optimized weights of the interaction path,A weight value representing each of the original interaction paths,Representing the dynamic adjustment coefficients between the paths,Representing the associated complexity between nodes in the path,Representing the priority factor of the task node,Regularization parameters representing adjustment coefficients in the path.
As a further aspect of the present invention, the calculation force demand quantization module includes:
The calculation force parameter capturing submodule extracts calculation force demand parameters of task nodes based on the task and resource interaction intensity data set, analyzes resource load records, positions the task nodes, classifies calculation demand data and generates a task node calculation force demand data table;
The resource matching weighting submodule analyzes the availability parameters of the resource nodes based on the task node calculation force demand data table, matches the calculation demands of the tasks and the resource nodes, adjusts the resource allocation amount and obtains a task resource matching weight data table;
And the task computing power distribution submodule extracts a resource distribution weight value based on the task resource matching weight data table, adjusts task computing power distribution and resource distribution and generates a task computing power demand distribution table.
As a further aspect of the present invention, the resource allocation module includes:
the distribution relation identification submodule extracts the calculation force demand data and distribution parameters of the task nodes and the resource nodes based on the task calculation force demand distribution table, analyzes the association information, matches the calculation force distribution relation and generates a task resource distribution relation table;
The allocation sequence adjusting submodule extracts task priority data based on the task resource allocation relation table, analyzes load conflict between tasks and resource nodes, sorts task allocation sequence, adjusts resource node mapping relation and generates a task allocation priority sequence table;
And the initial resource allocation submodule analyzes the allocation sequence and the resource node load parameter in the task path based on the task allocation priority list, calculates the adjusted initial allocation quantity of the resource nodes and establishes an initial scheduling resource allocation scheme.
As a further scheme of the present invention, the adjusted initial allocation amount of the resource node is calculated according to the formula:
;
Wherein,Representing the adjusted initial allocation amount of the resource node,Representative taskIs used for the weight coefficient of the (c),Representative taskIs set according to the priority of (1),Representing resource nodesIs used for the current load of the (c),Representing the load adjustment factor and,Representing the conventional adjustment coefficient of the device,Representing the total number of resource nodes.
As a further aspect of the present invention, the scheduling dynamic adjustment module includes:
The execution state monitoring submodule extracts the execution state of the task node and the idle weight of the resource node based on the initial scheduling resource allocation scheme, monitors the execution time and the completion percentage, records the dynamic association data of the task and the resource node and generates a task and resource state monitoring table;
The intelligent task resource matching sub-module extracts the computational power requirement of an unallocated task and the residual capacity of a resource node based on the task and resource state monitoring table, compares the computational power requirement of the task node with the computational capacity of the resource node, updates the task allocation order, and generates a task and resource dynamic matching sequence table;
And the intelligent dynamic scheduling sub-module analyzes the dynamic allocation data of the resources based on the dynamic matching sequence table of the tasks and the resources, adjusts the allocation weight and the computing capacity distribution of the resource nodes and generates a dynamic scheduling and scheduling result of the tasks.
On the other hand, the intelligent scheduling method based on the calculation force demand prediction is executed based on the intelligent scheduling system based on the calculation force demand prediction, and comprises the following steps:
S1, extracting task node execution time length, priority, dependency strength, CPU occupation, memory requirement and bandwidth distribution according to task description information, analyzing the proportion of the execution time length to the dependency strength, normalizing the CPU occupation rate and the memory requirement, matching the relation between the bandwidth distribution and the priority, arranging a priority resource matching sequence, and generating an initial matching weight table of the task node and the resource node;
S2, extracting direct dependency intensity value and path length data of task nodes, screening and sequencing connection intensity values of path task nodes, inducing path dependency distribution and resource occupation relation, analyzing the proportion difference between load parameters of the resource nodes and path requirements, and generating a task and resource distribution balance data set based on the initial matching weight table of the task nodes and the resource nodes;
s3, extracting calculation force demand parameters of task nodes, CPU occupancy rate of the resource nodes and memory bearing values based on the task and resource distribution balanced data set, carrying out multidimensional comparison of calculation force demands and resource occupancy, classifying idle state values of the resource nodes and accumulated matching parameters of priorities of the task nodes, screening path task node calculation force demand distribution adaptation relation, and generating a task calculation force demand and resource node distribution adaptation table;
S4, analyzing task node priority parameters and resource node idle distribution parameters based on the task computing power demand and resource node distribution adaptation table, adjusting task node resource priority distribution sequences, reconstructing resource node load distribution weight and resource matching paths, screening task node resource load conditions in the paths, and constructing an initial task path and resource distribution scheme;
And S5, screening calculation force demand parameters of unassigned task nodes and idle state values of the task nodes based on the initial task paths and the resource allocation scheme, adjusting the resource priority sequence in the task node paths, re-analyzing the idle time slot distribution of the resource nodes and the matching state of the task node loads, and generating a task dynamic scheduling and optimizing result.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
By accurately modeling and analyzing the task structure, the dependency relationship, the calculation force demand and the resource allocation, the method realizes the refinement and dynamic optimization of the resource scheduling, factors such as the task execution time, the priority, the resource demand and the like are integrated into a multidimensional association network between task nodes, so that a comprehensive and fine task resource interaction model is formed, the interaction model not only can quantify the dependency strength among tasks, but also can reasonably evaluate the interaction influence among the tasks according to the priority relationship, the intellectualization and the accuracy of the resource allocation are enhanced, the calculation force resource can reach the optimal proportion with the task demand through the weighted analysis and the matching algorithm, the real-time dynamic scheduling process between the tasks and the resources can be flexibly adjusted according to the task execution progress and the idle condition of the resource nodes, the resource conflict and waste are effectively reduced, the resource utilization rate is improved, the change of the task demand and the resource state can be responded quickly, the resource allocation is more flexible and intelligent, the problems of resource idle and scheduling lag are solved, the utilization rate of the calculation resource is effectively improved, the efficiency of the task scheduling is optimized, the efficiency is improved, the allocation efficiency of the task scheduling is further improved, the overall operation efficiency is improved under the concurrent environment of the multi-task allocation, and the allocation system is reasonably used, and the stability is guaranteed.
Detailed Description
The technical scheme of the invention is described below with reference to the accompanying drawings.
In embodiments of the invention, words such as "exemplary," "such as" and the like are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the term use of an example is intended to present concepts in a concrete fashion. Furthermore, in embodiments of the present invention, the meaning of "and/or" may be that of both, or may be that of either, optionally one of both.
In the embodiments of the present invention, "image" and "picture" may be sometimes used in combination, and it should be noted that the meaning of the expression is consistent when the distinction is not emphasized. "of", "corresponding (corresponding, relevant)" and "corresponding (corresponding)" are sometimes used in combination, and it should be noted that the meaning of the expression is consistent when the distinction is not emphasized.
In embodiments of the present invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1, and the meaning of the expression is consistent when de-emphasizing the distinction.
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
The embodiment of the invention provides an intelligent scheduling system based on calculation force demand prediction, as shown in a schematic diagram of the intelligent scheduling system based on calculation force demand prediction in fig. 1-2, the system comprises:
The task structure analysis module extracts execution time length, priority, dependency strength, CPU occupation, memory requirement and bandwidth distribution data according to the task description information to obtain task nodes and resource nodes, performs weight calculation on matching of task and resource attributes, and sorts task dependency paths and resource distribution conditions to obtain a task and resource association network model;
The dependency relationship assessment module is used for extracting connection weight values among task nodes and identifying dependency path lengths based on a task and resource association network model, quantifying direct and indirect dependency strengths, assessing interaction influence among the nodes through priority relationships among the tasks, and inducing interaction weight to obtain a task and resource interaction strength data set;
The calculation force demand quantification module extracts calculation force demand parameters of task nodes based on a task and resource interaction intensity data set, performs hierarchical weighted analysis by combining the availability of the resource nodes and the priority of the tasks, identifies matching values of the demands and the resource allocation capacity of the task nodes, and obtains a task calculation force demand allocation table by combining task paths and resource load conditions;
The resource allocation module is used for identifying the allocation relation between the tasks and the resource nodes according to the task priority and the demand allocation matching value ordering based on the task computing power demand allocation table, adjusting the allocation sequence according to the idle condition of the resource nodes and constructing an initial scheduling resource allocation scheme;
The scheduling dynamic adjustment module monitors the execution state of task nodes and the idle weight of the resource nodes based on an initial scheduling resource allocation scheme, compares the calculation power requirement and the resource remaining capacity of unassigned tasks in real time, fills idle time slots by adjusting the matching sequence of the tasks and the resource nodes, and combines task execution feedback to generate a task dynamic scheduling optimization result.
The task and resource association network model comprises task nodes, resource nodes, task and resource attribute matching weights, task dependency paths and resource distribution conditions, the task and resource interaction strength data set comprises connection weights among task nodes, dependency path lengths, direct dependency strengths, indirect dependency strengths and interaction weights among tasks, the task computing power demand distribution table comprises task node computing power demand parameters, resource node availability, task priority level weighting values, task path matching values and resource load conditions, the initial scheduling resource distribution scheme comprises task and resource node distribution relationships, task priority ordering, demand distribution matching values and resource node idle conditions, and the task dynamic scheduling and optimization results comprise task node execution states, resource node idle weights, unassigned task computing power demands, resource surplus capacity and task execution feedback.
Specifically, as shown in fig. 2 and 3, the task structure analysis module includes:
the task information extraction sub-module analyzes execution time, priority, dependency strength, CPU occupation, memory demand and bandwidth distribution according to task description information, classifies parameters according to task priority and sorts, marks attribute relations among tasks, and generates a task basic attribute set;
Firstly, calculating execution time length, priority, dependency strength, CPU occupation, memory requirement and bandwidth distribution of each task, which are basic data of task scheduling, wherein the specific operation comprises the steps of extracting key attributes from task description by using a data analysis technology, determining the level of the priority through text analysis, analyzing the execution time length according to the complexity and the emergency degree of task content, quantitatively analyzing the attributes to obtain the attributes, sequencing the attributes according to the priority, ensuring that the tasks can be executed according to the emergency degree and the importance sequence by using a customized sequencing algorithm, classifying each basic attribute of the tasks, marking the relationship among the tasks, such as the dependency strength as relative numerical weight, forming a task basic attribute set which reflects the sequence relationship among the tasks and the resource requirement degree, and providing basis for resource allocation.
The resource distribution matching submodule is used for matching CPU, memory and bandwidth parameters in the resource nodes based on the task basic attribute set, extracting idle state nodes in the resource distribution, distributing the resource node parameters according to task priorities, adjusting the distribution sequence, integrating the task priorities and the resource distribution conditions and generating a resource matching result;
The method comprises the steps of combining the real-time state of current resource nodes, carrying out node-by-node matching, sequentially matching idle nodes according to the priority of each task in execution, filtering CPU resources meeting requirements and recording allocation states through calculating the real-time idle rate of the nodes, simultaneously defining available memory intervals according to the memory requirements in task description and the residual distribution of memory capacity in actual nodes, optimizing the occupation ratio of the nodes on the premise of ensuring the task resource requirements, gradually adjusting the bandwidth allocation sequence according to the data flow related to the task input and output paths and the current throughput rate in node network distribution in the bandwidth parameter matching process, reserving data transmission bandwidth for high-priority tasks preferentially to form a resource allocation priority sequence, adjusting the corresponding relation between the task priority and node resource allocation after integration, eliminating resource nodes which cannot be matched, and finally outputting a complete resource matching result.
The task resource association modeling sub-module sorts the dependency paths between the tasks and the resources based on the resource matching result, extracts the associated resource node distribution in the task dependency relationship, distributes weight according to the dependency strength and integrates the weight into model data to obtain a task and resource association network model;
And extracting strongly dependent resource nodes in the associated paths by arranging the corresponding relation between task distribution and resource occupation, for example, if a certain task needs to be cooperatively completed by depending on a plurality of resource nodes, dividing weights for the associated nodes according to the dependency strength, marking the resource importance weights of each task path by extracting the dependency relation item by item in the task description, aggregating the strongly dependent nodes into core paths by combining the dependency strength distribution weights in the resource node distribution integration process, marking weak dependent nodes as redundant paths, carrying out splitting operation on the nodes with the excessively high dependency weights according to the distribution balance of the resource nodes, avoiding the problem that the execution efficiency of the task is reduced due to the fact that the resources are concentrated in a single path, generating an associated network model of the task and the resource after the model integration is completed, analyzing the resource conflict among the tasks by the graph structure of the model, and further providing basis for optimizing intelligent scheduling, so that the efficient matching and real-time distribution of the task and the resource are realized.
Specifically, as shown in fig. 2 and 4, the dependency evaluation module includes:
The weight acquisition submodule extracts connection weights among task nodes based on the task and resource association network model, invokes weight data among nodes in the network model, analyzes weight distribution in a connection path, classifies the weight data and performs statistics to generate a node connection weight data table;
Extracting connection weights among task nodes one by one from a model, analyzing all node connection paths in an associated network model, recording weight distribution conditions on each path, classifying the weights according to direct association strength and resource allocation proportion among the task nodes, archiving the path connection weights according to task priorities in the classifying process to form a high-priority weight set, a medium-priority weight set and a low-priority weight set, calculating weight density of each connection path according to accumulation and frequency distribution of weight distribution when counting weight data, checking resource occupation amount and path weight item by item according to actual distribution proportion of paths to ensure accuracy of a counting result, removing invalid path data according to actual resource use condition of the paths, and finally generating a node connection weight data table to provide accurate numerical support for follow-up path dependence and interaction analysis.
The path dependence quantization sub-module extracts the accumulated weight of each task node in the path based on the node connection weight data table, analyzes the direct connection strength between the nodes, identifies the dependence strength, and counts the weight difference between the nodes to obtain a dependence strength quantization result;
Sequentially analyzing the weight accumulation condition among task nodes, extracting the weight distribution of each path in execution, calculating the accumulated weight according to the resource allocation proportion of the nodes to form an accumulated weight sequence of the nodes in the path, recording the weight change rate on the path one by one in the process of analyzing the direct connection strength among the nodes, marking the part with obvious weight fluctuation in the path through the weight change trend of the nodes, calculating the weight difference of each path on the basis of the dependency strength quantization, generating a path dependency strength matrix according to the weight difference, wherein the path dependency strength matrix comprises inter-node dependency strength and connection path weight change data, matching the task priority with the path accumulated weight when counting the dependency weights among the nodes, defining a critical path and a secondary path, combining the resource consumption proportion of the path distribution, generating a dependency strength quantization result, and providing an accurate quantization basis for interaction influence analysis.
The interaction influence analysis submodule analyzes priority relation data among task nodes based on the dependency intensity quantization result, calculates optimization weight of interaction paths, analyzes interaction weight change among nodes, adjusts interaction influence path relation, and obtains a task and resource interaction intensity data set;
Calculating the optimization weight of the interaction path according to the formula:
;
Wherein,Representing the optimized weights of the interaction path,A weight value representing each of the original interaction paths,Representing the dynamic adjustment coefficients between the paths,Representing the associated complexity between nodes in the path,Representing the priority factor of the task node,Regularization parameters representing adjustment coefficients in the path;
Formula details and formula calculation derivation process:
The formula is used for calculating the optimized weight of the interaction path between task nodesThe obtained result is used for evaluating and adjusting the interaction strength between the task and the resource;
The weight value of each path in the original interaction path is used for quantifying the dependency relationship strength among the task nodes, and the dependency relationship with medium strength among the nodes is reflected by monitoring the communication frequency and data transmission capacity among the task nodes and obtaining the data transmission capacity, wherein the set value is 0.75;
The dynamic adjustment coefficient between the paths is represented, adjustment is carried out according to the interactive variation weight between the paths, the interactive variation condition between task nodes is monitored in real time, the set value is 1.2, and the interactive intensity of the current path is increased;
The association complexity among the nodes in the path is shown, the association complexity is used for quantifying the coupling degree of tasks among the nodes, the direct and indirect dependency quantity among the nodes is calculated by analyzing the dependency graph among the tasks, the set value is 3, the multiple dependency relationship among the nodes is reflected, and the coupling degree is higher;
the priority factor of the task node is represented, the difference of the priorities among the task nodes is reflected, the priority parameter set in the task scheduling strategy is used for obtaining, the set value is 2, and the priority of the task node is higher;
Regularization parameters representing adjustment coefficients in paths are used for balancing adjustment amplitude of weights among paths, and through empirical parameter setting, influence of excessive deviation of the path weights to single node adjustment on overall optimality of a result is avoided, and the set value is 0.5;
substituting the parameters into a formula to calculate:
;
resultsThe method shows that in the interaction path between the current task nodes, the dynamically adjusted optimization weight is about 0.245, the result is used for evaluating the interaction strength between the task and the resource, the higher the numerical value is, the greater the interaction strength is, the task scheduling strategy can be further adjusted through the optimization weight, the resource allocation is optimized, and the overall performance of the system is improved.
Specifically, as shown in fig. 2 and 5, the calculation force demand quantization module includes:
the calculation force parameter capturing submodule extracts calculation force demand parameters of task nodes based on a task and resource interaction intensity data set, analyzes resource load records, positions the task nodes, classifies calculation demand data and generates a task node calculation force demand data table;
Extracting calculation force demand parameters from task nodes one by one, analyzing the interaction intensity of each task node, positioning the calculation force demand condition of the current node according to the historical resource call record of the task, extracting the resource consumption distribution of the task, such as CPU occupation time length, memory demand quantity, network bandwidth throughput and the like, classifying the calculation demand data of different tasks, counting the computation intensive task and the data intensive task separately during classification so as to mark the different priority demands of the task on the calculation force, dividing the task into a high calculation force demand node and a low calculation force demand node according to the classification result of the calculation force demand, generating a calculation force demand data table of the task node by combining the priority of the task and the resource utilization efficiency, recording the calculation force demand specific value of each task node in the data table, marking the corresponding relation with the resource distribution, and providing an explicit calculation force demand basis for subsequent resource matching.
The resource matching weighting submodule analyzes the availability parameters of the resource nodes based on the task node calculation force demand data table, matches the calculation demands of the tasks and the resource nodes, adjusts the resource allocation amount and obtains the task resource matching weight data table;
Checking idle states of each resource node one by one, combining real-time parameters such as CPU core number, memory capacity, bandwidth utilization rate and the like, confirming availability conditions of the nodes, carrying out hierarchical matching on the resource nodes according to task computing power requirements, distributing tasks with high computing power requirements to the resource nodes with higher performance preferentially, distributing tasks with low computing power to the nodes with more available resources, gradually adjusting distribution proportion according to the utilization rate of the resource nodes in the distribution process, ensuring load balance of node resources, obtaining a task resource matching weight data table, recording weight values between each task node and the resource nodes, representing distributed priority and computing power requirement adaptation degree by the weight values, providing basis for subsequent computing power distribution, rejecting redundant paths of resource distribution, and ensuring efficient and accurate resource distribution of all tasks.
The task computing power distribution submodule extracts a resource distribution weight value based on a task resource matching weight data table, adjusts task computing power distribution and resource distribution, and generates a task computing power demand distribution table;
The method comprises the steps of adjusting the calculation power distribution and the resource node distribution of a task, analyzing the suitability of the calculation power demand and the resource node of the task one by one in execution, preferentially distributing the task with higher weight value to the node with optimal availability, simultaneously avoiding centralized distribution of the high calculation power task, preventing the resource node from reducing the dispatching efficiency of the whole system due to the overload of the calculation power, optimizing the calculation power resource configuration of the task through redistribution of the weight value, recording the calculation power resource data of the task distribution, such as the CPU time, the memory capacity and the bandwidth resource proportion distributed by each node, marking the node position of the task execution, and dynamically adjusting an allocation table to adapt to the change of the real-time task and the improvement of the resource utilization efficiency, thereby ensuring the dynamic balance of the task and the resource and improving the whole calculation performance of intelligent dispatching.
Specifically, as shown in fig. 2 and 6, the resource allocation module includes:
The distribution relation identification submodule extracts the calculation force demand data and distribution parameters of the task nodes and the resource nodes based on the task calculation force demand distribution table, analyzes the association information, matches the calculation force distribution relation and generates a task resource distribution relation table;
Analyzing each task node and the corresponding resource node, recording the relation between the calculation force demand and the actual allocation quantity, analyzing the association information of the task and the resources, marking the node with the calculation force demand exceeding the resource allocation quantity, classifying and archiving the condition of unbalanced allocation, mapping the allocation priority of the task according to the actual calculation force utilization rate of the resource node when the calculation force allocation relation is matched, preferentially matching the task node with the calculation force demand to the node with lower resource utilization rate, avoiding redundant waste of the node resources, generating a task resource allocation relation table aiming at the allocated completed task resource mapping relation, recording the actual resource occupation data of each task node, including CPU allocation time period, memory use quantity, bandwidth consumption proportion and the like, marking the direct connection strength of the task node and the resource node, and providing accurate allocation basis for the subsequent task sequence adjustment.
The allocation sequence adjusting submodule extracts task priority data based on a task resource allocation relation table, analyzes load conflict between tasks and resource nodes, sorts task allocation sequences, adjusts resource node mapping relation and generates a task allocation priority sequence table;
By sequencing the priorities of the tasks, the load conflict between the tasks and the resource nodes is analyzed item by item, for example, when the allocation resources of a certain high-priority task are occupied by a low-priority task, the resource nodes are released preferentially for the high-priority task, the efficient utilization of the resources is realized by adjusting the allocation sequence of the tasks, when the allocation sequence of the tasks is sequenced, a conflict matrix of the task resources is generated according to the task resource demand and the resource node load rate, a task path with larger conflict is marked as a critical path needing to be adjusted, when the resource node mapping relation is adjusted, the task with lower priority is adjusted to the node with lower load or the allocation sequence is moved backwards, the resource allocation stability of the high-priority task is ensured, finally, a task allocation priority list is generated, the adjusted task execution sequence and the resource node occupation condition are recorded, and a scheduling strategy for optimizing the sequence is provided for intelligent scheduling.
The initial resource allocation submodule analyzes the allocation sequence and the resource node load parameter in the task path based on the task allocation priority list, calculates the initial allocation quantity of the adjusted resource nodes and establishes an initial scheduling resource allocation scheme;
Calculating the initial allocation amount of the adjusted resource nodes according to the formula:
;
Wherein,Representing the adjusted initial allocation amount of the resource node,Representative taskIs used for the weight coefficient of the (c),Representative taskIs set according to the priority of (1),Representing resource nodesIs used for the current load of the (c),Representing the load adjustment factor and,Representing the conventional adjustment coefficient of the device,Representing the total number of resource nodes;
Formula details and formula calculation derivation process:
The formula is used to calculate the initial allocation amount of the resource nodeThe purpose being task-based priorityLoad with resource nodesCalculating the amount of resources to be allocated to each resource node, and accurately calculating the allocation amount of resources by weighing the task priority sequence, the resource load and the adjustment coefficient by each parameter in the formula through a hierarchical analysis method;
task ofThe weight coefficient of a task, which represents the importance of the task in overall scheduling, is obtained by comprehensively evaluating the complexity, required resources and time requirements of the task, is quantified usually by using an Analytic Hierarchy Process (AHP), and is setThe complexity of (a) is estimated to be 0.7, the required resource is estimated to be 0.6, the task time requirement is estimated to be 0.5, and the weight coefficient is calculated;
Task ofThe priority of the task is determined by the deadline of the task, the dependency relationship of the task and the resource demand period, and the task is setThe result of the priority calculation of (a) is 8 (full 10 points), then,
Resource nodeThe load of the node is represented by the current load degree of the node, which is generally the ratio of the number of tasks running on the node to the maximum bearing capacity of the node, and the resource node is setThe current load of (a) is 0.75 (75% load), then;
The load adjustment coefficient is used for adjusting the influence of the load on the distribution amount, is set based on the evaluation of the actual demand and the load influence, is set to be 0.9, and indicates that the influence degree of the load on the final distribution amount is higher, and is set to be 0.9;
conventional adjustment coefficients for normalizing the load difference of each resource node to avoid excessive deviation are generally determined based on the difference degree of the load of the resource node when setting the coefficients, and are setRepresenting moderately balancing the load difference of the resource nodes;
The total number of resource nodes, which represents the number of resource nodes involved in the scheduling scheme, is set to a total of 3 resource nodes, i;
Substituting the parameters into a formula to calculate:
It is known that:,,,,,,,,,,,;
First, the molecular fraction is calculated:
;
;
;
Then, the denominator part is calculated:
;
;
finally, substituting the complete formula to calculate:
;
The result shows that the initial allocation amount of the resource node is 8.57 units, which means that the node needs to allocate 8.57 units of resources according to the comprehensive evaluation of the task priority and the load of the resource node, and the value integrates the emergency degree of the task, the load condition of the resource node and other regulating factors, so that the reasonable allocation of the resources among a plurality of nodes can be ensured.
Specifically, as shown in fig. 2 and 7, the scheduling dynamic adjustment module includes:
The execution state monitoring submodule extracts the execution state of the task node and the idle weight of the resource node based on the initial scheduling resource allocation scheme, monitors the execution time and the completion percentage, records the dynamic association data of the task and the resource node, and generates a task and resource state monitoring table;
Recording the execution states of task nodes one by one, including the execution time, the real-time completion percentage and the current resource consumption condition of the tasks, marking the execution progress of each task by collecting the calculation load of the nodes in the task execution process at fixed time, updating the change trend of the idle weight gradually according to the calculation capacity and the load level of the current resource nodes when the idle weight of the resource nodes is monitored, analyzing the dynamic idle proportion of the resource nodes at different moments, associating the real-time execution state of the tasks with the load change of the resource nodes in the dynamic associated data recording process of the tasks and the resource nodes, generating a comprehensive data table including the task execution time, the completion percentage, the resource consumption and the node idle state, integrating the monitoring results to generate a task and resource state monitoring table, defining the dynamic progress of the task nodes and the real-time distribution condition of the resource nodes, and providing basic data support for the follow-up scheduling optimization.
The intelligent task resource matching submodule extracts the computational power requirement of an unallocated task and the residual capacity of a resource node based on the task and resource state monitoring table, compares the computational power requirement of the task node with the computational capacity of the resource node, updates the task allocation order, and generates a task and resource dynamic matching order table;
The calculation power of the node with the lowest load and highest availability of the resource nodes is screened by comparing the calculation power demand of the node without the task allocation with the remaining calculation power of the resource nodes, in the adjustment process, the task with the high calculation power demand is preferentially allocated to the node with the stronger calculation power and higher idle weight, the task with the low calculation power demand is properly delayed to be allocated or allocated to the redundant resource part of the node, the priority change condition of the task nodes and the actual allocation state of the corresponding resource nodes are recorded one by one, the adjusted priority allocation path is marked, the allocation order is dynamically adjusted according to the execution progress of the task and the load of the resource nodes, the distribution of the calculation power demand of the task and the remaining capacity of the resource are matched one by one, so that the real-time sequence of the task allocation is optimized, and the accuracy and the execution efficiency of calculation power scheduling are improved.
The intelligent dynamic scheduling sub-module analyzes the dynamic allocation data of the resources based on the dynamic matching sequence table of the tasks and the resources, adjusts the allocation weight and the computing power distribution of the resource nodes, and generates a dynamic scheduling result of the tasks;
The method is characterized in that the current computing capacity distribution of the resource nodes and the computing power demand of the task nodes are combined, the computing capacity of the resource nodes is redistributed, the distribution weight of each node is dynamically adjusted, for example, for the task nodes with suddenly increased computing power demand, high-performance resource nodes are timely distributed to ensure the smooth completion of the task computing process, meanwhile, the resource distribution proportion of the low-priority task nodes is adapted to be adjusted downwards so as to release more computing power, in the resource distribution weight adjustment process, the real-time load change of the nodes and the task demand trend are subjected to correlation analysis, the computing power distribution balance among the nodes is optimized, the occurrence of resource overload or idle condition is prevented, and the distribution optimization condition of task paths, the dynamic tuning strategy of the resource nodes and the final computing power occupation proportion of the node distribution are recorded one by one when the task dynamic scheduling tuning result is generated, so that the intelligent and efficient tuning result is ensured.
Referring to fig. 8, the intelligent scheduling method based on the calculation force demand prediction is executed based on the intelligent scheduling system based on the calculation force demand prediction, and includes the following steps:
S1, extracting task node execution time length, priority, dependency strength, CPU occupation, memory requirement and bandwidth distribution according to task description information, analyzing the proportion of the execution time length to the dependency strength, normalizing the CPU occupation rate and the memory requirement, matching the relation between the bandwidth distribution and the priority, arranging a priority resource matching sequence, and generating an initial matching weight table of the task node and the resource node;
S2, extracting direct dependence intensity value and path length data of task nodes, screening and sequencing connection intensity values of path task nodes, inducing path dependence distribution and resource occupation relation, analyzing the proportion difference between load parameters of the resource nodes and path requirements, and generating a task and resource distribution balance data set;
S3, extracting calculation force demand parameters of task nodes, CPU occupancy rate of the resource nodes and memory bearing values based on a task and resource distribution balanced data set, carrying out multidimensional comparison of calculation force demands and resource occupancy, classifying idle state values of the resource nodes and accumulated matching parameters of priorities of the task nodes, screening path task node calculation force demand distribution adaptation relation, and generating a task calculation force demand and resource node distribution adaptation table;
S4, analyzing task node priority parameters and resource node idle distribution parameters based on task computing power requirements and a resource node distribution adaptation table, adjusting task node resource priority distribution sequences, reconstructing resource node load distribution weight values and resource matching paths, screening task node resource load conditions in the paths, and constructing an initial task path and a resource distribution scheme;
And S5, screening calculation force demand parameters of unassigned task nodes and idle state values of the task nodes based on the initial task path and the resource allocation scheme, adjusting the resource priority sequence in the task node path, re-analyzing the idle time slot distribution of the resource nodes and the load matching state of the task nodes, and generating a task dynamic scheduling and optimizing result.
The foregoing is merely illustrative embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think about variations or substitutions within the technical scope of the present invention, and the invention should be covered. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.