Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The flow diagrams depicted in the figures are merely illustrative and not necessarily all of the elements and operations/steps are included or performed in the order described. For example, some operations/steps may be further divided, combined, or partially combined, so that the order of actual execution may be changed according to actual situations.
It is also to be understood that the terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1, fig. 1 is a schematic flowchart of a resource management method of a server operating system according to an embodiment of the present application. The method comprises the specific steps of S101-S106.
S101, performing double-layer graph modeling on processes and resource data in a server operating system to obtain a process competition graph layer and a resource dependency graph layer.
Illustratively, the dual-layer graph modeling includes a first level and a second level. In the first level, each process in the server operating system is taken as a node, and the resource competition relationship among the processes is taken as an edge, so that a process competition graph is formed. In the second level, various resources (such as CPU, memory, I/O and the like) are taken as nodes, and the dependency relationship among the resources is taken as an edge, so that a resource dependency graph is formed. The two levels are connected through a resource allocation relation, namely if a certain process is using or requesting a certain resource, a connection edge is established between the corresponding process node and the resource node. And giving process control block information to nodes in the process competition layer, giving resource attribute information to the nodes in the resource dependence layer, and giving resource usage information to interlayer connection edges. The double-layer graph structure can express the competition relationship between processes and the dependency relationship between resources, and provides more comprehensive system state information for subsequent resource allocation decisions. For each process node, the information of a Process Control Block (PCB) is recorded, including a process ID, a priority, a current state (ready, running, blocking) and the like, for each resource node, the attribute information of the type, the total amount, the allocated amount and the like is recorded, and for the connecting side, the weight information of the resource usage amount or the request amount and the like is recorded. Through the double-layer graph model, the limitation that the traditional resource graph only focuses on a single-layer relationship is avoided, and the multi-dimensional relationship in the system can be simultaneously depicted.
S102, utilizing a preset graph neural network to perform feature extraction on node information of a process competition layer and node information of a resource dependency layer to obtain node feature vectors containing competition relations and distribution states.
For each node in the process competition layer, the information of the adjacent nodes is aggregated, and meanwhile, an attention mechanism is introduced to distinguish the importance of different adjacent nodes, namely, the important adjacent nodes acquire higher weight in the process of information aggregation, the neighbor information acquired by aggregation is fused with the characteristic information of the node, and the new characteristic representation of the node is acquired through nonlinear transformation. In this process, special feature extraction modules are designed for the process layer and the resource layer respectively to process the features of different types of nodes. In the information transfer between two layers, a cross-layer attention mechanism is designed so that a server operating system can adaptively learn the association strength between a process and a resource. After multi-layer feature extraction, the feature vector of each node contains rich context information, which reflects local competition relationship and global resource allocation state.
And S103, carrying out deadlock probability calculation through the multi-layer perceptron according to the node feature vector to obtain a system deadlock probability value and a process priority adjustment scheme.
Illustratively, the features are input into a multi-layer perceptron that is made up of multiple fully connected layers for learning the nonlinear relationship between the features, the output of the multi-layer perceptron being a probability value that indicates the likelihood of a deadlock occurring in the system. When the predicted deadlock probability exceeds a preset threshold, triggering a priority dynamic adjustment mechanism, wherein for each process participating in resource competition, a comprehensive score is calculated based on the current occupied number of resources, the type of the requested resources, the running time and other factors. Processes with lower scores will be de-prioritized and may be suspended temporarily if necessary. Meanwhile, in order to avoid that the process cannot be executed for a long time, an aging mechanism is introduced, that is, the process with the reduced priority gradually restores its priority over time. Compared with the traditional passive deadlock processing method, the active adjustment mechanism based on prediction can discover potential deadlock risks earlier and take preventive measures.
And S104, performing resource allocation strategy learning according to the node feature vector and the deadlock probability value to obtain a first resource allocation scheme.
Illustratively, a resource allocation policy generator is designed based on the feature extraction result of S102 and the deadlock prediction result of S103. The generator employs a reinforcement learning framework to model the resource allocation problem as a sequential decision process. Specifically, the system state is represented by the characteristics extracted by the GNN, the action space is all possible resource allocation schemes, and the reward function comprehensively considers factors such as system throughput, resource utilization rate, deadlock risk and the like. By interacting with the environment, the policy network learns the optimal resource allocation policy step by step. In order to improve the reliability of the decision, a Monte Carlo Tree Search (MCTS) method is introduced, and before making a distribution decision, a scheme with the highest expected benefit is selected by simulating a plurality of possible distribution sequences and the results thereof. The method for combining reinforcement learning and tree searching can obviously reduce the computational complexity while guaranteeing the decision quality.
S105, carrying out multi-level queue allocation on the process according to the first resource allocation scheme to obtain a second resource allocation scheme.
The adaptive scheduling queue is constructed based on the first resource allocation scheme generated in the fourth step, a multi-stage adaptive scheduling queue system is constructed, and finally a second resource allocation scheme is obtained. Specifically, the processes are dynamically allocated to different scheduling queues according to the priority and resource demand characteristics of the processes. Each queue in the first resource allocation scheme adopts different scheduling strategies and time slice lengths, the higher priority queue allocates shorter time slices to ensure response speed, and the lower priority queue allocates longer time slices to improve throughput. Specifically, a queue level adaptive mechanism is introduced, performance indexes (such as average waiting time, completion time, etc.) of each queue are periodically evaluated, and scheduling parameters of the queues are dynamically adjusted according to the evaluation results. For example, when the average waiting time of a certain queue is too long, the time slice length of the queue is appropriately increased or the scheduling priority thereof is increased. The self-adaptive mechanism can automatically adjust the scheduling strategy according to the system load condition, and the overall balance of the system is maintained.
S106, performing resource management on the server according to the second resource allocation scheme.
The method comprises the steps of carrying out resource allocation and scheduling management on scheduling queues of all levels, setting a queue rotation time slice and a preemption threshold, and monitoring and counting the running states of processes in the queues, wherein the indexes comprise monitoring the use condition of resources, response time and the like. And dynamically adjusting the scheduling strategy according to the monitoring data, such as adjusting the time slice size, priority weight and the like, executing the process migration across the queues, and realizing load balancing. And carrying out deadlock detection and processing according to the resource occupation condition. And constructing a feedback control mechanism, and optimizing the management strategy according to the system performance evaluation index.
The embodiment of the application provides a resource management method of a server operating system, which comprises the steps of carrying out double-layer graph modeling on processes and resource data in the server operating system to obtain a process competition layer and a resource dependence layer, carrying out feature extraction on node information of the process competition layer and node information of the resource dependence layer by utilizing a preset graph neural network to obtain node feature vectors containing competition relations and allocation states, carrying out deadlock probability calculation according to the node feature vectors to obtain a process priority adjustment scheme by a multi-layer perceptron, carrying out resource allocation strategy learning according to the node feature vectors and the process priority adjustment scheme to obtain a first resource allocation scheme, carrying out multi-level queue allocation on the processes according to the first resource allocation scheme to obtain a second resource allocation scheme, and carrying out resource management on the server according to the second resource allocation scheme. In the method, the process competition relationship and the resource dependency relationship are represented in a layered manner through double-layer graph modeling, a process competition graph layer and a resource dependency graph layer are constructed to comprehensively capture competition states and resource allocation states among concurrent processes, the node information in a double-layer graph structure is subjected to feature extraction by adopting a graph neural network, the complex competition relationships and allocation states are encoded into feature vectors, deadlock probability calculation is carried out on the node feature vectors based on a multi-layer perceptron, quantitative evaluation and prevention of the deadlock risk of a system are realized by combining a process priority adjustment scheme, a resource allocation strategy is automatically adjusted according to the node feature vectors and the deadlock probability value through a resource allocation strategy learning mechanism, the resource allocation decision is more flexible and accurate, a multi-level queue allocation mechanism is adopted to dynamically schedule the processes, a layered scheduling architecture based on priority is established, an operating system can flexibly adjust the scheduling strategy according to process characteristics and resource requirements, and the performance and operation stability of the server operating system are improved.
In order to more clearly describe the technical scheme of the present application, the technical scheme of the present application will be described through a specific embodiment, and it should be noted that the specific embodiment is used for expanding the technical scheme of the present application, and is not limited to the present application.
In some embodiments, performing double-layer graph modeling on process and resource data in a server operating system to obtain a process competition graph layer and a resource dependency graph layer, including:
And carrying out feature vector coding on the process and the resource data to obtain a first node feature matrix and a second node feature matrix. The method comprises the steps of converting a process name into a unique digital ID, normalizing CPU utilization rate, converting memory utilization condition into a proportional value, carrying out numerical mapping on a process priority, and generating a one-hot code of a process state. The method comprises the steps of carrying out classified numbering on various system resources, converting the use amount of the resources into percentages, carrying out numerical value processing on the access rights of the resources, generating class codes of resource types, and constructing numerical value representation of resource dependency relations. Combining all process characteristics into a first node characteristic matrix, combining all resource characteristics into a second node characteristic matrix, ensuring the consistency of characteristic dimensions of the two matrixes, and verifying the integrity and the correctness of a coding result.
And calculating the association degree between the nodes according to the first node feature matrix and the second node feature matrix to obtain a first node association degree matrix and a second node association degree matrix. Specifically, the analysis items corresponding to the association degree comprise resource competition conditions, communication frequency, dependency relationship strength and shared resources, and the association degree scores among the processes are generated by carrying out weighted calculation on the analysis items. Take process P1, process P2, and process P3, and resource R1 and resource R2 as examples. The association degree calculation is performed based on four dimensions, and the weight allocation of each dimension is as follows, namely resource competition (0.3), communication frequency (0.3), dependency relationship (0.2) and shared resource (0.2). The process association degree is calculated, namely the association condition of the process P1 and the process P2 is that CPU resources are frequently competing (scoring 0.8), the process P1 depends on the output of the process P2 (scoring 0.7) and shares the same configuration file (scoring 0.5) after 5 times per minute of communication (scoring 0.6 after normalization). The process association of process P1 and process P2 is 0.8×0.3+0.6×0.3+0.7×0.2+0.5×0.2=0.66. The resource association degree is calculated, and the assumption is that the association condition of the resource R1 (CPU) and the resource R2 (memory) is that when a process uses the CPU in a large amount, more memory is often occupied (competition score of 0.7), the CPU and the memory (communication score of 0.8) are often required to be considered simultaneously when the resource is scheduled, and the memory allocation depends on the CPU scheduling result (dependency score of 0.6) and jointly serves the same group of processes (sharing score of 0.7). Then the resource association of resource R1 and resource R2 is 0.7 x 0.3+0.8 x 0.3+0.6 x 0.2+0.7 x 0.2 = 0.71. The association degree of the integrated process forms a first node association degree matrix, and the association degree of the integrated resource forms a second node association degree matrix.
And carrying out asymmetric matrix construction on the process competition relationship and the resource dependency relationship according to the first node association degree matrix and the first node association degree matrix to obtain an initial adjacent matrix of the double-layer graph. Specifically, the competition resources among the processes are identified, the asymmetry of the competition strength is calculated, a process competition relation matrix is constructed, the asymmetric characteristics of the matrix are ensured, and the rationality of the competition relation is verified. Analyzing the dependence direction among the resources, calculating the asymmetry of the dependence strength, constructing a resource dependence relation matrix, maintaining the transitivity of the dependence relation, and verifying the integrity of the dependence relation. The two relation matrixes are combined into an initial adjacent matrix, the correctness of the matrix structure is ensured, the validity of matrix elements is verified, and the connectivity of the matrix is checked. And (3) carrying out sparsification and ReLU activation on the initial adjacent matrix to obtain an optimized sparse adjacent matrix. Specifically, a correlation threshold is set, weak correlation is filtered, important structural information is reserved, matrix storage space is reduced, and calculation efficiency is improved. And applying a ReLU function to the matrix elements, eliminating negative influence, highlighting strong association relation, improving nonlinear expression capacity of the model and ensuring numerical stability after activation. Checking whether the sparsity is proper or not, and verifying whether the important relation is reserved or not, so that the optimization is ensured not to influence the model effect.
And normalizing the double-layer graph structure according to the optimized sparse adjacent matrix and the node characteristic matrix to obtain a process competition graph layer and a resource dependency graph layer. Specifically, the node characteristics are normalized, so that the consistency of characteristic dimensions is ensured, the dimension influence is eliminated, the stability of the model is improved, and the subsequent analysis and treatment are facilitated. Normalizing the optimized adjacency matrix, maintaining the topological characteristic of the graph structure, ensuring reasonable weight distribution, improving the expression capability of the graph layer and facilitating graph convolution operation. And generating a process competition layer, generating a resource dependence layer, verifying the correctness of the layer structure, ensuring reasonable association between layers, and providing a basis for subsequent analysis.
In some embodiments, the method comprises the steps of carrying out feature extraction on node information of a process competition layer and node information of a resource dependency layer by utilizing a preset graph neural network to obtain node feature vectors containing competition relations and allocation states, carrying out 1X 1 standard convolution kernel projection on the node information of the process competition layer and the node information of the resource dependency layer to obtain initial feature representation of potential space, carrying out self-adaptive learning on hidden relations among nodes according to the initial feature representation to obtain a graph adjacency matrix, wherein the graph adjacency matrix comprises graph inflow relations and graph outflow relations, carrying out mixed jump propagation on the graph adjacency matrix according to the graph inflow relations and the graph outflow relations to obtain space dependency features of the nodes, carrying out time sequence feature extraction on the space dependency features by expanding a convolution filter and a gating mechanism to obtain time dependency features of the nodes, carrying out residual connection and jump connection on the space dependency features to obtain fused multi-level features, and carrying out feature dimension reduction on the feature according to the fused multi-level features to obtain the node feature vectors containing the competition relations and the allocation states.
By way of example, initial features are obtained by standard convolution projection of two layers of process competition and resource dependence, spatial dependence features are captured by hybrid jump propagation in combination with an adaptive learning adjacency matrix, and timing features are extracted by using an expansion convolution and gating mechanism. Through a multi-level feature fusion mechanism of residual connection and jump connection, effective modeling of node competition relationship and resource allocation state is realized. The structural design can simultaneously consider the characteristic dependence of space and time dimensions, and improves the comprehensiveness and accuracy of characteristic extraction.
In some embodiments, according to the graph inflow relationship and the graph outflow relationship, propagating the graph adjacency matrix through mixed hops to obtain the spatial dependency characteristic of the node, including:
And respectively carrying out normalization processing on the graph inflow relation and the graph outflow relation to obtain an inflow standardized adjacent matrix and an outflow standardized adjacent matrix. Specifically, for the ingress relation of one node v, all ingress weights connected thereto need to be divided by the ingress sum of the node so that all ingress weights sum to 1. Similarly, for the outbound relationship, the outbound weight is divided by the outbound sum. This process can avoid gradient explosion or vanishing problems in numerical calculation. For example, a node has three edges, the weights are 2, 3 and 5, and the weights become 0.2, 0.3 and 0.5 after normalization. Typically calculated using the formula D(-1) a, where D is the degree matrix and a is the adjacency matrix.
And carrying out K-order neighborhood information aggregation processing on the node characteristics according to the inflow standardized adjacency matrix to obtain an inflow characteristic sequence of the node. Specifically, the K-order neighborhood represents the set of all nodes that can be reached through K steps starting from the central node. By iteratively aggregating the information of K-order neighbors, a larger range of context information can be obtained. For example, when k=2, not only directly connected nodes but also indirectly connected second-order neighbor nodes are considered. In particular implementations, an iterative algorithm similar to PageRank may be used, with each step updating the node's feature representation.
And carrying out K-order neighborhood information aggregation processing on the node characteristics according to the outbound degree standardized adjacency matrix to obtain an outbound characteristic sequence of the node. Specifically, this step is similar to the previous step, but the direction of the outgoing edge of the node is of interest. By analyzing the outbound relation of the node, the influence of the node on other nodes can be understood. For example, in a resource allocation scenario, if one process node has many outgoing edges pointing to different resource nodes, indicating that the process needs multiple resources, this information is important to prevent deadlock.
And carrying out weighted average processing on the inflow characteristic sequence and the outflow characteristic sequence to obtain initial space dependence characteristics. In particular, gating mechanisms (e.g., GRU or LSTM) may be used to dynamically adjust weights for the incoming and outgoing signature sequences, enabling flexibility in biasing information in different directions in different scenarios. The specific calculation formula of the weighted average of the initial spatial dependence characteristic Hinitial is Hinitial=αHin+(1-α)Hout, wherein alpha is a weight parameter which can be learned, Hin is an inflow characteristic of the inflow characteristic sequence, and Hout is an outflow characteristic of the outflow characteristic sequence.
And carrying out feature fusion processing through a nonlinear transformation layer according to the initial space dependence features to obtain the space dependence features of the nodes. In particular, multi-layer perceptron (MLP) or other neural network structures are commonly used as the nonlinear transformation layer. For example, a multi-layer fully connected network may be constructed using a ReLU as an activation function. Thus, complex interactive relations among the features can be learned, and the space dependent features H of the nodes with more expressive force can be obtained. In particular, it may be expressed as h=mlp (Hinitial), where MLP may be a multi-layer neural network structure.
Through respectively normalizing the relationship between the input degree and the output degree and carrying out K-order neighborhood aggregation, the multi-scale characteristic representation of the nodes in different directions can be respectively obtained, and the local topological structure is considered and the global information with a larger range is contained. And then, the two-way features are fused through weighted average and nonlinear transformation is introduced, so that the expression capability of the features is further enhanced, the obtained space dependence features can more comprehensively describe the position information and the connection mode of the nodes in the graph structure, and the two-way space dependence relationship among the nodes is effectively captured.
In some embodiments, according to the node feature vector, performing deadlock probability calculation by a multi-layer perceptron to obtain a system deadlock probability value and a process priority adjustment scheme, including:
And carrying out normalization and dimension reduction on the node feature vector, and compressing the feature dimension to 50% of the original dimension through a principal component analysis method to obtain a compressed feature vector. Specifically, the collected node feature vectors are normalized, and all feature values are mapped to the same numerical range (usually between 0 and 1) in a unified way, so that dimensional differences among different features can be eliminated. For example, if the original features include the number of processes (1-100 range) and memory usage (0-100% range), they are all converted to the same scale after normalization. The feature dimension can be reduced while retaining the data main information by using Principal Component Analysis (PCA) for dimension reduction. Assuming that the original feature vector has 20 dimensions, the dimension reduction by PCA is followed by a compression to 10 dimensions. By the method, the complexity of subsequent calculation can be reduced, redundant information and noise among the features can be removed, and the generalization capability of the model is improved. The importance of each feature will be quantified by PCA, preserving the most important feature combinations.
And constructing a three-layer perceptron network structure according to the compressed feature vector, randomly initializing the connection weight between the input layer and the hidden layer, and initializing the connection weight between the hidden layer and the output layer by adopting an Xavier method to obtain an initialization weight matrix. Specifically, the three-layer perceptron comprises an input layer, a hidden layer and an output layer, wherein the number of neurons of the input layer is equal to the number of features after dimension reduction, such as 10 neurons. The number of neurons of the hidden layer typically needs to be determined experimentally, possibly set to 1.5 times the input layer, i.e. 15 neurons. The output layer then contains neurons representing different probabilities of deadlock, possibly 2 (representing the probabilities of deadlock and non-deadlock). The weight initialization is very important for model training, namely, the connection from an input layer to a hidden layer is randomly initialized and the range can be between-0.5 and 0.5, and the connection from the hidden layer to an output layer is made by using an Xavier initialization method, and the weight range can be automatically adjusted according to the size of the layer, so that reasonable propagation of signals in a network is ensured.
And carrying out forward propagation calculation on the initialized weight matrix, carrying out nonlinear transformation on hidden layer neuron output by utilizing a ReLU activation function, and carrying out probability normalization processing on an output layer result by utilizing a Softmax function to obtain a system deadlock probability value. Specifically, the data passes through the input layer first, then multiplies with the first layer weight, and passes to the hidden layer. In the hidden layer, the data is processed using a ReLU activation function that will set negative values to 0 and positive values remain unchanged, which can increase the nonlinear expression capacity of the model. The data then continues to propagate forward to the output layer, and finally the output is converted to a probability distribution by the Softmax function. For example, if the calculated output of a system state feature is 0.8, 0.2, the probability that the system is in a deadlock state is 80% and the probability that the system is not in a deadlock state is 20%.
And determining a deadlock risk factor of each process according to the system deadlock probability value and the process resource dependency relationship. Specifically, the calculation of the risk factor needs to consider the resource dependency graph among the processes, evaluate the position and effect of each process in the possible deadlock loop, and therefore, need to extract the current process resource dependency relationship from the system. The process resource dependency relationship comprises the current resource occupation condition of each process, a resource request relationship and the waiting time of the process, and a specific deadlock risk factor is calculated for each process by combining the system deadlock probability value and the process resource dependency relationship. For example, if the system deadlock probability is 80%, a process occupies multiple key resources at the same time, and other processes are waiting for the resources, the key resources of the process are set as deadlock risk factors.
And taking the deadlock risk factor, the resource occupation amount of each process and the waiting time of each process as scoring parameters, and obtaining a process priority scoring sequence through weighted summation operation. Specifically, the deadlock risk factor obtained in the last step is taken as a main scoring parameter, and the higher the risk factor, the lower the score can be obtained. Meanwhile, the process scores of the more key resources occupied by the process are possibly lower in consideration of the resource occupation amount of the process, so that the process scores can be promoted to release the resources as soon as possible. In addition, the waiting time of the process is considered, and the process score can be higher when the waiting time is longer, so as to prevent starvation phenomenon. These parameters are weighted and summed by setting different weight coefficients to obtain the final score for each process. For example, possible weight allocations are a deadlock risk factor of 50%, a resource occupation of 30%, a latency of 20%. And scoring the priority of each process according to the scoring process, and scoring according to the priority of each process to obtain a process priority scoring sequence.
And sequencing and segmenting the process priority scoring sequence, and mapping the scoring value to a discrete priority level interval by adopting a piecewise linear mapping method to obtain an initial priority adjustment scheme. Specifically, the processes in the process priority scoring sequence are ordered, and the relative positions of all the processes are determined. The priority class section is set, for example, to be classified into 5 classes, i.e., highest priority, high priority, medium priority, low priority, and lowest priority. Successive scoring values are mapped into these discrete priority levels by a piecewise linear mapping method. For example, the highest scoring 20% of the processes may be mapped to the highest priority, the next 20% mapped to the high priority, and so on. The mapping method ensures the balance of priority allocation and prevents a large number of processes from being concentrated on a certain priority.
And according to the initial priority adjustment scheme and the history adjustment record, calculating the compensation factors of the processes with the reduced priorities continuously for many times, and superposing the compensation factors with the current priority level to obtain the process priority adjustment scheme. Specifically, it is first necessary to examine the historical priority adjustment record of each process in the initial priority adjustment scheme to identify those processes that are reduced in priority multiple times in succession. For example, a time window (e.g., the last 10 scheduling periods) may be set, counting the number of times a process is de-prioritized within this window. If a process is continuously prioritized more than a preset threshold (e.g., 3 times) within this time window, a compensation factor needs to be calculated for it. The compensation factor may be calculated in an exponentially increasing manner, i.e. the larger the number of successive downgraded times, the larger the compensation factor. The specific formula may be that the compensation factor=base compensation value (1+number of degradation times 0.2). For example, if the base compensation value is 1 and a process is downgraded 4 times in succession, its compensation factor is 1×1+4×0.2=1.8.
When the calculated compensation factors are overlapped with the current priority level, the overall balance of the system needs to be considered. The manner of stacking may be to multiply the compensation factor by the current priority score, or to convert the compensation factor to an additional priority level boost. For example, if the backoff factor is 1.8, the current process is at the lowest priority (level 1), then it may be promoted to medium priority (level 3). In order to prevent the compensation mechanism from over-prioritizing certain processes, a maximum boost level limit needs to be set, such as only up to 2 priority levels.
By the mechanism, the situation that certain processes cannot get execution opportunities for a long time due to continuous priority reduction can be avoided, and therefore fairness of the system is guaranteed. Meanwhile, the current load condition and deadlock risk of the system also need to be considered in the calculation of the compensation factors, the compensation strength can be properly reduced when the system pressure is high, and a larger compensation space can be provided when the system is idle, so that the dynamic adjustment of the compensation mechanism is realized. In addition, when the compensated process gets enough execution machine, the compensation factor should be gradually attenuated until the normal level is recovered, so that the long-term influence of the compensation mechanism on the system can be prevented.
In some embodiments, according to the node feature vector and the process priority adjustment scheme, performing resource allocation policy learning to obtain a first resource allocation scheme, including:
And carrying out feature fusion on the node feature vector and the process priority adjustment scheme to obtain a fusion feature matrix. Specifically, the node feature vector contains information such as process competition relation, resource use condition and the like, the node feature vector and the process priority adjustment scheme are mapped to the same dimension space, and in the same dimension space, the attention mechanism is adopted to calculate the correlation weight among the features, so that the important features are given higher weight. And (3) retaining original characteristic information by using a residual connection structure, performing nonlinear transformation by using a multi-layer perceptron, and performing splicing operation on the transformed characteristics and priority information to obtain a fusion characteristic matrix containing complete context information.
And performing reinforcement learning training on the resource allocation strategy according to the fusion feature matrix to obtain strategy model parameters. Specifically, a deep reinforcement learning model based on an Actor-Critic architecture is constructed, wherein an Actor network is responsible for generating resource allocation actions, and the Critic network evaluates action values. Defining reward functions to consider multiple indexes such as system throughput, resource utilization rate and the like, adopting an experience playback mechanism to store historical decision data, and using a strategy gradient algorithm to update model parameters. And outputting the trained strategy model parameters through a multi-training iterative optimization strategy network.
And performing iterative optimization calculation on the strategy model parameters to obtain an optimized resource allocation strategy. The method comprises the steps of performing model pruning, removing redundant parameters, reducing complexity of the model, compressing parameter bit widths of the model by using a quantization technology, adjusting and optimizing super parameters by using a Bayesian optimization method, evaluating generalization performance of the model by cross verification, and restraining model output by combining expert rules to ensure reliability of strategies. The whole process is continuously optimized in an iterative mode until the strategy performance meets the requirement.
And performing action evaluation according to the optimized resource allocation strategy to obtain a resource allocation sequence. The method comprises the steps of building a simulation environment to simulate a real system state, inputting the current system state to obtain allocation actions of a strategy network, calculating expected return and risk degrees of the actions, conducting Monte Carlo tree search on a plurality of candidate actions, evaluating long-term benefits, screening out an optimal resource allocation action sequence, and converting the action sequence into specific resource allocation instructions.
And carrying out constraint inspection on the resource allocation sequence to obtain a preliminary allocation scheme. The method comprises the steps of checking total resource constraint required by a resource allocation sequence, ensuring that allocated resources do not exceed system capacity, verifying mutually exclusive resource constraint, avoiding resource conflict, checking priority constraint, ensuring that a high-priority process obtains resources preferentially, analyzing time sequence constraint, ensuring reasonable allocation sequence, evaluating load balancing constraint, avoiding uneven allocation of resources, and generating a preliminary allocation scheme meeting all constraint conditions.
And carrying out security verification according to the preliminary allocation scheme to obtain a first resource allocation scheme. The method comprises the steps of analyzing deadlock risks by using a formal verification method, detecting hidden danger of resource leakage, evaluating system stability, verifying real-time requirements and analyzing fault tolerance of a scheme. Through comprehensive security verification, the final resource allocation scheme can meet performance requirements and ensure safe and stable operation of the system.
In some embodiments, according to a first resource allocation scheme, performing multi-level queue allocation on a process to obtain a second resource allocation scheme, including:
Grouping the resource allocation sequences, constructing multidimensional resource vectors according to the resource types corresponding to the resource allocation sequences and the resource quantity corresponding to the resource allocation sequences, and carrying out quantization calculation on the resource vectors to obtain a resource weight matrix;
According to the resource weight matrix and the process priority adjustment scheme, carrying out layered construction on process queues, dividing the process into a plurality of priority queues by adopting a layered tree structure, and distributing time slice quota to each queue node to obtain an initial queue structure model;
carrying out dynamic load calculation on the process in the initial queue structure model, taking the CPU utilization rate, the memory occupancy rate and the IO request frequency as load parameters, and obtaining a process load index sequence through an exponential weighted average method;
According to the process load index sequence, the processes in the queue are quantitatively ordered, and the positions of the processes in the queue are locally adjusted by adopting a multi-objective optimization algorithm, so that an optimized queue ordering scheme is obtained;
feedback control is carried out on the optimized queue ordering scheme, the scheduling period and time slice allocation of the queues are dynamically adjusted through a proportional-integral-derivative controller, and a compensation coefficient is calculated by combining with historical execution records, so that a queue scheduling parameter set is obtained;
And carrying out global coordination on the multi-stage queue system according to the queue scheduling parameter set, distributing the processor time in a layered rotation mode, and establishing a resource yielding and preempting mechanism among the queues to obtain a second resource distribution scheme.
In the method, the resources are quantitatively evaluated and grouped, a hierarchical tree-shaped queue structure based on priority is established, dynamic load calculation is carried out through parameters such as CPU utilization rate, memory occupancy rate and IO request frequency, the process position in the queue is adjusted by utilizing a multi-objective optimization algorithm, finally, the scheduling parameters are dynamically adjusted by adopting a PID controller, and a preemption mechanism is yielded by matching with hierarchical rotation and resources among the queues, so that dynamic optimization of resource allocation is realized, and the resource utilization efficiency and task processing performance of a server system can be remarkably improved.
Referring to fig. 2, fig. 2 is a schematic block diagram of a resource management system of a server operating system according to an embodiment of the present application, where a resource management system 200 of the server operating system is used to execute the foregoing resource management method of the server operating system. The resource management system 200 of the server operating system may be configured in a server.
The server may be an independent server, may be a server cluster, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, a content delivery network (Content Delivery Network, CDN), and basic cloud computing services such as big data and an artificial intelligence platform.
As shown in fig. 2, the resource management system 200 of the server operating system includes a layer modeling module 201, a feature extraction module 202, a process fool-proofing module 203, an allocation calculation module 204, a resource allocation module 205, and a policy enforcement module 206.
The layer modeling module 201 is configured to perform double-layer graph modeling on process and resource data in the server operating system to obtain a process competition layer and a resource dependency layer;
the feature extraction module 202 is configured to perform feature extraction on node information of the process competition layer and node information of the resource dependent layer by using a preset graph neural network, so as to obtain a node feature vector including a competition relationship and an allocation state;
the process fool-proofing module 203 is configured to calculate, according to the node feature vector, a deadlock probability by using a multi-layer perceptron, to obtain a process priority adjustment scheme;
The allocation calculation module 204 is configured to perform resource allocation policy learning according to the node feature vector and the process priority adjustment scheme, so as to obtain a first resource allocation scheme;
A resource allocation module 205, configured to perform multi-level queue allocation on a process according to the first resource allocation scheme, to obtain a second resource allocation scheme;
And the policy enforcement module 206 is configured to perform resource management on the server according to the second resource allocation scheme.
The embodiment of the application provides electronic equipment, which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor is used for executing the computer program and realizing the resource management method of the server operating system according to any one of the embodiment of the application when the computer program is executed.
The embodiment of the application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes a resource management method of a server operating system according to any one of the embodiments of the application.
While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.