Disclosure of Invention
The embodiment of the invention provides a multi-robot cooperative control method and system based on edge calculation, which can solve the problems in the prior art.
In a first aspect of an embodiment of the present invention,
The utility model provides a multi-robot cooperative control method based on edge calculation, which comprises the following steps:
The method comprises the steps that environmental characteristics and search and rescue requirements of a search and rescue area are obtained through an unmanned aerial vehicle, the search and rescue area is divided into a plurality of subareas, a subarea search and rescue task is generated, capacity parameters and current states of all the ground robots are obtained at the same time, and an optimal matching scheme of the ground robots and the subarea search and rescue task is generated according to the subarea search and rescue task and the capacity parameters and the current states of all the ground robots;
Transmitting the sub-region search and rescue task and the optimal matching scheme to corresponding ground robots, adopting different search and rescue strategies aiming at different types of ground robots, modeling a plurality of different types of ground robots into a discrete event system subsystem, and generating a heterogeneous robot collaborative search and rescue strategy by defining formal language and lexicon of collaborative control of each ground robot;
and each ground robot executes the search and rescue task in the corresponding subarea according to the collaborative search and rescue strategy of the heterogeneous robots, monitors the use condition of the computing resources and the network condition of the unmanned aerial vehicle node in real time, starts a computation unloading decision when the use condition of the resources or the network condition is detected to meet the preset condition, and unloads part of the search and rescue tasks to other computing nodes according to the decision result to continue executing the search and rescue action until the search and rescue task is completed.
In an alternative embodiment of the present invention,
According to the sub-region search and rescue task, the capacity parameters and the current state of each ground robot, the generation of the optimal matching scheme of the ground robots and the sub-region search and rescue task comprises the following steps:
according to the capacity parameters and the current state of each ground robot, calculating the utility value of each ground robot for executing different sub-region search and rescue tasks;
According to the utility value of each ground robot executing different sub-region search and rescue tasks, simultaneously considering task completion time and energy consumption index construction scheme matching objective functions;
Constructing a multi-objective optimization model of a multi-robot task allocation problem based on a scheme matching objective function, wherein a decision variable is an allocation matrix of a ground robot and a sub-region search and rescue task, and constraint conditions comprise that at least one robot is allocated to each sub-region and each robot can only be allocated to one sub-region;
and obtaining an optimal task allocation scheme by solving a multi-objective optimization model of the multi-robot task allocation problem.
In an alternative embodiment of the present invention,
The method for obtaining the optimal task allocation scheme by solving the multi-objective optimization model of the multi-robot task allocation problem comprises the following steps:
representing the distribution matrix of the ground robot and the sub-region search and rescue task as a real-value matrix, and randomly generating a certain number of particles, wherein each particle represents a possible task distribution scheme;
calculating the fitness value of each particle on the scheme matching objective function, comparing the current position of each particle with the dominant relationship of the individual optimal solution, and updating the individual optimal solution according to the obtained first comparison result;
for each particle, updating the speed and the position of the particle according to the current speed, the updated individual optimal solution and the updated global optimal solution to obtain updated particles;
combining the original particles and the updated particles to obtain an expanded population, and performing non-dominant sorting on the expanded population to obtain non-dominant solution sets of different grades;
And repeating the iteration until the termination condition is met, outputting the current non-dominant solution set as an optimal solution set, and selecting a final optimal task allocation scheme from the obtained optimal solution set according to the preference of a decision maker.
In an alternative embodiment of the present invention,
Modeling a plurality of different types of ground robots as a discrete event system subsystem, and generating a heterogeneous robot collaborative search and rescue strategy by defining formal languages and lexicons of collaborative control of the ground robots comprises:
generating a series of language samples with cooperative communication interaction and behaviors according to task requirements and capabilities of each ground robot, and storing the language samples in a cooperative control language corpus;
Preprocessing a cooperative control language corpus to form a standardized data set, and converting a language sample into a language sample embedding vector by utilizing a word embedding technology;
Constructing a semantic neural network model based on an encoder-decoder architecture, wherein the semantic neural network model comprises an encoder and a decoder, a task demand encoder and a ground robot capacity encoder based on a graph neural network are introduced into the encoder, task demand graphs and ground robot capacity maps are encoded, and embedded representations of each task node and ground robot node are obtained;
The language samples are embedded into vectors and input into a constructed semantic neural network model, a language symbol string cooperatively controlled by the heterogeneous robots is generated, a semantic verification mechanism based on a knowledge graph is introduced, semantic consistency check is carried out on the language symbol string, and formal description of a cooperative search and rescue strategy of the heterogeneous robots is finally output.
In an alternative embodiment of the present invention,
When the resource use condition or the network condition is detected to meet the preset condition, starting a calculation unloading decision, unloading part of search and rescue tasks to other calculation nodes according to the decision result, and continuing to execute search and rescue actions until the search and rescue tasks are completed, wherein the steps include:
Modeling a calculation unloading decision problem as an unloading optimization model, wherein a task unloading objective function is used for minimizing task execution time and energy consumption, constraint conditions are task completion time and energy consumption limit, and decision variables are unloading decisions of each task and selection of unloading objective nodes;
Calculating an unloading optimization model by adopting an optimization solution algorithm to obtain an optimal calculation unloading scheme, and unloading part of search and rescue tasks from the unmanned aerial vehicle to the selected ground robot nodes according to the optimal calculation unloading scheme;
The selected ground robot node receives the data of the unloading task, executes the calculation of the unloading task and returns the calculation result of the unloading task to the unmanned aerial vehicle;
And integrating the received task unloading calculation result with other locally executed task results by the unmanned aerial vehicle to form a complete search and rescue scheme, and continuously commanding the ground robot to execute until the search and rescue task is completed.
In an alternative embodiment of the present invention,
The formula for the task offload objective function is as follows:
;
Wherein J represents an unloading optimization target, M represents the number of search and rescue tasks, i represents the search and rescue task index, N represents the number of ground robot nodes, J represents the ground robot node index, xij represents a binary variable, tij represents the time required for task i to execute on ground robot node J, ti represents the time required for task i to execute on a local node, λ represents a trade-off coefficient between time and energy consumption, eij represents the energy consumed by task i to execute on ground robot node J, and ei represents the energy consumed by task i to execute on a local node.
In an alternative embodiment of the present invention,
The method further comprises the steps of:
constructing a local task state model of the unmanned aerial vehicle node and the backup node, wherein the input of the local task state model comprises attribute characteristics of search and rescue tasks, an allocated search and rescue task list and the completion progress of each search and rescue task, and the output is the completion probability of the search and rescue tasks and the estimated remaining time;
Constructing a federal learning frame between the unmanned aerial vehicle nodes and the backup nodes, wherein in the task execution process, the unmanned aerial vehicle nodes and the backup nodes respectively utilize locally acquired task state data, obtain model updating quantity through gradient calculation, and transmit the model updating quantity to the federal learning frame to obtain a global updating scheme;
updating the local task state model by using a global updating scheme to obtain a new local task state model, and repeating iteration until the local task state model converges;
The unmanned aerial vehicle node periodically sends heartbeat packets to the backup node, the backup node monitors the state of the unmanned aerial vehicle through the heartbeat packets, and when the heartbeat packets are not received for a plurality of times continuously, the unmanned aerial vehicle is judged to have faults, and task migration is automatically triggered;
And when the task is migrated, the backup node continuously executes the task by using the local task state model of the backup node, after the task is migrated, the model parameters of the local task state model of the backup node are sent to the federation learning frame, and the received model parameters are distributed to other backup nodes through the federation learning frame.
In a second aspect of an embodiment of the present invention,
There is provided a multi-robot cooperative control system based on edge computation, characterized by comprising:
The first unit is used for acquiring the environmental characteristics and the search and rescue requirements of the search and rescue area through the unmanned aerial vehicle, dividing the search and rescue area into a plurality of subareas, generating subarea search and rescue tasks, simultaneously acquiring the capacity parameters and the current state of each ground robot, and generating an optimal matching scheme of the ground robots and the subarea search and rescue tasks according to the subarea search and rescue tasks and the capacity parameters and the current state of each ground robot;
The second unit is used for transmitting the sub-region search and rescue task and the optimal matching scheme to the corresponding ground robots, adopting different search and rescue strategies aiming at different types of ground robots, modeling a plurality of different types of ground robots into a discrete event system subsystem, and generating a heterogeneous robot collaborative search and rescue strategy by defining formal language and lexicon of collaborative control of each ground robot;
And the third unit is used for executing the search and rescue task in the corresponding subarea through each ground robot according to the cooperative search and rescue strategy of the heterogeneous robots, monitoring the use condition of the computing resources and the network condition of the unmanned aerial vehicle node in real time, starting a calculation unloading decision when the use condition of the resources or the network condition is detected to meet the preset condition, unloading part of the search and rescue task to other computing nodes according to the decision result, and continuing to execute the search and rescue action until the search and rescue task is completed.
In a third aspect of an embodiment of the present invention,
There is provided an electronic device including:
A processor;
A memory for storing processor-executable instructions;
wherein the processor is configured to invoke the instructions stored in the memory to perform the method described previously.
In a fourth aspect of an embodiment of the present invention,
There is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method as described above.
In the embodiment, the unmanned aerial vehicle is utilized to obtain the environmental characteristics and the requirements of the search and rescue area, the search and rescue area is reasonably divided into a plurality of subareas, and the optimal matching scheme of the ground robot and the subareas is generated based on the multi-objective optimization model, so that the robot resources can be fully utilized, and the search and rescue efficiency and quality are improved. Modeling different types of ground robots as discrete event system subsystems, and generating heterogeneous robot collaborative search and rescue strategies by defining formal languages and lexical methods, so that collaborative control among different robots is realized, respective advantages are brought into play, and search and rescue capability is improved. The method comprises the steps of monitoring the service condition of computing resources of unmanned aerial vehicle nodes and network conditions in real time, starting a computing unloading decision when resources are tense, unloading part of search and rescue tasks to other computing nodes for continuous execution, avoiding resource waste and search and rescue interruption, and improving the utilization efficiency of computing resources. By introducing advanced technologies such as intelligent algorithm, multi-robot cooperation, calculation unloading and the like, the search and rescue efficiency and quality can be improved, the utilization of calculation resources is optimized, the intelligent level and autonomy of the system are enhanced, and the system has high practical value and is advanced.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
Fig. 1 is a flow chart of a multi-robot cooperative control method based on edge calculation according to an embodiment of the present invention, as shown in fig. 1, the method includes:
S101, acquiring environmental characteristics and search and rescue requirements of a search and rescue area through an unmanned aerial vehicle, dividing the search and rescue area into a plurality of subareas, generating a subarea search and rescue task, simultaneously acquiring capacity parameters and current states of all the ground robots, and generating an optimal matching scheme of the ground robots and the subarea search and rescue task according to the subarea search and rescue task and the capacity parameters and the current states of all the ground robots.
Illustratively, first, an adaptive region partitioning algorithm is designed based on topographical features and search and rescue requirements. The unmanned aerial vehicle acquires the topographic image and the elevation data of the search and rescue area through the airborne sensor, and different topographic types (such as plain, hills, mountains and the like) and key ground features (such as buildings, roads, water areas and the like) are identified by utilizing the image segmentation and feature extraction technology. And then, combining the priority of the search and rescue task and the target distribution information, and adopting a clustering algorithm based on a graph to carry out self-adaptive division on the search and rescue area. The algorithm is used for aggregating the regions with high similarity into a sub-region by constructing a region similarity matrix and considering a plurality of factors such as terrain complexity, barrier density, search and rescue priority and the like. And finally, optimizing the size and shape of the sub-area according to the perception range and the flying height of the unmanned aerial vehicle, and generating an area division scheme suitable for collaborative search and rescue of multiple robots.
In an alternative embodiment of the present invention,
According to the sub-region search and rescue task, the capacity parameters and the current state of each ground robot, the generation of the optimal matching scheme of the ground robots and the sub-region search and rescue task comprises the following steps:
according to the capacity parameters and the current state of each ground robot, calculating the utility value of each ground robot for executing different sub-region search and rescue tasks;
According to the utility value of each ground robot executing different sub-region search and rescue tasks, simultaneously considering task completion time and energy consumption index construction scheme matching objective functions;
Constructing a multi-objective optimization model of a multi-robot task allocation problem based on a scheme matching objective function, wherein a decision variable is an allocation matrix of a ground robot and a sub-region search and rescue task, and constraint conditions comprise that at least one robot is allocated to each sub-region and each robot can only be allocated to one sub-region;
and obtaining an optimal task allocation scheme by solving a multi-objective optimization model of the multi-robot task allocation problem.
Illustratively, first, utility values for each robot to perform different sub-region search and rescue tasks are calculated based on capability parameters (e.g., mobility, endurance, load capacity, etc.) and current states (e.g., location, power, fault conditions, etc.) of the robots. For example, a plurality of ground robots and a plurality of subareas are provided, wherein utility values can be calculated by adopting a weighted summation mode through various index values (such as coverage area, discovery target probability, electric quantity consumption and the like) when the robots execute tasks.
According to utility values of different sub-region search and rescue tasks executed by each ground robot, a scheme matching objective function is constructed by considering task completion time and energy consumption indexes, a multi-objective optimization model of a multi-robot task allocation problem is constructed based on the scheme matching objective function, wherein decision variables are allocation matrixes of the ground robots and the sub-region search and rescue tasks, constraint conditions comprise that at least one robot is allocated to each sub-region, and each robot can only be allocated to one sub-region.
And solving the constructed optimization model by adopting an optimization algorithm to obtain an optimal decision variable matrix, and providing an optimal matching scheme of the ground robot and the sub-region search and rescue task according to the value of the decision variable matrix. According to the matching scheme, deploying the ground robot to execute the search and rescue task. And collecting actual data in the task execution process, and feeding back the actual data to the utility value calculation and the model parameter adjustment for subsequent optimization.
In the embodiment, an optimal matching scheme of the robot and the task can be obtained by constructing a multi-target optimization model and taking an allocation matrix of the ground robot and the sub-region search and rescue task as a decision variable. The scheme can fully consider the capacity parameter and the current state of each robot, furthest develop the potential of the robots and improve the search and rescue efficiency and quality. By only distributing the constraint of one sub-area to each robot, the repeated work of a plurality of robots in the same area is avoided, and the cooperation efficiency among the robots is improved. Meanwhile, at least one constraint of the robot is allocated to each sub-area, so that timely search and rescue of all areas can be ensured.
In an alternative embodiment of the present invention,
The method for obtaining the optimal task allocation scheme by solving the multi-objective optimization model of the multi-robot task allocation problem comprises the following steps:
representing the distribution matrix of the ground robot and the sub-region search and rescue task as a real-value matrix, and randomly generating a certain number of particles, wherein each particle represents a possible task distribution scheme;
calculating the fitness value of each particle on the scheme matching objective function, comparing the current position of each particle with the dominant relationship of the individual optimal solution, and updating the individual optimal solution according to the obtained first comparison result;
for each particle, updating the speed and the position of the particle according to the current speed, the updated individual optimal solution and the updated global optimal solution to obtain updated particles;
combining the original particles and the updated particles to obtain an expanded population, and performing non-dominant sorting on the expanded population to obtain non-dominant solution sets of different grades;
And repeating the iteration until the termination condition is met, outputting the current non-dominant solution set as an optimal solution set, and selecting a final optimal task allocation scheme from the obtained optimal solution set according to the preference of a decision maker.
Illustratively, the allocation matrix of robots and tasks is represented as a real-valued matrix, where the number of behavioural robots of the matrix is listed as the number of tasks. The elements in the matrix represent the probabilities of assigning robot i to task j.
A plurality of particles are randomly generated, each particle representing one possible task allocation scheme. For each particle, its fitness value on a time target and an energy consumption target is calculated. And comparing the dominant relationship between the current position of each particle and the individual optimal solution thereof, and updating the individual optimal solution according to the obtained first comparison result, and comparing the individual optimal solution of all particles with the global optimal solution, and updating the global optimal solution according to the obtained second comparison result.
Generating a variation vector according to a multi-target differential evolution algorithm, randomly selecting three different particles in a population for each particle, and generating the variation vector, namely the current speed by step length control of differential variation;
The method comprises the steps of updating particle speed and position according to a multi-target particle swarm optimization algorithm, updating speed and position according to the current speed, an individual optimal solution and a global optimal solution of each particle through inertia weight, acceleration constant and random number, carrying out boundary processing on newly generated particle positions, limiting elements in a position matrix within a given range, carrying out normalization processing to ensure that the sum of elements in each row is 1, merging original particles and newly generated particles to obtain an expanded population, and carrying out non-dominant sorting on the expanded population to obtain non-dominant solution sets of different grades.
N particles are selected from the non-dominant solution set as a new generation population. Particles with higher grades and more uniform distribution can be preferentially selected according to indexes such as crowding degree distance, aggregation distance and the like. And judging whether a termination condition is met, such as reaching the maximum iteration number, reaching the preset threshold value of the solution quality, and the like. And if the result is met, outputting a non-dominant solution set as an optimal solution, otherwise, returning to the particle fitness calculation step, continuing iterative optimization, and finally selecting a final optimal task allocation scheme from the obtained optimal solution set according to the preference of a decision maker.
In this embodiment, different regions of the solution space can be effectively explored through a parallel search mechanism of the particle swarm algorithm, so that global optimization capability is improved, and the situation of sinking into local optimization is avoided. Through directly performing non-dominant sorting in the target space, the multi-target optimization problem can be effectively processed, and meanwhile, a plurality of indexes such as task completion time, energy consumption and the like are optimized, so that a more comprehensive and reasonable solution set is obtained. The output is a non-dominant solution set, and a decision maker can flexibly select an optimal task allocation scheme meeting the requirements according to actual preference.
S102, transmitting sub-region search and rescue tasks and an optimal matching scheme to corresponding ground robots, adopting different search and rescue strategies aiming at different types of ground robots, modeling a plurality of different types of ground robots into a discrete event system subsystem, and generating a heterogeneous robot collaborative search and rescue strategy by defining formal languages and lexicons of collaborative control of the ground robots;
The method comprises the steps of designing targeted search strategies aiming at heterogeneous robots of different types. For example, for robots equipped with visual sensors, a probabilistic graph-based target search algorithm is employed. Specifically, according to visual images and a deep learning technology, a probability occupation grid map of a search and rescue area is constructed, and the probability of targets existing in each grid is represented. And then, deducing an optimal search path by using Bayesian inference and information entropy theory, so that the robot reaches the grid with the highest target existence probability in the shortest time. In the searching process, the robot dynamically updates the probability map according to the real-time detected target information, and correspondingly adjusts the searching path. For another example, for robots equipped with infrared sensors, a heat source directed fast search strategy is employed. The robot detects the heat source distribution in the environment in real time by utilizing an infrared thermal imaging technology, and identifies a suspected target area. And then, combining the characteristics of the position, the area, the temperature and the like of the target area, planning an optimal searching route, and carrying out key searching and careful investigation on the suspected target area.
The discrete event system subsystem comprises state variables which represent the motion state (such as movement, searching, obstacle avoidance and the like) of the robot, event variables which represent the task events (such as target discovery, region searching completion and the like) and communication events (such as receiving task instructions, sending state information and the like) of the robot.
On the basis, formal languages and lexicons of cooperative control of the heterogeneous robots are defined, and information interaction and behavior coordination among multiple robots are standardized. For example, when the vision robot finds a suspicious target, it sends a "target found" event to the infrared robot requesting it to go to the area for heat source detection, and when the infrared robot confirms the existence of the target, it sends a "target confirm" event to the central controller requesting to schedule other robots to go to support. And finally, formalized analysis is carried out on the cooperative control strategy of the multiple robots, and experimental verification is carried out by using a simulation tool, so that the safety and the effectiveness of cooperative search and rescue of the heterogeneous robots are ensured.
In an alternative embodiment of the present invention,
Modeling a plurality of different types of ground robots as a discrete event system subsystem, and generating a heterogeneous robot collaborative search and rescue strategy by defining formal languages and lexicons of collaborative control of the ground robots comprises:
generating a series of language samples with cooperative communication interaction and behaviors according to task requirements and capabilities of each ground robot, and storing the language samples in a cooperative control language corpus;
Preprocessing a cooperative control language corpus to form a standardized data set, and converting a language sample into a language sample embedding vector by utilizing a word embedding technology;
Constructing a semantic neural network model based on an encoder-decoder architecture, wherein the semantic neural network model comprises an encoder and a decoder, a task demand encoder and a ground robot capacity encoder based on a graph neural network are introduced into the encoder, task demand graphs and ground robot capacity maps are encoded, and embedded representations of each task node and ground robot node are obtained;
The language samples are embedded into vectors and input into a constructed semantic neural network model, a language symbol string cooperatively controlled by the heterogeneous robots is generated, a semantic verification mechanism based on a knowledge graph is introduced, semantic consistency check is carried out on the language symbol string, and formal description of a cooperative search and rescue strategy of the heterogeneous robots is finally output.
The method comprises the steps of firstly cleaning and marking a collected cooperative control language corpus, matching a language sample with corresponding task requirements and robot capabilities to form a standardized data set, mapping words in the language sample into language sample embedded vectors by Word2Vec and other Word embedding technologies to serve as input features of a neural network, and extracting and encoding features of the task requirements and the robot capabilities to form a structured input representation.
The method comprises the steps of designing a sequence-to-sequence neural network model based on a transducer architecture, utilizing a self-attention mechanism to realize long-distance dependence modeling of an input sequence and an output sequence, introducing a task requirement and robot capacity encoder based on a graph neural network into an encoder part of the transducer, and embedding structural information of tasks and robots into a language generation process, wherein the graph attention network can encode a task requirement graph and a robot capacity graph to obtain embedded representation of each task node and robot node. The embedded representations are fused with the output of the transducer encoder, so that the task and the structural information of the robot can be introduced into the language generation process.
In the decoder part of the transducer, a copy mechanism based on a Pointer Network (Pointer Network) is introduced, which decides by means of a distribution of attention which element of positions to copy from the input sequence. The pointer network calculates an attention profile over a time step of the decoder based on the current decoder and encoder outputs, wherein the attention profile represents the probability of copying each position from the input sequence over a time step, and the decoder decides whether to generate a word from the vocabulary or to copy a word from the input sequence based on the resulting attention profile.
And (3) performing end-to-end supervised learning training on the designed neural network model by using a standardized data set, optimizing model parameters, introducing a reward mechanism based on reinforcement learning in the training process, and optimizing the model in a fine granularity according to indexes such as quality and diversity of generated language. Regularization technologies such as Early Stopping and Dropout are adopted to prevent the model from being over-fitted and improve the generalization capability of the model.
The trained neural network model is deployed into an online reasoning service, a real-time language generation function is realized, and post-processing is carried out on the generated language symbol string, including formatting, error checking, grammar correcting and the like, so that the generated language is ensured to meet the requirement of cooperative control.
And a semantic verification mechanism based on a knowledge graph is introduced, so that semantic consistency check is performed on the generated language, and the accuracy and the interpretability of language generation are improved. Specifically, a knowledge graph related to the collaborative search and rescue task of the robot needs to be constructed first. The knowledge graph is a structured knowledge base, and contains knowledge of entities (such as robot type, task type, etc.), relationships among the entities, and attributes of the entities. The method can be extracted from the existing domain knowledge base or corpus, and can also be semi-automatically/automatically constructed by the knowledge graph automatic construction technology. And carrying out semantic analysis on the generated language description, identifying the entities mentions contained in the language description, and linking the entities to corresponding entity nodes in the knowledge graph. Semantic parsing may be implemented using natural language processing techniques such as named entity recognition, relationship extraction, and the like. And constructing a series of semantic constraint rules according to the domain knowledge and task requirements. These rules define the semantic constraints that should be satisfied between the elements of the language description, entities, relationships, etc. For example, whether a certain robot type can execute a certain task type, whether a precedence dependence exists between different tasks, etc. And verifying the results of semantic analysis and entity linking with constraint rules. And checking whether the entity and the relation in the language description meet the predefined semantic constraint, so as to judge whether the description is semantically consistent and reasonable. For portions where semantic inconsistencies are detected, correction suggestions may be given or regeneration mechanisms triggered to output semantically consistent cooperative control policy descriptions.
In the embodiment, by constructing a semantic neural network model and encoding task requirements and robot capacity into graph embedded vector input, various heterogeneous information can be comprehensively considered, and a cooperatively controlled language symbol string description is generated, so that efficient cooperative control of heterogeneous robots is realized. The language description generation problem is modeled as an encoder-decoder framework, the formal language description of the robot collaborative strategy is generated end to end, manual rules are not needed, and the complexity of strategy design is reduced. The semantic verification mechanism based on the knowledge graph is introduced, so that the semantic rationality and consistency of the generation strategy can be checked, and the generated formal description is ensured to meet the actual requirements. The encoder end utilizes the structured representation of the task demand and the robot capability of the graph neural network learning, the decoder end introduces a pointer network replication mechanism to capture the alignment relationship between input and output, the expression capability of a model is improved, the automatic generation of the cooperative control strategy of the heterogeneous robot can be realized, the rationality of strategy description is ensured, and the intelligent cooperative control method has the interpretation, generalization capability and self-adaptability and is an efficient intelligent cooperative control method.
S103, each ground robot executes the search and rescue task in the corresponding subarea according to the cooperative search and rescue strategy of the heterogeneous robots, simultaneously monitors the use condition of the computing resources and the network condition of the unmanned aerial vehicle node in real time, starts a computation unloading decision when the use condition of the resources or the network condition is detected to meet the preset condition, and unloads part of the search and rescue tasks to other computing nodes according to the decision result to continue executing the search and rescue actions until the search and rescue task is completed.
The computing resource use condition mainly comprises indexes such as CPU occupancy rate, memory occupancy rate, storage space occupancy rate and the like. The network condition mainly comprises indexes such as network bandwidth, transmission delay, packet loss rate and the like.
The ground robot obtains the use condition of the computing resources by periodically sending a resource detection request to the unmanned plane node. After receiving the request, the unmanned aerial vehicle node evaluates the use condition of the computing resource of the unmanned aerial vehicle node and returns the evaluation result to the ground robot. And the ground robot judges whether the computing resources of the unmanned aerial vehicle nodes meet the preset conditions according to the collected evaluation results. The preset conditions can be set according to the requirements of search and rescue tasks and the performance requirements of the system, for example, the CPU occupancy rate is not more than 80%, the memory occupancy rate is not more than 70% and the like.
The ground robot evaluates indexes such as network bandwidth, transmission delay, packet loss rate and the like by periodically detecting network conditions between the ground robot and unmanned aerial vehicle nodes. The detection method can adopt ICMP, TCP/UDP and other network diagnosis protocols. And the ground robot judges whether the network condition meets the preset condition according to the detection result. The preset conditions can be set according to the requirements of search and rescue tasks on the real-time performance and reliability of data transmission, for example, the network bandwidth is not lower than 1Mbps, the transmission delay is not more than 50ms, the packet loss rate is not more than 1%, and the like.
The goal of the calculation offloading decision is to select an optimal calculation offloading scheme to offload part of the search and rescue tasks from the unmanned aerial vehicle nodes to other calculation nodes so as to improve the execution efficiency and quality of the search and rescue tasks.
In an alternative embodiment of the present invention,
When the resource use condition or the network condition is detected to meet the preset condition, starting a calculation unloading decision, unloading part of search and rescue tasks to other calculation nodes according to the decision result, and continuing to execute search and rescue actions until the search and rescue tasks are completed, wherein the steps include:
Modeling a calculation unloading decision problem as an unloading optimization model, wherein a task unloading objective function is used for minimizing task execution time and energy consumption, constraint conditions are task completion time and energy consumption limit, and decision variables are unloading decisions of each task and selection of unloading objective nodes;
Calculating an unloading optimization model by adopting an optimization solution algorithm to obtain an optimal calculation unloading scheme, and unloading part of search and rescue tasks from the unmanned aerial vehicle to the selected ground robot nodes according to the optimal calculation unloading scheme;
The selected ground robot node receives the data of the unloading task, executes the calculation of the unloading task and returns the calculation result of the unloading task to the unmanned aerial vehicle;
And integrating the received task unloading calculation result with other locally executed task results by the unmanned aerial vehicle to form a complete search and rescue scheme, and continuously commanding the ground robot to execute until the search and rescue task is completed.
The computational offloading decision process is triggered, for example, when the ground robot detects that the computational resource usage or network conditions of the unmanned aerial vehicle node meet a preset condition. The goal of the calculation offloading decision is to select an optimal calculation offloading scheme to offload part of the search and rescue tasks from the unmanned aerial vehicle nodes to other calculation nodes so as to improve the execution efficiency and quality of the search and rescue tasks.
The computational offload decision problem can be modeled as an optimization problem. The optimization objective is to minimize task execution time and energy consumption under constraint conditions such as meeting task completion time and energy consumption limit. Decision variables include the offload decision (binary variable representing whether or not to offload) for each task and the choice of offload target node (integer variable representing to which compute node to offload).
Alternatively, a heuristic or approximation algorithm may be employed for the solution. Common heuristic algorithms include genetic algorithms, particle swarm optimization algorithms, ant colony algorithms, and the like. Taking genetic algorithm as an example, unloading decision variables can be encoded into chromosomes, and the population is continuously optimized through genetic operations such as selection, crossover, mutation and the like, and finally the optimal solution is converged.
According to the optimal calculation unloading scheme obtained by the optimization solution, the unmanned aerial vehicle node unloads part of search and rescue tasks to the selected ground robot node. The offloading process includes the transfer of task data and the migration of task computations.
In the task data transmission stage, the unmanned aerial vehicle node transmits task related data (such as map information of a search and rescue area, target feature information and the like) to the target node through a wireless network. In order to improve the efficiency and reliability of data transmission, techniques such as block transmission, multi-channel parallel transmission and the like can be adopted. At the same time, the transmitted data may be encrypted and digitally signed in order to ensure confidentiality and integrity of the data.
In the task calculation migration stage, the target node immediately starts to execute task calculation after receiving task data. The computation process may involve complex algorithms and models such as object detection, path planning, decision optimization, etc. To accelerate the computation process, acceleration devices such as GPUs, FPGAs, etc. of the target node may be utilized. And after the calculation is completed, the target node returns a calculation result to the unmanned aerial vehicle node.
After receiving the calculation result of the unloading task, the unmanned aerial vehicle node integrates the result with other task results executed locally to form a complete search and rescue scheme. According to the search and rescue scheme, the unmanned aerial vehicle node continuously commands the ground robot to execute search and rescue actions until the search and rescue task is completed.
Optionally, in the whole process of computing unloading, a sound fault-tolerant mechanism and an abnormal processing strategy are established, so that task execution failure caused by node faults, network interruption and the like is prevented.
In an alternative embodiment of the present invention,
;
Wherein J represents an unloading optimization target, M represents the number of search and rescue tasks, i represents the search and rescue task index, N represents the number of ground robot nodes, J represents the ground robot node index, xij represents a binary variable, tij represents the time required for task i to execute on ground robot node J, ti represents the time required for task i to execute on a local node, λ represents a trade-off coefficient between time and energy consumption, eij represents the energy consumed by task i to execute on ground robot node J, and ei represents the energy consumed by task i to execute on a local node.
In the embodiment, by constructing the optimization model and solving the optimal unloading scheme, the computing resources can be reasonably allocated, tasks are unloaded to idle nodes for execution, the computing resource utilization efficiency of the whole system is improved, and resource waste is avoided. According to the real-time detected unmanned aerial vehicle node calculation load condition, whether task unloading is needed or not can be dynamically decided, dynamic load balancing of the system is achieved, and running stability is improved. The objective function of the optimization model considers the task execution time and the energy consumption, and the optimal scheme obtained by solving can shorten the task completion time delay, reduce the energy consumption and improve the time and the energy utilization efficiency. The technical scheme based on calculation unloading optimization can improve the utilization efficiency of calculation resources, improve time delay and energy efficiency, ensure the continuity of task execution, support heterogeneous calculation coordination, enhance the reliability of a system, and has good flexibility and expansibility, thereby being an effective mobile edge calculation solution.
In an alternative embodiment of the present invention,
The method further comprises the steps of:
constructing a local task state model of the unmanned aerial vehicle node and the backup node, wherein the input of the local task state model comprises attribute characteristics of search and rescue tasks, an allocated search and rescue task list and the completion progress of each search and rescue task, and the output is the completion probability of the search and rescue tasks and the estimated remaining time;
Constructing a federal learning frame between the unmanned aerial vehicle nodes and the backup nodes, wherein in the task execution process, the unmanned aerial vehicle nodes and the backup nodes respectively utilize locally acquired task state data, obtain model updating quantity through gradient calculation, and transmit the model updating quantity to the federal learning frame to obtain a global updating scheme;
updating the local task state model by using a global updating scheme to obtain a new local task state model, and repeating iteration until the local task state model converges;
The unmanned aerial vehicle node periodically sends heartbeat packets to the backup node, the backup node monitors the state of the unmanned aerial vehicle through the heartbeat packets, and when the heartbeat packets are not received for a plurality of times continuously, the unmanned aerial vehicle is judged to have faults, and task migration is automatically triggered;
And when the task is migrated, the backup node continuously executes the task by using the local task state model of the backup node, after the task is migrated, the model parameters of the local task state model of the backup node are sent to the federation learning frame, and the received model parameters are distributed to other backup nodes through the federation learning frame.
Illustratively, the unmanned plane node and the backup node respectively construct their own local task state models for characterizing the state and progress of task execution. The input of the model may include the attribute characteristics of the task, the list of assigned subtasks, the progress of completion of each subtask, etc., and the output may be the probability of completion of the task, the estimated time remaining, etc. The model structure can adopt a deep neural network, a graph neural network and the like, and the specific model architecture and super parameters can be optimized according to the characteristics of tasks and the data distribution.
And constructing a federal learning framework between the unmanned aerial vehicle nodes and the backup nodes, and coordinating model training and updating among different nodes. The framework mainly comprises a communication module which is responsible for transmitting data such as model parameters, gradients and aggregation results among nodes and adopts safe communication protocols such as SSL, TLS and the like, an aggregation module which is responsible for aggregating model updating quantities received from other nodes and can adopt an aggregation algorithm such as FedAvg, fedProx and the like, and an updating module which is responsible for locally utilizing the aggregated updating quantities to update the local model and can adopt an optimization algorithm such as gradient descent, adam and the like.
In the task execution process, the unmanned plane node and the backup node train and update the local model by utilizing the locally acquired task state data. Specifically, each node independently utilizes local data to calculate gradient or weight updating quantity of a model, each node sends the calculated updating quantity to an aggregation module of a federal learning framework, the aggregation module aggregates the received updating quantity to obtain a global model updating scheme, and each node updates the local model by utilizing the global updating scheme after sending the global updating scheme to each node to obtain a new local model. Repeating the steps until the model converges or reaches the preset iteration times.
When the unmanned aerial vehicle node fails and needs to perform task migration, the backup node can directly use the local model of the backup node to continue to execute tasks without acquiring complete task state data from the failed node. In order to ensure that the model of the backup node is consistent with the model of the fault node, after the task migration is completed, the backup node can send own model parameters to the federal learning framework and distribute the model parameters to other nodes through the aggregation module so as to realize the global synchronization of the models.
In the embodiment, by constructing the local task state model, the task completion probability and the estimated remaining time can be output according to the search and rescue task attribute, the distribution condition and the completion progress, and decision support is provided for task scheduling and management. By utilizing the federal learning framework, the unmanned aerial vehicle nodes and the backup nodes can update the parameters of the local task state model in real time in the task execution process, so that the model is continuously learned, and the accuracy of task state prediction is improved. When the unmanned aerial vehicle node fails, the backup node can automatically take over task execution and continue to execute by utilizing the local task state model of the backup node, so that the fault tolerance and reliability of the system are improved. The method realizes cooperative work of heterogeneous computing resources of the unmanned aerial vehicle nodes and the backup nodes, fully utilizes all computing capacity in the system, reduces network communication overhead, and simultaneously, each node only needs to update a local model, reduces computing overhead and realizes balance of computing and communication overhead.
Fig. 2 is a schematic structural diagram of a multi-robot cooperative control system based on edge computation according to an embodiment of the present invention, as shown in fig. 2, the system includes:
The first unit is used for acquiring the environmental characteristics and the search and rescue requirements of the search and rescue area through the unmanned aerial vehicle, dividing the search and rescue area into a plurality of subareas, generating subarea search and rescue tasks, simultaneously acquiring the capacity parameters and the current state of each ground robot, and generating an optimal matching scheme of the ground robots and the subarea search and rescue tasks according to the subarea search and rescue tasks and the capacity parameters and the current state of each ground robot;
The second unit is used for transmitting the sub-region search and rescue task and the optimal matching scheme to the corresponding ground robots, adopting different search and rescue strategies aiming at different types of ground robots, modeling a plurality of different types of ground robots into a discrete event system subsystem, and generating a heterogeneous robot collaborative search and rescue strategy by defining formal language and lexicon of collaborative control of each ground robot;
And the third unit is used for executing the search and rescue task in the corresponding subarea through each ground robot according to the cooperative search and rescue strategy of the heterogeneous robots, monitoring the use condition of the computing resources and the network condition of the unmanned aerial vehicle node in real time, starting a calculation unloading decision when the use condition of the resources or the network condition is detected to meet the preset condition, unloading part of the search and rescue task to other computing nodes according to the decision result, and continuing to execute the search and rescue action until the search and rescue task is completed.
In a third aspect of an embodiment of the present invention,
There is provided an electronic device including:
A processor;
A memory for storing processor-executable instructions;
wherein the processor is configured to invoke the instructions stored in the memory to perform the method described previously.
In a fourth aspect of an embodiment of the present invention,
There is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method as described above.
The present invention may be a method, apparatus, system, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for performing various aspects of the present invention.
It should be noted that the above embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solution described in the above embodiments may be modified or some or all of the technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present invention.