Disclosure of Invention
The invention aims to solve the problems that the existing mining robot has signal interference or limited communication signal coverage range when in use, so that a remote control signal is unstable and even is disconnected, and meanwhile, the abnormality of the robot cannot be processed in time.
The aim of the invention can be achieved by the following technical scheme:
a mining robot remote control system comprises a robot end, a cloud end and a control terminal;
a robot end; the method is used for executing various tasks, and specifically comprises the steps of carrying out inspection and data acquisition on various devices in the mine;
and (3) cloud end. The system is used for managing and processing data uploaded by the analysis robot end;
a control terminal; the remote control robot terminal is used for executing different operations;
further comprises:
the detection module is used for detecting the condition of direct information transmission between the robot end and the cloud end as well as between the robot end and the control terminal;
the method specifically comprises the following steps:
obtaining a dbm (dbm) duty ratio, an SNR (signal to noise ratio) value, a packet loss rate and a communication delay of information transmission in a system, and analyzing and calculating the obtained data to obtain a confidence value;
comparing the calculated reliability value with a preset reliability threshold, and recording the point position moved by the robot end when the calculated reliability value of the robot end is lower than the preset reliability threshold in the inspection movement process; summarizing marked point positions in all routes of the robot inspection;
the marked point positions are bypassed according to a bypass rule, and the specific bypass rule comprises the following steps:
forming two semicircular detour routes by taking the marking point position as the circle center and taking the preset distance as the radius, verifying the obstacles on the two semicircular detour routes, and selecting the other semicircular route for detour when the obstacle exists in one semicircular detour route; when the two semicircular detour routes have barriers, increasing the preset distance until the detour route is determined;
testing the determined detour route, judging whether the reliability value on the detour route is lower than the preset reliability threshold value or not by controlling the robot end to move on the detour route, judging the point position of the robot end at the moment, and detour again according to the detour rule; the reliability value tested in the whole inspection route and the changed detour route of the robot end is higher than a preset reliability threshold value, so that the communication between the control terminal and the robot end is ensured to be normal and stable;
the optimizing module is used for optimizing the fluency of information transmission in the system;
the detection of the determined optimization operation comprises the following steps:
acquiring the latest routing inspection route of the robot end, sending routing inspection signaling to the robot end through the control terminal, and carrying out routing inspection through the latest routing inspection route after the robot end receives the routing inspection signaling;
in the process of inspecting the robot end, acquiring the confidence value of the robot end through a detection module, wherein a specific acquisition method adopts an intermittent time acquisition method or an intermittent distance acquisition method, so that a group of continuous confidence value groups are obtained, and the confidence values of the group are averaged through an average calculation formula;
acquiring a calculated average value, comparing the average value with a preset normal value, and executing optimization operation when the average value is smaller than the preset normal value;
and the early warning module is used for early warning the abnormality of the robot end.
Further, the specific operation steps of the detection module for obtaining the dbm duty ratio, the SNR value, the packet loss rate and the communication delay of the information transmission in the system are as follows:
detecting signal intensity of signal transmission between a robot end and a cloud end, and judging quality of a communication link; setting a full signal dbm value, and representing the signal strength by the ratio of the detected real-time dbm value to the full signal dbm;
collecting the signal-to-noise ratio SNR of data transmission between the robot end and the cloud end, namely the ratio of a signal to background noise;
the method comprises the steps of sending a data packet to a robot end through a cloud, detecting the packet loss condition of the data packet in the communication process, obtaining the packet loss rate of the data transmission, performing multiple tests according to the same steps, removing abnormal results, and performing average value calculation;
the control signaling is sent through the control terminal, the transmission delay of the control signaling sent by the control terminal after being sent to the robot end through the cloud is measured, and the control signaling is represented by the delay time; the communication delay is represented by calculating an average of a plurality of delay times using the transmission of a plurality of test control signaling.
Further, the specific operation steps of the optimizing operation in the optimizing module are as follows:
firstly, establishing a standby communication channel, wherein the standby communication channel is any one of a wireless network, satellite communication and a wired network;
the method comprises the steps of monitoring an existing communication channel in real time, and when abnormality occurs, directly switching signal transmission into a standby communication channel to provide a standby communication path for a robot end;
then, a signal relay station is added in a robot-end inspection area, so that the signal coverage area is enlarged, and the dead zone and signal interference of the signal receiving and transmitting of the robot end are reduced;
a spread spectrum encoder and a decoder are arranged for spreading and decoding the control signaling spectrum;
the spread spectrum coder converts the control signaling into a high-speed spread spectrum signal, and the decoder restores the received spread spectrum signal into the original control signaling;
selecting a corresponding spread spectrum sequence, wherein the spread spectrum sequence has autocorrelation and cross correlation, and specifically comprises a pseudo random noise code sequence or a Gold code sequence so as to ensure that control signaling is recovered to be normal after spreading;
multiplying the original control signaling and the spread spectrum sequence to realize the spectrum spreading of the control signal, and completing the bit-by-bit multiplication between the control signal and the spread spectrum sequence;
coding a control signaling transmitted by a control terminal and adding redundancy sinking information; and an algorithm for checking and correcting codes is arranged at the robot end, and errors in the transmission process of the control signaling are detected and repaired.
Further, the specific operation steps of the early warning module for early warning are as follows:
acquiring image and sound data acquired by a robot end, and analyzing and processing the image and sound data;
when the acquired image data and sound data are analyzed and then the warning signaling appears, warning operation is executed; the specific warning operation comprises the following steps:
after receiving the warning signaling, determining that the robot end acquires abnormal data in the data, uploading the abnormal data and the occurrence time to the manual inspection terminal for secondary manual inspection, and judging whether the inspection stopping maintenance is needed;
after the uploading of the abnormal data is completed, setting waiting time, and when feedback is not received after the waiting time is exceeded, sending prompt information to the manual inspection mobile terminal, specifically summarizing the abnormal data and sending the abnormal data to the manual inspection mobile terminal, prompting inspection personnel to check the abnormal information in time, so that larger loss is avoided;
when the manual inspection terminal executes related instructions within the set waiting time range, the robot terminal is controlled to perform corresponding operation, and then the operation is recorded;
simultaneously, the robot end is automatically controlled to move towards the maintenance end point, the current position of the robot end is detected, the route planning is carried out by combining the position of the maintenance end point, and after the route planning is completed, the robot end is controlled to move towards the maintenance end point;
in the moving process of the robot, searching the mobile equipment of maintenance personnel in a circular area by taking a maintenance end point as a circle center and a preset distance as a radius; judging whether maintenance personnel exist in the search area;
when maintenance personnel exist, a maintenance prompt is sent to a mobile terminal of the maintenance personnel, and specific contents comprise abnormal data and time of the robot and a specific time point for returning the robot to a maintenance terminal; data support is made for maintenance personnel to make corresponding maintenance arrangement;
when the mobile equipment of the maintenance personnel is not searched, the search area is enlarged, or when the mobile equipment of the maintenance personnel is not searched, prompt information is sent to the terminal of the management personnel, the maintenance personnel is subjected to task arrangement through the management personnel, the robot of the maintenance end point is maintained timely, and the time waste is avoided, so that the operation efficiency of the robot is influenced.
Further, the specific operation steps of the early warning module for analyzing the image data are as follows:
firstly, carrying out preliminary pretreatment on the obtained influence data, including noise reduction, image enhancement and size normalization adjustment;
and analyzing the preprocessed image data, specifically:
object detection: identifying each object in the image data, and generating a warning signal if an unexpected object appears or an object which should exist is absent;
behavior recognition: the motion of the robot is identified through the image data transmitted by the robot end, and the gesture is estimated; judging a specific motion track and a specific behavior mode of the robot according to the motion recognition and the gesture estimation of the normal robot, and generating a warning signaling when the robot deviates from the expected behavior;
and (3) image comparison: comparing the image of the current frame in the image data with the previous normal image, checking whether the images have obvious differences, quantifying the differences of the images by using an image feature extraction and matching method, such as structural similarity evaluation or key point matching, and generating a warning signaling when the differences of the images are larger than a preset value;
motion detection: and identifying a motion area in a picture in the image data by utilizing a motion detection algorithm, and generating a warning signaling when large-area and intense motion occurs in a static background or due moving objects suddenly disappear or the motion track is abnormal.
Further, the specific operation steps of the pre-warning module for analyzing the sound data are as follows:
firstly, analyzing and detecting the volume change amplitude in sound data;
when huge noise occurs, the change amplitude of the volume is rapidly enlarged, and when the change amplitude of the volume exceeds a preset amplitude threshold, the abnormal volume is secondarily judged;
comparing the abnormal volume with the volume change amplitude generated at the moment of normal inspection of the robot, continuing to monitor when the difference is lower than a preset value, and generating an alarm signaling when the difference exceeds the preset value.
Compared with the prior art, the invention has the beneficial effects that:
(1) According to the invention, the inspection route of the inspection robot is planned by detecting the communication condition of the system, so that signal interference of mine topography is effectively avoided, communication delay can be reduced, instantaneity between the robot and a remote operation center is improved, faster and more accurate instruction transmission is realized, waiting time of the robot for executing instructions is reduced, and working efficiency is improved;
(2) According to the invention, the fault detection method is integrated, the state of the robot can be monitored in real time, once an abnormal or fault condition is found, the system can generate early warning information timely and send the early warning information to related personnel, meanwhile, the route to a maintenance endpoint can be planned, the maintenance personnel is reminded of reaching the first time, the occurrence of unexpected conditions caused by the fault of the robot can be effectively avoided, and meanwhile, the working efficiency of the robot is improved.
Detailed Description
The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present disclosure is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the present disclosure and claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As shown in fig. 1, the mining robot remote control system comprises a robot end, a cloud end, a control terminal, a detection module, an optimization module and an early warning module;
the robot end is used for executing various tasks, and the specific execution tasks comprise inspection and data acquisition of various devices in the mine; the cloud end is used for managing and processing data uploaded by the analysis robot end; the control terminal is used for remotely controlling the robot end to execute different operations.
The detection module is used for detecting the condition of direct information transmission between the robot end and the cloud end as well as between the robot end and the control terminal;
judging the quality of a communication link by detecting the signal intensity of signal transmission between the robot end and the cloud end; setting a full signal dbm value, and representing the signal strength by the ratio of the detected real-time dbm value to the full signal dbm; where dbm is a unit representing a power level and represents a decibel value relative to 1 milliwatt, dbm is commonly used to measure the strength of a signal in wireless communications and network devices, the higher the dbm value of the signal, the stronger the signal strength, the better the communication quality, and if the lower the dbm value, the weaker the signal strength. Collecting signal-to-noise ratio (SNR) of data transmission between the robot end and the cloud end, wherein the SNR is the ratio between the received effective signal intensity and the background noise intensity, and a lower SNR indicates that communication is interfered or noise interference is larger; the method comprises the steps of sending a data packet to a robot end through a cloud, detecting the packet loss condition of the data packet in the communication process, obtaining the packet loss rate of the data transmission, performing multiple tests according to the same steps, eliminating abnormal results, and then performing average value calculation to improve the accuracy of packet loss rate monitoring;
the control signaling is sent by the control terminal, the transmission delay of the control signaling sent by the control terminal after being sent to the robot end by the cloud is measured, the communication is rapidly and stably represented by the delay time, the control response and the instantaneity of the robot end are affected by the lower delay, and in order to improve the measurement accuracy, the sending of a plurality of test control signaling can be adopted, so that the average value of a plurality of delay times is calculated to represent the communication delay;
calibrating the obtained dbm duty ratio, SNR value, packet loss rate and communication delay as DM, SR, DB and YS respectively, and substituting the normalized values into the formula:to obtain a confidence value XD, where +.>The preset weight coefficient of the dbm duty ratio, the preset weight coefficient of the SNR value, the preset weight coefficient of the packet loss rate and the preset weight coefficient of the communication delay are respectively taken as 1.112, 0.996, 1.023 and 1.032;
comparing the calculated reliability value XD with a preset reliability threshold, and recording the point position moved by the robot end when the calculated reliability value XD is lower than the preset reliability threshold in the process of inspection movement of the robot end; summarizing marked point positions in all routes of the robot inspection;
the marked point positions are bypassed according to a bypass rule, and the specific bypass rule comprises the following steps:
forming two semicircular detour routes by taking the marking point position as the circle center and taking the preset distance as the radius, verifying the obstacles on the two semicircular detour routes, and selecting the other semicircular route for detour when the obstacle exists in one semicircular detour route; when the two semicircular detour routes have barriers, increasing the preset distance until the detour route is determined; testing the determined detour route, judging whether the reliability value on the detour route is lower than the preset reliability threshold value or not by controlling the robot end to move on the detour route, judging the point position of the robot end at the moment, and detour again according to the detour rule; and the reliability value tested in the whole inspection route and the changed detour route of the robot end is higher than a preset reliability threshold value, so that the communication between the control terminal and the robot end is ensured to be normal and stable.
The optimizing module is used for optimizing the fluency of information transmission in the system;
acquiring the latest routing inspection route of the robot end, sending routing inspection signaling to the robot end through the control terminal, and carrying out routing inspection through the latest routing inspection route after the robot end receives the routing inspection signaling; in the process of inspecting the robot end, acquiring the confidence value of the robot end through a detection module, wherein a specific acquisition method can adopt an intermittent time acquisition method or an intermittent distance acquisition method, so that a group of continuous confidence value groups are obtained, and the confidence values are averaged through an average calculation formula; acquiring a calculated average value, comparing the average value with a preset normal value, and executing optimization operation when the average value is smaller than the preset normal value;
the specific optimization operation comprises the following steps:
firstly, establishing a standby communication channel, wherein the standby communication channel can be any one of a wireless network, satellite communication and a wired network; the existing communication channel is monitored in real time, and once abnormality occurs, signal transmission is directly switched to the standby communication channel, so that a standby communication path is provided for the robot end, and the reliability of communication is improved; adding a signal relay station to the robot terminal inspection area, expanding the signal coverage area, and reducing the blind area and signal interference of the robot terminal signal receiving and transmitting; a spread spectrum encoder and a decoder are arranged for spreading and decoding the control signaling spectrum; the spread spectrum coder converts the control signaling into a high-speed spread spectrum signal, and the decoder restores the received spread spectrum signal into the original control signaling; selecting a corresponding spread spectrum sequence, wherein the spread spectrum sequence has autocorrelation and cross correlation, and specifically comprises a pseudo random noise code sequence or a Gold code sequence so as to ensure that control signaling can be recovered to be normal after spreading; multiplying the original control signaling and the spread spectrum sequence to realize the spectrum spreading of the control signal, and completing the bit-by-bit multiplication between the control signal and the spread spectrum sequence; coding a control signaling transmitted by a control terminal, adding redundancy sinking information, arranging an algorithm for checking and correcting the coding at a robot end, and detecting and repairing errors in the transmission process of the control signaling.
The early warning module is used for early warning of the abnormality of the robot end;
acquiring image and sound data acquired by a robot end, and analyzing and processing the image and sound data;
the analysis for the image data specifically includes the following steps:
firstly, carrying out preliminary pretreatment on the obtained influence data, including noise reduction, image enhancement and size normalization adjustment, and then analyzing the pretreated image data, wherein the specific analysis steps are as follows:
object detection: identifying each object in the image data, and generating a warning signal if an unexpected object appears or an object which should exist is absent; behavior recognition: the motion of the robot is identified through the image data transmitted by the robot end, and the gesture is estimated; judging a specific motion track and a specific behavior mode of the robot according to the motion recognition and the gesture estimation of the normal robot, and generating a warning signaling when the robot deviates from the expected behavior; and (3) image comparison: comparing the image of the current frame in the image data with the previous normal image, checking whether the images have obvious differences, quantifying the differences of the images by using an image feature extraction and matching method, such as structural similarity evaluation or key point matching, and generating a warning signaling when the differences of the images are larger than a preset value; motion detection: identifying a motion area in a picture in the image data by utilizing a motion detection algorithm, and generating a warning signaling when large-area and intense motion occurs in a static background or due moving objects suddenly disappear or motion track is abnormal;
the specific steps of analysis for sound data are as follows:
firstly, analyzing and detecting the volume change amplitude in sound data; when huge noise appears, the change amplitude of the volume is rapidly enlarged, when the change amplitude of the volume exceeds a preset amplitude threshold, the abnormal volume is secondarily judged, the abnormal volume is compared with the change amplitude of the volume generated at the moment when the robot normally patrols and examines, when the difference is lower than a preset value, monitoring is continued, when the difference exceeds the preset value, warning signaling is generated, when the obtained image data and sound data are analyzed, warning operation is executed, and the specific warning operation comprises the following steps:
after receiving the warning signaling, determining that the robot end acquires abnormal data in the data, uploading the abnormal data and the occurring time to the manual inspection terminal for secondary manual inspection, judging whether stopping maintenance is needed, setting waiting time after the uploading of the abnormal data is completed, and sending prompt information to the manual inspection mobile end when feedback is not received after the waiting time is exceeded, wherein the abnormal data can be summarized and the abnormal data can be sent to the manual inspection mobile end, prompting inspection personnel to check the abnormal information in time, and avoiding larger loss; when a manual inspection terminal executes related instructions within a set waiting time range, after controlling a robot terminal to perform corresponding operation, recording the operation, simultaneously automatically controlling the robot terminal to move towards a maintenance endpoint, detecting the current position of the robot terminal, combining the position of the maintenance endpoint, planning a route, controlling the robot terminal to move towards the maintenance endpoint after the route planning is completed, and in the moving process of the robot, searching the mobile equipment of maintenance personnel in a circular area by taking the maintenance endpoint as a circle center and taking a preset distance as a radius, judging whether maintenance personnel exist in the searching area, and when the maintenance personnel exist, sending maintenance prompt to the mobile terminal of the maintenance personnel, wherein specific contents comprise abnormal data and time of the robot, and also comprising specific time points for returning the robot to the maintenance terminal, and making data support for the corresponding maintenance arrangement of the maintenance personnel; when the mobile equipment of the maintenance personnel is not searched, the search area is enlarged, or when the mobile equipment of the maintenance personnel is not searched, prompt information is sent to the terminal of the management personnel, the maintenance personnel is subjected to task arrangement through the management personnel, the robot of the maintenance end point is maintained timely, and the time waste is avoided, so that the operation efficiency of the robot is influenced.
The preferred embodiments of the invention disclosed above are intended only to assist in the explanation of the invention. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.