Detailed Description
The embodiment of the application provides a coal mine safety supervision method and system, which solve the technical problem that the timeliness of safety accident management is poor because the operation behaviors of workers cannot be monitored in real time in the prior art. Through accident correlation analysis, a sensitive operation area and sensitive operation behaviors to be monitored are determined, then any one operator is identified through operation video streaming, and when the sensitive operation area and the sensitive operation behaviors are triggered, early warning is timely carried out, so that the technical effect of improving coal mine safety supervision timeliness is achieved.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.
Embodiment one:
as shown in fig. 1, an embodiment of the present application provides a coal mine safety supervision method, where the method includes:
s10: acquiring a coal mine operation scene for accident correlation analysis, and acquiring a sensitive operation area and sensitive operation behaviors, wherein the sensitive operation behaviors comprise implicit sensitive behaviors and explicit sensitive behaviors;
as shown in fig. 2, step S10 includes the steps of:
s11: collecting first accident record data by taking the coal mine operation scene as a constraint condition, wherein the coal mine operation scene refers to an operation scene at any moment corresponding to any preset coal mine production task, and the first accident record data comprises an accident type, an occurrence area and operation behaviors;
s12: traversing the first accident record data based on the trigger area to cluster, and obtaining a first clustering result of the accident record data;
s13: counting a first area trigger frequency and a second area trigger frequency of a first clustering result of the accident record data until an Mth area trigger frequency;
S14: screening the first area triggering frequency and the second area triggering frequency until the M-th area triggering frequency meets a first triggering frequency threshold value, and setting the areas as the sensitive operation areas;
s15: cleaning the first accident record data only belonging to the accident type of the sensitive operation area, and obtaining second accident record data;
s16: and carrying out behavior association analysis on the second accident record data to acquire the sensitive operation behavior.
Specifically, the coal mine operation scene refers to an operation scene at any moment corresponding to any preset coal mine production task, and is exemplified by a coal mine transportation task in a coal mine transportation area, a power distribution debugging task in a power distribution area, a ventilation equipment debugging task in a ventilation area, and the like. Further, different coal mine operation scenarios may be determined, with associated areas where operators are prohibited from entering, and actions where operators are not allowed, such as: in the conveying scene, an area which is used for prohibiting the operators from entering is arranged below the conveying channel, and the operators are not allowed to act by touching the conveying objects on the conveying channel by hands.
The area where the operators are prohibited from entering is set as a sensitive operation area, the behavior where the operators are not allowed is set as a sensitive operation behavior, further, the implicit sensitive behavior refers to the behavior of the operators with the action amplitude smaller than or equal to the amplitude threshold, and the explicit sensitive behavior refers to the behavior of the operators with the action amplitude larger than the amplitude threshold. The action amplitude refers to the size of a range which can be covered by any action, and the action with larger action amplitude can be distinguished from the action with smaller action amplitude by dividing the action amplitude, because the action with larger action amplitude is usually easy to identify for feature extraction, and is regarded as an explicit sensitive behavior; and the action with smaller action amplitude is difficult to identify, so that the action is regarded as implicit sensitive action. The identification modes of the two are greatly different, so that the two are required to be stored differently for processing.
Furthermore, the sensitive operation area and the sensitive operation behavior can be set by expert personnel in a self-defining way, and can also be determined by data mining through an intelligent algorithm. The embodiment of the application preferably uses an accident-correlation analysis algorithm to determine the sensitive operation area and the sensitive operation behavior, and the detailed flow is as follows:
a first step of: the coal mine safety accident history data set in the corresponding scene is searched based on big data by taking a production area, a production task type and a task execution link as constraint information through a coal mine operation scene, and is stored as first accident record data, wherein any one piece of first accident record data comprises an accident type, an occurrence area and operation behaviors, and the preferable storage mode is as follows:，/>characterizing the ith incident record data, +.>Representing the accident type of the ith accident record data, < >>Characterizing the occurrence area of the ith incident record data, where the occurrence area refers to the area inside the work area, refers to the coverage coordinate area of the incident occurrence, +.>Characterization of the ith AccidentThe job behavior of the data is recorded.
And a second step of: determining a sensitive operation area: when the number of data records collected by the first event record data meets the threshold value of the number of records of the data records, cluster analysis is carried out on the first event record data based on the occurrence areas, namely when the coincidence proportion of the occurrence areas of any two pieces of first event record data is larger than or equal to the threshold value of the coincidence proportion, the occurrence areas of any two pieces of first event record data are regarded as one type, otherwise, the occurrence areas of any two pieces of first event record data are regarded as two types, and the coincidence proportion is preferably calculated by: and calculating the coincidence proportion=coincidence area/(coincidence area+non-coincidence area), and obtaining a first clustering result of the accident record data, wherein any one category of the first clustering result corresponds to a plurality of similar accident occurrence areas, and the first clustering result belongs to one category and is regarded as one area. The first area trigger frequency refers to the number of clustered record data of any one category, and M categories correspond to the first area trigger frequency and the second area trigger frequency until the Mth area trigger frequency. Further, a user preferably sets a first trigger frequency threshold value for deciding whether the trigger frequency is a sensitive operation area by user definition, and the area corresponding to the cluster from the first area trigger frequency to the second area trigger frequency until the M-th area trigger frequency is greater than or equal to the first trigger frequency threshold value is screened to be regarded as the sensitive operation area. And the area with too low frequency is regarded as accidental accident, and the sensitive area is not counted.
And a third step of: determining sensitive operation behaviors: because the sensitive operation area is an area which is forbidden to enter, when entering, the user action which occurs in the sensitive operation area is alarmed, and therefore the user action does not need to be processed, the first accident record data which only belongs to the accident type of the sensitive operation area is cleaned, the second accident record data which does not belong to the operation action of the sensitive operation area is obtained, and the sensitive operation action with higher triggering frequency is extracted.
And through an accident correlation analysis algorithm, data mining is carried out based on the recorded data of the small sample, a relatively objective sensitive operation area and sensitive operation behaviors are rapidly determined, and monitoring reference data is provided for subsequent safety supervision.
Step S16 includes the steps of:
s161: performing primary clustering on the second accident record data according to the accident type to obtain a second clustering result of the accident record data;
s162: traversing the second clustering result of the accident record data according to the operation behavior characteristics to perform secondary clustering, and obtaining a third clustering result of the accident record data;
s163: counting the first accident type triggering frequency and the second accident type triggering frequency of the second clustering result of the accident record data until the Nth accident type triggering frequency, wherein the accident triggering frequency refers to the number of record data strips in the first clustering result of the accident record data;
S164: counting a third class result of the accident record data, and acquiring a first behavior trigger frequency and a second behavior trigger frequency of a first accident type until a Q-th behavior trigger frequency;
s165: counting a third class result of the accident record data, and acquiring a first behavior trigger frequency and a second behavior trigger frequency of an Nth accident type until an L-th behavior trigger frequency;
s166: extracting a first set of behavior trigger frequencies up to an nth set of behavior trigger frequencies for common trigger behaviors of the first event type up to the nth event type based on the first behavior trigger frequency, the second behavior trigger frequency up to the Q-th behavior trigger frequency, and the first behavior trigger frequency, the second behavior trigger frequency up to the L-th behavior trigger frequency;
s167: constructing a first row of data sequence with the first event type trigger frequency and the first set of behavior trigger frequencies;
s168: constructing an nth data sequence according to the nth accident type triggering frequency and the nth group behavior triggering frequency;
s169: constructing a correlation analysis matrix according to the first row data sequence to the N row data sequence to perform gray correlation analysis, and obtaining a correlation analysis result;
S16A: setting the job behavior feature of which the association degree analysis result is greater than or equal to an association degree threshold value as the sensitive job behavior of the corresponding accident type, including,
s16A1: the operation behavior features comprise explicit behavior features and implicit behavior features, wherein the explicit behavior features refer to behavior features with action amplitude larger than an amplitude threshold, and the implicit behavior features refer to behavior features with action amplitude smaller than or equal to the amplitude threshold;
s16A2: when the operation behavior feature belongs to the explicit behavior feature, setting the operation behavior feature as the explicit sensitive behavior of the corresponding accident type;
s16A3: and when the operation behavior characteristic belongs to the implicit behavior characteristic, setting the operation behavior characteristic as the implicit sensitive behavior corresponding to the accident type.
Specifically, a preferred procedure for performing the correlation analysis based on the second incident record data is as follows:
cluster analysis is performed on the second accident record data according to the accident types, namely, the second accident record data of the same accident type are regarded as one type, different accident types are regarded as one type, and the accident types comprise: the types of harmful gas leakage, falling of conveyed objects, equipment wetting and the like, different accident types have unique numbers, second accident record data are obtained, and second accident record data of the same accident type are stored in any one second accident record data.
And traversing the second clustering result of the accident record data according to the operation behavior characteristics to perform secondary clustering, and obtaining a third clustering result of the accident record data, wherein the operation behavior characteristics refer to the operation characteristics of operators in the accident record data storage. Traversing accident record data, aggregating accident record data with consistent operation behaviors of any second aggregation result into one class, aggregating accident record data with inconsistent operation behaviors into two classes, wherein whether the operation behaviors are consistent refers to whether the user action forms are consistent in the same task scene in the same area.
Further, the motion characteristics are preferably characterized by positioning information of each joint of the user, and a certain motion characteristic should be characterized by a section of positioning sequence of each joint; further, the waist joints of the positioning sequences of the joints of any two motion characteristics are overlapped, and the waist joints are overlapped because the activity of the waist joints is small. Comparing the positioning deviation degree of other joints, counting the sum of the positioning differences of all joints at each moment, counting the sum of the positioning differences at a plurality of moments, setting the sum as the deviation coefficient of two action characteristics, and considering the action characteristics as consistent when the deviation coefficient is smaller than or equal to the deviation coefficient threshold value. Any one of the third class of accident record data results stores accident record data with the same accident type and the same action characteristics.
The first accident-type trigger frequency, the second accident-type trigger frequency, and up to the nth accident-type trigger frequency refer to the number of record data pieces in any one of the classes of accident-record data second-class results, each class corresponding to one accident type. The first behavior trigger frequency, the second behavior trigger frequency and the Q-th behavior trigger frequency refer to the number of recorded data of the behavior characteristic clustering result corresponding to the first event type of the third category result of the accident record data; the first behavior trigger frequency and the second behavior trigger frequency up to the L-th behavior trigger frequency refer to the number of record data pieces of the behavior characteristic clustering result corresponding to the N-th accident type of the third class result of the accident record data. Traversing the second accident type triggering frequency in the same way until the N-1 accident type triggering frequency obtains various behavior characteristic triggering frequencies of all accident types.
Further, the common trigger behavior refers to a plurality of action feature information triggered by the first accident type through the nth accident type; according to the counted characteristic trigger frequencies of the N groups of various types of behaviors from the first accident type to the N-th accident type, extracting the trigger frequencies of the common trigger behaviors for storage, and storing the trigger frequencies of the common trigger behaviors extracted from the first action trigger frequency of the first accident type, the second action trigger frequency and the Q-th action trigger frequency as a first group of action trigger frequencies, wherein the preferred storage form of the first group of action trigger frequencies is as follows: (common behavior feature type 1: trigger frequency 1, common behavior feature type 2: trigger frequency 2, common behavior feature type 3: trigger frequency 3, …,. Common behavior feature type Z: trigger frequency Z), Z is the total number of common trigger behaviors. Based on the same principle, traversing the second accident type until the Nth accident type is counted, and acquiring a second group of behavior trigger frequencies until the Nth group of behavior trigger frequencies.
Further, a first line data sequence is constructed according to a first event type trigger frequency and a first group of action trigger frequencies, wherein in the first line data sequence, the first event type trigger frequency is a first column attribute, and various action features in the first group of action trigger frequencies are sequentially a second column attribute to a Z column attribute; and processing the second group of behavior trigger frequencies until the Nth group of behavior trigger frequencies in the same way to obtain a second data sequence until the Nth data sequence.
Further, based on the first row data sequence up to the nth row data sequence, a correlation analysis matrix is constructed as follows:wherein->The accident triggering frequency from the first row of data sequence to the nth row of data sequence is represented, and the rest is the characteristic triggering frequency of various types of behaviors.
And (3) taking the common trigger behavior and the first accident type as constraint information until the Nth accident type, repeating g times of acquisition of accident record data to construct g correlation analysis matrixes, and finishing the g correlation analysis matrixes to obtain the following steps:wherein->A correlation analysis matrix characterizing an nth incident type,the accident trigger frequency of g relevance analysis matrix records representing the nth accident type, and the rest are the common trigger behaviors recorded in the g relevance analysis matrix records And the trigger frequency of the (C) is greater than or equal to N and greater than or equal to 1, and N is an integer.
Further, gray correlation analysis is performed based on the correlation analysis matrix, and the gray correlation analysis formula is as follows:
，
wherein,representing the association coefficient of elements in the ith column and the kth row, wherein i is more than or equal to 2 and less than or equal to Z+1, i is an integer, k is more than or equal to 1 and less than or equal to g, and k is an integer; />Characterization Accident trigger frequency using any row minus absolute value of behavior trigger frequency, +.>Characterization pass->Minimum in g rows after treatment, < >>Characterization pass->Maximum in g rows after treatment, +.>Default to 0.5 for resolution;
and processing the g relevance analysis matrixes based on a gray relevance analysis formula, obtaining a plurality of relevance coefficients of any common trigger behavior of any accident type, solving a mean value, and setting the mean value as a relevance analysis result of the corresponding trigger behavior. And setting the job behavior characteristics of which the correlation analysis result is greater than or equal to a correlation threshold as the sensitive job behaviors corresponding to the accident types, wherein the correlation threshold refers to a preset sensitive behavior decision correlation.
Further, any one action feature has an action amplitude, the action feature with the action amplitude larger than the amplitude threshold is set as an explicit action feature, and the action feature with the action amplitude smaller than or equal to the amplitude threshold is set as an implicit action feature; when the operation behavior characteristics of the sensitive operation behaviors belong to the explicit behavior characteristics, setting the operation behavior characteristics as the explicit sensitive behaviors corresponding to the accident types; when the operation behavior characteristics of the sensitive operation behaviors belong to the implicit behavior characteristics, setting the implicit sensitive operation behaviors corresponding to the accident types. By determining the sensitive operation behavior in the above manner, a data basis is provided for analysis of the abnormal behavior in the later step.
As shown in fig. 3, the embodiment of the present application includes:
S16B1: extracting a first set of independent behavior trigger frequencies up to an nth set of independent behavior trigger frequencies for unique trigger behaviors of the first event type up to the nth event type based on the first behavior trigger frequency, the second behavior trigger frequency up to the Q-th behavior trigger frequency, and the first behavior trigger frequency, the second behavior trigger frequency up to the L-th behavior trigger frequency;
S16B2: setting the behavior characteristics of the first group of independent behaviors with the triggering frequency larger than or equal to a second triggering frequency threshold as a first accident type sensitive operation behavior;
S16B3: setting the behavior characteristics of the N independent behavior trigger frequency which is greater than or equal to a second trigger frequency threshold as N accident type sensitive operation behaviors;
S16B4: adding the first accident-type sensitive job behavior to the nth accident-type sensitive job behavior into the sensitive job behavior.
Specifically, any one accident type has, in addition to the common trigger behavior, a unique trigger behavior, and a first set of independent behavior trigger frequencies of the first accident type up to the unique trigger behavior of the nth accident type are extracted up to the nth set of independent behavior trigger frequencies; setting the behavior characteristics of the first group of independent behaviors with the triggering frequency larger than or equal to a second unique triggering frequency threshold as the first accident type sensitive operation behaviors, wherein the second triggering frequency threshold is a preset triggering frequency threshold; repeating the processing, and acquiring the first accident-type sensitive operation behaviors until the Nth accident-type sensitive operation behaviors. Further, the implicit sensitive behavior and the explicit sensitive behavior are respectively classified for the first accident type sensitive operation behavior and the nth accident type sensitive operation behavior, so that the final sensitive operation behavior is obtained. The integrity of the sensitive behavior is ensured through the analysis of the frequency of the unique triggering behavior.
S20: when a first coal mine area triggers the coal mine operation scene, acquiring an operation video stream of a first operator through an image acquisition device;
specifically, the first coal mine area refers to any one coal mine area, the coal mine operation scene refers to the scene which is subjected to sensitive behavior and sensitive area analysis, when the first coal mine area is subjected to coal mine operation scene, an operation video stream of a first operator is acquired through the image acquisition device, the first operator refers to any one operator of the coal mine operation scene, and the operation video stream refers to operation monitoring video information acquired in real time by the image acquisition device. And by collecting the operation video stream, the sensitive operation area and the sensitive operation behavior can be conveniently identified and analyzed in the later step.
S30: positioning analysis is carried out on the first operator according to the operation video stream, and positioning information of the operator is obtained;
step S30 includes the steps of:
s31: acquiring the working clothes number information of a first worker, wherein the working clothes number information is at least deployed in any three areas of a helmet, arms, chest front, shoulders and back;
s32: acquiring a first frame image of the operation video stream until an O frame image;
S33: acquiring a plurality of pieces of worker positioning information of the first frame image, traversing the plurality of pieces of worker positioning information to carry out number identification, and acquiring a work clothes number identification result;
s34: comparing the work clothes number information with the work clothes number identification result to obtain first frame positioning information;
s35: repeating the analysis until the O frame image, fusing the first frame positioning information of the first operator until the O frame positioning information, and acquiring the operator positioning information;
s36: wherein the step of traversing the plurality of worker positioning information to carry out number identification, obtaining the number identification result of the work clothes comprises,
s361: collecting working clothes image sets in multiple postures, numbering and identifying the working clothes image sets, and obtaining working clothes numbering and identifying information;
s362: carrying out text region identification on one half of the working clothes image set to obtain a text region identification result;
s363: taking the working clothes image set as input data, and taking the character area identification result as supervision data to perform semi-supervision training to train a character identification layer, wherein the character identification layer is used for performing character area identification on the working clothes image set;
S364: taking the character area identification result as input data, taking the work clothes number identification information as supervision data, performing full supervision training, and training a character recognition layer;
s365: and fully connecting the output node of the character identification layer with the input node of the character identification layer to obtain a number identification model, traversing the positioning information of the plurality of operators to carry out number identification, and obtaining the number identification result of the working clothes.
Specifically, the worker positioning information refers to information representing the position of the first worker, and the flow of positioning analysis by identifying the first worker from the worker video stream is as follows:
the method comprises the steps of acquiring working clothes number information of a first operator, wherein the working clothes number information is at least deployed in any three areas of a helmet, arms, chest, shoulders and back, any one operator has unique working clothes number information, and the accuracy of image acquisition can be ensured by deploying in any three areas of the helmet, arms, chest, shoulders and back. And decomposing the operation video stream frame by frame to obtain first frame images to O frame images which are arranged according to the time sequence. Further, a plurality of operator positioning information of the first frame image is collected, a plurality of operator positioning information is traversed to carry out number recognition, a working clothes number recognition result is obtained, the working clothes number recognition result refers to a result obtained by carrying out number recognition on images of a plurality of operators based on the first frame image until an O-th frame image, the operator identity information and the operator positioning information can be combined through the working clothes number recognition result, real-time early warning is carried out after the step, and the plurality of operator positioning information can be provided through portable equipment worn by each operator.
Further, the number identification process is as follows:
collecting a plurality of working clothes image sets, numbering and identifying the working clothes image sets, and obtaining working clothes numbering and identifying information, wherein the working clothes numbering and identifying information refers to the identified numbering data; and carrying out character area identification on one half of the working clothes image set to obtain character area identification results, wherein the character area identification results refer to numerical identification of character areas of the working clothes image, so that the computer identification is facilitated. Only half of the reason for marking is that semi-supervised training is required, so only half of the data is marked.
Further, the working clothes image set is used as input data, the character area identification result is used as supervision data, semi-supervision training is carried out based on a convolutional neural network, and a character identification layer is trained, wherein the character identification layer is used for carrying out character area identification on the working clothes image set; and taking the character region identification result as input data, taking the working clothes number identification information as supervision data, and performing full supervision training based on a convolutional neural network to train a character recognition layer. Because the intelligent model training and constructing process is already a mature technology, a mature gradient descent training method is preferably adopted, and the description is omitted.
And fully connecting the output node of the character identification layer with the input node of the character identification layer, acquiring a number identification model, traversing the plurality of pieces of operator positioning information to carry out number identification, acquiring the number identification result of the working clothes, wherein the number identification model can be used for carrying out number identification on image data of the plurality of pieces of operator positioning information, comparing the number identification result of the working clothes with the number identification result of the plurality of pieces of working clothes to obtain first frame positioning information of a first operator of a first frame image, further repeatedly analyzing until the O frame image, fusing the first frame positioning information of the first operator until the O frame positioning information, and thus obtaining the operator positioning information representing a positioning change sequence of the first operator. The personnel positioning information and the numbers can be matched one by one through the number identification, and the real-time positioning data of staff with any number can be determined. And the early warning of abnormal operation areas is convenient.
S40: when the worker positioning information belongs to the sensitive operation area, generating a first supervision identification of the first worker based on the sensitive operation area;
s50: performing action recognition on the first operator according to the operation video stream to acquire an explicit operation behavior and an implicit operation behavior;
Step S50 includes the steps of:
s51: acquiring joint number information of the first operator, and extracting a joint positioning sequence of the joint number information based on the operation video stream;
s52: determining a joint point rotation amplitude change curve according to the joint positioning sequence;
s53: determining a maximum rotation amplitude value according to the rotation amplitude change curve of the joint point;
s54: when the maximum value of the rotation amplitude is larger than or equal to an amplitude threshold value, adding actions corresponding to the joint numbers into the explicit operation behaviors;
s55: and when the maximum value of the rotation amplitude is smaller than the amplitude threshold value, adding the action corresponding to the joint number into the implicit operation behavior.
Specifically, the explicit operation behavior refers to an operation feature belonging to an explicit operation for performing operation recognition on the first operator according to an operation video stream, and the implicit operation behavior refers to an operation feature belonging to an implicit operation for performing operation recognition on the first operator according to an operation video stream. The detailed determination flow is as follows:
the joint number information refers to the body joint number of the operator, and one joint number of the operator corresponds to one joint only. The joint positioning sequence refers to a positioning change sequence of joint number information extracted based on a job video stream. The articulation point rotation amplitude variation curve refers to an articulation point rotation amplitude variation curve, the rotation amplitude maximum value refers to the maximum angle of rotation of the articulation point rotation amplitude variation curve in a certain direction, when the rotation amplitude maximum value is greater than or equal to an amplitude threshold value, actions corresponding to the articulation numbers are added into explicit operation behaviors, and when the rotation amplitude maximum value is less than the amplitude threshold value, actions corresponding to the articulation numbers are added into implicit operation behaviors.
S60: generating a second supervision identification of the first operator based on the explicit sensitive behavior when the operator positioning information does not belong to the sensitive operation area and when the explicit operation behavior belongs to the explicit sensitive behavior;
s70: when the explicit operation behavior does not belong to the explicit sensitive behavior and the implicit operation behavior belongs to the implicit sensitive behavior, generating a third supervision identification of the first operator based on the implicit sensitive behavior;
s80: and generating early warning information according to the first supervision identifier or the second supervision identifier or the third supervision identifier, and sending the early warning information to the portable user side of the first operator.
Specifically, when the worker positioning information belongs to a sensitive operation area, generating a first supervision identification representing abnormality of the first worker positioning area based on the sensitive operation area; generating a second supervision identification representing a larger-amplitude behavior abnormality of the first operator based on the explicit sensitive behavior when the worker positioning information does not belong to the sensitive operation area and the explicit operation behavior belongs to the explicit sensitive behavior; when the explicit operation behavior does not belong to the explicit sensitive behavior and the implicit operation behavior belongs to the implicit sensitive behavior, generating a third supervision identification representing small-amplitude action abnormality of the first operator based on the implicit sensitive behavior. Generating real-time early warning information according to the first supervision identification or the second supervision identification or the third supervision identification, and sending the real-time early warning information to a portable user side of a first operator for safety early warning, and warning to stop corresponding abnormal operation behaviors and forbid entering an abnormal area.
The embodiment of the application further comprises:
s91: acquiring a first joint number positioning sequence of the explicit operation behavior;
s92: acquiring a second joint numbering and positioning sequence of the explicit sensitive behavior;
s93: constructing an action deviation coefficient analysis formula:wherein S characterizes the degree of deviation of actions of explicit job behavior and explicit sensitive behavior, +.>The positioning distance of the jth frame image of the jth joint number for representing the explicit sensitive behavior and the explicit operation behavior, T represents the total number of explicit joints, J represents the total number of image frames,/->Distance threshold characterizing the t-th joint number, < >>Characterizing infinity;
s94: overlapping and positioning a first characteristic region of an operator of any frame of the operation video stream and the sample video stream of the explicit sensitive behavior, and analyzing the degree of motion deviation based on the motion deviation coefficient analysis formula to obtain a motion deviation coefficient, wherein the first characteristic region refers to a hip region and a waist region;
s95: when the action deviation coefficient is smaller than or equal to the action deviation coefficient threshold value, the explicit operation behavior is considered to belong to the explicit sensitive behavior;
s96: and when the action deviation coefficient is larger than the action deviation coefficient threshold, the action deviation coefficient is regarded as that the explicit operation behavior does not belong to the explicit sensitive behavior.
Specifically, the method for determining whether the action features are consistent has been briefly described in the above embodiments, and is described in quantization herein by taking explicit job behavior and explicit sensitive behavior consistency determination as an example:
based on the above knowledge, the behavior features are feature data of the stored joint action sequence, a first joint number positioning sequence of the explicit operation behavior is obtained, a second joint number positioning sequence of any explicit sensitive behavior is obtained, and an action deviation coefficient analysis formula is constructed:wherein S characterizes the degree of deviation of actions of explicit job behavior and explicit sensitive behavior, +.>The positioning distance of the jth frame image of the jth joint number for representing the explicit sensitive behavior and the explicit operation behavior, T represents the total number of explicit joints, J represents the total number of image frames,/->Distance threshold characterizing the t-th joint number, < >>Characterizing infinity; and carrying out superposition positioning on a first characteristic region of an operator of any frame of a working video stream and a sample video stream of explicit sensitive behaviors, and carrying out motion deviation degree analysis based on the motion deviation coefficient analysis formula to obtain a motion deviation coefficient, wherein the preferable first characteristic region refers to a hip region and a waist region because joints of the hip region and the waist region are fewer, and the sample video stream refers to video data corresponding to the explicit sensitive behaviors. Obtaining motion deviation coefficients of the explicit operation behavior and the explicit sensitive behavior, and considering the explicit operation behavior and the explicit sensitive behavior as the motion deviation coefficients are smaller than or equal to the motion deviation coefficient threshold value The operation behavior belongs to the explicit sensitive behavior, the action deviation coefficient threshold is a preset deviation coefficient threshold, preferably, the manager carries out user-defined setting, the larger the deviation coefficient threshold is, the larger the fault tolerance deviation is, and when the action deviation coefficient is larger than the action deviation coefficient threshold, the explicit operation behavior is regarded as not belonging to the explicit sensitive behavior.
In summary, the coal mine safety supervision method and system provided by the embodiment of the application have the following technical effects:
1. the embodiment of the application provides a coal mine safety supervision method and system, which solve the technical problem that the timeliness of safety accident management is poor because the operation behaviors of workers cannot be monitored in real time in the prior art. Through accident correlation analysis, a sensitive operation area and sensitive operation behaviors to be monitored are determined, then any one operator is identified through operation video streaming, and when the sensitive operation area and the sensitive operation behaviors are triggered, early warning is timely carried out, so that the technical effect of improving coal mine safety supervision timeliness is achieved.
Embodiment two:
based on the same inventive concept as one coal mine safety supervision method in the foregoing embodiments, as shown in fig. 4, an embodiment of the present application provides a coal mine safety supervision system, where the system includes:
The accident association analysis module 100 is used for acquiring a coal mine operation scene to perform accident association analysis, and acquiring a sensitive operation area and sensitive operation behaviors, wherein the sensitive operation behaviors comprise implicit sensitive behaviors and explicit sensitive behaviors;
the image acquisition module 200 is used for acquiring an operation video stream of a first operator through the image acquisition device when the first coal mine area triggers the coal mine operation scene;
the positioning analysis module 300 is configured to perform positioning analysis on the first operator according to the operation video stream, and obtain positioning information of the operator;
a first supervision identification module 400, configured to generate a first supervision identification of the first operator based on the sensitive work area when the operator positioning information belongs to the sensitive work area;
the action recognition module 500 is configured to perform action recognition on the first operator according to the job video stream, and obtain an explicit job behavior and an implicit job behavior;
a second supervision identification module 600, configured to generate a second supervision identification of the first operator based on the explicit sensitive behavior when the operator positioning information does not belong to the sensitive operation area and when the explicit operation behavior belongs to the explicit sensitive behavior;
A third supervision identification module 700, configured to generate a third supervision identification of the first operator based on the implicit sensitive behavior when the explicit operation behavior does not belong to the explicit sensitive behavior and the implicit operation behavior belongs to the implicit sensitive behavior;
the early warning information sending module 800 is configured to generate early warning information according to the first supervision identifier, the second supervision identifier, or the third supervision identifier, and send the early warning information to the portable user side of the first operator.
Further, the accident association analysis module 100 is configured to perform the following steps:
collecting first accident record data by taking the coal mine operation scene as a constraint condition, wherein the coal mine operation scene refers to an operation scene at any moment corresponding to any preset coal mine production task, and the first accident record data comprises an accident type, an occurrence area and operation behaviors;
traversing the first accident record data based on the trigger area to cluster, and obtaining a first clustering result of the accident record data;
counting a first area trigger frequency and a second area trigger frequency of a first clustering result of the accident record data until an Mth area trigger frequency;
Screening the first area triggering frequency and the second area triggering frequency until the M-th area triggering frequency meets a first triggering frequency threshold value, and setting the areas as the sensitive operation areas;
cleaning the first accident record data only belonging to the accident type of the sensitive operation area, and obtaining second accident record data;
and carrying out behavior association analysis on the second accident record data to acquire the sensitive operation behavior.
Further, the accident association analysis module 100 is configured to perform the following steps:
performing primary clustering on the second accident record data according to the accident type to obtain a second clustering result of the accident record data;
traversing the second clustering result of the accident record data according to the operation behavior characteristics to perform secondary clustering, and obtaining a third clustering result of the accident record data;
counting the first accident type triggering frequency and the second accident type triggering frequency of the second clustering result of the accident record data until the Nth accident type triggering frequency, wherein the accident triggering frequency refers to the number of record data strips in the first clustering result of the accident record data;
counting a third class result of the accident record data, and acquiring a first behavior trigger frequency and a second behavior trigger frequency of a first accident type until a Q-th behavior trigger frequency;
Counting a third class result of the accident record data, and acquiring a first behavior trigger frequency and a second behavior trigger frequency of an Nth accident type until an L-th behavior trigger frequency;
extracting a first set of behavior trigger frequencies up to an nth set of behavior trigger frequencies for common trigger behaviors of the first event type up to the nth event type based on the first behavior trigger frequency, the second behavior trigger frequency up to the Q-th behavior trigger frequency, and the first behavior trigger frequency, the second behavior trigger frequency up to the L-th behavior trigger frequency;
constructing a first row of data sequence with the first event type trigger frequency and the first set of behavior trigger frequencies;
constructing an nth data sequence according to the nth accident type triggering frequency and the nth group behavior triggering frequency;
constructing a correlation analysis matrix according to the first row data sequence to the N row data sequence to perform gray correlation analysis, and obtaining a correlation analysis result;
setting the job behavior feature of which the association degree analysis result is greater than or equal to an association degree threshold value as the sensitive job behavior of the corresponding accident type, including,
The operation behavior features comprise explicit behavior features and implicit behavior features, wherein the explicit behavior features refer to behavior features with action amplitude larger than an amplitude threshold, and the implicit behavior features refer to behavior features with action amplitude smaller than or equal to the amplitude threshold;
when the operation behavior feature belongs to the explicit behavior feature, setting the operation behavior feature as the explicit sensitive behavior of the corresponding accident type;
and when the operation behavior characteristic belongs to the implicit behavior characteristic, setting the operation behavior characteristic as the implicit sensitive behavior corresponding to the accident type.
Further, the coal mine safety supervision system is further configured to perform the following steps:
extracting a first set of independent behavior trigger frequencies up to an nth set of independent behavior trigger frequencies for unique trigger behaviors of the first event type up to the nth event type based on the first behavior trigger frequency, the second behavior trigger frequency up to the Q-th behavior trigger frequency, and the first behavior trigger frequency, the second behavior trigger frequency up to the L-th behavior trigger frequency;
setting the behavior characteristics of the first group of independent behaviors with the triggering frequency larger than or equal to a second triggering frequency threshold as a first accident type sensitive operation behavior;
Setting the behavior characteristics of the N independent behavior trigger frequency which is greater than or equal to a second trigger frequency threshold as N accident type sensitive operation behaviors;
adding the first accident-type sensitive job behavior to the nth accident-type sensitive job behavior into the sensitive job behavior.
Further, the positioning analysis module 300 is configured to perform the following steps:
acquiring the working clothes number information of a first worker, wherein the working clothes number information is at least deployed in any three areas of a helmet, arms, chest front, shoulders and back;
acquiring a first frame image of the operation video stream until an O frame image;
acquiring a plurality of pieces of worker positioning information of the first frame image, traversing the plurality of pieces of worker positioning information to carry out number identification, and acquiring a work clothes number identification result;
comparing the work clothes number information with the work clothes number identification result to obtain first frame positioning information;
repeating the analysis until the O frame image, fusing the first frame positioning information of the first operator until the O frame positioning information, and acquiring the operator positioning information;
wherein the step of traversing the plurality of worker positioning information to carry out number identification, obtaining the number identification result of the work clothes comprises,
Collecting working clothes image sets in multiple postures, numbering and identifying the working clothes image sets, and obtaining working clothes numbering and identifying information;
carrying out text region identification on one half of the working clothes image set to obtain a text region identification result;
taking the working clothes image set as input data, and taking the character area identification result as supervision data to perform semi-supervision training to train a character identification layer, wherein the character identification layer is used for performing character area identification on the working clothes image set;
taking the character area identification result as input data, taking the work clothes number identification information as supervision data, performing full supervision training, and training a character recognition layer;
and fully connecting the output node of the character identification layer with the input node of the character identification layer to obtain a number identification model, traversing the positioning information of the plurality of operators to carry out number identification, and obtaining the number identification result of the working clothes.
Further, the action recognition module 500 is configured to perform the following steps:
acquiring joint number information of the first operator, and extracting a joint positioning sequence of the joint number information based on the operation video stream;
Determining a joint point rotation amplitude change curve according to the joint positioning sequence;
determining a maximum rotation amplitude value according to the rotation amplitude change curve of the joint point;
when the maximum value of the rotation amplitude is larger than or equal to an amplitude threshold value, adding actions corresponding to the joint numbers into the explicit operation behaviors;
and when the maximum value of the rotation amplitude is smaller than the amplitude threshold value, adding the action corresponding to the joint number into the implicit operation behavior.
Further, the coal mine safety supervision system is further configured to perform the following steps:
acquiring a first joint number positioning sequence of the explicit operation behavior;
acquiring a second joint numbering and positioning sequence of the explicit sensitive behavior;
constructing an action deviation coefficient analysis formula:wherein S characterizes the degree of deviation of actions of explicit job behavior and explicit sensitive behavior, +.>The positioning distance of the jth frame image of the jth joint number for representing the explicit sensitive behavior and the explicit operation behavior, T represents the total number of explicit joints, J represents the total number of image frames,/->Distance threshold characterizing the t-th joint number, < >>Characterizing infinity;
overlapping and positioning a first characteristic region of an operator of any frame of the operation video stream and the sample video stream of the explicit sensitive behavior, and analyzing the degree of motion deviation based on the motion deviation coefficient analysis formula to obtain a motion deviation coefficient, wherein the first characteristic region refers to a hip region and a waist region;
When the action deviation coefficient is smaller than or equal to the action deviation coefficient threshold value, the explicit operation behavior is considered to belong to the explicit sensitive behavior;
and when the action deviation coefficient is larger than the action deviation coefficient threshold, the action deviation coefficient is regarded as that the explicit operation behavior does not belong to the explicit sensitive behavior.
Any of the steps of the methods described above may be stored as computer instructions or programs in a non-limiting computer memory and may be called by a non-limiting computer processor to identify any of the methods to implement embodiments of the present application, without unnecessary limitations.
Further, the first or second element may not only represent a sequential relationship, but may also represent a particular concept, and/or may be selected individually or in whole among a plurality of elements. It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the present application and the equivalents thereof, the present application is intended to cover such modifications and variations.