System and method for identifying performance of manual assembly task based on wearable equipmentTechnical Field
The invention relates to a performance identification system, in particular to a performance identification system for a manual assembly task based on a wearable device.
Background
With the development of social economy, the number of large-scale production assembly enterprises is increasing day by day, and the population of production line assembly operators is expanding day by day. As an important member of a production assembly system, assembly workers are often affected by field fatigue at the workplace. Studies have shown that individuals experience a decline in cognitive ability and manual performance in fatigue states, including stability during movement, flexibility of the fingers and fingertips, and the like. The yield and quality of the product are also affected by the physiological fatigue of the operators, and there is evidence that 20-40% of quality problems are caused by human error in operations requiring delicate manual work.
For companies with a large number of employees, work efficiency reduction, product quality reduction, lack of duty, and the like due to physiological fatigue are not negligible. In order to improve the current situation, besides improving the management mode and the management standard of the production field and strengthening the training of technical workers, the realization of real-time monitoring on the assembly performance through scientific and technological means has become a consensus of production enterprises and engineering technology.
In recent years, various attempts and researches are made for realizing real-time supervision on assembly performance, but in the past, radio frequency identification technologies such as RFID are generally adopted to label materials, and after an assembly product passes through a quality inspection work station, assembly performance data are synchronized to the cloud end so as to be fed back. This feedback mode lacks attention from the assembler himself and is slow. How to realize the real-time evaluation of the assembly performance of the production line becomes a technical problem to be solved urgently at present.
Disclosure of Invention
The invention aims to provide an identification system capable of predicting the assembly efficiency of an assembly worker in real time, which takes physiological signals of the assembly worker as input to predict the assembly efficiency.
The technical scheme of the invention is as follows.
The invention provides a manual assembly task performance identification system based on a wearable device in a first aspect, which comprises a display terminal, the wearable device and a computing terminal, wherein the display terminal, the wearable device and the computing terminal are arranged in the system
The display terminal comprises an interaction device used for receiving operation input and feeding back an evaluation result;
the computing terminal is used for filtering and extracting statistical characteristics of signal data reflecting the assembling operation state of an operator, and monitoring and evaluating the signal data; the monitoring and evaluating process uses at least one machine learning method and a network learning model;
the wearable device is used for being worn on the body of an operator during assembly operation, establishing data connection with the display terminal and the computing terminal, acquiring signal data, sending the signal data to the computing terminal, receiving a detection evaluation result of the computing terminal, and feeding the result back to the operator through the display terminal.
Preferably, the signal data acquired by the wearable device comprises an arm muscle surface electromyographic signal of the operator.
Preferably, the signal data further includes an arm movement acceleration signal, an arm movement angular velocity signal, and an arm rotation angle signal.
A second aspect of the present invention provides a performance evaluation method using the performance recognition system for a manual assembly task according to any one of the first aspects of the present invention, including the steps of:
step S1, inputting operator information and evaluation algorithm type through an interactive device of the display terminal, and collecting signal data reflecting the assembling operation state of the operator through the wearable equipment;
step S2, transmitting the signal data to a computing terminal;
step S3, filtering and extracting statistical characteristics of the signal data, monitoring and evaluating, and sending the evaluation result to a display terminal; the monitoring and evaluating process uses at least one machine learning method and a network learning model according to the input evaluation algorithm type;
and step S4, displaying corresponding performance indexes according to the evaluation result through an interactive device.
Preferably, the process of filtering and extracting statistical features from the signal data in step S3 includes the following steps:
s3.1, extracting signal data of a specific assembly task according to the starting time and the ending time of each assembly task;
s3.2, performing noise reduction smoothing processing on the myoelectric signals of the surfaces of the muscles of the arms by using a first filter, and performing noise reduction smoothing processing on the acceleration signals of the movement of the arms and the angular velocity signals of the arms by using a second filter;
s3.3, dividing the processed arm muscle surface electromyographic signals by utilizing sliding time windows, and calculating the mean value and the root mean square of the signals in each time window to obtain an arm muscle surface electromyographic signal mean value time sequence and an arm muscle surface electromyographic signal root mean square time sequence;
step S3.4, dividing the processed arm motion acceleration signals, arm angular velocity signals and arm corner signals by utilizing a sliding time window, and calculating the mean value and the root mean square of the signals in each time window, thereby obtaining two groups of time sequences for each signal, namely an arm motion acceleration signal mean value time sequence, an arm motion acceleration signal root mean square time sequence, an arm angular velocity signal mean value time sequence, an arm angular velocity signal root mean square time sequence, an arm corner signal mean value time sequence and an arm corner signal root mean square time sequence;
and S3.5, extracting the parameter distribution in the description period of the description statistic of each time sequence.
Preferably, the first filter in step S3.2 is a fourth order Butterworth filter of 30Hz, and the second filter is a third order median filter.
Preferably, the sliding time window in step S3.3 has a length of 0.25S and the overlap length of 0.075S.
Preferably, the sliding time window in step S3.4 has a length of 0.3S and the overlap length of 0.1S.
Preferably, the description statistics in step S3.5 include: mean, variance, median, mode, kurtosis, and skewness.
The third aspect of the present invention provides a training method for a network model in the performance recognition system based on the hand-assembly task of the wearable device according to any one of the first aspect of the present invention, comprising the following steps:
step S01, constructing an ASSEMBLY data set ASSEMBLY by using the operator information and the data collected by the wearable device in the step S1;
step S02, labeling all data of the data set ASSEMBLY partially to form an ASSEMBLY _ Output of the performance judged manually; meanwhile, forming a data set ASSEMBLY _ Input for the wearable device to acquire the physiological signal characteristics;
step S03, randomly splitting data in the data Set ASSEMBLY, wherein 70% of the data form a training data Set Train _ Set, and the other 30% of the data form a testing data Set Test _ Set;
step S04, training by using two machine learning methods GBDT and LDA and a RNN deep learning network model through a training data Set Train _ Set to obtain models GBDT _ trained, LDA _ trained and RNN _ trained;
step S05, obtaining an assessment data Output Set ASSEMBLY _ Trained _ Output after classifying the Input of ASSEMBLY _ Input and GBDT _ trailing, LDA _ trailing and RNN _ trailing models in the training data Set Trained _ Set;
step S06, using errors of ASSEMBLY _ TRAINED _ OUTPUT in the performance evaluation data Output Set ASSEMBLY _ TRAINED _ OUTPUT and the ASSEMBLY _ OUTPUT in the training data Set Train _ Set to enhance GBDT _ trailing, LDA _ trailing and RNN _ trailing, and obtaining enhanced machine learning models GBDT _ inproved, LDA _ inproved and RNN _ inproved;
step S07, testing the performance of the enhanced machine learning models GBDT _ improved, LDA _ improved and RNN _ improved by using a Test data Set Test _ Set, and finishing training if the Test result is qualified; if not, the parameters of the three network models are adjusted, and the process is executed again from step S03.
Through the technical scheme, the invention can obtain the following beneficial effects.
The work activity information of an operator is collected by using wearable equipment (including sensor equipment configured on a workstation), and is further identified and recorded through a machine learning algorithm and communication equipment, so that the physiological fatigue in a workplace is identified in real time and is intervened.
The invention compares 6 types of algorithm models under three assembly scenes in a production line, screens out three machine learning and deep learning network models with the highest prediction accuracy, and compared with other machine learning models, the model used in the invention has the advantages that: the prediction accuracy of LDA machine learning in a stability task station in a production assembly line is up to more than 90%, and the prediction accuracy of other machine learning models is only about 80%; the prediction accuracy of the RNN deep learning model in a flexible task station in a production assembly line is up to more than 88%, and the prediction accuracy of other machine learning models is only about 70%; the prediction accuracy of the GBDT machine learning model in a screw assembly station in a production assembly line is up to more than 99%, and the prediction accuracy of other machine learning models is only about 87%.
Drawings
FIG. 1 is a partial block diagram of embodiment 1 of the present invention;
FIG. 2 is a flowchart of embodiment 2 of the present invention;
FIG. 3 is a flow chart of a signal filtering and statistical feature extraction method according to embodiment 3 of the present invention;
fig. 4 is a flowchart of a deep learning network model building method according to embodiment 3 of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for purposes of illustration and explanation only and are not intended to limit the present invention.
Example 1
The embodiment provides a wearable manual assembly task performance identification system which comprises a display terminal, wearable equipment and a computing terminal, wherein the wearable equipment is connected with the computing terminal and the display terminal through Bluetooth.
As shown in fig. 1, the display terminal includes an interactive device for accepting an operation input and feeding back an evaluation result. The interface of the interaction device comprises a worker basic information acquisition module, a real-time data display module, a classification algorithm selection module and an evaluation result feedback module.
The computing terminal comprises a computer provided with performance recognition software and is used for filtering and extracting statistical characteristics of signal data reflecting the assembling operation state of an operator, monitoring and evaluating; the process of monitoring and evaluating uses at least one machine learning method and a network learning model. The performance identification software comprises a signal filtering module, a statistical feature extraction module and a monitoring and evaluating module, wherein the monitoring and evaluating module comprises a machine learning and deep learning network model. The output end of the signal filtering module is connected with the input end of the statistical characteristic extraction module, and the output end of the statistical characteristic extraction module is connected with the input end of the monitoring evaluation module.
The wearable device is used for being worn on the body of an operator during assembly operation, is in data connection with the display terminal and the computing terminal, can collect signal data, sends the signal data to the computing terminal, receives a detection evaluation result of the computing terminal, and feeds the detection evaluation result back to the operator through the display terminal. The wearable device is connected with the input end of the real-time data display module of the display terminal through data connection on one hand, and is connected with the input end of the signal filtering module of the computing terminal through data connection on the other hand.
In a preferred embodiment, the computing terminal is implemented by a desktop computer with a bluetooth function and a Matlab platform, the display terminal is a touch display with a UI graphical interface based on the Matlab platform, and the wearable device is a Myo bracelet. Those skilled in the art will appreciate that any suitable implementation known in the art may be used with the present invention, and may be readily changed by the user if desired. In particular, the wearable device for the present invention must contain metal electrodes capable of acquiring surface myoelectricity and IMU sensors capable of acquiring arm movement signals.
Example 2
The present embodiment provides an assembly performance evaluation method. The method of the present embodiment is implemented by the system of embodiment 1. As shown in fig. 2, the method of the present embodiment includes the following steps performed in sequence.
And step S1, data acquisition. Basic information such as the age, the sex, the height and the weight of an assembly worker is acquired through a worker basic information acquisition module of an interaction device of the display terminal, and surface myoelectricity, arm movement triaxial acceleration, arm movement triaxial angular velocity data, arm movement triaxial angle and the like of the assembly worker are acquired in real time through wearable equipment and transmitted to the computing terminal and the display terminal. A user selects a specific evaluation algorithm type from the three machine learning and deep learning network models by using a classification algorithm selection module.
Step S2, data transmission. And sending the data collected in the step S1 to a signal filtering module included in the computing terminal by means of data transmission such as bluetooth.
Step S3, data processing. The data processing process in the step comprises the following three aspects: the signal filtering module carries out noise reduction smoothing processing on the received data, the statistical characteristic extraction module carries out segmentation processing on the smooth data passing through the signal filtering module into two groups of time sequences by using a sliding time window method, and then the statistical characteristics of the two groups of time sequences are obtained and input to the monitoring and evaluation module; and the monitoring and evaluating module performs performance detection on the arm muscle surface myoelectricity, the arm movement acceleration data, the arm angular velocity data and the arm corner data in the received data by using the selected evaluating algorithm, outputs an evaluation value and outputs the evaluation value on a screen through a display terminal.
As shown in fig. 3, the signal filtering and statistical feature extraction includes the following steps.
And S3.1, extracting four input signals of a specific assembly task by using the start time and the end time of each assembly task recorded by the assembly system, and recording the four input signals as an arm muscle surface electromyogram signal _ i, an arm motion acceleration signal _ i, an arm angular velocity signal _ i and an arm rotation angle signal _ i.
And S3.2, performing noise reduction smoothing processing on the arm muscle surface electromyographic signal _ i by using a 30Hz fourth-order Butterworth filter to obtain an arm muscle surface electromyographic signal _ i _ filtered, and performing noise reduction smoothing processing on the arm motion acceleration signal _ i and the arm angular velocity signal _ i by using a third-order median filter to obtain an arm motion acceleration signal _ i _ filtered, an arm angular velocity signal _ i _ filtered and an arm corner signal _ i _ filtered.
And S3.3, dividing the arm muscle surface electromyographic signals _ i _ filtered by using a sliding time window, and calculating a mean value and a root mean square of the signals in each time window to obtain two groups of time sequences, namely the arm muscle surface electromyographic signals _ i _ filtered _ average and the arm muscle surface electromyographic signals _ i _ filtered _ RMS. In a preferred embodiment, the time window length is 0.25s and the overlap length is 0.075 s.
Step S3.4, the sliding time window is used to divide the arm motion acceleration signal _ i _ filtered, the arm angular velocity signal _ i _ filtered, and the arm rotation angle signal _ i _ filtered, and the mean and root mean square of the signals are calculated in each time window, so as to obtain two sets of time sequences for each signal, namely, the arm motion acceleration signal _ i _ filtered _ average, the arm motion acceleration signal _ i _ filtered _ RMS, the arm angular velocity signal _ i _ filtered _ average, the arm angular velocity signal _ i _ filtered _ RMS, the arm rotation angle signal _ i _ filtered _ average, and the arm rotation angle signal _ i _ filtered _ RMS. In a preferred embodiment, the time window length is 0.3s and the overlap length is 0.1 s.
Step S3.5, extracting the parameter distribution in the description period of 6 description statistics of each time sequence, wherein the parameter distribution comprises the following steps: mean, variance, median, mode, kurtosis, and skewness as outputs of the signal filtering and statistical features.
Step S4, performance assessment feedback. The evaluation result feedback module of the display terminal displays the corresponding performance index, namely high, medium or low according to the received evaluation value.
Example 3
The embodiment provides a training method of the machine learning and deep learning network model in embodiment 1.
As shown in fig. 4, the method for training a machine learning and deep learning network model includes the following steps.
In step S01, an asset data set is provided, that is, the data collected by the basic information collection module of the worker and the wearable device in step S1 includes the age, sex, height, weight, muscle surface myoelectricity of the arm, arm movement acceleration data, arm angular velocity data, arm rotation angle, etc. of the worker.
Step S02, manually labeling all data of the ASSEMBLY data set to form an ASSEMBLY data set for manually judging performance, namely an ASSEMBLY _ Output data set; meanwhile, a data set of the wearable device for acquiring the physiological signal characteristics is formed, namely the data set ASSEMBLY _ Input.
Step S03, randomly splitting the asset data Set into 70%, wherein 70% of the asset data Set constitutes a training Set Train _ Set, and 30% of the asset data Set constitutes a Test Set Test _ Set.
Step S04, training by using two machine learning methods GBDT and LDA and an RNN deep learning model and a data Set Train _ Set to obtain models GBDT _ trained, LDA _ trained and RNN _ trained;
step S05, classifying the contribution _ Input and GBDT _ Trained, LDA _ Trained, RNN _ Trained models in the data Set Train _ Set to obtain a performance evaluation data Output Set, i.e. the data Set contribution _ Trained _ Output.
Step S06, using errors of the data sets ASSEMBLY _ Trained _ Output and ASSEMBLY _ Output to enhance GBDT _ trailing, LDA _ trailing and RNN _ trailing, and obtaining enhanced machine learning models GBDT _ enhanced, LDA _ enhanced and RNN _ enhanced;
step S07, testing the performance of the GBDT _ improved, LDA _ improved and RNN _ improved assembly performance evaluation machine learning and deep learning network models by using a data Set Test _ Set, and ending the process of training the machine learning and deep learning network models if the Test result is qualified; if not, the parameters of the three types of machine learning and deep learning network models are adjusted, and the process is executed again from step S03.
The existing machine learning and deep learning network models are various in types, and common models comprise SVM \ KNN \ LDA \ GBDT \ RNN \ Boost and the like, the invention compares 6 types of algorithm models under three assembly scenes in a production line, screens out three machine learning and deep learning network models with the highest prediction accuracy, and compared with other machine learning models, the model used in the invention has greater advantages: the prediction accuracy of LDA machine learning in a stability task station in a production assembly line is up to more than 90%, and the prediction accuracy of other machine learning models is only about 80%; the prediction accuracy of the RNN deep learning model in a flexible task station in a production assembly line is up to more than 88%, and the prediction accuracy of other machine learning models is only about 70%; the prediction accuracy of the GBDT machine learning model in a screw assembly station in a production assembly line is up to more than 99%, and the prediction accuracy of other machine learning models is only about 87%.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents and are included in the scope of the present invention.