Disclosure of Invention
The invention provides a method and a system for processing industrial Internet of things edge data in a light-weight manner, which can solve the defects of high industrial field mass data transmission and storage cost and insufficient instantaneity.
Other objects and advantages of the present invention will be further appreciated from the technical features disclosed in the present invention.
In order to achieve one or a part of or all of the purposes or other purposes, the method for processing the edge data of the industrial Internet of things in a lightweight way comprises the steps of sampling industrial field devices through sampling equipment, wherein a sampling result comprises device vibration frequency and industrial image data, processing the collected sampling result in an edge lightweight way, processing the collected device vibration frequency by using a one-dimensional neural network structure, extracting and reinforcing key information in the input device vibration frequency by using the one-dimensional neural network to output characteristic values, detecting the collected industrial image data in real time, identifying a defect area and reducing image transmission quality, sensing the state of data collection and lightweight processing equipment, dynamically scheduling according to the state of the equipment, compressing the lightweight processing result when the equipment is normal, outputting the compressed lightweight processing result, triggering full data transmission and sending an alarm when the state of the industrial field devices is abnormal.
The edge light processing further comprises the steps of processing the collected industrial equipment text logs, understanding complex semantics by using a BERT distillation model, mapping log texts into a low-dimensional semantic space, screening high-information-content logs by using a TF-IDF algorithm, and reducing the text amount needing advanced processing.
The one-dimensional neural network structure comprises at least three convolution layers and at least two full-connection layers, wherein the three convolution layers are sequentially and recursively connected, the output of the former convolution layer is the input of the latter convolution layer, the output formula of each convolution layer is Yi＝f(Wi*X+bi \), Yi is the output of the ith convolution layer, X is input data, Wi is the convolution kernel of the ith convolution layer, bi is the offset of the ith convolution layer, "/and" \ "represents convolution operation, the convolution layer operation extracts primary characteristics, the non-linear expression capacity of the extracted primary characteristics is enhanced through ReLU activation function processing, and the two full-connection layers integrate and output the output of the last convolution layer.
The two full-connection layers are recursively connected, an output formula of each full-connection layer is Oi＝f(WfciZ+bfci, wherein Oi is output of the ith full-connection layer, Wfci is a weight matrix of the ith full-connection layer, bfci is bias of the ith full-connection layer, and the last full-connection layer finally outputs 128-dimensional fault feature vectors.
The method comprises the steps of processing collected industrial image data, detecting an industrial image in real time, identifying a defect area and reducing image transmission quality, processing the image by using a YOLO model, wherein the YOLO model comprises an input end, a backbone network, a neck network, a detection head and an output end, the industrial image data is input to the input end of the YOLO model, the backbone network is a convolutional neural network backbone with residual connection, the input industrial image is processed to output a multi-scale feature map, the neck network upsamples, downsamples and splices the output multi-scale feature map through a bidirectional feature pyramid to generate an enhanced feature map, the detection head generates a prediction tensor on multiple scales, and the prediction tensor is analyzed by the output module to output a detection result comprising the defect position and the confidence.
The industrial image data input into the YOLO model is further subjected to segmentation processing, which comprises dividing the input industrial image data into s×s grids, predicting B bounding boxes each, each bounding box containing position information (x, y, w, h), wherein x, y are coordinates of a bounding box center relative to the grids, and w, h are wide and high.
The method comprises the steps of dividing an input image based on different scale feature images to generate feature images with different resolutions, processing the feature images by a backbone network, outputting feature images with larger sizes with high resolution and more detail information, outputting feature images with smaller sizes with low resolution and rich semantic information, and outputting a prediction tensor with the shape of (S, S, B x (5+C) \), wherein S is the dividing density of each grid, B is the number of boundary boxes of each grid, and C is the confidence.
Sensing the state of the data acquisition and lightweight processing equipment, including sensing the CPU utilization rate, the memory occupancy rate, the network bandwidth and the delay jitter parameters of the equipment, and selecting a data transmission strategy based on the state of the equipment.
The dynamic scheduling method further comprises the steps of constructing a distributed computing cluster by using a D2D communication protocol and the adjacent edge nodes, calculating the load scores of the edge nodes, processing local tasks by the edge nodes preferentially, and migrating tasks of the overload nodes through a gRPC protocol when the transmission load index of the local edge nodes exceeds a threshold value.
The cloud collaborative processing method comprises the steps of firstly, initializing global model parameters w0 by the cloud, secondly, sending the global model parameters wt to each edge node in each round of iteration t, carrying out model training on an objective function of each edge node on a local data set again by the cloud, updating the local model parameters wt, sending the updated local model parameters back to the cloud by the edge node cloud, and polymerizing the model parameters of each edge node to obtain new global model parameters wt+1.
The locally trained model parameters comprise weights of a one-dimensional neural network structure and anchor frames of a YOLO model, and the FedProx algorithm iteratively trains the iteration cycle of each edge node model parameter to be less than 24 hours.
And predicting the state of the equipment to be sampled by using a Kalman filtering model, and dynamically adjusting the sampling frequency of the vibration sensor.
According to the industrial Internet of things edge data lightweight processing system provided by the other technical scheme of the invention, the industrial Internet of things edge data lightweight processing method is used for executing the industrial Internet of things edge data lightweight processing method, and the industrial Internet of things edge data lightweight processing system comprises a multi-mode data acquisition module, wherein the multi-mode data acquisition module is used for sampling parameters of industrial field devices based on sampling thresholds; the system comprises an edge lightweight processing engine, a dynamic scheduling module, a cloud collaborative optimization module, a model parameter training module, a dynamic scheduling module and a sampling threshold value dynamic adjustment module, wherein the edge lightweight processing engine is used for processing the collected equipment vibration frequency by using a one-dimensional neural network structure, extracting and strengthening key information in the equipment vibration frequency to output characteristic values, detecting the collected industrial image data in real time, identifying a defect area and reducing image transmission quality, the dynamic scheduling module is used for sensing the state of data acquisition and lightweight processing equipment and transmitting the lightweight processing result compressed into the characteristic values to the edge preservation module according to the state of the equipment, the full data are transmitted to a cloud server when the state of the industrial field equipment is abnormal, and sending a warning signal to a local controller, the cloud collaborative optimization module is used for collecting historical data of the cloud server, training model parameters of local training of each edge node by the collected historical data, and dynamically adjusting the sampling threshold value based on historical false report and missing report data.
Compared with the prior art, the method has the advantages that 1, the method realizes the extraction of the characteristics through the edge light-weight processing, senses the state of the data acquisition and light-weight processing equipment, and performs the dynamic compression processing of the data transmission according to the state of the equipment, so that redundant data can be greatly reduced, and network bandwidth and cloud storage resources are saved. 2. According to the invention, most of key analysis and control tasks are transferred to the edge end, so that delay is effectively reduced, and the real-time requirement of an industrial field is met. 3. According to the invention, the state of the data acquisition and lightweight processing equipment is sensed, the data transmission mode is selected based on the network bandwidth and the delay jitter parameters, meanwhile, the tasks of the edge nodes are scheduled according to the load through the load balancing algorithm of the D2D communication, the task processing efficiency is improved, and the power consumption can be reduced.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments, as illustrated in the accompanying drawings.
Detailed Description
The foregoing and other features, aspects, and advantages of the present invention will become more apparent from the following detailed description of a preferred embodiment, which proceeds with reference to the accompanying drawings. The directional terms mentioned in the following embodiments, such as up, down, left, right, front or rear, etc., are only referring to the directions of the attached drawings. Thus, the directional terminology is used for purposes of illustration and is not intended to be limiting of the invention.
Example 1
The first embodiment provides an industrial Internet of things edge data lightweight processing method, which comprises the steps of sampling industrial field equipment through sampling equipment, wherein a sampling result comprises equipment vibration frequency and industrial image data, performing edge lightweight processing on the acquired sampling result, processing the collected equipment vibration frequency by using a one-dimensional neural network structure, extracting and strengthening key information in the input equipment vibration frequency to output a characteristic value, detecting the collected industrial image data in real time, identifying a defect area and reducing image transmission quality, sensing the state of data acquisition and lightweight processing equipment, dynamically scheduling according to the state of the equipment, compressing and outputting the lightweight processing result when the equipment is normal, and triggering full-quantity data transmission and sending alarm when the state of the equipment is abnormal.
The following explains a detailed explanation of the method for processing the edge data of the industrial internet of things in the first embodiment.
The industrial Internet of things processing system comprises industrial field equipment, a sampling module, an edge computing module, a data transmission module and a cloud server. The sampling device samples the industrial field device, and the first sampling in this embodiment includes collecting the vibration frequency of the industrial device, collecting the image data of the industrial device, and collecting the text log of the industrial device. Wherein the collection of the vibration frequency of the industrial equipment may be the collection of the vibration frequency of the bearing equipment. In order to reduce sampling data and avoid occupation of redundant sampling data to memory space and cloud space, the invention sets a sampling threshold value during sampling. Taking vibration frequency sampling as an example, in order to improve the sampling accuracy, a vibration frequency sampling range may be set, for example, the vibration sensor re-collects vibration frequency data in a frequency range of 10Hz-2kHz, vibration outside the above frequency is not highly associated with faults, or occurrence probability is extremely low so that the vibration sensor does not collect the vibration frequency data, so that the sampled data volume can be effectively reduced.
The sampling equipment supports multi-protocol conversion, specifically, when the sampling equipment performs data transmission, the sampling equipment supports real-time conversion from Modbus RTU/ASCII, OPC UA, profibus DP and other protocols to MQTT. The sampling equipment can process communication mechanisms of different protocols, convert data into MQTT information, and facilitate unified processing of an Internet of things platform. And meanwhile, a unified data format, such as JSON-LD, can be defined, and other data formats can be defined, wherein the defined data formats can comprise metadata such as device ID, timestamp, data type, original value and the like. Standardized data is defined, so that subsequent data analysis and integration application are facilitated.
Because the acquired data volume is larger, if the acquired data is directly uploaded to the cloud, the cloud storage memory is tight, and meanwhile, the transmission bandwidth requirement of the transmission of a large amount of data is correspondingly increased. Therefore, the first embodiment performs edge light processing on the collected industrial equipment data before data transmission to achieve compression and feature extraction of the data, specifically includes processing collected equipment vibration frequencies by using a one-dimensional neural network structure, extracting and strengthening key information in the input equipment vibration frequencies to output feature values by using the one-dimensional neural network, detecting the collected industrial image data in real time, identifying defective areas and reducing image transmission quality, and processing collected industrial equipment text logs.
When the one-dimensional neural network structure is used for processing the collected vibration frequency of the equipment, the one-dimensional neural network structure (CNN structure) comprises at least three layers of convolution layers and at least two full-connection layers, wherein the three layers of convolution layers are connected in a recursion mode, the output of the former convolution layer is the input of the latter convolution layer, the output formula of each convolution layer is Yi＝f(Wi*X+bi \), Yi is the output of the ith convolution layer, X is input data, Wi is the convolution kernel of the ith convolution layer, bi is the offset of the ith convolution layer, "X" and "\" of the ith convolution layer represent convolution operation, the convolution layer operation extracts primary characteristics, the non-linear expression capacity of the extracted primary characteristics is enhanced through the ReLU activation function processing, and the two full-connection layers integrate the output of the last convolution layer and output the final convolution layer. The method comprises the steps of recursively connecting two full-connection layers, wherein an output formula of each full-connection layer is Oi＝f(WfciZ+bfci, Oi is output of an ith full-connection layer, Wfci is a weight matrix of the ith full-connection layer, bfci is bias of the ith full-connection layer, and a last full-connection layer spreads output results of a last convolution layer into a one-dimensional vector to finally output 128-dimensional fault feature vectors. The input vibration information is extracted through the one-dimensional neural network structure, so that the compression of input data can be realized, and the compression ratio can reach 8:1.
When the collected industrial image data is processed, a defective area is identified and image transmission quality is reduced by detecting the industrial image in real time, the method comprises the steps of processing an image by using a YOLO model, wherein the YOLO model comprises an input end, a backbone network, a neck network, a detection head and an output end, the industrial image data is input to the input end of the YOLO model, the backbone network is a convolution neural network backbone with residual connection, the input industrial image is processed to output a multi-scale feature map, the neck network carries out upsampling, downsampling and splicing operations on the output multi-scale feature map through a bi-directional feature pyramid (for example, fcat = Concat (F1, fup), the Fup is an upsampled feature map, the Fcat is a spliced feature map, an enhanced feature map is generated, the detection head generates a prediction tensor on multiple scales, the prediction tensor is analyzed through the output module, and a detection result comprising the defect position and the confidence is output.
In order to facilitate the processing of industrial images, the industrial image data input into the YOLO model is further subjected to segmentation processing, which comprises dividing the input industrial image data into s×s grids (the YOLO model is divided based on feature maps of different sizes in practical application), predicting B bounding boxes by each grid, and each bounding box contains position information (x, y, w, h), wherein x, y is the coordinate of the center of the bounding box relative to the grid, and w, h is the width and height.
In order to segment different types of images, an input image is segmented based on different scale feature images (the 'different scale feature images' refer to feature data with different multi-layer resolution and semantic information output after model processing, wherein a large scale image retains details, a small target is adapted for fine defect detection, the small scale image compresses semantics, a large target is adapted for global defect analysis, light weight and full scene defect identification is realized at an industrial edge end through layering utilization), and a prediction tensor output by a detection head is tensor with the shape of (S, S, B× (5+C) \), wherein S is the dividing density of each grid, B is the quantity of boundary boxes of each grid, and C is the confidence.
This embodiment mentions less than 10Mb after the YOLO model quantization while supporting real-time detection of 1080p video.
When the collected industrial equipment text logs are processed, the BERT distillation model is used for understanding complex semantics, log texts are mapped to a low-dimensional semantic space, and a TF-IDF algorithm is used for screening high-information-content logs, so that the amount of the texts needing to be deeply processed is reduced. Parameters of the text log processed by the BERT distillation model can be reduced by 90%, 32-dimensional semantic vectors are generated, the TF-IDF algorithm filters redundant log entries, and key events are reserved. The BERT distillation model and the TF-IDF algorithm are used for processing the text logs, the high information content logs are screened, and the amount of the text needing to be deeply processed is reduced as the prior art, so that the text processing method is not repeated here.
In the first embodiment, when performing dynamic scheduling of data, sensing the state of the data acquisition and lightweight processing device, including sensing the CPU utilization, the memory occupancy, the network bandwidth, and the delay jitter parameters of the device, and performing dynamic scheduling according to the state of the device, when the device is normal, compressing the lightweight processing result, outputting (transmitting the compressed characteristic value, and transmitting the transmission rate of 10 KB/min), and when detecting that the device state is abnormal, triggering the transmission of the full data and sending an alarm.
Specifically, the transmitted full data are transmitted to a cloud server, the compressed data of the light processing result are transmitted to edge equipment for buffering, and the transmitted abnormal alarm information is transmitted to a local controller.
In dynamic scheduling, because the loads of the edge nodes are different, in order to balance the loads of different edge nodes (the node is intelligent computing equipment on the edge side of the industrial Internet of things, such as a factory production line intelligent gateway, a workshop edge server and the like, bears data acquisition, processing and collaborative computing tasks and is an edge computing network basic unit, the edge node refers to a computing and storage unit which is arranged near the network edge position of a data source or terminal equipment), the embodiment firstly uses a D2D communication protocol to construct a distributed computing cluster of the local edge node and the adjacent edge node, calculates the load score of each edge node, and the edge nodes preferentially process the local task, and migrates the task of the overload node through a gRPC protocol when the transmission load index of the local edge node exceeds a threshold value.
Balancing different edge nodes, comprising the following steps:
S1D 2D cluster construction and load scoring
S1-1, constructing a cluster, namely, periodically sending a D2D discovery message (containing equipment ID and resource/network state) by a local node, establishing connection between adjacent adaptation nodes, and selecting a cluster coordination node to overall schedule.
S1-2, load grading, namely collecting data and calculating and quantifying the load of the node by the coordination node according to a formula (load grading = CPU utilization rate x 0.4+ memory occupancy rate x 0.3+ task queue length x 0.2+ network bandwidth occupancy rate x 0.1) every 5 seconds.
S2, local task processing
The node built-in scheduler follows the priority of local task > cooperative task, the local task enters into high-priority queue to execute preferentially, and the preemption threshold is set, if the local task occupies more than 70% of the resources for 30 seconds, the resources are automatically released to the cooperative task.
S3: gRPC task migration
When the transmission load index exceeds the threshold value (more than 85%), triggering the migration specifically comprises
S3-1, an overload node gRPC migration request containing task types, states and resource requirements;
S3-2, selecting a target node with load score <30 and resource adaptation by the coordination node;
S3-3, the overload node and the target node transmit tasks through gRPC flow + ProtoBuf, the target node uses a lightweight container to restart the tasks, and after the task is transferred, a new task is directly routed to the target node until the load of the overload node drops.
S3-4, abnormal rollback, namely returning the task to the original node/the standby node when the target node fails, and guaranteeing service continuity.
S3-5, security enhancement, namely, D2D encrypts with an industrial-grade certificate, gRPC enables TLS1.3, and data leakage is prevented.
In order to train each model parameter and update the sampling threshold of the sampling device, the method for processing the edge data of the industrial internet of things in a lightweight manner further comprises a cloud collaborative processing step, specifically, includes collecting historical data of a cloud server, wherein the historical data transmitted to the cloud server is full transmission data based on the above, the full transmission data is data when abnormality of the device is detected, and optimization training can be performed on the model based on the data (normal state data has little meaning on model improvement, and abnormal state data has more reference meaning on model improvement). The method comprises the steps of training locally trained model parameters of all edge nodes through collected historical data, dynamically adjusting sampling thresholds (for example, the collected abnormal threshold of vibration signals is changed from 3 sigma to be self-adaptive) based on the historical false alarm and missing report data, sampling by sampling equipment according to the adjusted sampling thresholds, carrying out cloud collaborative processing, carrying out iterative training on the model parameters of all the edge nodes by adopting FedProx algorithm to reduce the specificity of balancing the local data of all the edge nodes, initializing global model parameters w0 by the cloud, sending global model parameters wt to all the edge nodes by the cloud in each iteration t, carrying out model training on an objective function of all the edge nodes by each edge node on a local data set, updating the local model parameters wt, sending the updated local model parameters back to the cloud by the edge nodes, and aggregating the model parameters of all the edge nodes by the cloud to obtain new global model parameters wt+1, and repeating the steps until convergence conditions are met. The model parameters of the local training comprise weights of the one-dimensional neural network structure and anchor frames of the YOLO model, and the iteration period of the FedProx algorithm for iteratively training the model parameters of each edge node is less than 24 hours.
When the model optimization training is carried out, cloud server data can be collected regularly to optimize the model, for example, cloud historical data can be collected once every month to train the model.
When equipment sampling is carried out, in order to enable the sampling to be more accurate, a Kalman filtering model is used for predicting the state of equipment to be sampled, and the sampling frequency of the vibration sensor is dynamically adjusted. Meanwhile, when the cloud is cooperatively processed, the Kalman filtering model can be trained based on historical data, and the sampling frequency of the vibration sensor can be further adjusted.
According to the industrial Internet of things edge data light-weight processing method, feature extraction is achieved through edge light-weight processing, states of data acquisition and light-weight processing equipment are perceived, dynamic compression processing of data transmission is carried out according to the states of the equipment, redundant data can be greatly reduced, and network bandwidth and cloud storage resources are saved.
According to the invention, most of key analysis and control tasks are transferred to the edge end, so that delay is effectively reduced, and the real-time requirement of an industrial field is met.
According to the invention, the state of the data acquisition and lightweight processing equipment is sensed, the data transmission mode is selected based on the network bandwidth and the delay jitter parameters, meanwhile, the tasks of the edge nodes are scheduled according to the load through the load balancing algorithm of the D2D communication, the task processing efficiency is improved, and the power consumption can be reduced.
Example two
The second embodiment provides a system for processing the edge data of the industrial internet of things in a light manner, which is used for executing the method for processing the edge data of the industrial internet of things in the first embodiment, and referring to fig. 1, the system comprises a multi-mode data acquisition module, wherein the multi-mode data acquisition module samples parameters of industrial field devices based on sampling thresholds;
the edge light processing engine is used for processing the collected equipment vibration frequency by using a one-dimensional neural network structure, extracting and strengthening key information in the equipment vibration frequency to output a characteristic value, detecting the collected industrial image data in real time, identifying a defect area and reducing the image transmission quality;
The device comprises a dynamic scheduling module, an edge preservation module, a cloud server, a local controller, a data processing module and a data processing module, wherein the dynamic scheduling module senses the state of data acquisition and light weight processing equipment, dynamically schedules according to the state of the equipment, and compresses light weight processing results into characteristic values to be transmitted to the edge preservation module when the equipment is normal;
The cloud collaborative optimization module is used for collecting historical data of the cloud server, training model parameters of local training of each edge node through the collected historical data (training a model used by an edge lightweight processing engine based on the historical data to update parameters of the model, training a model used by the dynamic scheduling module based on the historical data to update corresponding model parameters), and dynamically adjusting a sampling threshold based on historical false alarm and missing report data.
The cloud collaborative optimization module mainly illustrates how to train parameters of each model and update sampling thresholds, and to update what model is, corresponding training and updating can be performed according to the specific model type actually used by each module.
The method and system for processing the edge data of the industrial Internet of things in light of weight are described in detail, and specific examples are applied to illustrate the structure and working principle of the method and the system, and the description of the embodiments is only used for helping to understand the method and the core idea of the method. It should be noted that it will be apparent to those skilled in the art that various improvements and modifications can be made to the present invention without departing from the principles of the invention, and such improvements and modifications fall within the scope of the appended claims.