CN109447162B - A real-time behavior recognition system based on Lora and Capsule and its working method - Google Patents

Abstract

Translated fromChinese

本发明涉及一种基于Lora和Capsule的实时行为识别系统及其工作方法，该系统包括行为信息物理层、行为信息接入层、行为信息平台层、行为信息应用层。本发明将行为信息接入层的传输采用Lora节点、Lora基站，实现远距离、低功耗的行为信息传输；对行为信息不确定性进行了不一致和不完备性方面的处理，提高行为信息的可信度；采用Capsule自动获取有用特征以及特征之间的空间关系来进行行为识别，在精度方面有了很大的提升。

The invention relates to a Lora and Capsule-based real-time behavior recognition system and its working method. The system includes a behavior information physical layer, a behavior information access layer, a behavior information platform layer, and a behavior information application layer. The present invention adopts Lora node and Lora base station for the transmission of behavior information access layer, so as to realize long-distance and low-power consumption behavior information transmission; Credibility: Capsule is used to automatically obtain useful features and the spatial relationship between features for behavior recognition, which has greatly improved the accuracy.

Description

Real-time behavior recognition system based on Lora and Capsule and working method thereof

Technical Field

The invention relates to a real-time behavior recognition system based on Lora and Capsule and a working method thereof, belonging to the technical field of artificial intelligence and pattern recognition.

Background

The behavior recognition system is a system for realizing behaviors through a reasonable model by acquiring behavior information of a person. With the development and maturity of advanced technologies such as internet of things, artificial intelligence, big data and cloud computing, more and more scholars begin to pay attention to the research of behavior recognition direction. Behavior recognition has become a good research direction in the research field of artificial intelligence and pattern recognition, and the development of wearable devices provides good opportunities for human behavior recognition. Nowadays, human behavior recognition technology has been primarily applied in the fields of games, human motion analysis, smart home, human-computer interaction, medical diagnosis and monitoring, and the like.

Behavior information required by the behavior recognition system mainly comes from the following two aspects:

1. visual-based behavior information — visual behavior information is collected by the camera device.

2. Sensor-based behavior information — sign behavior information is collected by intelligent hardware.

At present, the mainstream behavior recognition technology in the market is mainly used for recognizing behavior information based on vision. Although the two behavior information acquisition modes can realize real-time identification of human body behaviors through corresponding algorithms, the mode based on visual behavior information acquisition has certain disadvantages, and a lot of behavior information cannot be acquired in a blind area of the camera equipment or a scene with a dark environment. For example: when fighting in a bathroom, a manager cannot know what happens in the bathroom, and therefore behavior recognition by means of single visual behavior information has many drawbacks.

In addition, the transmission technology of information in the current behavior recognition system mainly takes bluetooth, ZigBee, WiFi, 3G, 4G as the main technology. The mainstream transmission technology cannot be compatible in terms of transmission distance and power consumption, and therefore, a transmission technology with low power consumption and long transmission distance is required to realize real-time behavior recognition. Loa is a radio modem technology promulgated by Semtech corporation. The technology has four categories of a Lora-WAN protocol, a Lora proprietary protocol, a CLASS protocol and data transparent transmission, and has great advantages in the aspects of power consumption, ad hoc network and the like compared with other low-power-consumption wide-area Internet of things technologies.

The model for the system to perform behavior recognition mainly adopts algorithms such as machine learning and deep learning. At present, the mainstream machine learning mainly comprises a K nearest neighbor algorithm (KNN), a Support Vector Machine (SVM), a random forest, a neural network and the like; the mainstream deep learning algorithm mainly includes a Deep Neural Network (DNN), a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Deep Belief Network (DBN), and the like. The mainstream algorithm carries out behavior recognition according to the behavior information characteristics, so that the input behavior information can be classified into a certain type of behaviors as long as certain characteristics exist. For example: behavior information 1100 and 0011 represents standing and sitting, respectively, 1010 and 0101 represent fighting behaviors, and a mainstream algorithm may judge behavior information with only two 1 and two 0 characteristics as the fighting behavior, so if the input behavior information is characterized by 0101, 1100, 0011 or 1010, all of which are considered as fighting behaviors, but 1100 or 0011 actually represents standing or sitting behaviors. The main reason is that the existing mainstream algorithm only considers whether the behavior information contains certain features, but not the spatial characteristics of the features, so that misjudgment can be caused to a certain extent, and the accuracy of behavior identification is reduced. Geoffrey Hinton introduced 2017 the concept of a capsule network that can identify not only whether behavior information has certain characteristics, but also the spatial relationships between the characteristics. The capsule network algorithm model adopted by the invention is improved in the aspect of accuracy of behavior identification.

Disclosure of Invention

Aiming at the unicity of the current behavior information acquisition, the particularity of a transmission mode and the misjudgment of a prediction model, the invention provides a real-time behavior recognition system based on Lora and Capsule.

The invention also discloses a working method of the system.

Summary of the invention:

real-time behavior identification system based on Lora and Capsule, at first, filter and design intelligent hardware through equipment quality (QoD) parameter, gather the behavior information through intelligent equipment, then, transmit the behavior information through the Lora node, the Lora basic station will receive the behavior information this moment. Because the behavior information at this time is the original behavior information, the system needs to perform uncertainty detection on the original behavior information, and the information with incompleteness or inconsistency in the behavior information is processed by methods such as context prediction filling, 0 complementing, deleting and the like, so that the reliability of the behavior information is improved. And then, standardizing the behavior information subjected to uncertainty processing and intercepting the behavior information based on a time series, wherein the standardization is used for improving the accuracy and generalization capability of the model, and the interception of the behavior information through a sliding window mechanism is used for normalizing the input of the model and improving the accuracy of the model. Then, training the behavior information set with the label under the built network architecture model, and finding the optimal model while continuously optimizing the loss function. And transmitting the behavior information acquired in real time into the model to realize real-time behavior identification. Finally, the system adjusts the preset threshold value and the model parameter according to the user quality of experience (QoE) and the quality of service (QoS), so that the stability, the accuracy and the applicability of the system are improved.

The invention provides a feasible scheme for real-time behavior recognition based on the sensor, makes up for the defects caused by the video-based behavior recognition, and lays a foundation for improving the accuracy of the behavior recognition by multi-source behavior information. The application of the deep learning model based on the Capsule network in the aspect of behavior recognition also greatly improves the accuracy of behavior recognition.

The technical scheme of the invention is as follows:

a real-time behavior recognition system based on Lora and Capsule comprises a behavior information physical layer, a behavior information access layer, a behavior information platform layer and a behavior information application layer which are sequentially connected;

the behavior information physical layer is configured to: sensing, collecting, storing and transmitting user behavior information from the environment, wherein the behavior information comprises: acceleration, angular velocity, heart rate;

the behavior information access layer is configured to: networking and transmitting the collected behavior information through a low-power-consumption wide-area Internet of things;

the behavior information platform layer is used for: sequentially carrying out uncertainty detection, standardization and interception based on a time sequence on the behavior information, training a behavior information set with a label under a built network architecture model, and finding out an optimal model while continuously optimizing a loss value; the uncertainty detection means that: incomplete or inconsistent information in the behavior information is processed through a context prediction filling, 0 complementing and deleting method, so that the reliability of the behavior information is improved; the standardization is to carry out normalization processing on numerical data, so that the accuracy and generalization capability of the model are improved; intercepting based on the time sequence means intercepting behavior information through a sliding window mechanism so as to ensure the normalization of model input and improve the accuracy of the model;

the behavior information application layer is used for: and adjusting the stability and the adaptability of the whole real-time behavior recognition system.

The invention provides a relatively optimized system in four aspects of transmission technology, information processing, behavior recognition, behavior application and the like, overcomes the defects that the real-time performance of the recognition in the current market is poor and the recognition cannot be used in a specific area, further improves the accuracy of the behavior recognition, and ensures that the system has stability.

According to the invention, preferably, the behavior information physical layer is a behavior information acquisition module, and the behavior information acquisition module comprises a sensor module and a plurality of intelligent hardware modules; the sensor module comprises a plurality of sensors of different types, the intelligent hardware module is respectively connected with the sensors of different types, and the intelligent hardware module is used for controlling the sensors to sense behavior information of different types of users and storing the sensed behavior information.

In the behavior information acquisition module, the selection of the sub-modules and the design of the intelligent equipment are carried out according to QoD parameters, application scenes and user requirements of the sub-modules, and QoD parameters mainly comprise sampling frequency, service life, precision and the like of the sensor module.

And the intelligent hardware module adopts Lora to transmit the behavior information. The transmission technology is a technology which makes full use of the transmission capability of different channels to form a complete transmission system so that information can be reliably transmitted. With the progress of society and the development of wireless technology, the convenience of wireless transmission is further amplified on the premise of low requirement on packet loss rate. At present, the mainstream wireless technologies mainly include WiFi, bluetooth, ZigBee, 3G, 4G, and the like, and each wireless technology is in a situation that transmission distance and power consumption are not compatible, but in order to realize real-time behavior recognition, transmission of behavior information requires a wireless technology with low power consumption and long transmission distance. Low power local area network (LPWAN) is the main technology to address the current situation, so the present invention uses Lora to transmit behavior information.

According to the present invention, preferably, the behavior information access layer is a behavior information transmission module, and the behavior information transmission module includes a behavior information sending module and a behavior information receiving module; the behavior information sending module is connected with the intelligent hardware module and used for sending behavior information to the behavior information receiving module.

Further preferably, the information sending module is a Lora node, and the behavior information receiving module is a Lora base station.

According to the preferred embodiment of the invention, the behavior information platform layer is a behavior information preprocessing module, and the behavior information preprocessing module comprises a behavior information detection module, a behavior information uncertainty elimination module, a behavior information processing module and a network architecture module which are sequentially connected;

the behavior information detection module comprises an inconsistency detection/quantification unit and an incomplete detection/quantification unit;

the behavior information uncertainty eliminating module comprises an inconsistency eliminating unit and an imperfection eliminating unit;

the behavior information processing module comprises a behavior information standardization unit and a behavior information sliding window unit which are sequentially connected;

the network architecture module comprises a convolution layer unit, a first Capsule layer unit, a second Capsule layer unit and a full connection layer unit which are sequentially connected;

the behavior information receiving module, namely a gateway, is connected with the behavior information detecting module;

the behavior information received by the behavior information receiving module, namely the original behavior information, is input into the behavior information detecting module, the original behavior information is subjected to uncertainty detection through the inconsistency detecting/quantifying unit and the incompleteness detecting/quantifying unit, the inconsistency detecting/quantifying unit detects whether different types of behavior information at the same moment are objected, and the incompleteness detecting/quantifying unit detects whether the perceived behavior information at the same moment is lost;

if the behavior information is found to have uncertainty, the uncertainty is eliminated through the imperfection eliminating unit and the inconsistency eliminating unit, the imperfection eliminating unit processes the loss condition existing in the perception behavior information at the same moment through an eliminating method, a 0 complementing method and a context prediction filling method, the inconsistency eliminating unit processes the inconsistency information through voting, an QoD optimal principle of hardware, a D-S (Dempster-Shafer) evidence theory and a fuzzy set, and the inconsistency information enters the behavior information standardizing unit; if the behavior information is found to have no uncertainty, directly entering the behavior information standardization unit; the behavior information standardization unit and the behavior information sliding window unit are used for processing, and the behavior information standardization unit is used for processing through a standardization and normalization method, so that the identification accuracy and the applicability are improved; the behavior information sliding window unit intercepts the behavior information based on a time sequence by adjusting the size of the sliding window and the sliding mode of the sliding window;

inputting the processed behavior information into a trained network architecture model, and realizing behavior recognition through the network architecture model; the convolution layer unit extracts features from the behavior information and converts a feature scalar into a vector, and the Capsule layer I unit is used for converting the input behavior information into behavior information with spatial characteristics; the second unit of the Capsule layer processes the behavior information through a dynamic routing protocol; and the full connection layer unit converts the behavior information characteristics into ordered one-dimensional characteristics, and finally, all the characteristics are operated through a Softmax classifier to identify the current behavior.

The network architecture module mainly works to recognize according to behavior information, and in the fields of artificial intelligence and mode recognition, the strong artificial intelligence can be really realized by the proposal of machine learning, and the proposal of deep learning has a great progress in the aspect of recognition rate. However, the emphasis of both the machine learning model and the deep learning model is whether some feature values are included in the input information. The network architecture based on the Capsule adopted in the invention not only pays attention to the characteristics of the behavior information, but also adds the spatial relationship of the behavior information characteristics, thereby improving the accuracy of behavior identification.

The behavior preprocessing module is mainly used for improving the credibility of the behavior information through preprocessing the behavior information. Compared with some systems which directly perform behavior recognition on the original behavior information, the method and the system have great improvement in the aspects of stability, accuracy and the like after the behavior information is preprocessed. The invention mainly carries out uncertainty analysis on the original information and correspondingly processes the category and the degree of uncertainty of the behavior information. In the aspect of information standardization, the invention provides a normalization method and a normalization method. And intercepting the behavior information based on the time sequence by adjusting the size of the sliding window and the sliding mode of the sliding window.

According to the present invention, preferably, the behavior information application layer includes a behavior information threshold setting module and a behavior application layer adjusting module, and the behavior application layer adjusting module includes a behavior identification unit, a user feedback unit and an error correction unit, which are connected in sequence;

the behavior information threshold setting module is used for adjusting a threshold in the behavior information uncertainty eliminating module so as to determine whether the monitoring data has uncertainty or not, and adjusting the uncertainty processing module to select a data processing mode; the behavior identification unit is used for identifying the current behavior in real time; the user feedback unit adjusts the preset threshold value and the parameters of the network architecture module according to different scenes and user requirements, and improves the applicability of the system to a certain degree; the error correction unit continuously adjusts the network architecture module to make the network architecture module in the optimal state all the time.

The error correction unit adjusts parameters when the recognition error rate is high, and the user feedback unit adjusts system parameters to be suitable for different scenes.

The real-time behavior recognition system based on Lora and Capsule and the working method thereof comprise the following steps:

step S01: sensor sensing behavioral information

Screening different manufacturers and different types of sensors according to original behavior information required by behavior recognition and QoD parameters of the sensors, wherein the QoD parameters of the sensors comprise: sampling frequency, lifetime, accuracy, for example: the method comprises the following steps that a sensor with high sampling frequency and high precision can be adopted for sensing behavior information of a user needing important monitoring, and a sensor with common sampling frequency and common precision can be adopted for sensing the behavior information of a common user; the sensor senses different types of behavior information of the user;

step S02: designing an intelligent hardware module

Selecting a proper intelligent hardware module according to the scheme requirement, controlling each sensor through the intelligent hardware module, and acquiring behavior information required by a behavior recognition system;

step S03: transmission of behavioral information

According to the requirement of real-time behavior recognition, some wireless transmission modes which are mainstream at present, such as 3G, 4G, ZigBee, Bluetooth and the like, can be excluded. The low-power-consumption wide area network is a more suitable transmission mode, as NB-IoT is about to be put into civilian use, the system adopts the Lora node to send the behavior information in consideration of privacy;

step S04: reception of behavioural information

Adopting a Lora base station to receive the behavior information; the sending equipment of the behavior information adopts a Lora node, and the receiving of the corresponding behavior information adopts a Lora base station.

Step S05: uncertainty detection of behavioral information

Setting a threshold range of the behavior information, for example, setting the accuracy of the behavior information to be not less than 85%, when the accuracy of the behavior information is less than 85%, regarding the information as uncertain behavior information, performing inconsistency detection/quantization and incomplete detection/quantization on the original behavior information sequentially through an inconsistency detection/quantization unit and an incomplete detection/quantization unit to obtain a detection result, when the original behavior information is inconsistent and incomplete, executing step S05, otherwise, executing step S06; the original behavior information refers to different types of behavior information of the user sensed by the sensor in step S01;

step S06: uncertainty elimination of behavioral information

The method comprises the following steps that an incomplete eliminating unit processes behavior information by different methods through a threshold value of uncertainty detection of the behavior information, when the accuracy of the behavior information is 85% -90%, a context prediction filling method is adopted for the behavior information, when the accuracy of the behavior information is 90% -95%, a 0 complementing method is adopted for the behavior information, and when the accuracy of the behavior information is 95% -100%, a deleting method is adopted for the behavior information;

the inconsistency elimination unit processes the inconsistency information, and the processing method comprises voting, QoD optimal principle of hardware, D-S (Dempster-Shafer) evidence theory and fuzzy set; the reliability of the original behavior information is improved;

step S07: processing of behavioral information

Standardizing the behavior information with higher credibility through a behavior information standardization unit; the standardization of behavior information uses different standardization approaches for different types of data, including: for data of the class-type characteristics, adopting one-hot coding (one-hot coding) standardization; normalization processing standardization is adopted for data of numerical characteristics; for the data of the ordered type features, the ordered type numerical code is adopted for standardization; standardization can enable the system to have good expansibility.

Referring to the preset parameters of the user, the preset parameters of the user comprise: the size of the sliding window and the sliding mode of the window are used for performing sliding window processing on the behavior information after the standardized processing through a behavior information sliding window unit, so that the behavior information is changed into an information block which is input into a network architecture module;

step S08: behavior information network architecture

Constructing a four-layer network architecture model through the convolution layer unit, the first Capsule layer unit, the second Capsule layer unit and the full-connection layer unit, and referring to parameters set by a user, wherein the parameters set by the user mainly comprise: inputting a series of parameters such as the current situation, the size of the convolutional layer kernel, the number and the like of data, training behavior information with a label through a plurality of iterations, and continuously optimizing model parameters and a dynamic routing protocol in a Capsule layer unit by reducing a loss function in the training process to finally obtain a network architecture model with high recognition rate;

step S09: identification of behavioral information

Inputting the behavior information acquired in real time into a trained network architecture model to perform real-time identification on the current behavior;

step S10: error detection

Judging whether the current behavior identification has errors, if so, executing the step S11, otherwise, executing the step S12;

step S11: error correction

The error correction unit adjusts the threshold range of the behavior information and the corresponding parameters of the behavior information processing module; the behavior information threshold range comprises an uncertainty detection threshold range, and the corresponding parameters of the behavior information processing module comprise the size of a sliding window in a behavior information sliding window unit and the sliding mode of the window; when the recognition error is large, the threshold range of the behavior information is appropriately increased, and the size of the sliding window and the sliding mode of the window are reduced.

Step S12: user feedback detection

And judging whether the system has user feedback information, if so, executing step 13.

Step S13: user feedback

And the user feedback unit performs feedback adjustment on the threshold range of the behavior information and the corresponding parameters of the behavior information processing module.

Preferably, according to the present invention, the step S08,

the network architecture module comprises a convolution layer unit, a first Capsule layer unit, a second Capsule layer unit and a full connection layer unit which are sequentially connected;

setting the number of convolution kernels in convolution layer unit to be N₁Each convolution kernel is 1 × Nuclear _ Size₁Step length of L₁；

Setting the number of convolution kernels in one unit of Capsule layer to be N₂Each convolution kernel is 1 × Nuclear _ Size₂Step length of L₂；

Setting the Output length of a second unit of the Capsule layer as Num _ Output dimension behavior information, wherein Vec _ Lenv behavior information characteristics are adopted in each dimension;

setting the Output Length in the full connection layer unit as Output _ Length;

the method comprises the following steps:

(1) inputting behavior information with the Size of Batch _ Size multiplied by 1 multiplied by Window _ Size multiplied by 3, wherein Batch _ Size refers to the number of the behavior information which runs in the network architecture module at a time, and Window _ Size refers to the length of the network architecture module which is input each time;

(2) after the behavior information of the Size of Batch _ Size × 1 × Window _ Size × 3 passes through the convolutional layer unit, the input behavior information is converted from a scalar to a vector by formula (i):

in the formula (I), X_iThe behavior information is subjected to uncertainty, standardization and sliding window processing based on time series; w_ijThe weight parameter refers to the weight parameter of the convolutional layer unit, and the initial value is a random number which generates truncated normal distribution by default;

b_jthe offset parameter of the convolutional layer unit is defined, and the default value is 0.0;

n represents the number of convolution kernels;

Y_jis representative of convolutional layer output;

the output information size is:

where it is necessary to ensure that the result of the fraction in the preceding formula is a positive integer. The output result at this moment is vector behavior information, which meets the input requirement of the Capsule network;

(3) order to

Encapsulating the M groups of convolution layers in a Capsule network, and acquiring behavior information Y of the vector_jInputting the behavior information into a Capsule layer one unit, and converting the input behavior information into behavior information with spatial characteristics through a formula (II);

in the formula (II), W_jlIs the weight parameter and initial value of a unit of the Capsule layerDefaults to a random number that generates a truncated normal distribution;

b_lthe offset parameter is a bias parameter of a first unit of a Capsule layer, and the default value of the initial value is 0.0;

the squnah () function is a new nonlinear function, similar to the previous common nonlinear functions such as tanh (), relu (), and the squnah () function is a nonlinear process oriented to vector information; while other non-linear functions are primarily directed to the processing of scalar information;

the vector behavior information characteristics are output by a Capsule network;

the size of the information output after passing through a Capsule layer unit is as follows:

(4) inputting the behavior information with the space characteristic into a Capsule layer two unit, and processing the behavior information through dynamic routing protocols, namely formulas (III) and (IV);

in the formulae (III) and (IV),

b_ikthe dynamic routing weight of the ith neuron in the first Capsule layer unit and the kth neuron in the second Capsule layer unit is referred to;

b_ijthe dynamic routing weight of the ith neuron in the first Capsule layer unit and the jth neuron in the second Capsule layer unit is referred to;

refers to the output of each Capsule layer;

S_jthe behavior information characteristics are output after the Capsule layer two unit passes through a dynamic routing protocol.

Is a vector output that indicates the network architecture;

the size of the information output after the processing of the second unit of the Capsule layer is as follows: batch _ Size × Num _ Output × Vec _ Lenv × 1;

(5) converting the behavior information from a vector to a scalar through a full connection layer unit;

the size of the information output after passing through the full connection layer unit is as follows:

Batch_Size×Output_Length×1；

(6) adding a Softmax classifier, and performing classification identification on the behavior information through the Softmax classifier; the behavior information characteristics with the information Size of Batch _ Size × Output _ Length × 1 are subjected to solution of each behavior probability through a classifier, and the behavior with the maximum corresponding probability value, namely the behavior with the maximum probability value, is found out as the final recognition result of the network architecture module.

The invention has the beneficial effects that:

1. the practicability is as follows:

the real-time human behavior recognition has higher requirements on transmission media and accuracy, and the invention well realizes low power consumption and long-distance transmission of behavior information; meanwhile, the method has certain advantages in the aspect of accuracy of behavior identification.

2. Self-adaptability:

aiming at different application scenes, parameters in the system are adjusted through a user feedback unit (QoE) and an error correction unit (QoS), the adaptability of the system is improved, and personalized and intelligent services are provided for users. Wherein the adjustable parameters include: the threshold value of the inconsistency detection/quantization unit and the threshold value of the incomplete detection/quantization unit, the standardization mode in the behavior information standardization unit, the size and the sliding mode of the sliding window in the behavior information sliding window unit, the iteration times, the learning rate, the training iteration times and other parameters in the network architecture module.

3. High reliability:

after the behavior information based on the sensor is added to the single information source based on the visual behavior information, the behavior recognition system is more complete; compared with the behavior recognition of a mainstream model algorithm, the method has the advantage that the accuracy is further improved. Has good stability in real-time.

Drawings

FIG. 1 is a schematic diagram of a main module structure and a connection relationship of a real-time behavior recognition system based on Lora and Capsule.

FIG. 2 is a schematic diagram of module composition and connection relationship realized by the real-time behavior recognition system based on Lora and Capsule.

FIG. 3 is a schematic workflow diagram of the real-time behavior recognition system based on Lora and Capsule according to the present invention.

FIG. 4 is a schematic diagram of behavior recognition of the real-time behavior recognition system based on Lora and Capsule according to the present invention.

FIG. 5 is a schematic diagram of the working principle of a first Capsule layer unit and a second Capsule layer unit in the behavior recognition of the real-time behavior recognition system based on Lora and Capsule.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more clearly and completely understood, the technical solutions of the present invention are described below with reference to the following embodiments and the accompanying fig. 1-5 of the specification, and it is obvious that the specific embodiments described herein are only used for explaining the present invention and are not used for limiting the present invention.

Example 1

A real-time behavior recognition system based on Lora and Capsule is shown in figure 1 and comprises a behavior information physical layer, a behavior information access layer, a behavior information platform layer and a behavior information application layer which are sequentially connected;

the behavior information physical layer is used for: sensing, collecting, storing and transmitting user behavior information from the environment, wherein the behavior information comprises: acceleration, angular velocity, heart rate;

the behavior information access layer is used for: networking and transmitting the collected behavior information through a Lora technology in a low-power-consumption wide-area Internet of things;

the behavior information platform layer is used for: sequentially carrying out uncertainty detection, standardization and interception based on a time sequence on the behavior information, training a behavior information set with a label under a built network architecture model, and finding out an optimal model while continuously optimizing a loss value; the uncertainty detection means that: incomplete or inconsistent information in the behavior information is processed through a context prediction filling, 0 complementing and deleting method, so that the reliability of the behavior information is improved; the standardization is to carry out normalization processing on numerical data, so that the accuracy and generalization capability of the model are improved; intercepting based on the time sequence means intercepting behavior information through a sliding window mechanism so as to ensure the normalization of model input and improve the accuracy of the model;

the behavior information application layer is used for: and adjusting the stability and the adaptability of the whole real-time behavior recognition system.

The invention provides a relatively optimized system in four aspects of transmission technology, information processing, behavior recognition, behavior application and the like, overcomes the defects that the real-time performance of the recognition in the current market is poor and the recognition cannot be used in a specific area, further improves the accuracy of the behavior recognition, and ensures that the system has stability.

Example 2

A real-time behavior recognition system based on Lora and Capsule as described in embodiment 1, as shown in fig. 2, is different in that,

the behavior information physical layer is a behavior information acquisition module, and the behavior information acquisition module comprises a sensor module and a plurality of intelligent hardware modules; the sensor module comprises a plurality of sensors of different types, the intelligent hardware module is respectively connected with the sensors of different types and is used for controlling the sensors to sense behavior information of different types of users and storing the sensed behavior information.

In the behavior information acquisition module, the selection of the sub-modules and the design of the intelligent equipment are carried out according to QoD parameters, application scenes and user requirements of the sub-modules, and QoD parameters mainly comprise sampling frequency, service life, precision and the like of the sensor module.

And the intelligent hardware module adopts Lora to transmit the behavior information. The transmission technology is a technology which makes full use of the transmission capability of different channels to form a complete transmission system so that information can be reliably transmitted. With the progress of society and the development of wireless technology, the convenience of wireless transmission is further amplified on the premise of low requirement on packet loss rate. At present, the mainstream wireless technologies mainly include WiFi, bluetooth, ZigBee, 3G, 4G, and the like, and each wireless technology is in a situation that transmission distance and power consumption are not compatible, but in order to realize real-time behavior recognition, transmission of behavior information requires a wireless technology with low power consumption and long transmission distance. Low power local area network (LPWAN) is the main technology to address the current situation, so the present invention uses Lora to transmit behavior information.

The behavior information access layer is a behavior information transmission module, and the behavior information transmission module comprises a behavior information sending module and a behavior information receiving module; the behavior information sending module is connected with the intelligent hardware module and used for sending the behavior information to the behavior information receiving module.

The information sending module is a Lora node, and the behavior information receiving module is a Lora base station. In view of the requirements of real-time performance and scenes, the invention selects the Lora nodes and the Lora base stations as transmission media.

The behavior information platform layer is a behavior information preprocessing module, and the behavior information preprocessing module comprises a behavior information detection module, a behavior information uncertainty elimination module, a behavior information processing module and a network architecture module which are connected in sequence;

the behavior information detection module comprises an inconsistency detection/quantification unit and an incomplete detection/quantification unit; the behavior information uncertainty eliminating module comprises an inconsistency eliminating unit and an imperfection eliminating unit; the behavior information processing module comprises a behavior information standardization unit and a behavior information sliding window unit which are sequentially connected; the network architecture module comprises a convolution layer unit, a first Capsule layer unit, a second Capsule layer unit and a full connection layer unit which are sequentially connected;

the behavior information receiving module is a gateway connection behavior information detection module;

the behavior information received by the behavior information receiving module, namely the original behavior information, is input into the behavior information detection module, the original behavior information is subjected to uncertainty detection through the inconsistency detection/quantification unit and the incomplete detection/quantification unit, the inconsistency detection/quantification unit detects whether different types of behavior information exist objections at the same moment, and the incomplete detection/quantification unit detects whether the perceived behavior information at the same moment is lost;

if the behavior information is found to have uncertainty, the uncertainty is eliminated through an imperfection eliminating unit and the inconsistency eliminating unit, the imperfection eliminating unit processes the loss condition of the perception behavior information at the same moment through a deleting method, a 0 complementing method and a context prediction filling method, the inconsistency eliminating unit processes the inconsistency information through voting, the QoD optimal principle of hardware, the D-S (Dempster-Shafer) evidence theory and the fuzzy set mode, and the inconsistency information enters a behavior information standardizing unit; QoC indexes are indexes used for describing behavior information quality, including completeness, credibility, updating degree and the like; if the behavior information is found to have no uncertainty, directly entering a behavior information standardization unit; the behavior information standardization unit and the behavior information sliding window unit are used for processing, and the behavior information standardization unit is used for processing through a standardization and normalization method, so that the identification accuracy and the applicability are improved; the behavior information sliding window unit intercepts the behavior information based on the time sequence by adjusting the size of the sliding window and the sliding mode of the sliding window;

inputting the processed behavior information into a trained network architecture model, and realizing behavior recognition through the network architecture model; the convolution layer unit extracts features from the behavior information and converts a feature scalar into a vector, and the Capsule layer I unit is used for converting the input behavior information into behavior information with spatial characteristics; the second unit of the Capsule layer processes the behavior information through a dynamic routing protocol; and finally, operating all the characteristics through a Softmax classifier, and identifying the current behavior.

The network architecture module mainly works to recognize according to behavior information, and in the fields of artificial intelligence and mode recognition, the strong artificial intelligence can be really realized by the proposal of machine learning, and the proposal of deep learning has a great progress in the aspect of recognition rate. However, the emphasis of both the machine learning model and the deep learning model is whether some feature values are included in the input information. The network architecture based on the Capsule adopted in the invention not only pays attention to the characteristics of the behavior information, but also adds the spatial relationship of the behavior information characteristics, thereby improving the accuracy of behavior identification.

The behavior preprocessing module is mainly used for improving the credibility of the behavior information through preprocessing the behavior information. Compared with some systems which directly perform behavior recognition on the original behavior information, the method and the system have great improvement in the aspects of stability, accuracy and the like after the behavior information is preprocessed. The invention mainly carries out uncertainty analysis on the original information and correspondingly processes the category and the degree of uncertainty of the behavior information. In the aspect of information standardization, the invention provides a normalization method and a normalization method. And intercepting the behavior information based on the time sequence by adjusting the size of the sliding window and the sliding mode of the sliding window.

The behavior information application layer comprises a behavior information threshold setting module and a behavior application layer adjusting module, and the behavior application layer adjusting module comprises a behavior identification unit, a user feedback unit and an error correction unit which are sequentially connected;

the behavior information threshold setting module is used for adjusting a threshold in the behavior information uncertainty eliminating module so as to determine whether the monitoring data has uncertainty or not, and adjusting the uncertainty processing module to select a data processing mode; the behavior identification unit is used for identifying the current behavior in real time; the user feedback unit adjusts the preset threshold value and the parameters of the network architecture module according to different scenes and user requirements, and improves the applicability of the system to a certain degree; the error correction unit continuously adjusts the network architecture module to make the network architecture module in the optimal state all the time. The behavior identification unit is mainly used for identifying behaviors according to the behavior information and a reasonable model; the QoS index value refers to the service quality, corresponding adjustment information is generated according to the service quality and then fed back to the behavior information preprocessing module; the method comprises the steps of obtaining a QoE index value of a user on the whole application service, generating feedback information and transmitting the feedback information to a behavior information preprocessing module; the QoE index value is a user scoring index used for expressing the satisfaction degree of the user on the application service, and mainly serves as adjusting a preset QoC index value.

Example 3

In the real-time behavior recognition system based on Lora and Capsule and the working method thereof in the embodiment 2, as shown in fig. 3, for example, when fighting behavior is recognized, criminals in prisons may slightly differ from common persons in terms of psychology and physiology, and may have a biased behavior in terms of treating problems. In order to prevent serious influence and harm caused by overexcitation, the system acquires the behavior information of a criminal in one day through the acceleration sensor S1, the angular velocity sensor S2 and the heart rate sensor S3, improves the reliability of the information after information preprocessing, and then carries out real-time behavior recognition through a trained model. The prison manager can set different parameters according to different scenes and different criminals to perform real-time behavior recognition. The method comprises the following specific steps:

step S01: obtaining QoD parameters

The main QoD parameters include: the precision of sensor, the material of sampling interval and bracelet. The accuracy of the sensor is 0.94, 0.80 and 0.88 respectively, the sampling interval is 0.02s-1s respectively, and the material is mainly composed of rubber, alloy and other materials.

Step S02: design collection device

The intelligent hardware is designed according to parameters of the sensor QoD, requirements of criminals and the security level, for key supervision objects, the behavior information acquisition equipment can be designed by adopting materials with high sampling frequency, high identification precision and low possibility of damage, and for good-performance and light supervision objects, the design can be designed by adopting materials with low sampling frequency, common identification and low manufacturing cost.

Step S03: transmission of behavioral information

According to the confidentiality and the size of the activity range, the Lora node is adopted to send the behavior information;

step S04: reception of behavioural information

Adopting a Lora base station to receive the behavior information; the sending equipment of the behavior information adopts a Lora node, and the receiving of the corresponding behavior information adopts a Lora base station.

Step S05: uncertainty detection of behavioral information

The incompleteness detection/quantification unit sets the threshold to be 0.85, namely 85% of the behavior information received every second is missing, and the original behavior information is considered to be incomplete;

the inconsistency detection/quantification unit sets the threshold value to be 0.8, namely, when the similarity of the behavior information received every second is lower than 0.8, the inconsistency of the group of original behavior information is judged.

And analyzing the original behavior information by carrying out inconsistency detection/quantification and incomplete detection/quantification on the original behavior information according to the set threshold range of the behavior information, and executing the step S06 when the original behavior information is found to have uncertainty such as inconsistency, incompleteness and the like, or executing the step S07.

Step S06: uncertainty elimination of behavioral information

If the original behavior information is found to be incomplete, the system can delete the incomplete behavior information, or complement 0 to the incomplete information, or fill the incomplete behavior information according to context information prediction, and the system can default to select to fill the incomplete behavior information according to the context information prediction;

if the original behavior information is found to have inconsistency, the system can modify the inconsistency information according to a voting principle, or modify the inconsistency information according to an QoD optimal principle of information acquisition hardware, or calculate the credibility of the inconsistency information by using a D-S evidence theory method for modification, and the system selects the inconsistency information according to the voting principle by default.

After the uncertainty of the behavior information is eliminated, the reliability of the original behavior information is greatly improved, and the reliability is provided for the processing of the subsequent behavior information and the identification of the behavior.

Step S07: processing of behavioral information

The standardization unit of the behavior information mainly standardizes the same type of behavior information, the standardization which can be used in the system is mainly a normalization method or a normalization method, and the default standardization of the system is the normalization method;

the behavior information sliding window unit mainly intercepts behavior information based on a time sequence, the system mainly provides two parameters of the size of a sliding window and the sliding mode, the size of the sliding window is 40, 60, 80 and 100, the sliding mode mainly comprises sliding based on a half time sequence and sliding based on a whole time sequence, the default size of the sliding window of the system is 80, and the sliding mode is sliding based on a half time sequence.

Step S08: behavior information network architecture

A four-layer network architecture module is constructed by a convolutional layer unit, a Capsule layer one unit, a Capsule layer two unit and a full connection layer unit, behavior information with labels is trained through N iterations with reference to some parameters preset by a user, loss functions are continuously optimized in the training process to optimize module parameters and dynamic routing protocols in the Capsule layer unit, and finally the module with high recognition rate is obtained. The training set can select the behavior information of all people or select the behavior information of a certain person to carry out a training module, and then carry out behavior recognition on the certain person. Because the scheme needs huge behavior information and needs larger resource support, the proposal is only used for partial heavy criminals, and the system selects the behavior information database of the whole prison criminal by default to train the module. As shown in fig. 4, a specific implementation flow of the modules used in this example is as follows:

setting the number of convolution kernels in a convolution layer unit to be 256, the size of each convolution kernel to be 1 multiplied by 41 and the step length to be 1;

setting the number of convolution kernels in a Capsule layer unit to be 32, the size of each convolution kernel to be 1 multiplied by 21 and the step length to be 2;

setting the output length of a second unit of the Capsule layer as 8-dimensional behavior information, wherein each dimension adopts 16 behavior information characteristics;

setting the length of output in the full connection layer unit to be 6;

the method comprises the following steps:

(1) inputting behavior information of 5 × 1 × 80 × 3 size;

(2) after behavior information of a size of 5 × 1 × 80 × 3 passes through the convolutional layer unit, the input behavior information is converted from a scalar to a vector by formula (i):

in the formula (I), X_iThe behavior information is subjected to uncertainty, standardization and sliding window processing based on time series; w_ijThe weight parameter refers to the weight parameter of the convolutional layer unit, and the initial value is a random number which generates truncated normal distribution by default;

b_jthe offset parameter of the convolutional layer unit is defined, and the default value is 0.0;

n represents the number of convolution kernels;

Y_jis representative of convolutional layer output;

the output information size is: 5 × 1 × 40 × 256; where it is necessary to ensure that the result of the fraction in the preceding formula is a positive integer. The output result at this moment is vector behavior information, which meets the input requirement of the Capsule network;

(3) as shown in fig. 5, the above 8 groups of convolutional layers are encapsulated in a Capsule, the result output by the convolutional layer unit is input to a Capsule layer one unit, and the input behavior information is converted into behavior information with spatial characteristics by formula (ii);

in the formula (II), W_jlThe weight parameter is a weight parameter of a unit of a Capsule layer, and an initial value is a random number which generates truncation normal distribution by default;

b_lthe offset parameter is a bias parameter of a first unit of a Capsule layer, and the default value of the initial value is 0.0;

the squnah () function is a new nonlinear function, similar to the previous common nonlinear functions such as tanh (), relu (), and the squnah () function is a nonlinear process oriented to vector information; while other non-linear functions are primarily directed to the processing of scalar information;

the vector behavior information characteristics are output by a Capsule network;

the size of the information output after passing through a Capsule layer unit is as follows: 5X 320X 8X 1;

(4) taking the output result of the first Capsule layer unit as the input information of the second Capsule layer unit, and processing the behavior information through dynamic routing protocols, namely formulas (III) and (IV);

in the formulae (III) and (IV),

b_ikthe dynamic routing weight of the ith neuron in the first Capsule layer unit and the kth neuron in the second Capsule layer unit is referred to;

b_ijthe dynamic routing weight of the ith neuron in the first Capsule layer unit and the jth neuron in the second Capsule layer unit is referred to;

refers to the output of each Capsule layer;

S_jthe behavior information characteristics are output after the Capsule layer two unit passes through a dynamic routing protocol.

Is a vector output that indicates the network architecture;

the size of the information output after the processing of the second unit of the Capsule layer is as follows: 5X 12X 16X 1;

(5) converting the behavior information from a vector to a scalar through a full connection layer unit;

the size of the information output after passing through the full connection layer unit is as follows: 5X 192X 1;

(6) adding a Softmax classifier, and performing classification identification on the behavior information through the Softmax classifier; the behavior information features with the information size of 5 multiplied by 192 multiplied by 1 are subjected to the solution of each behavior probability through a classifier, and the corresponding behavior with the maximum probability value, namely the behavior with the maximum probability value, is found out as the final recognition result of the network architecture module.

The system adjustable parameters mainly comprise parameters such as dynamic routing iteration times, learning rate, training iteration times and the like, and the dynamic routing iteration times are set to be 1-10; the learning rate is set to 0.1, 0.01, 0.001; and the number of training iterations is set to 1-50. The default parameters of the system are 5, 0.01 and 40 in sequence.

Step S09: identification of behavioral information

Inputting the behavior information acquired in real time into a trained network architecture model to perform real-time identification on the current behavior;

step S10: error detection

Judging whether the current behavior identification has errors, if so, executing the step S11, otherwise, executing the step S12;

step S11: error correction

The error correction unit adjusts the threshold range of the behavior information and the corresponding parameters of the behavior information processing module; the behavior information threshold range comprises an uncertainty detection threshold range, and the corresponding parameters of the behavior information processing module comprise the size of a sliding window in a behavior information sliding window unit and the sliding mode of the window; when the identification errors are more, the threshold range of the behavior information is properly increased, and the size of the sliding window and the sliding interval of the window are reduced;

step S12: user feedback detection

And judging whether the system has user feedback information, if so, executing step 13.

Step S13: user feedback

And performing feedback information according to different environments of different users, and adjusting some parameters preset by the users to adjust the behavior information processing module and the network architecture module. Parameters that may be adjusted include: the threshold value of an inconsistency detection/quantification unit and the threshold value of an incomplete detection/quantification unit in the original behavior information detection module, the standardization mode in a behavior information standardization unit and the sliding window size and sliding mode in a behavior information sliding window unit in the behavior information processing module, the iteration times, the learning rate, the training iteration times and the like in the network architecture module.

Claims (4)

1. A real-time behavior recognition system based on Lora and Capsule is characterized by comprising a behavior information physical layer, a behavior information access layer, a behavior information platform layer and a behavior information application layer which are sequentially connected;

the behavior information physical layer is configured to: sensing, collecting, storing and transmitting user behavior information from the environment, wherein the behavior information comprises: acceleration, angular velocity, heart rate;

the behavior information access layer is configured to: networking and transmitting the collected behavior information through a low-power-consumption wide-area Internet of things;

the behavior information platform layer is used for: sequentially carrying out uncertainty detection, standardization and interception based on a time sequence on the behavior information, training a behavior information set with a label under a built network architecture model, and finding out an optimal model while continuously optimizing a loss value; the uncertainty detection means that: incomplete or inconsistent information in the behavior information is processed through a context prediction filling, 0 complementing and deleting method, so that the reliability of the behavior information is improved; the standardization is to carry out normalization processing on numerical data; intercepting based on time series refers to intercepting behavior information through a sliding window mechanism;

the behavior information application layer is used for: adjusting the stability and the adaptivity of the whole real-time behavior recognition system;

the behavior information physical layer is a behavior information acquisition module, and the behavior information acquisition module comprises a sensor module and a plurality of intelligent hardware modules; the sensor module comprises a plurality of sensors of different types, the intelligent hardware module is respectively connected with the sensors of different types, and the intelligent hardware module is used for controlling the sensors to sense behavior information of different types of users and storing the sensed behavior information;

the behavior information access layer is a behavior information transmission module, and the behavior information transmission module comprises a behavior information sending module and a behavior information receiving module; the behavior information sending module is connected with the intelligent hardware module and used for sending behavior information to the behavior information receiving module;

the behavior information sending module is a Lora node, and the behavior information receiving module is a Lora base station;

the behavior information platform layer is a behavior information preprocessing module, and the behavior information preprocessing module comprises a behavior information detection module, a behavior information uncertainty elimination module, a behavior information processing module and a network architecture module which are connected in sequence;

the behavior information detection module comprises an inconsistency detection/quantification unit and an incomplete detection/quantification unit;

the behavior information uncertainty eliminating module comprises an inconsistency eliminating unit and an imperfection eliminating unit;

the behavior information processing module comprises a behavior information standardization unit and a behavior information sliding window unit which are sequentially connected;

the network architecture module comprises a convolution layer unit, a first Capsule layer unit, a second Capsule layer unit and a full connection layer unit which are sequentially connected;

the behavior information receiving module, namely a gateway, is connected with the behavior information detecting module;

the behavior information received by the behavior information receiving module, namely the original behavior information, is input into the behavior information detecting module, the original behavior information is subjected to uncertainty detection through the inconsistency detecting/quantifying unit and the incompleteness detecting/quantifying unit, the inconsistency detecting/quantifying unit detects whether different types of behavior information at the same moment are objected, and the incompleteness detecting/quantifying unit detects whether the perceived behavior information at the same moment is lost;

if the behavior information is found to have uncertainty, the uncertainty is eliminated through the imperfection eliminating unit and the inconsistency eliminating unit, the imperfection eliminating unit processes the loss condition of the perception behavior information at the same moment through an eliminating method, a 0 complementing method and a context prediction filling method, the inconsistency eliminating unit processes the inconsistency information through voting, an QoD optimal principle of hardware, a D-S evidence theory and a fuzzy set, and the inconsistency information enters the behavior information standardizing unit; if the behavior information is found to have no uncertainty, directly entering the behavior information standardization unit; the behavior information standardization unit and the behavior information sliding window unit are used for processing, and the behavior information standardization unit is used for processing through a standardization and normalization method, so that the identification accuracy and the applicability are improved; the behavior information sliding window unit intercepts the behavior information based on a time sequence by adjusting the size of the sliding window and the sliding mode of the sliding window;

inputting the processed behavior information into a trained network architecture model, and realizing behavior recognition through the network architecture model; the convolution layer unit extracts features from the behavior information and converts a feature scalar into a vector, and the Capsule layer I unit is used for converting the input behavior information into behavior information with spatial characteristics; the second unit of the Capsule layer processes the behavior information through a dynamic routing protocol; and the full connection layer unit converts the behavior information characteristics into ordered one-dimensional characteristics, and finally, all the characteristics are operated through a Softmax classifier to identify the current behavior.

2. The real-time Lora and Capsule-based behavior recognition system of claim 1,

the behavior information application layer comprises a behavior information threshold setting module and a behavior application layer adjusting module, and the behavior application layer adjusting module comprises a behavior identification unit, a user feedback unit and an error correction unit which are sequentially connected;

the behavior information threshold setting module is used for adjusting a threshold in the behavior information uncertainty eliminating module so as to determine whether the monitoring data has uncertainty or not, and adjusting the uncertainty processing module to select a data processing mode; the behavior identification unit is used for identifying the current behavior in real time; the user feedback unit adjusts the preset threshold value and the parameters of the network architecture module according to different scenes and user requirements, and the error correction unit continuously adjusts the network architecture module to enable the network architecture module to be in an optimal state all the time.

3. The method of claim 2, wherein the Lora and Capsule based real-time behavior recognition system comprises the steps of:

step S01: sensor sensing behavioral information

Screening different manufacturers and different types of sensors according to original behavior information required by behavior recognition and QoD parameters of the sensors, wherein the QoD parameters of the sensors comprise: sampling frequency, service life and precision, and sensing different types of behavior information of a user by a sensor;

step S02: designing an intelligent hardware module

Selecting a proper intelligent hardware module according to the scheme requirement, controlling each sensor through the intelligent hardware module, and acquiring behavior information required by a behavior recognition system;

step S03: transmission of behavioral information

Adopting Lora nodes to send behavior information;

step S04: reception of behavioural information

Adopting a Lora base station to receive the behavior information;

step S05: uncertainty detection of behavioral information

Setting a threshold range of behavior information, executing a step S05 when the original behavior information is inconsistent and incomplete, otherwise, executing a step S06; the original behavior information refers to different types of behavior information of the user sensed by the sensor in step S01;

step S06: uncertainty elimination of behavioral information

The method comprises the following steps that an incomplete eliminating unit processes behavior information by different methods through a threshold value of uncertainty detection of the behavior information, when the accuracy of the behavior information is 85% -90%, a context prediction filling method is adopted for the behavior information, when the accuracy of the behavior information is 90% -95%, a 0 complementing method is adopted for the behavior information, and when the accuracy of the behavior information is 95% -100%, a deleting method is adopted for the behavior information;

the inconsistency elimination unit processes the inconsistency information, and the processing method comprises voting, QoD optimal principle of hardware, D-S evidence theory and fuzzy set; the reliability of the original behavior information is improved;

step S07: processing of behavioral information

Standardizing the behavior information with higher credibility through a behavior information standardization unit; the standardization of behavior information uses different standardization approaches for different types of data, including: adopting one-hot coding standardization aiming at the data of the class type characteristics; normalization processing standardization is adopted for data of numerical characteristics; for the data of the ordered type features, the ordered type numerical code is adopted for standardization;

referring to the preset parameters of the user, the preset parameters of the user comprise: the size of the sliding window and the sliding mode of the window are used for performing sliding window processing on the behavior information after the standardized processing through a behavior information sliding window unit, so that the behavior information is changed into an information block which is input into a network architecture module;

step S08: behavior information network architecture

Constructing a four-layer network architecture model through a convolutional layer unit, a Capsule layer one unit, a Capsule layer two unit and a full-connection layer unit, training behavior information with labels through a plurality of iterations according to parameters set by a user, continuously optimizing model parameters and a dynamic routing protocol in the Capsule layer unit by reducing a loss function in the training process, and finally obtaining the network architecture model with high recognition rate;

step S09: identification of behavioral information

Inputting the behavior information acquired in real time into a trained network architecture model to perform real-time identification on the current behavior;

step S10: error detection

Judging whether the current behavior identification has errors, if so, executing the step S11, otherwise, executing the step S12;

step S11: error correction

The error correction unit adjusts the threshold range of the behavior information and the corresponding parameters of the behavior information processing module; the behavior information threshold range comprises an uncertainty detection threshold range, and the corresponding parameters of the behavior information processing module comprise the size of a sliding window in a behavior information sliding window unit and the sliding mode of the window;

step S12: user feedback detection

Judging whether the system has user feedback information, if so, executing step S13;

step S13: user feedback

And the user feedback unit performs feedback adjustment on the threshold range of the behavior information and the corresponding parameters of the behavior information processing module.

4. The working method of the real-time behavior recognition system based on Lora and Capsule according to claim 3, wherein in the step S08, the network architecture module comprises a convolutional layer unit, a Capsule layer one unit, a Capsule layer two unit, and a fully connected layer unit which are connected in sequence;

setting the number of convolution kernels in convolution layer unit to be N₁Each convolution kernel is 1 × Nuclear _ Size₁Step length of L₁；

Setting the number of convolution kernels in one unit of Capsule layer to be N₂Each convolution kernel is 1 × Nuclear _ Size₂Step length of L₂；

Setting the Output length of a second unit of the Capsule layer as Num _ Output dimension behavior information, wherein Vec _ Lenv behavior information characteristics are adopted in each dimension;

setting the Output Length in the full connection layer unit as Output _ Length;

the method comprises the following steps:

(1) inputting behavior information with the Size of Batch _ Size multiplied by 1 multiplied by Window _ Size multiplied by 3, wherein Batch _ Size refers to the number of the behavior information which runs in the network architecture module at a time, and Window _ Size refers to the length of the network architecture module which is input each time;

(2) after the behavior information of the Size of Batch _ Size × 1 × Window _ Size × 3 passes through the convolutional layer unit, the input behavior information is converted from a scalar to a vector by formula (i):

in the formula (I), X_iThe behavior information is subjected to uncertainty, standardization and sliding window processing based on time series; w_ijThe weight parameter refers to the weight parameter of the convolutional layer unit, and the initial value is a random number which generates truncated normal distribution by default;

b_jthe offset parameter of the convolutional layer unit is defined, and the default value is 0.0;

n represents the number of convolution kernels;

Y_jis representative of convolutional layer output;

the output information size is:

(3) order to

Encapsulating the M groups of convolution layers in a Capsule network, and acquiring behavior information Y of the vector_jInputting the behavior information into a Capsule layer one unit, and converting the input behavior information into behavior information with spatial characteristics through a formula (II);

in formula (II), M ═ M, W_jlThe weight parameter is a weight parameter of a unit of a Capsule layer, and an initial value is a random number which generates truncation normal distribution by default;

b_lthe offset parameter is a bias parameter of a first unit of a Capsule layer, and the default value of the initial value is 0.0;

the squnah () function is a new nonlinear function;

the vector behavior information characteristics are output by a Capsule network;

the size of the information output after passing through a Capsule layer unit is as follows:

(4) inputting the behavior information with the space characteristic into a Capsule layer two unit, and processing the behavior information through dynamic routing protocols, namely formulas (III) and (IV);

in the formulae (III) and (IV),

b_ikthe dynamic routing weight of the ith neuron in the first Capsule layer unit and the kth neuron in the second Capsule layer unit is referred to;

b_ijthe dynamic routing weight of the ith neuron in the first Capsule layer unit and the jth neuron in the second Capsule layer unit is referred to;

refers to the output of each Capsule layer;

S_jbehavior information characteristics output after the Capsule layer two unit passes through a dynamic routing protocol are represented;

is a vector output that indicates the network architecture;

the size of the information output after the processing of the second unit of the Capsule layer is as follows: batch _ Size × Num _ Output × Vec _ Lenv × 1;

(5) converting the behavior information from a vector to a scalar through a full connection layer unit;

the size of the information output after passing through the full connection layer unit is as follows:

Batch_Size×Output_Length×1；

(6) adding a Softmax classifier, and performing classification identification on the behavior information through the Softmax classifier; the behavior information characteristics with the information Size of Batch _ Size × Output _ Length × 1 are subjected to solution of each behavior probability through a classifier, and the behavior with the maximum corresponding probability value, namely the behavior with the maximum probability value, is found out as the final recognition result of the network architecture module.

Images