Summary of the invention
According to the one aspect of the application, a kind of safety defense monitoring system is provided, is greatly improved under unmanned public safetyReal time monitoring is become the monitoring of specific behavior motion detection by the efficiency for monitoring security protection.A security personnel are most in actual sceneIt can only monitor 5 unmanned retail shops and working strength is very big, can easily monitor 40 using a security personnel same after this programmeA unmanned retail shop, efficiency have 8 times of promotions.
The application combines deep learning and safety monitoring, identifies that some specific behaviors are dynamic according to business demand trainingMake, solves the problems, such as monitoring efficiency very well in conjunction with software systems, and attempt the identification of more complicated action behavior.
The safety defense monitoring system includes:
Video acquisition unit, for obtaining the video information of area to be monitored；
Action recognition unit is electrically connected with the video acquisition unit, the action recognition unit view for identificationThe video information that frequency acquiring unit obtains；
Display unit is electrically connected with the video acquisition unit and/or the action recognition unit, and the display unit is usedIn the information for the video information and/or action recognition unit output for showing the video acquisition unit acquisition.
Optionally, the action recognition unit runs deep learning processing system neural network based；
The deep learning processing system neural network based includes constructing first nerves according to preset tactic patternNetwork；
The preset tactic pattern includes: input layer, at least two convolutional layers, at least two pond layers and output layer.
Optionally, the first nerves network includes sequentially connected input layer, the first convolutional layer, the first pond layer,Two convolutional layers, the second pond layer, full articulamentum and output layer.
Optionally, the video acquisition unit includes: monitoring camera, and the monitoring camera obtains area to be monitoredAction video data；
Using the action video data training first nerves network, nervus opticus network is obtained.
Optionally, the video acquisition unit includes: multiple monitoring cameras, and multiple monitoring cameras are obtained wait superviseThe action video data in region are controlled, and are arranged by monitoring area without dead angle monitor mode；
Using the action video data training first nerves network, nervus opticus network is obtained.
On convenience store top surface and multiple cameras are installed at surrounding, interval, are adopted with carrying out image to customer from multiple anglesCollection is realized without monitoring dead angle.Each camera is to customer from entering in shop to the everything progress video capture left in shopRecord.
Optionally, described using the action video data training first nerves network, obtain nervus opticus networkThe step of include:
Obtain the first action video data set that the monitoring camera is collected into the area to be monitored；
Specifies behavior is labeled as to each video data in the first action video data set, obtains the first mark viewFrequency data set；
The first mark sets of video data is inputted into the first nerves network, propagated forward or back-propagating step by step,Obtain nervus opticus network.
Optionally, the safety defense monitoring system work step includes:
The video acquisition unit obtains the action video of area to be monitored, and the action video is sent to the movementRecognition unit；
The action recognition unit is obtained described dynamic using video acquisition unit described in the nervus opticus Network RecognitionMake whether contain specifies behavior in video, if recognizing the specifies behavior, the movement containing the specifies behavior is regardedFrequency and action mark data are sent to the display unit；
The display unit shows the action video containing the specifies behavior and movement labeled data.
Optionally, the action recognition unit is local computer or remote server；It is described by the action video andThe step of action mark data are sent to the action recognition unit, comprising:
The video acquisition unit obtains the action video of area to be monitored, and the action video is sent to and is located at remotelyThe action recognition unit on server；And/or
The video acquisition unit obtains the action video of area to be monitored, and the action video is sent to and is located locallyThe action recognition unit of computer.
Optionally, the action recognition unit includes skeleton pose identification module, the skeleton pose identification module: is used forMultiple image of i-th user acquired in monitoring camera to the setting of e-th of angle at jth moment and preceding j-n periodSequence carries out framework characteristic extraction respectively, obtains several framework characteristic figures, by the time sequencing of video capture, the skeleton is specialAfter sign figure arrangement, the motion profile of all skeleton joint points of i-th user's whole body is obtained as i-th user's skeleton pose information；
The skeleton pose identification module is connect with the video acquisition cell data.
N herein refers to the total duration for obtaining multiple image sequence.
It detects to obtain multiple users in image, and each user's body upper joint point position is identified and is positioned.It realizesDetection and tracking is carried out to human skeleton motion postures multiple in video simultaneously on Spatial Dimension and time dimension.Guarantee movementThe time continuity of posture avoids the problem that the irregular jump of interframe.Human motion posture information be in time series it is continuous,It only include the image information of user's skeletal joint motion profile.
Skeleton motion gesture recognition module not only detects multiple users in monitor video, while positioning each userArtis position can accurately identify taken article in user hand so as to track the motion profile of user's both hands, mouthImage or feed commodity image.It avoids interfering caused by commodity on shelf caused by camera angle problem is Chong Die with user and askTopic improves the accuracy judged commodity operation behavior user.
Skeleton motion gesture recognition module is solved only to detect single-frame images in previous video object detection, can not be obtainedThe trail change information for taking target athletic posture between video sequence, is referred to for monitoring personnel.
Optionally, the action recognition unit includes user behavior analysis module and alarm monitoring module, user behavior pointAnalysis module: for detecting the image obtained in each video comprising the i-th user, the image comprising the i-th user is collected as the i-th identityImage set generates the i-th user behavior sequence map by the i-th identity image set, and to the i-th user behavior sequence map and the boneFrame posture information carries out behavioural analysis, according to the behavioural analysis as a result, the alarm monitoring module is in the display unitShow the i-th user behavior sequence map and the skeleton pose information and behavioural analysis result；
The user behavior analysis module connects with the skeleton pose identification module, the alarm monitoring module data respectivelyIt connects, the alarm monitoring module is connect with the display unit data.
For user in convenience store's duration of stay, behavior is complicated and changeable, many determinations of the nature of the act cannot by a video image orOne section of video of person judges (as stolen behavior, including commodity of taking from shop, no clearing behavior and behavior of going out,It is possible that constituting pilferage behavior), in the application by obtain, analyze in each scene same user from get started to going out period justIn sharp shop, multiple angle camera all-the-way tracking videos obtain the continuous Motion Portrait brand and behavior sequence map of the user, toComprehensive analysis and judgement are made in the behavior at family.More correct judgement is made with the behavior to user.
Optionally, the behavioural analysis include: judge whether the i-th user is commodity of taking, if it is judged that be it is yes, thenJudge whether the i-th user settles accounts in information desk, if it is judged that otherwise to judge whether the i-th user gos out, if it is determined that knotFruit be it is yes, then in the behavior sequence map and the skeleton pose and warning message for showing the i-th user, if it is judged that beingIt is no, then it does not show；
Or
The behavioural analysis include: judge whether the i-th user takes commodity, if it is judged that be it is yes, then judge the i-th useWhether family settles accounts and gos out in information desk, if it is judged that being yes, the then behavioural analysis of the stopping to the i-th user.
User behavior trajectory diagram of the user in shop is drawn by skeleton pose identification module and user behavior analysis modulePicture, and pay close attention to behavior sequence map of user between shelf, checkout station and entrance.Such as: when detecting userAfter shelf take commodity, directly go to doorway, do not settle accounts movement or close to checkout stand then determine its there are larger theft risk,At this time by acquired image transmitting to monitoring room, and alarm is issued, pop up the monitored picture, so that monitoring personnel is to particular rowTo be judged, if it is judged that then starting anti-theft process for theft.Solution monitoring personnel, which takes a significant amount of time, to be browsedThe video of normal Shopping Behaviors user, improves monitoring efficiency.
Alarm monitoring module herein includes display module, is set in monitoring room, can be according to judging result in display mouldDisplay needs the image sequence of artificial judgment in block.
In the application safety defense monitoring system monitoring method the following steps are included:
The action video of video acquisition unit acquisition area to be monitored；
The action recognition chief of the Xiongnu in Acient China extracts the skeleton pose information of the i-th user in action video, action video described in Detection and ExtractionIn include the image of the i-th user, and collect as the i-th identity image set, the i-th user behavior sequence generated by the i-th identity image setMap, and behavioural analysis is carried out to the i-th user behavior sequence map and the bone posture information；
The behavioural analysis includes analyzing and determining to the motion profile and skeleton pose information of the i-th user, according to sentencingDisconnected result shows the motion profile and skeleton pose and warning message of the i-th user in alarm monitoring module；
The behavioural analysis include: judge whether the i-th user takes commodity, if it is judged that be it is yes, then judge the i-th useWhether family settles accounts in information desk, if it is judged that otherwise to judge whether the i-th user gos out, if it is judged that be it is yes, thenShow the behavior sequence map and the skeleton pose and warning message of the i-th user, if it is judged that be it is no, then do not show；
Or
The behavioural analysis include: judge whether the i-th user takes commodity, if it is judged that be it is yes, then judge the i-th useWhether family settles accounts and gos out in information desk, if it is judged that being yes, the then behavioural analysis of the stopping to the i-th user.
According to the another aspect of the application, a kind of unmanned convenience store is provided, comprising in above-mentioned safety defense monitoring systemAt least one, the area to be monitored of the safety defense monitoring system include the inlet of the unmanned convenience store, exit, shelf,At least one region in information desk, outside.
Optionally, the action recognition unit of the safety defense monitoring system trail for identification into behavior, carry pet intoEnter behavior, abnormal behavior of taking, eat/drink behavior, static behavior in shop, in behavior of persistently walking about outside the unmanned convenience storeAt least one behavior.
Optionally, the safety defense monitoring system is applied to the unmanned convenience store to realize at least one of following application:
(A) anti-trailing: having the ability that identification is trailed, and reminds user, stops enabling and/or alarm；
(B) pet identifies: recognizing the entrance of carrying pet or pet is strayed into, remind or alarm；
(C) internal abnormality movement monitoring；
(D) external abnormal operation monitoring.
The beneficial effect that the application can generate includes:
Safety defense monitoring system provided by the present application greatly improves the efficiency that security protection is monitored under unmanned public safety,Real time monitoring becomes the monitoring of specific behavior motion detection.In actual scene a security personnel can only at most monitor 5 nobody zeroIt sells shop and working strength is very big, can easily monitor 40 unmanned retail shops, efficiency using a security personnel same after this programmeThere are 8 times of promotions.
Embodiment 1
As the specific embodiment of the application, safety defense monitoring system provided by the present application includes: video acquisitionUnit 2, action recognition unit 4 and display unit 6.
Specifically, video acquisition unit 2 is used to obtain the video information of area to be monitored.
Action recognition unit 4 is electrically connected with video acquisition unit 2, the video acquisition unit 2 for identification of action recognition unit 4The video information of acquisition.
Referring to Fig. 1~3, display unit 6 is electrically connected with video acquisition unit 2 and/or action recognition unit 4.
Referring to fig. 2, as an optional embodiment, video acquisition unit 2 respectively with action recognition unit 4 and displayUnit 6 is electrically connected, and action recognition unit 4 and display unit 6 are electrically connected.
Display unit 6 is used to show what the video information that video acquisition unit 2 obtains and/or action recognition unit 4 exportedInformation.
As an optional embodiment, action recognition unit 4 runs deep learning processing system neural network basedSystem；
Deep learning processing system neural network based includes: to construct first nerves net according to preset tactic patternNetwork；
Preset tactic pattern includes: input layer, at least two convolutional layers, at least two pond layers and output layer.
As an optional embodiment, first nerves network includes sequentially connected input layer, the first convolutional layer,One pond layer, the second convolutional layer, the second pond layer, full articulamentum and output layer.Two convolutional layers pair are at least used in the applicationAcquired image is handled, feature needed for extracting.
As an optional embodiment, video acquisition unit 2 includes monitoring camera.Monitoring camera is obtained wait superviseControl the action video data in region；
Using action video data training first nerves network, nervus opticus network is obtained.
Nervus opticus is obtained using action video data training first nerves network as an optional embodimentNetwork includes:
Obtain the first action video data set that monitoring camera is collected into true environment；
Specifies behavior is labeled as to each video data in the first action video data set, obtains the first mark video countsAccording to collection；
First mark sets of video data is inputted into first nerves network, propagated forward or back-propagating, obtain second step by stepNeural network.
As an optional embodiment, safety defense monitoring system work step includes:
Video acquisition unit 2 obtains the action video of area to be monitored, and action video is sent to action recognition unit 4；
Whether contain in the action video that action recognition unit 4 is obtained using nervus opticus Network Recognition video acquisition unit 2There is specifies behavior, when recognizing specifies behavior, the action video containing specifies behavior and action mark data is sent to aobviousShow unit 6；
Display unit 6 shows the action video containing specifies behavior and action mark data.
As an optional embodiment, action recognition unit 4 is local computer or remote server；Movement is regardedFrequency is sent to action recognition unit 4, comprising:
Video acquisition unit 2 obtains the action video of area to be monitored, and action video is sent to positioned at remote serverAction recognition unit 4；And/or
Video acquisition unit 2 obtains the action video of area to be monitored, and action video is sent to and is located locally computerAction recognition unit 4.