A kind of intelligent safety and defence system of wisdom new cityTechnical field:
The invention belongs to security device manufacturing technology fields, and in particular to a kind of intelligent safety and defence system of wisdom new city, InCarry out that there is high accuracy in person recognition.
Background technique:
Video monitoring is the important component of safety and protection system, it is a kind of stronger integrated system of prevention ability.Each key area of wisdom new city, all deployment 4G video monitoring camera, realization social security and city are managed for emphasis place, regionIntelligent HD video construction is managed, 4G IP Camera can support mobile phone, computer remote real-time video monitoring；Video monitoring withIt is intuitive, accurate, timely abundant with the information content and is widely used in many occasions.In recent years, with computer, network andThe rapid development of image procossing, transmission technology, there has also been significant progresses for Video Supervision Technique.Not with Video Supervision TechniqueDisconnected development, application is also more and more extensive, also, requirement of the user to video monitoring is also higher and higher, more and more at presentUser is desirable with video monitoring system and realizes long-range monitoring, the case where to understand monitored site in real time.Due to video monitoringSystem is the processing for video data, is related to the operations such as a series of algorithm calling, compression, transmission and massive store, rightThe requirement of server performance itself is very high, can become the bottleneck that efficiency improves.However, if not using above-mentioned front monitoring front-end and prisonThe one-to-one configuration mode of control terminal, the communication interface of open a front monitoring front-end and multiple monitor terminals, although can be oppositeThe load of some servers is reduced, but there are biggish security risks.Therefore, the present invention seeks design to provide a kind of wisdom newThe intelligent safety and defence system in city can effectively improve video treatment effeciency, while improve the levels of precision of monitoring.
Summary of the invention:
It is an object of the invention to overcome drawbacks described above of the existing technology, seeks design and a kind of wisdom new city is providedIntelligent safety and defence system.
To achieve the goals above, a kind of intelligent safety and defence system of wisdom new city of the present invention passes through following technical sideCase is realized: data acquisition subsystem includes access control system, video monitoring system；Data processing submodule includes that face is knownProcessing module is invaded in other module, abnormal behaviour analysis module, anomalous event monitoring modular, region；Cloud storage subsystem includesVideo data module, smart machine data memory module；Alarm system includes abnormal alarm module；Public administration platform includesThere is video management platform；According to preset strategy, each system carries out linkage collaborative work, and the present invention is by hardware device, video tubePlatform, mass data, intelligence AI parser, connection cloud combine, and realize intelligent safety and defence system.
Data acquisition subsystem includes a variety of smart machines of access control system, video monitoring system and installation；
Access control system is the 4G face snap video camera disposed in the bayonet position of key area, to realize ultra high densityFace snap, in the case where not contacted directly with equipment, video camera with fixed frequency acquire face dynamic menu, to eachThe facial image application face recognition algorithms that frame detects, the portrait picture of different angle is separated from background, is savedInto face database；Face recognition algorithms realize face characteristic extraction, Face datection, Face detection, face tracking, Characteristic Contrast,The process of face early warning；When portrait moves in the range of camera is shot, face recognition algorithms can automatically be carried out portraitIt tracks, whether it is the same person that the registered portrait in the portrait and face database captured does comparison and verifies, because of face databaseIn store the face picture of different angle, face recognition algorithms support side face and pitch angle detection, and with 99.99% faceRecognition accuracy, face information compare successfully, and gate mouth is open, and the linkage of alarm system is realized in comparing result failure, prompt pre-Alert information, gate mouth can not be opened；
4G face snap video camera embeds Intelligent human-face recognizer module, realizes recognition of face, 4G in video camera front endFace snap video camera acquires face picture with fixed frequency, and Intelligent human-face recognizer module can filter a large amount of unrelated images lettersBreath only retains the image comprising face picture and handle and image data is transferred to connection cloud by 4G network depositingStorage；
The 4G face snap video camera of embedded Intelligent human-face recognizer module, edge side (edge calculations are conducive toReal-time data analysis and intelligent processing, edge side are equivalent to a module, close to device end, realize data filtering andAnalysis, can be improved the utilization efficiency of data, and intelligent video AI Processing Algorithm keeps whole system intelligent in conjunction with edge calculationsIt is higher) data of processing acquisition nearby on the picture pick-up device of front end, cloud is uploaded to without mass data, by data source header sideProximal end service is provided, application program is initiated in edge side, can generate faster web services response, reduce time delay and transmissionCost greatly reduces the transmission cost in broadband and the load of connection cloud storage；
Intelligent human-face recognizer module in access control system is provided with self-study module, can be with the hair of testing staffThe variations such as type, the colour of skin, age dynamic updates face database, remains that the face template of database is newest, so as to everyIt is secondary correctly to identify face；
The 4G candid camera installed in access control system is equally applicable to the candid photograph to suspect vehicle, by the vehicle vehicle of candid photographBoard information, candid photograph time, candid photograph place etc. information upload to connection cloud by TCP network protocol and carry out unified storage, andEstablish the personal information database in emphasis place；
Video monitoring system: the collected data of 4G network monitoring video camera are mostly non-structured, abundantThese data are excavated it is necessary to carry out the structuring of video data processing, using video structural technology (including space-time dividing skillArt, Feature Extraction Technology, Identifying Technique of Object, deep learning technology), management is uniformly accessed into collected video resource, it is realExisting vehicle structure, pedestrian's structuring, human face structure, anomalous event occur to video monitoring regional, (region intrusion event is stumbledLine detecting event, crowd massing event, delay event of hovering) it is analyzed, by the intellectual analysis to video data, in magnanimityKey message is extracted in video pictures, the semantic description of style of writing of going forward side by side originally uses video preferably；
The video data volume is huge, if photographic technique uploads to cloud center, needs to occupy massive band width, and cloud is handledData are also required to the regular hour, increase user's request response time, human body behavior are analyzed from monitor video and to exceptionBehavior carries out alarm and needs to have high requirement to the processing response time of video, is handled and is calculated beyond the clouds and is not existingIt is real.
4G monitoring device front end bayonet is embedded with intelligent AI Processing Algorithm module, to carry out preliminary video image screeningProcessing enables magnanimity monitoring data to provide processing service nearby in edge side, screens out a large amount of unrelated video image informations；It willMonitor video data are transferred to connection cloud through 4G network by image data after treatment, and video management platform can be directCloud video data is accessed, entire video management center can carry out real time inspection to entire monitoring area and historical record is looked intoIt askes, understands monitoring area dynamic change；Video data is stored in corresponding structuring number after intelligence structureization processing, classificationAccording to warehouse, such as human face photo database, facial feature database, behavior picture and property data base, vehicle image and feature database,Comprehensive different data warehouse and associated video segment warehouse can establish corresponding search engine, realize to all kinds of dataThe depth information in warehouse excavates, and improves the efficiency that data are handled beyond the clouds.
In order to analyze the moving target in video, and different types of target is classified to be put into inhomogeneityIn the database of type, facilitate the intelligent retrieval in later period, need to video carry out structuring processing, to the moving target in video intoRow segmentation uses, and specific implementation is as follows:
Data structured processing is carried out to video image using the flooding mechanism of unsupervised Video segmentation, in video sequence x={ x1,...,xFF frame xi, initial prospect conspicuousness estimation is distributed on i ∈ { 1,2 ..., F }, using super-pixel by each frameImage is divided into a group node, and uses the semantic relation in the estimation of overall situation Neighborhood Graph and coded frame with interframe, is added by oneWeigh row random neighborhood matrixIndicate global Neighborhood Graph, wherein N is the node total number in video.Execute each nodeInitialization prospect conspicuousness estimationDiffusion be by present node estimation and Neighborhood matrix G repetition matrix phaseMultiply to realize, as the t times diffusing step vt=Gvt-1.Unsupervised Video Segmentation Neighborhood matrix G based on diffusion and justBeginningization v0As unique input；
Saliency foreground moving, u are indicated using uiAnd piRespectively indicate the conspicuousness prospect and picture in the i-th frame imageElement individually handles each frame x to calculate conspicuousness estimationi, wherein i ∈ { 1,2 ..., F }, gives a frame image xi,Reference image vegetarian refreshments piIntensity value,It indicates in the i-th frame and i+1 frame with pixel piObject fortune is measured for unitDynamic light stream vector, βiIndicate the boundary pixel set at frame i.
It is based onWithThe conspicuousness sport foreground in video frame i is calculated, appointing for the i-th frame image is each measuredWhat pixel piWith boundary set βi.First distanceCalculate the pixel p observed on boundaryiWith public flow direction itBetween minimal flow direction difference；Second distanceCalculate pixel piPath cost function between boundary pixel；WithAll capture pixel piSimilitude between the movement and background motion at place；
Calculate the difference in light stream direction: pixel p of the light stream direction difference at video frame iiAnd border betaiPublic light stream sideTo；Gather firstly, k-means is used to cluster boundary light stream direction for K, wherein k ∈ { 1 ..., K }, by cluster centre pointIt covers in set, is accomplished by
WhereinIndicate distribution pixel piTo the index in cluster k, r represents the series connection of all target variables；It willKiIt is updated to only comprising being assigned with the center more than 1/6 boundary pixel, in the case where giving cluster centre, by captured frame iPixel piOptical-flow between the key light stream direction observed in the boundary frame i, is realized by following formula:
It calculates minimum potential barrier distance: calculating pixel p in video frame iiWith the minimum potential barrier distance D of boundary pixelbd,i, pass throughFollowing formula is calculated:
WhereinIndicate path, edge aggregation connects pixel piWith boundary pixel s ∈ βi, by being calculated most in frame iSmall spanning tree, edge weights wi(e) it is used to minimum spanning tree and potential barrier distance in calculation formula (1), by calculating adjacent pixelMaximum light stream direction difference value obtains:E=(pi,qi) it is two pixel ps of connectioniAnd qiSide；Minimum spanning tree is calculated using the method in 4 connection fields.The potential barrier distance of point-to-point transmission is calculated as point-to-point transmission minimum to generateThe difference of the maximum side right and minimum side right on path is set, the minimum value conduct of potential barrier distance should between any point on the point and boundaryThe minimum potential barrier distance of point；
It is significant to calculate foreground moving type: by two distance metrics after normalizationWithTo obtain in video frame iThe conspicuousness foreground target u of pixelationi；
Neighborhood Graph is calculated by calculating Neighborhood matrix G=T × E × V, wherein T is inter-frame information, and E is frame information,V is long-range (long-range) components.
Interframe timing information: interframe timing information is obtained by extracting Optic flow information, to prevent the light stream at moving boundariesEstimation inaccuracy, while self-consistent property, super-pixel is connected between the light stream vectors of adjacent interframe；To calculate light streamNeighborhood matrix T, continuous two needles image xiAnd xi+1, each frame image includes pixel piAnd pi+1, calculate the forward light of each pixelFlow field fi(pi) and backward optical flow field bi+1(pi+1), optical flow field obtained by calculation defines video frame xiIn pixel piBefore placeTo confidence scoreRealize that formula is as follows:
Backward confidence scoreIt is accomplished by
Wherein σ2For hyper parameter；Confidence score measures pixel piWith pass through pi+fi(pi) optical flow field followed into frame xi+1Later as a result, and passing through pi+fi(pi)+bi+1(pi+fi(pi)) return to frame xiIn result；It is calculated using confidence scoreTwo super-pixel s in frame i and frame i+1i,kAnd si+1,mBonding strength, realized by formula:
Wherein δ () indicates target function, | si,k| and | si+1,k| it represents in si,kAnd si+1,mThe pixel quantity at place；Above formulaMiddle first item compares with super-pixel si,kStart, si+1,mThe bonding strength of end be derived from si,kAnd si+1,mOverall strength；Section 2It is similar to indicate；
Spatial information prevents the diffusion of vision edge in frame in frame, if do not separated by strong edge, allows information sameIt is propagated between adjacent super-pixel in one frame, for the space E for finding edge perception, uses the side of trained frame and interframe firstEdge response calculates confidence score A (s) by the method summed to attenuation function using skirt response for all super-pixel, realIt is now as follows:
Wherein G (p) ∈ [0,1] indicates the skirt response at pixel p, σwHyper parameter is represented with ε；
Edge, which is calculated, by above-mentioned marginal information perceives Neighborhood matrix E:
If si,kSpatially close to si+1,m, that is, super picture of the centre distance less than 1.5 times between two super-pixelPlain size square root；
If having visual similarity between two super-pixel, the long connection of view-based access control model similitude allow far from the time orIt is propagated between super-pixel spatially, long connection propagation information can be made more efficient by neighbour's image, to calculate viewFeel similarity matrix V, these super-pixel and super-pixel si,mRelationship is the closest, k nearest neighbor search is first carried out, to eachSuper-pixel si,m, s is found in the time range of r framei,mK arest neighbors, feature space using Euclidean distance calculateThe distance between super-pixel；
The feature f (s) of super-pixel is calculated by connecting LBP the and RGB histogram of pixel in the super-pixel being calculated,It simultaneously further include the x and y coordinates of HOG feature and super-pixel center；
By defining visual similarity matrix, make super-pixel si,mK arest neighbors pass through N (si,m) reference, visual similarityMatrix is expressed as:
Wherein σ is hyper parameter, and f (s) indicates the character representation of super-pixel s；
After carrying out structuring processing to multi-path monitoring video image, a large amount of hashes in image are not only reduced, are obtainedUseful data, and reduce the load capacity of cloud storing data.In the accuracy of target identification, due to human body behaviorNon-rigid characteristic so that behavior have diversity and variability, the image processing algorithm can to Different Individual generate it is sameA kind of human body behavior has relatively high accuracy, enables using in practice.In order to meet actual application demand, depending onThe processing speed of frequency image 30 frame images of processing per second, can achieve the effect that quasi real time.
The foreground target detected by algorithm needs to judge prospect mesh by IOU and accuracy formula evaluation indexThe IOU ratio of target accuracy, target and realistic objective that algorithm detects is bigger, and the foreground detection effect for representing the algorithm is got overGood, accuracy is higher.
Multi-channel video monitor picture can be by video management platform real time inspection, to the key area in different monitoring rangeDifferent grades of fence is set as security area, is calculated per the insertion intelligence AI Human bodys' response of monitoring camera front end all the wayWhether the fence of method analysis monitored picture setting there are abnormal vehicle, pet, residue, the personnel that cross the border, non-working personIt, will be before analysis treated result feedback to management platform after the processing of AI Human bodys' response algorithmic preliminaries Deng invasion objectEnd, if detecting abnormal behaviour (fall down, hover) and anomalous event (assemble, fight, residue, fence cross the border), video letterAutomatic spring is abnormal the picture of behavior and anomalous event by breath management platform, and carries out video recording storage, and administrative staff verifyThe case where whether warning message belongs to erroneous judgement, however, it is determined that be abnormal, staff is handled in time；Video monitoring system and reportAlert system links, and by the way of the automatic spring warning message in monitored picture, had both improved security personnel and has handled policeThe correctness of feelings can also solve a case rapidly for security personnel and provide firsthand information and evidence；
Video background service provides the video monitoring service function based on cloud computing, and passes through protocol interface and external callPerson communicates, while 4G monitoring camera can support mobile phone, computer remote to check real-time video monitoring picture；
4G monitors photography/videography machine for complex application context, software and algorithm that can be different according to displaying on-demand loading,By cooperateing between multi-feature extraction and identification, multiple-camera, the collaboration between cloud is held to double up intellectual analysis efficiency, it is built-inRealtime graphic quality testing and evaluation of properties, and have self perception and scene adaptability learning ability, allow algorithm and application notDisconnected iteration and evolution；
A variety of smart machines include intelligent closestool sitting alarm set, fuel gas alarm, infrared detector, interior UWB fixedA variety of smart machines such as position system；Intelligent closestool sitting alarm set has sitting reminding module, and to going to the toilet, sitting personnel are carried outAbnormity prompt, to prevent accident occurs；Fuel gas alarm measures indoor smokescope, if smokescope is beyond defaultValue, alarm sound an alarm；IR intrusion detector carries out infrared wiring to great security protection region, when in the region, appearance can be movedInfrared ray will be offended when object, infrared probe detects exception, issues alarm, a variety of intelligent bases of indoor locating system arrangementIt stands, personnel, which carry positioning label, can determine the position of personnel；
Data processing module: data processing system carries out collected data through embedded intelligent algorithm to initial dataScreening Treatment filters irrelevant information, completes to retain useful letter to various treatment processes such as the processing, arrangement, calculating of informationBreath, and stored, information is uploaded to by connection cloud by 4G network.
Connection cloud: connection cloud mainly stores the information of various types of data formats, including monitor video data,Intelligent AI algorithm (Intelligent human-face recognizer module, abnormal behaviour recognizer, anomalous event recognizer), database purchaseInformation (various smart machine basic information managements, face information database, the storage of various warning messages)；Through each smart machineAfter edge data processing, connection cloud is uploaded to, connection cloud stores a plurality of types of data, carries out cloud reception by 4G networkInformation from different intelligent equipment, a variety of data are unified in the storage of connection cloud, facilitate the retrieval and inquiry in later period.
Alarm module:
Local area network is formed by router between smart machine in representative region scene, includes to enclose to monitoring areaRegion including wall, important entrance, elevator, corridor, strategic road, Underground Garage, alarm system need and IR intrusion detector, cigaretteSense alarm, access control system and monitoring system link, if detecting exception, alarm song, weight immediately occurs by loudspeakerThe different fence grades of point monitoring area setting, detect region invader or fence cross the border etc. abnormal behaviours andWhen anomalous event, there is pop-up alarm, gate inhibition's recognition of face of important entrance setting in video management platform, once reach alarmGate inhibition's condition, will automatic trigger alarm system.
Video management platform: management platform realizes the intelligent management to whole system, connects connection cloud, multitude of video numberIt is stored in connection cloud according to the data of, smart machine acquisition, video management platform and connection cloud communicate to connect, and are able to carry outIt checks, video management platform includes following module:
1) basic information management: the personnel in key area are managed；
2) equipment state management includes the intellective video monitoring device of installation, infrared alarm equipment, gas alarm equipment, determinesThe working conditions such as position equipment are managed, and are configured to the attribute of equipment, are realized that the additions and deletions of equipment change and look into, the database of useFor MySQL；
3) it region invasion management: alarms immediately once triggering in the region of setting；
4) abnormal behaviour/incident management: saving type, place and time that the abnormal behaviour/event occurred every time occurs,Convenient for statistics；
5) alarm indication and confirmation: the alarm screen of video monitoring management platform pop-up needs staff to confirm,The case where to prevent there is erroneous detection or missing inspection；
6) historical record is inquired: including monitor video inquiry, alarm logging inquiry；To the alarm logging of preservation with the time,The conditions such as event type, behavior type, scene are inquired；
7) recognition of face is shown: showing the face picture that the access control system of entrance is captured, the facial image of acquisition and peopleFace picture in face library is compared；
8) fence is arranged: different fence grades is arranged in different zones；
9) real-time video monitoring: the module has the function of adopting, passes, deposits, seeing, obtains per the video flowing monitored all the wayLocation transmits camera acquired image data by 4G network, and data are stored in connection cloud, can be appointed in front endWherein monitoring carries out real-time display to meaning calling all the way, and to original video picture by being shown after intelligence AI algorithm process by switchingShow button, the monitored picture that can show that treated, and behavior label and behavior accuracy to people in frame out, real-time video are drawnException is detected in face, pops up abnormal alarm dialog box, while saving abnormal picture, waits relevant staff to handle abnormal；4G camera hardware is carried in the front end of video management platform, collected data is transmitted to background video management platform, in real timeMonitor video picture, the AI intelligent algorithm of rear end can analyze the human body behavior in video.
Compared with prior art, the present invention what is obtained has the beneficial effect that:
(1) video data structure that monitoring camera can acquire camera in the intelligent AI algorithm that marginal end is embedded inProcessing is different types of target, facilitates the inquiry of user and transfers.Since data are first handled in marginal end, screen out a large amount ofThe data of redundancy, then it is transferred to cloud, greatly reduce the load pressure of cloud data.
(2) using the methods of video segmentation of conspicuousness movement guidance in image, mesh is realized in the segmentation to foreground targetTarget subdivision, compared to other foreground object segmentation methods, video of this method in the case where handling complex background and nonrigidThere is significant effect when moving target.
(3) this method is different from supervised learning method, does not need to select great amount of samples and mark interested in sampleTarget, this method oneself can learn the target divided the image into out and sort out.
Summary, body design is skillfully constructed, and safe and convenient to use, image processing efficiency and accuracy rate greatly improve, and answersWith environmental-friendly, wide market.
Detailed description of the invention:
Fig. 1 is image processing flow schematic diagram of the present invention.
Fig. 2 is video monitoring equipment of the present invention and connection control planning schematic illustration.
Fig. 3 is video management platform feature schematic illustration of the present invention.
Fig. 4 is overall topological structure schematic illustration of the present invention.
Fig. 5 is function structure schematic illustration of the present invention.
Specific embodiment:
The invention will be further described by way of example and in conjunction with the accompanying drawings.
Embodiment 1:
A kind of intelligent safety and defence system for wisdom new city that the present embodiment is related to is accomplished in that data acquisition systemSystem, includes access control system, video monitoring system；Data processing submodule includes face recognition module, abnormal behaviour analysisProcessing module is invaded in module, anomalous event monitoring modular, region；Cloud storage subsystem includes video data module, intelligently setsStandby data memory module；Alarm system includes abnormal alarm module；Public administration platform includes video management platform；According toPreset strategy, each system carry out linkage collaborative work, and the present invention is by hardware device, video management platform, mass data, intelligenceAI parser, connection cloud combine, and realize intelligent safety and defence system.
Data acquisition subsystem includes a variety of smart machines of access control system, video monitoring system and installation；
Access control system is the 4G face snap video camera disposed in the bayonet position of key area, to realize ultra high densityFace snap, in the case where not contacted directly with equipment, video camera with fixed frequency acquire face dynamic menu, to eachThe facial image application face recognition algorithms that frame detects, the portrait picture of different angle is separated from background, is savedInto face database；Face recognition algorithms realize face characteristic extraction, Face datection, Face detection, face tracking, Characteristic Contrast,The process of face early warning；When portrait moves in the range of camera is shot, face recognition algorithms can automatically be carried out portraitIt tracks, whether it is the same person that the registered portrait in the portrait and face database captured does comparison and verifies, because of face databaseIn store the face picture of different angle, face recognition algorithms support side face and pitch angle detection, and with 99.99% faceRecognition accuracy, face information compare successfully, and gate mouth is open, and the linkage of alarm system is realized in comparing result failure, prompt pre-Alert information, gate mouth can not be opened；
4G face snap video camera embeds Intelligent human-face recognizer module, realizes recognition of face, 4G in video camera front endFace snap video camera acquires face picture with fixed frequency, and Intelligent human-face recognizer module can filter a large amount of unrelated images lettersBreath only retains the image comprising face picture and handle and image data is transferred to connection cloud by 4G network depositingStorage；
The 4G face snap video camera of embedded Intelligent human-face recognizer module, edge side (edge calculations are conducive toReal-time data analysis and intelligent processing, edge side are equivalent to a module, close to device end, realize data filtering andAnalysis, can be improved the utilization efficiency of data, and intelligent video AI Processing Algorithm keeps whole system intelligent in conjunction with edge calculationsIt is higher) data of processing acquisition nearby on the picture pick-up device of front end, cloud is uploaded to without mass data, by data source header sideProximal end service is provided, application program is initiated in edge side, can generate faster web services response, reduce time delay and transmissionCost greatly reduces the transmission cost in broadband and the load of connection cloud storage；
Intelligent human-face recognizer module in access control system is provided with self-study module, can be with the hair of testing staffThe variations such as type, the colour of skin, age dynamic updates face database, remains that the face template of database is newest, so as to everyIt is secondary correctly to identify face；
The 4G candid camera installed in access control system is equally applicable to the candid photograph to suspect vehicle, by the vehicle vehicle of candid photographBoard information, candid photograph time, candid photograph place etc. information upload to connection cloud by TCP network protocol and carry out unified storage, andEstablish the personal information database in emphasis place；
Video monitoring system: the collected data of 4G network monitoring video camera are mostly non-structured, abundantThese data are excavated it is necessary to carry out the structuring of video data processing, using video structural technology (including space-time dividing skillArt, Feature Extraction Technology, Identifying Technique of Object, deep learning technology), management is uniformly accessed into collected video resource, it is realExisting vehicle structure, pedestrian's structuring, human face structure, anomalous event occur to video monitoring regional, (region intrusion event is stumbledLine detecting event, crowd massing event, delay event of hovering) it is analyzed, by the intellectual analysis to video data, in magnanimityKey message is extracted in video pictures, the semantic description of style of writing of going forward side by side originally uses video preferably；
The video data volume is huge, if photographic technique uploads to cloud center, needs to occupy massive band width, and cloud is handledData are also required to the regular hour, increase user's request response time, human body behavior are analyzed from monitor video and to exceptionBehavior carries out alarm and needs to have high requirement to the processing response time of video, is handled and is calculated beyond the clouds and is not existingIt is real.
4G monitoring device front end bayonet is embedded with intelligent AI Processing Algorithm module, to carry out preliminary video image screeningProcessing enables magnanimity monitoring data to provide processing service nearby in edge side, screens out a large amount of unrelated video image informations；It willMonitor video data are transferred to connection cloud through 4G network by image data after treatment, and video management platform can be directCloud video data is accessed, entire video management center can carry out real time inspection to entire monitoring area and historical record is looked intoIt askes, understands monitoring area dynamic change；Video data is stored in corresponding structuring number after intelligence structureization processing, classificationAccording to warehouse, such as human face photo database, facial feature database, behavior picture and property data base, vehicle image and feature database,Comprehensive different data warehouse and associated video segment warehouse can establish corresponding search engine, realize to all kinds of dataThe depth information in warehouse excavates, and improves the efficiency that data are handled beyond the clouds.
In order to analyze the moving target in video, and different types of target is classified to be put into inhomogeneityIn the database of type, facilitate the intelligent retrieval in later period, need to video carry out structuring processing, to the moving target in video intoRow segmentation uses, and specific implementation is as follows:
Data structured processing is carried out to video image using the flooding mechanism of unsupervised Video segmentation, in video sequence x={ x1,...,xFF frame xi, initial prospect conspicuousness estimation is distributed on i ∈ { 1,2 ..., F }, using super-pixel by each frameImage is divided into a group node, and uses the semantic relation in the estimation of overall situation Neighborhood Graph and coded frame with interframe, is added by oneWeigh row random neighborhood matrixIndicate global Neighborhood Graph, wherein N is the node total number in video.Execute each nodeInitialization prospect conspicuousness estimationDiffusion be by present node estimation and Neighborhood matrix G repetition matrix phaseMultiply to realize, as the t times diffusing step vt=Gvt-1.Unsupervised Video Segmentation Neighborhood matrix G based on diffusion and justBeginningization v0As unique input；
Saliency foreground moving, u are indicated using uiAnd piRespectively indicate the conspicuousness prospect and picture in the i-th frame imageElement individually handles each frame x to calculate conspicuousness estimationi, wherein i ∈ { 1,2 ..., F }, gives a frame image xi,Reference image vegetarian refreshments piIntensity value,It indicates in the i-th frame and i+1 frame with pixel piObject fortune is measured for unitDynamic light stream vector, βiIndicate the boundary pixel set at frame i.
It is based onWithThe conspicuousness sport foreground in video frame i is calculated, appointing for the i-th frame image is each measuredWhat pixel piWith boundary set βi.First distanceCalculate the pixel p observed on boundaryiWith public flow direction itBetween minimal flow direction difference；Second distanceCalculate pixel piPath cost function between boundary pixel；WithAll capture pixel piSimilitude between the movement and background motion at place；
Calculate the difference in light stream direction: pixel p of the light stream direction difference at video frame iiAnd border betaiPublic light stream sideTo；Gather firstly, k-means is used to cluster boundary light stream direction for K, wherein k ∈ { 1 ..., K }, by cluster centre pointIt covers in set, is accomplished by
WhereinIndicate distribution pixel piTo the index in cluster k, r represents the series connection of all target variables；It willKiIt is updated to only comprising being assigned with the center more than 1/6 boundary pixel, in the case where giving cluster centre, by captured frame iPixel piOptical-flow between the key light stream direction observed in the boundary frame i, is realized by following formula:
It calculates minimum potential barrier distance: calculating pixel p in video frame iiWith the minimum potential barrier distance D of boundary pixelbd,i, pass throughFollowing formula is calculated:
WhereinIndicate path, edge aggregation connects pixel piWith boundary pixel s ∈ βi, by being calculated most in frame iSmall spanning tree, edge weights wi(e) it is used to minimum spanning tree and potential barrier distance in calculation formula (1), by calculating adjacent pixelMaximum light stream direction difference value obtains:
E=(pi,qi) it is two pixel ps of connectioniAnd qiSide；Using 4 connectionsThe method in field calculates minimum spanning tree.The potential barrier distance of point-to-point transmission is calculated as the maximum on point-to-point transmission minimum spanning tree pathThe difference of side right and minimum side right, on the point and boundary between any point potential barrier distance minimum value as the point minimum potential barrier away fromFrom；
It is significant to calculate foreground moving type: by two distance metrics after normalizationWithTo obtain in video frame iThe conspicuousness foreground target u of pixelationi；
Neighborhood Graph is calculated by calculating Neighborhood matrix G=T × E × V, wherein T is inter-frame information, and E is frame information,V is long-range (long-range) components.
Interframe timing information: interframe timing information is obtained by extracting Optic flow information, to prevent the light stream at moving boundariesEstimation inaccuracy, while self-consistent property, super-pixel is connected between the light stream vectors of adjacent interframe；To calculate light streamNeighborhood matrix T, continuous two needles image xiAnd xi+1, each frame image includes pixel piAnd pi+1, calculate the forward light of each pixelFlow field fi(pi) and backward optical flow field bi+1(pi+1), optical flow field obtained by calculation defines video frame xiIn pixel piBefore placeTo confidence scoreRealize that formula is as follows:
Backward confidence scoreIt is accomplished by
Wherein σ2For hyper parameter；Confidence score measures pixel piWith pass through pi+fi(pi) optical flow field followed into frame xi+1Later as a result, and passing through pi+fi(pi)+bi+1(pi+fi(pi)) return to frame xiIn result；It is calculated using confidence scoreTwo super-pixel s in frame i and frame i+1i,kAnd si+1,mBonding strength, realized by formula:
Wherein δ () indicates target function, | si,k| and | si+1,k| it represents in si,kAnd si+1,mThe pixel quantity at place；Above formulaMiddle first item compares with super-pixel si,kStart, si+1,mThe bonding strength of end be derived from si,kAnd si+1,mOverall strength；Section 2It is similar to indicate；
Spatial information prevents the diffusion of vision edge in frame in frame, if do not separated by strong edge, allows information sameIt is propagated between adjacent super-pixel in one frame, for the space E for finding edge perception, uses the side of trained frame and interframe firstEdge response calculates confidence score A (s) by the method summed to attenuation function using skirt response for all super-pixel, realIt is now as follows:
Wherein G (p) ∈ [0,1] indicates the skirt response at pixel p, σwHyper parameter is represented with ε；
Edge, which is calculated, by above-mentioned marginal information perceives Neighborhood matrix E:
If si,kSpatially close to si+1,m, that is, super picture of the centre distance less than 1.5 times between two super-pixelPlain size square root；
If having visual similarity between two super-pixel, the long connection of view-based access control model similitude allow far from the time orIt is propagated between super-pixel spatially, long connection propagation information can be made more efficient by neighbour's image, to calculate viewFeel similarity matrix V, these super-pixel and super-pixel si,mRelationship is the closest, k nearest neighbor search is first carried out, to eachSuper-pixel si,m, s is found in the time range of r framei,mK arest neighbors, feature space using Euclidean distance calculateThe distance between super-pixel；
The feature f (s) of super-pixel is calculated by connecting LBP the and RGB histogram of pixel in the super-pixel being calculated,It simultaneously further include the x and y coordinates of HOG feature and super-pixel center；
By defining visual similarity matrix, make super-pixel si,mK arest neighbors pass through N (si,m) reference, visual similarityMatrix is expressed as:
Wherein σ is hyper parameter, and f (s) indicates the character representation of super-pixel s；
After carrying out structuring processing to multi-path monitoring video image, a large amount of hashes in image are not only reduced, are obtainedUseful data, and reduce the load capacity of cloud storing data.In the accuracy of target identification, due to human body behaviorNon-rigid characteristic so that behavior have diversity and variability, the image processing algorithm can to Different Individual generate it is sameA kind of human body behavior has relatively high accuracy, enables using in practice.In order to meet actual application demand, depending onThe processing speed of frequency image 30 frame images of processing per second, can achieve the effect that quasi real time.
The foreground target detected by algorithm needs to judge prospect mesh by IOU and accuracy formula evaluation indexThe IOU ratio of target accuracy, target and realistic objective that algorithm detects is bigger, and the foreground detection effect for representing the algorithm is got overGood, accuracy is higher.
Multi-channel video monitor picture can be by video management platform real time inspection, to the key area in different monitoring rangeDifferent grades of fence is set as security area, is calculated per the insertion intelligence AI Human bodys' response of monitoring camera front end all the wayWhether the fence of method analysis monitored picture setting there are abnormal vehicle, pet, residue, the personnel that cross the border, non-working personIt, will be before analysis treated result feedback to management platform after the processing of AI Human bodys' response algorithmic preliminaries Deng invasion objectEnd, if detecting abnormal behaviour (fall down, hover) and anomalous event (assemble, fight, residue, fence cross the border), video letterAutomatic spring is abnormal the picture of behavior and anomalous event by breath management platform, and carries out video recording storage, and administrative staff verifyThe case where whether warning message belongs to erroneous judgement, however, it is determined that be abnormal, staff is handled in time；Video monitoring system and reportAlert system links, and by the way of the automatic spring warning message in monitored picture, had both improved security personnel and has handled policeThe correctness of feelings can also solve a case rapidly for security personnel and provide firsthand information and evidence；
Video background service provides the video monitoring service function based on cloud computing, and passes through protocol interface and external callPerson communicates, while 4G monitoring camera can support mobile phone, computer remote to check real-time video monitoring picture；
4G monitors photography/videography machine for complex application context, software and algorithm that can be different according to displaying on-demand loading,By cooperateing between multi-feature extraction and identification, multiple-camera, the collaboration between cloud is held to double up intellectual analysis efficiency, it is built-inRealtime graphic quality testing and evaluation of properties, and have self perception and scene adaptability learning ability, allow algorithm and application notDisconnected iteration and evolution；
A variety of smart machines include intelligent closestool sitting alarm set, fuel gas alarm, infrared detector, interior UWB fixedA variety of smart machines such as position system；Intelligent closestool sitting alarm set has sitting reminding module, and to going to the toilet, sitting personnel are carried outAbnormity prompt, to prevent accident occurs；Fuel gas alarm measures indoor smokescope, if smokescope is beyond defaultValue, alarm sound an alarm；IR intrusion detector carries out infrared wiring to great security protection region, when in the region, appearance can be movedInfrared ray will be offended when object, infrared probe detects exception, issues alarm, a variety of intelligent bases of indoor locating system arrangementIt stands, personnel, which carry positioning label, can determine the position of personnel；
Data processing module: data processing system carries out collected data through embedded intelligent algorithm to initial dataScreening Treatment filters irrelevant information, completes to retain useful letter to various treatment processes such as the processing, arrangement, calculating of informationBreath, and stored, information is uploaded to by connection cloud by 4G network.
Connection cloud: connection cloud mainly stores the information of various types of data formats, including monitor video data,Intelligent AI algorithm (Intelligent human-face recognizer module, abnormal behaviour recognizer, anomalous event recognizer), database purchaseInformation (various smart machine basic information managements, face information database, the storage of various warning messages)；Through each smart machineAfter edge data processing, connection cloud is uploaded to, connection cloud stores a plurality of types of data, carries out cloud reception by 4G networkInformation from different intelligent equipment, a variety of data are unified in the storage of connection cloud, facilitate the retrieval and inquiry in later period.
Alarm module:
Local area network is formed by router between smart machine in representative region scene, includes to enclose to monitoring areaRegion including wall, important entrance, elevator, corridor, strategic road, Underground Garage, alarm system need and IR intrusion detector, cigaretteSense alarm, access control system and monitoring system link, if detecting exception, alarm song, weight immediately occurs by loudspeakerThe different fence grades of point monitoring area setting, detect region invader or fence cross the border etc. abnormal behaviours andWhen anomalous event, there is pop-up alarm, gate inhibition's recognition of face of important entrance setting in video management platform, once reach alarmGate inhibition's condition, will automatic trigger alarm system.
Video management platform: management platform realizes the intelligent management to whole system, connects connection cloud, multitude of video numberIt is stored in connection cloud according to the data of, smart machine acquisition, video management platform and connection cloud communicate to connect, and are able to carry outIt checks, video management platform includes following module:
1) basic information management: the personnel in key area are managed；
2) equipment state management includes the intellective video monitoring device of installation, infrared alarm equipment, gas alarm equipment, determinesThe working conditions such as position equipment are managed, and are configured to the attribute of equipment, are realized that the additions and deletions of equipment change and look into, the database of useFor MySQL；
3) it region invasion management: alarms immediately once triggering in the region of setting；
4) abnormal behaviour/incident management: saving type, place and time that the abnormal behaviour/event occurred every time occurs,Convenient for statistics；
5) alarm indication and confirmation: the alarm screen of video monitoring management platform pop-up needs staff to confirm,The case where to prevent there is erroneous detection or missing inspection；
6) historical record is inquired: including monitor video inquiry, alarm logging inquiry；To the alarm logging of preservation with the time,The conditions such as event type, behavior type, scene are inquired；
7) recognition of face is shown: showing the face picture that the access control system of entrance is captured, the facial image of acquisition and peopleFace picture in face library is compared；
8) fence is arranged: different fence grades is arranged in different zones；
9) real-time video monitoring: the module has the function of adopting, passes, deposits, seeing, obtains per the video flowing monitored all the wayLocation transmits camera acquired image data by 4G network, and data are stored in connection cloud, can be appointed in front endWherein monitoring carries out real-time display to meaning calling all the way, and to original video picture by being shown after intelligence AI algorithm process by switchingShow button, the monitored picture that can show that treated, and behavior label and behavior accuracy to people in frame out, real-time video are drawnException is detected in face, pops up abnormal alarm dialog box, while saving abnormal picture, waits relevant staff to handle abnormal；4G camera hardware is carried in the front end of video management platform, collected data is transmitted to background video management platform, in real timeMonitor video picture, the AI intelligent algorithm of rear end can analyze the human body behavior in video.