CN111160150A - Video monitoring crowd behavior identification method based on depth residual error neural network convolution - Google Patents
Video monitoring crowd behavior identification method based on depth residual error neural network convolutionDownload PDFInfo
- Publication number
- CN111160150A CN111160150ACN201911295819.4ACN201911295819ACN111160150ACN 111160150 ACN111160150 ACN 111160150ACN 201911295819 ACN201911295819 ACN 201911295819ACN 111160150 ACN111160150 ACN 111160150A
- Authority
- CN
- China
- Prior art keywords
- crowd
- behavior
- subclasses
- neural network
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
Technical Field
The invention relates to the technical field of crowd behavior recognition and analysis in a monitoring video, in particular to a video monitoring crowd behavior recognition method based on deep residual error neural network convolution.
Background
It is extremely difficult for human observers to monitor a large number of individuals for their behavior and activity from large camera topologies. The areas with abnormal behaviors are usually highly crowded urban areas, and the intelligent behavior detection technology for dense crowds extracts useful behavior pattern information, which is important for public safety, crowd management and timely key decision and support.
Existing research has focused primarily on sparse, mostly stage-like scenes with relatively little effort for reliably classifying and understanding human activities in real and very crowded scenes. In a full-size scenario, where the targets (people and objects of interest) are typically very small, research objects are primarily focused on target recognition tasks to detect or track individual people or vehicles, and are challenging to analyze population behavior, such as characterizing behavioral interactions of an aggregated population.
Currently, researchers have proposed two methods to analyze behavior in such complex scenarios. The first is to consider crowd and scene objects as a whole, where crowd objects, locations, scenes, are not individually identified or classified as individual objects such as their behavior or interaction, but rather are processed according to their overall appearance, and understanding crowd behavior without knowledge of individual behavior is often beneficial and simple. The second object-based approach, where each individual (person and/or object) is detected and segmented to perform motion and/or behavior analysis, is a very challenging task to perform complex segmentation and tracking of individuals in crowd-sourced videos.
Disclosure of Invention
The present invention is intended to solve the above technical problems to some extent.
In view of the above, the invention provides a method for identifying behaviors of video monitoring crowd based on deep residual error neural network convolution, which effectively extracts useful information of crowd behavior categories and can provide key decisions and support for public safety and crowd management in time.
In order to solve the technical problem, the invention provides a video monitoring crowd behavior identification method based on deep residual error neural network convolution, which is characterized by comprising the following steps of:
s1: the method comprises the following steps that a plurality of cameras collect crowd behavior videos and then extract frame images, and the extracted images are subjected to amplification and standard adjustment to form a data source;
s2: inputting a data source into a residual error neural network for image processing, extracting overall crowd behavior characteristics, and dividing the overall crowd behavior characteristics into different types of sub-crowd behavior characteristics based on the overall crowd behavior characteristics;
s3: creating subclasses in the behavior characteristics of each sub-population by using PCA, calculating a dispersion matrix in the subclasses to extract key subclasses, learning variances formed in the key subclasses, and optimizing the variances by using a Fisher criterion;
s4: regularization processing is carried out on the key sub-crowd behavior characteristics so as to model the optimized variance and obtain a modeled model characteristic value;
s5: judging the abnormal conditions of all subclasses based on the obtained characteristic value of each subclass, and outputting a safety index by combining the crowd behaviors of the subclasses;
s6: sending the safety index to a crowd behavior detection module, finding out a corresponding frame image extracted by the detection module for the crowd subclass with lower safety index, and detecting whether the video frame image has abnormal behavior by a background difference method and an area optical flow method;
s7: and finding a similarity measure between frame images of two different behavior groups by using dynamic time warping and a first neighbor classifier on a cosine distance measure so as to track the dynamic video of the abnormal crowd behavior.
Further, in step S4, the modeling the optimized variance, and obtaining the modeled eigenvalue includes the following steps:
s41: within subclass scatter matrix S for computation and descriptionwsCharacteristic value of
The associated feature vector, using the functional form 1/f, estimates the feature spectrum as:
1≤k≤rωswherein α and β are two constants to obtain an eigenspectrum of the initial part that models real numbers;
s42: by making
And
to determine α and β, to obtain
Model feature values for generating a close match to the true feature values are obtained.
Further, in step S3, the calculation formula of the scattering matrix is:
Further, in step S2, the residual neural network inputs the data source and is then fine-tuned and trained by the behavior characteristics of the crowd database.
Further, in the step S1, the data of the data augmentation includes gaussian noise, flipping and brightness.
Further, in step S1, the image normalization includes adjusting the image to a uniform pixel value, and normalizing each channel to zero mean and unit standard deviation.
The invention has the technical effects that: (1) useful crowd behavior pattern information is effectively extracted, and key decisions and support can be provided for public safety and crowd management in time.
(2) The scene type is not required, but the flow characteristic is required; the characteristic sequence for expressing the crowd behavior information is obtained by fully utilizing the bottom layer local motion characteristic information and combining the bottom layer local motion characteristic information with the high-level information of the space-time cube, and through a training model, uniform intensive crowd behaviors can be recognized, and abnormal behaviors can be detected. In some cases, detection and tracking of targets in high-density crowd scenarios lacks reliability. The method avoids the detection and tracking of the target, so that the method has good performance in various high-density population identification and is suitable for large and small populations.
Drawings
Fig. 1 is a flowchart of a method for identifying behaviors of a video surveillance crowd based on deep residual neural network convolution according to the present invention.
Detailed Description
The present invention is further described with reference to the following drawings and specific examples so that those skilled in the art can better understand the present invention and can practice the present invention, but the examples are not intended to limit the present invention.
As shown in fig. 1, a method for identifying behaviors of a video monitoring crowd based on convolution of a depth residual neural network includes the following steps: s1: the method comprises the following steps that a plurality of cameras collect crowd behavior videos and then extract frame images, and the extracted images are subjected to amplification and standard adjustment to form a data source; s2: inputting a data source into a residual error neural network for image processing, extracting overall crowd behavior characteristics, and dividing the overall crowd behavior characteristics into different types of sub-crowd behavior characteristics based on the overall crowd behavior characteristics; s3: creating subclasses in the behavior characteristics of each sub-population by using PCA, calculating a dispersion matrix in the subclasses to extract key subclasses, learning variances formed in the key subclasses, and optimizing the variances by using a Fisher criterion; s4: regularization processing is carried out on the key sub-crowd behavior characteristics so as to model the optimized variance and obtain a modeled model characteristic value; s5: judging the abnormal conditions of all subclasses based on the obtained characteristic value of each subclass, and outputting a safety index by combining the crowd behaviors of the subclasses; s6: sending the safety index to a crowd behavior detection module, finding out a corresponding frame image extracted by the detection module for the crowd subclass with lower safety index, and detecting whether the video frame image has abnormal behavior by a background difference method and an area optical flow method; s7: and finding a similarity measure between frame images of two different behavior groups by using dynamic time warping and a first neighbor classifier on a cosine distance measure so as to track the dynamic video of the abnormal crowd behavior.
According to the specific embodiment of the invention, the method for identifying the behaviors of the video monitoring crowd based on the depth residual error neural network convolution adopts monitoring videos provided by a certain downtown business area as training data, two monitoring videos of a daytime scene and a nighttime scene of 6 provided camera point positions are extracted as frame images, the time of each video is 10 minutes, one frame image is captured at intervals of 1s as a sample, fuzzy samples are eliminated through screening, 4000 pictures containing crowd aggregation are reserved, and the extracted images are subjected to amplification and standardized adjustment to form a data source;
inputting a data source into a residual error neural network for image processing, screening out overall crowd behavior characteristics with human body behaviors, and dividing the overall crowd behavior characteristics into different classes of sub-crowd behavior characteristics based on the overall crowd behavior characteristics, wherein the different classes of behaviors comprise framed people, queuing people, dining people and the like;
creating subclasses in a feature map of the behavior features of each sub-population by using PCA, calculating a dispersion matrix in the subclasses to extract key subclasses, learning variances formed in the key subclasses, and optimizing the variances by using a Fisher criterion; wherein, the calculation formula of the dispersion matrix is as follows:
where E is the number of behavioral characteristics of the sub-population, where HiIndicates the sub-number of the ith class, GijDenotes the number of samples, x, of the jth sub-class of the ith classijkIs the k-th image vector in the j-th sub-class of the i-th class,
is the sample mean, p, of the jth sub-class of the ith classiAnd
is the estimated prior probability; fisher objective function of
Is a general subclass scatter matrix;
regularization processing is carried out on the key sub-crowd behavior characteristics so as to model the optimized variance and obtain a modeled model characteristic value;
judging the abnormal conditions of all subclasses based on the obtained characteristic value of each subclass, and outputting a safety index by combining the crowd behaviors of the subclasses;
the safety index is sent to a crowd behavior detection module, the detection module finds out corresponding frame images extracted by the detection module for crowd subclasses with lower safety indexes, detects whether the video frame images have abnormal behaviors or not by a background difference method and an area optical flow method, judges whether the pictures have violent and terrorist behaviors such as crowding, treading, fighting and the like or not by adopting a maximum speed value, has better discrimination on the normal behaviors and the abnormal behaviors, and can carry out real-time monitoring;
the similarity measurement between frame images of two different behavior groups is found by using dynamic time warping and a first neighbor classifier on the cosine distance measurement, time conversion allowing matching of two behaviors is calculated to perform time alignment and normalization, and the same behavior cannot be completely reproduced by the same crowd is tracked.
As shown in fig. 1, the step S4 of modeling the optimized variance and obtaining the modeled eigenvalue includes the following steps:
s41: within subclass scatter matrix S for computation and descriptionwsCharacteristic value of
The associated feature vector, using the functional form 1/f, estimates the feature spectrum as:
1≤k≤rωswherein α and β are two constants to obtain an eigenspectrum of the initial part that models real numbers;
s42: by making
And
to determine α and β, to obtain
Model feature values for generating a close match to the true feature values are obtained.
As shown in fig. 1, in step S2, the residual neural network input data source is trimmed and trained by the behavior characteristics of the crowd database.
As shown in fig. 1, in step S1, the data to be augmented includes gaussian noise, flipping and brightness.
As shown in fig. 1, in step S1, image normalization includes adjusting the image to uniform pixel values, normalizing each channel to zero mean and unit standard deviation.
According to a specific embodiment of the present invention, all images are adjusted to 224 × 224, and each channel is normalized to zero mean and unit standard deviation.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (6)
1. A video monitoring crowd behavior identification method based on deep residual error neural network convolution is characterized by comprising the following steps:
s1: the method comprises the following steps that a plurality of cameras collect crowd behavior videos and then extract frame images, and the extracted images are subjected to amplification and standard adjustment to form a data source;
s2: inputting a data source into a residual error neural network for image processing, extracting overall crowd behavior characteristics, and dividing the overall crowd behavior characteristics into different types of sub-crowd behavior characteristics based on the overall crowd behavior characteristics;
s3: creating subclasses in the behavior characteristics of each sub-population by using PCA, calculating a dispersion matrix in the subclasses to extract key subclasses, learning variances formed in the key subclasses, and optimizing the variances by using a Fisher criterion;
s4: regularization processing is carried out on the key sub-crowd behavior characteristics so as to model the optimized variance and obtain a modeled model characteristic value;
s5: judging the abnormal conditions of all subclasses based on the obtained characteristic value of each subclass, and outputting a safety index by combining the crowd behaviors of the subclasses;
s6: sending the safety index to a crowd behavior detection module, finding out a corresponding frame image extracted by the detection module for the crowd subclass with lower safety index, and detecting whether the video frame image has abnormal behavior by a background difference method and an area optical flow method;
s7: and finding a similarity measure between frame images of two different behavior groups by using dynamic time warping and a first neighbor classifier on a cosine distance measure so as to track the dynamic video of the abnormal crowd behavior.
2. The method for identifying behaviors of video surveillance people based on convolution of depth residual neural network as claimed in claim 1, wherein in step S4, the step of modeling the optimized variance and obtaining the modeled eigenvalue comprises the steps of:
s41: within subclass scatter matrix S for computation and descriptionwsCharacteristic value of
The associated feature vector, using the functional form 1/f, estimates the feature spectrum as:
where α and β are two constants to obtain an eigenspectrum of the initial part that models real numbers;
s42: by making
And
to determine α and β, to obtain
Model feature values for generating a close match to the true feature values are obtained.
3. The method for identifying the behavior of the crowd monitored by the video based on the convolution of the depth residual neural network as claimed in claim 1, wherein in the step S3, the calculation formula of the scattering matrix is:
4. The method for identifying behaviors of monitoring people through video based on convolution of deep residual neural network as claimed in claim 1, wherein in step S2, the residual neural network is input to the data source and then fine-tuned and trained through behavior features of the crowd database.
5. The method for identifying behaviors of video surveillance people based on convolution of depth residual neural network as claimed in claim 1, wherein in step S1, the data of data augmentation includes gaussian noise, inversion and brightness.
6. The method for identifying the behavior of the crowd monitored by the video based on the convolution of the depth residual neural network as claimed in claim 1, wherein in the step S1, the image normalization comprises adjusting the image to be a uniform pixel value and normalizing each channel to be a zero mean and a unit standard deviation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911295819.4ACN111160150A (en) | 2019-12-16 | 2019-12-16 | Video monitoring crowd behavior identification method based on depth residual error neural network convolution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911295819.4ACN111160150A (en) | 2019-12-16 | 2019-12-16 | Video monitoring crowd behavior identification method based on depth residual error neural network convolution |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111160150Atrue CN111160150A (en) | 2020-05-15 |
Family
ID=70557189
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911295819.4APendingCN111160150A (en) | 2019-12-16 | 2019-12-16 | Video monitoring crowd behavior identification method based on depth residual error neural network convolution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111160150A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111768430A (en)* | 2020-06-23 | 2020-10-13 | 重庆大学 | A tracking method for expressway outfield vehicles based on multi-feature cascade matching |
| CN112651275A (en)* | 2020-09-01 | 2021-04-13 | 武汉科技大学 | Behavioral intelligence recognition system for stampede accidents in crowded places |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799863A (en)* | 2012-07-02 | 2012-11-28 | 中国计量学院 | Method for detecting group crowd abnormal behaviors in video monitoring |
| CN103235944A (en)* | 2013-05-15 | 2013-08-07 | 青岛科技大学 | Crowd flow division and crowd flow abnormal behavior identification method |
| CN107480578A (en)* | 2016-06-08 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of video detection system and method using crowd behaviour analysis |
| CN109064484A (en)* | 2018-03-21 | 2018-12-21 | 南京邮电大学 | Crowd movement's Activity recognition method with momentum Fusion Features is divided based on subgroup |
- 2019
- 2019-12-16CNCN201911295819.4Apatent/CN111160150A/enactivePending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799863A (en)* | 2012-07-02 | 2012-11-28 | 中国计量学院 | Method for detecting group crowd abnormal behaviors in video monitoring |
| CN103235944A (en)* | 2013-05-15 | 2013-08-07 | 青岛科技大学 | Crowd flow division and crowd flow abnormal behavior identification method |
| CN107480578A (en)* | 2016-06-08 | 2017-12-15 | 国家计算机网络与信息安全管理中心 | A kind of video detection system and method using crowd behaviour analysis |
| CN109064484A (en)* | 2018-03-21 | 2018-12-21 | 南京邮电大学 | Crowd movement's Activity recognition method with momentum Fusion Features is divided based on subgroup |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111768430A (en)* | 2020-06-23 | 2020-10-13 | 重庆大学 | A tracking method for expressway outfield vehicles based on multi-feature cascade matching |
| CN111768430B (en)* | 2020-06-23 | 2023-08-11 | 重庆大学 | A Vehicle Tracking Method Based on Multi-feature Cascade Matching in Outfield of Expressway |
| CN112651275A (en)* | 2020-09-01 | 2021-04-13 | 武汉科技大学 | Behavioral intelligence recognition system for stampede accidents in crowded places |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10248860B2 (en) | System and method for object re-identification | |
| US10346688B2 (en) | Congestion-state-monitoring system | |
| Hou et al. | People counting and human detection in a challenging situation | |
| CN101965576B (en) | Object matching for tracking, indexing, and search | |
| Conde et al. | HoGG: Gabor and HoG-based human detection for surveillance in non-controlled environments | |
| CN113963301A (en) | A video fire smoke detection method and system based on fusion of spatiotemporal features | |
| CN111353338B (en) | A method for improving energy efficiency based on video surveillance in business halls | |
| CN109829382B (en) | Abnormal target early warning tracking system and method based on intelligent behavior characteristic analysis | |
| CN101344922A (en) | A face detection method and device thereof | |
| Hu et al. | Parallel spatial-temporal convolutional neural networks for anomaly detection and location in crowded scenes | |
| Kongurgsa et al. | Real-time intrusion—detecting and alert system by image processing techniques | |
| CN110728252A (en) | Face detection method applied to regional personnel motion trail monitoring | |
| Hirzi et al. | Literature study of face recognition using the viola-jones algorithm | |
| Farhood et al. | Counting people based on linear, weighted, and local random forests | |
| Ghidoni et al. | Texture-based crowd detection and localisation | |
| Arshad et al. | Anomalous situations recognition in surveillance images using deep learning | |
| CN111160150A (en) | Video monitoring crowd behavior identification method based on depth residual error neural network convolution | |
| CN119649470A (en) | Personnel behavior analysis management system and method based on AI vision | |
| Hou et al. | Automated people counting at a mass site | |
| Padmaja et al. | Crowd abnormal behaviour detection using convolutional neural network and bidirectional LSTM | |
| CN118644804B (en) | Visual multi-target tracking method, system and electronic device based on deep learning | |
| CN104123569B (en) | Video person number information statistics method based on supervised learning | |
| Mantini et al. | Camera Tampering Detection using Generative Reference Model and Deep Learned Features. | |
| Shrivastav | A Real-Time Crowd Detection and Monitoring System using Machine Learning | |
| Boudhane et al. | Optical fish classification using statistics of parts |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20210527 Address after:224000 Building 1, intelligent terminal industry pioneer park, high tech Zone, Yandu District, Yancheng City, Jiangsu Province (d) Applicant after:Yancheng Jiyan Intelligent Technology Co.,Ltd. Address before:224000 Building 1, intelligent terminal industry pioneer park, high tech Zone, Yandu District, Yancheng City, Jiangsu Province Applicant before:Co., Ltd of intelligent terminal industrial research institute of Yancheng Jinlin University | |
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned | Effective date of abandoning:20240510 |