Movatterモバイル変換


[0]ホーム

URL:


CN114332163B - High-altitude parabolic detection method and system based on semantic segmentation - Google Patents

High-altitude parabolic detection method and system based on semantic segmentation
Download PDF

Info

Publication number
CN114332163B
CN114332163BCN202111636002.6ACN202111636002ACN114332163BCN 114332163 BCN114332163 BCN 114332163BCN 202111636002 ACN202111636002 ACN 202111636002ACN 114332163 BCN114332163 BCN 114332163B
Authority
CN
China
Prior art keywords
semantic segmentation
altitude parabolic
area
network
camera shake
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111636002.6A
Other languages
Chinese (zh)
Other versions
CN114332163A (en
Inventor
涂志刚
张正博
朱立远
古昊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHUfiledCriticalWuhan University WHU
Priority to CN202111636002.6ApriorityCriticalpatent/CN114332163B/en
Publication of CN114332163ApublicationCriticalpatent/CN114332163A/en
Application grantedgrantedCritical
Publication of CN114332163BpublicationCriticalpatent/CN114332163B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Landscapes

Abstract

The invention provides a high-altitude parabolic detection method and system based on semantic segmentation, comprising the steps of training a lightweight semantic segmentation network through knowledge distillation in advance, taking a camera shake-free frame from a monitoring video, inputting the semantic segmentation network to extract a building region as a candidate region for high-altitude parabolic detection; taking a plurality of frames without camera shake, performing binarization processing, and then performing background modeling by using a Gaussian mixture model to obtain a background image of the current scene; judging camera shake of the current frame image to be detected, if the camera does not shake, detecting motion in a building area of the current frame image by using a background difference method by using a background to obtain a moving object; denoising the obtained moving object; and (3) carrying out target tracking on the denoised candidate object by using a Hungary algorithm, judging a tracking track if the target object can be successfully tracked, and if the tracking track accords with the track of the high-altitude parabolic object, considering the target object as the high-altitude parabolic object and further obtaining the throwing position and the falling point position of the target object.

Description

High-altitude parabolic detection method and system based on semantic segmentation
Technical Field
The invention relates to semantic segmentation based on deep learning, and aims to effectively solve the problems of small targets, poor timeliness and complex scenes in high-altitude parabolic detection.
Background
The real-time monitoring and early warning treatment of the high-altitude parabolic articles have strong practicability. However, the performance of the current high-altitude parabolic detection method and system is not ideal, for example, the problems of insufficient real-time performance, high miss detection and error detection proportion and the like are generally faced.
The semantic segmentation combines image classification, target detection and image segmentation, the image is segmented into area blocks with certain semantic meaning through a certain method, the semantic category of each area block is identified, the semantic reasoning process from the bottom layer to the high layer is realized, and finally a segmented image with pixel-by-pixel semantic annotation is obtained. The image semantic segmentation method comprises a traditional method and a convolutional neural network-based method, wherein the traditional semantic segmentation method can be divided into a statistical-based method and a geometric-based method. With the development of deep learning, the semantic segmentation technology is greatly improved, and the semantic segmentation method based on the convolutional neural network is the most different from the traditional semantic segmentation method in that the network can automatically learn the characteristics of the image, perform end-to-end classification learning, and greatly improve the accuracy of semantic segmentation. The following three types of semantic segmentation models are commonly used:
1. Real-time semantic segmentation techniques. The network model for evaluating the semantic segmentation at the present stage mainly focuses on the accuracy, but as the requirement for the application to the real scene is higher and higher, shorter response time is required, so that the response time is shortened as much as possible on the basis of maintaining the high accuracy, and the direction of working in the future is considered.
2. Weak supervised or unsupervised semantic segmentation techniques. Aiming at the problem that a large number of marked data sets are needed to improve the precision of a network model, a weak supervision or unsupervised semantic segmentation technology is a trend of future development.
3. Semantic segmentation techniques for three-dimensional scenes. The data used for training in the existing semantic segmentation technology based on deep learning is mainly two-dimensional picture data, and meanwhile, the tested object is often a two-dimensional picture, but the environment facing the test object is a three-dimensional environment in actual application, so that the semantic segmentation technology is applied to the actual application, and research on the semantic segmentation technology of the three-dimensional data is needed in the future.
The semantic segmentation network parameters are large, the reasoning time is long, the real-time performance of the algorithm is affected, and a knowledge distillation method is generally adopted to carry out light weight treatment on the model. Knowledge distillation is intended to migrate hidden knowledge in a complex model (teacher network) to a simple model (student network), which in general has strong capabilities and behavior, while the student network is more compact. By knowledge distillation, it is desirable that the student network be as close as possible to or beyond the accuracy of the teacher network, so that similar predictive effects can be obtained with less complexity. Hinton, at DISTILLING THE Knowledge in a Neural Network, first proposed the concept of knowledge distillation to induce training of student networks by introducing a soft target (soft targets) of the teacher network. In recent years, many knowledge distillation methods have appeared, and different methods have different definitions for hidden information to be transferred in a network.
The high-altitude parabolic detection is a special moving object detection, and the moving object detection is a process of effectively extracting objects with space position change by subtracting redundant information in time and space in a video through a computer vision method, so that the high-altitude parabolic detection has been widely focused in many computer vision applications. The method mainly comprises the following three methods:
1. Background differencing. Background subtraction, also known as background subtraction, is commonly used to detect moving objects in video images, and is one of the mainstream methods for detecting moving objects at present. The basic principle is that the current frame in the image sequence and the background reference model (background image) which is already determined or acquired in real time are subtracted, the difference is found, and the area which is different from the background image pixel and exceeds a certain threshold value is calculated as a moving area, so that the characteristics of the position, the outline, the size and the like of the moving object are determined, and the method is very suitable for the static scene of a camera.
2. Optical flow method. The optical flow method utilizes the change of pixels in an image sequence on a time domain and the correlation between adjacent frames, and calculates the motion information of objects between the adjacent frames according to the corresponding relation between the previous frame and the current frame. The psychologist Gibson in the fifties of the twentieth century first proposed the basic concept of optical flow based on psychological experiments, while the algorithm that introduced the optical flow constraint equation, creatively linked gray scales to two-dimensional velocity fields by Horn, kanade, lucash and Schunck until the eighties, worked on optical flow calculations. In practical tests, although the method can detect the whole area of a moving object and is suitable for the static and moving situations of a camera, most optical flow calculation methods have huge calculated amount and complex structure, are easily influenced by illumination, object shielding or image noise and have poor robustness, so the method is generally not adopted by monitoring systems with higher requirements on precision and real-time performance.
3. A supervised approach based on deep learning. The supervised classification method represented by deep learning performs better than the unsupervised background difference method and optical flow method. But the deep learning-based approach has several drawbacks. If the generalization capability is poor, the performance is poor in a non-trained scene, and the performance can be obviously reduced; compared with an unsupervised method, the method based on deep learning lacks enough theoretical guarantee and has poor interpretation; a large amount of prior data needs to be collected, but the data collection difficulty of high-altitude parabolic objects is large.
The existing high-altitude parabolic method and system have the defects of poor anti-interference capability, difficult application of complex scenes, difficult detection of small targets, difficult object tracking and the like.
Disclosure of Invention
In order to solve the problems, the invention provides a high-altitude parabolic detection scheme based on semantic segmentation, which has strong anti-interference capability, can be suitable for complex scenes such as communities, office buildings and the like, can detect small targets and has good tracking effect.
The invention provides a high-altitude parabolic detection method based on semantic segmentation, which comprises the following steps of:
S1, training a lightweight semantic segmentation network through knowledge distillation in advance, taking a frame without camera shake from a monitoring video, inputting the frame into the semantic segmentation network to extract a building area as a candidate area for high-altitude parabolic detection;
s2, taking a plurality of frames without camera shake of the monitoring video, performing binarization processing, and then performing background modeling by using a Gaussian mixture model to obtain a background image of the current scene;
S3, judging camera shake of the current frame image to be detected, if the camera does not shake, detecting motion in a building area of the current frame image by using a background difference method by using the background obtained in the S2 to obtain a moving object, and then entering the step S4;
s4, denoising the moving object obtained in the step S3;
S5, carrying out target tracking on the denoised candidate object by using a Hungary algorithm, judging a tracking track if the target object can be successfully tracked, and if the tracking track accords with the track of the high-altitude parabolic object, considering the target object as the high-altitude parabolic object, and further obtaining the throwing position and the falling point position of the target object.
Moreover, the training of the lightweight semantic segmentation network through knowledge distillation is realized as follows,
In PspNet semantic segmentation networks, resNet101 is used as a teacher network, resNet is used as a student network, cosine similarity between every two pixels of a feature map of the teacher network is extracted, and then the student network learns the feature and the feature map of the teacher network in a knowledge distillation mode;
A self-encoder is pre-trained by using a teacher network, a feature map of the teacher network is input into the self-encoder to obtain high-order features of the self-encoder, a K-Means algorithm is used for clustering, a similarity relation between each pixel and a clustering center is extracted, and a result is transmitted into a student network for learning;
training a new countermeasure neural network in advance, inputting the student network into a generating network, judging whether the global features of the student network feature map are consistent with the global features of the teacher network feature map by using a judging network, if so, indicating that training is completed, otherwise, iterating training until the global features are consistent.
In addition, when taking a frame without camera shake and carrying out camera shake judgment on the current frame image to be detected and carrying out camera shake judgment in the same way, the following is realized,
Detecting a moving target by using a background difference method for a current input frame;
Calculating the number N of all moving target areas and the length x and the width y of the minimum outsourcing rectangle of each area, and further calculating the total S of the minimum outsourcing rectangles of the moving targets of the frame;
If N > T1, or the total area S > T2 of all the outsourcing rectangles, the frame is considered to have camera shake; t1 is a preset motion area total threshold for the jitter-free frame, and T2 is a preset minimum outsourcing rectangle total threshold for the jitter-free frame.
Moreover, the implementation of said step S4 comprises the following sub-steps,
S401, calculating an outsourcing rectangle of all moving objects, and taking the outsourcing rectangle as a moving area of a candidate parabolic object;
S402, if a plurality of motion areas are overlapped, obtaining an area with the largest motion area occupation ratio by using a non-maximum value inhibition method, and deleting other areas;
S403, calculating a center POINT coordinate set POINT of the area of the outsourcing rectangle of each motion area, and obtaining a circular area by taking the center of each motion area as an origin and taking a preset threshold r as a radius; and calculating the number N of POINTs in the set POINT appearing in the circular area and the distance D between the nearest two POINTs, if N < N, D < D, considering the circular area as not being a noise area, reserving all the motion areas in the circular area, wherein N is the maximum value of the number of the POINTs in the set POINT contained in a preset circle, and D is the distance of the nearest POINT in the set POINT contained in the preset circle.
Furthermore, said step S5 comprises the following sub-steps:
In a post Num frame of the current frame, carrying out target tracking by using a Hungary algorithm, if the target tracking can be successfully carried out, obtaining coordinates of candidate targets tracked in the post Num frame, respectively calculating a difference value between the coordinates and the targets of the previous frame, and if the difference value gradually becomes larger and the movement in the abscissa direction does not have mutation, considering the difference value as a movement track of a high-altitude parabolic object; wherein Num is a preset value.
The invention also provides a high-altitude parabolic detection system based on semantic segmentation, which is used for realizing the high-altitude parabolic detection method based on semantic segmentation.
Furthermore, the device comprises the following modules,
The first module is used for training a lightweight semantic segmentation network through knowledge distillation in advance, taking a frame without camera shake from a monitoring video, inputting the semantic segmentation network to extract a building area as a candidate area for high-altitude parabolic detection;
the second module is used for taking a plurality of camera shake-free frames of the monitoring video, performing binarization processing, and then performing background modeling by using a Gaussian mixture model to obtain a background image of the current scene;
The third module is used for judging camera shake of the current frame image to be detected, if the camera does not shake, the background obtained by the second module is used for detecting the motion of the current frame image in the building area by using a background difference method to obtain a moving object, and then the moving object enters the fourth module;
A fourth module, configured to denoise the moving object obtained by the third module;
and a fifth module, configured to perform target tracking on the denoised candidate object by using a hungarian algorithm, determine a tracking track if the candidate object can be successfully tracked, and consider the candidate object as a high-altitude parabolic object if the tracking track conforms to the track of the high-altitude parabolic object, so as to further obtain a throwing position and a falling point position of the candidate object.
Or comprises a processor and a memory, wherein the memory is used for storing program instructions, and the processor is used for calling the stored instructions in the memory to execute the high-altitude parabolic detection method based on semantic segmentation.
Or comprises a readable storage medium, wherein the readable storage medium is stored with a computer program, and the computer program realizes the high altitude parabolic detection method based on semantic segmentation when being executed.
Compared with the current high-altitude parabolic method and system, the technical scheme has the following beneficial effects:
1. according to the invention, semantic segmentation is innovatively introduced to determine the area of a building in the monitoring video, so that the area where the parabolic object possibly appears is determined, and the interference of the complex environment on high-altitude parabolic object detection is greatly reduced; in addition, the invention trains a lighter semantic segmentation network by using a knowledge distillation method, thereby greatly reducing the reasoning time and improving the real-time performance of the algorithm.
2. The method creatively removes the interference by using a plurality of methods, firstly removes part of interference by camera shake detection, secondly removes part of incomplete detection frames by using a non-maximum value inhibition method, and finally detects whether the judgment track of the target accords with the parabolic object or not, thereby obviously reducing the false detection rate of the high-altitude parabolic object.
Drawings
Fig. 1 is a flowchart of a high-altitude parabolic detection method based on semantic segmentation according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of an outer bounding rectangle and cross-ratios in accordance with an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention discloses a high altitude parabolic detection scheme based on semantic segmentation, which comprises the steps of firstly obtaining the range of buildings in a detection area through semantic segmentation, and then obtaining all moving targets in the building area by a background difference method as candidate ranges for high-altitude parabolic detection. The invention combines the motion characteristics of the paraboloid and Kalman filtering to track the candidate target of the high-altitude paraboloid, removes the influence caused by camera shake and ensures timeliness and accuracy. In the tracking process, if the situation that the candidate target cannot be tracked correctly occurs, the candidate frame can be deleted, so that the accuracy is further improved. The invention has low false alarm rate and can well detect and track small targets under the condition of ensuring precision and real-time performance.
As shown in fig. 1, the high-altitude parabolic detection method based on semantic segmentation provided by the embodiment of the invention comprises the following steps:
S1, extracting candidate areas for high-altitude parabolic detection in advance: the monitoring video can be subjected to semantic segmentation by taking one frame at preset intervals (preferably one hour), the building area is extracted as a candidate area for high-altitude parabolic detection,
In an embodiment, the step S1 includes the following sub-steps:
S101, in the PspNet semantic segmentation network, resNet101 is used as a teacher network, resNet is used as a student network, and knowledge distillation is achieved. Firstly, extracting cosine similarity between every two pixels of a feature map of a teacher network, and enabling a student network to learn the feature and the feature map of the teacher network in a knowledge distillation mode;
The invention preferably uses ResNet and ResNet to realize the required semantic segmentation, and the specific network structure is the prior art and will not be repeated.
S102, pre-training a self-encoder by using a teacher network, wherein an encoder part and a decoder part of the self-encoder are composed of three layers of convolutions. Inputting the characteristic diagram of the teacher network into a self-encoder to obtain the higher-order characteristic of the characteristic diagram, clustering by using a K-Means algorithm, extracting the similarity relation between each pixel and a clustering center, and transmitting the characteristic diagram into a student network for learning.
S103, training a new countermeasure neural network in advance, inputting a student network into a generation network, judging whether global features of a student network feature map are consistent with global features of a teacher network feature map by using a judging network, if so, indicating that training is completed, entering step S104, and if not, returning to S101 for training;
the number of the trained student network parameters is less than one seventh of that of a teacher network, and the accuracy is improved by 10% compared with that of a student network without knowledge distillation.
S104, judging camera shake of the current frame, if the camera shake does not occur, performing semantic segmentation according to the trained lightweight semantic segmentation network to obtain a building area as a high-altitude parabolic detection area, and if the camera shake occurs, taking down a frame of image to perform camera shake judgment until the high-altitude parabolic detection area is successfully extracted.
In the implementation, a lightweight neural network can be trained in advance by a knowledge distillation method, and semantic segmentation is performed by using the network.
Firstly, camera shake judgment is carried out on the current frame, the specific operation steps are that the frame at the current time tN and the previous frame tN-1 are taken to calculate the total number N of the areas of the moving object by using a background difference method, and the length and width set of all the outsourcing rectangles of all the moving areas { (x1,y1),(x2,y2),…,(xN,yN) }, and then the minimum outsourcing rectangle sum of the moving object of the frame is calculatedWhere i is the index of the motion region, i=1, 2, … N.
If N > T1 or S > T2, considering that the current frame has camera shake, not processing the frame image, and re-inputting the next frame image; if N < T1 and S < T2, the current frame is considered to have no camera shake, and the frame is semantically segmented.
Wherein, T1 is a preset total number threshold of motion areas without jitter frames, T2 is a preset minimum outsourcing rectangle sum threshold without jitter frames, and specific values of T1 and T2 are set according to actual application scenes. In specific implementation, the threshold value can be preset according to experience.
And inputting the selected camera shake-free frames into a trained lightweight semantic segmentation network, obtaining a series of feature images through multiple convolution and pooling processes, and then upsampling the feature images obtained by the last convolution layer by utilizing a deconvolution layer, so that the upsampled feature images are the same as the original images in size, and further, each pixel value on the feature images is predicted while spatial position information of the pixel value in the original images is reserved, and finally, the upsampled feature images are subjected to pixel-by-pixel classification to obtain a building background serving as a detection area of a high-altitude parabolic object.
S2, extracting a background image of the current scene in advance:
In implementation, due to the characteristics of the high-altitude parabolic detection scene, the background map of the current scene can be extracted at preset intervals (for example, at half an hour). The extraction process is that taking continuous x frames of the monitoring video, wherein x is preferably 5, judging camera shake, if the camera does not shake, performing binarization processing, then performing background modeling by using a Gaussian mixture model (MOG 2) of Opencv to obtain a background image of the current scene, if the camera shake occurs, re-taking the continuous x frames until the background image of the current scene is successfully obtained when the camera shake is judged not to exist, and then entering step S3.
S3, judging camera shake of the current frame image, if the camera does not shake, detecting motion in a building area of the current frame image by using a background difference method by using the background obtained in the S2 to obtain a moving object, and then entering the step S4; if the camera shakes, the frame image is not processed, and the next frame image is input again until it is judged that the camera shakes, the process proceeds to step S4.
In the implementation, the current frame shot by the camera can be extracted in real time according to the requirement of high-altitude parabolic detection.
The camera shake detection method in S2 and S3 is the same as that described in S1, namely:
detecting a moving target by using a background difference method for a current input frame;
Calculating the number N of all moving target areas and the length x and the width y of the minimum outsourcing rectangle of each area, and further calculating the total S of the minimum outsourcing rectangles of the moving targets of the frame;
If N > T1 or total area S > T2 of all the outsourcing rectangles, the frame is considered to have camera shake condition, and the frame image is not processed; if the number of the areas and the total outsourcing area do not exceed the threshold value, the frame is considered to have no camera shake, and the specific values of T1 and T2 are set according to the actual application scene.
S4, denoising the moving object obtained in the step S3, wherein the implementation mode of denoising operation is further provided as follows:
s401, calculating the outsourcing rectangles of all the moving objects, and taking the outsourcing rectangles as the moving areas of the candidate throws:
First, the left upper corner coordinates and right lower corner coordinates of the smallest outsourcing rectangle of all the motion areas are calculated, n motion areas are arranged, the left upper corner coordinates (x11,y11) and right lower corner coordinates (x12,y12) of the smallest outsourcing rectangle of the 1 st motion area, the left upper corner coordinates (z21,y21) and right lower corner coordinates (x22,y22) of the smallest outsourcing rectangle of the 2 nd motion area, the left upper corner coordinates (xn1,yn1) and right lower corner coordinates (xn2,yn2) of the smallest outsourcing rectangle of the … n-th motion area are calculated
Get the collection REC={((x11,y11),(x12,y12)),…,((xn1,yn1),(xn2,yn2))}.
S402, if a plurality of motion areas are overlapped, obtaining an area with the largest motion area occupation ratio by using a non-maximum value inhibition method, and deleting other areas:
Starting from the first outsourced rectangle in the set REC, calculating the intersection ratio IOU of each rectangle and other rectangles in sequence, if IOU >0.3, considering that the two rectangles are intersected, then calculating the moving pixel area occupation ratio V of the two rectangles respectively, reserving the rectangle with large moving pixel area occupation ratio, and deleting the rectangle with small moving pixel area occupation ratio.
Referring to fig. 2, the moving pixel area ratio V in the intersection ratio IOU and the outsourcing rectangle is calculated as follows:
inter=(min(xa2,xb2)-max(xa1,xb1))×(min(ya2,yb2)-max(ya1,yb1))
union=s1+s2-inter
s1=(xa2-xa1)×(ya2-ya1),s2=(xb2-xb1)×(yb2-yb1)
Wherein, (xa1,ya1),(xa2,ya2),(xb1,yb1),(xb2,yb2) is the upper left corner coordinates and lower right corner coordinates of the two outsourcing rectangles a and b, inter is the area where the two rectangles meet, union is the area where the two rectangles meet, Smove is the area of the moving region in the outsourcing rectangle, S is the area of the outsourcing rectangle, min (xa2,xb2) represents the minimum value of xa2 and xb2, and max (ya1,yb1) represents the maximum value of ya1 and yb1.
S403, calculating a center POINT coordinate set POINT of the area of the outsourcing rectangle of each motion area, and obtaining a circular area by taking the center of each motion area as an origin and taking a preset threshold r as a radius; the distance d between the POINT n in the set of center POINTs POINT of the bounding rectangle appearing in each circular region and the nearest two center POINTs is calculated, where d=0 when n < 2. If N < N, D < D, then the circular region is considered not a noise region, leaving all motion regions within the circular region. N is the maximum value of the number of POINTs in the set POINT contained in a preset circle, D is the distance between the nearest POINTs in the set POINT contained in the preset circle, and the specific values of N and D are set according to the actual application scene.
The specific implementation comprises the following operations:
Calculating a center POINT coordinate set POINT of the area of each motion area outsourcing rectangle:
POINT={(x1,y1),(x2,y2)…,(xn,yn)}
The same operation is performed on each POINT in the POINT, taking the POINT (xa,ya) as a circle center, taking the POINT as a circle center, taking the radius as r, calculating the number num1 of POINTs (including the circle center) in the set POINT in the circle, calculating the distance between every two POINTs (including the circle center) in the set POINT in the circle, and comparing to obtain the distance d of the nearest two POINTs, wherein d=0 when num1< 2. If num1< N1 and D < D, the motion area represented by the point is considered to accord with the characteristics of the high-altitude parabolic area, and the point is reserved, otherwise, the point is deleted. Wherein r is the radius of a preset circle, N1 is the maximum value of the number of POINTs in the preset circle containing set POINT, D is the distance between the nearest POINTs in the preset circle containing set POINT, and the specific values of N1 and D are set according to the actual application scene. The inter-point distance d is calculated as follows:
d=√((xa-xb)2+(ya-yb)2)
Where (xa,ya) and (xb,yb) are center point coordinates of the two bounding rectangles taken, and N1 and D are preset thresholds.
S5, carrying out target tracking on the denoised candidate object by using a Hungary algorithm. If the tracking track is successful, judging the tracking track, if the tracking track accords with the track of the high-altitude parabolic object, considering the tracking track as the high-altitude parabolic object, further obtaining the throwing position and the falling point position of the high-altitude parabolic object, and if the tracking track does not accord with the track of the high-altitude parabolic object, deleting the candidate object.
The tracking track judging mode is as follows:
Let num2 tracking points be obtained, the coordinates are (x1,y1),(x2,y2)…,(xnum2,ynum2) respectively, and the set point of all tracking points is obtained:
point={(x1,y1),(x2,y2)…,(xnum2,ynum2)}
Calculating the vertical distance d between all the previous and subsequent frames:
d={d1,d2,…,dnum2},
d1=y2-y1
d2=y3-y2
dnum2-1=ynum2-ynum2-1
If the trajectory coincides with the parabolic trajectory, the vertical distance satisfies d1<d2<...<dnum2-1, and the abscissa of every three consecutive tracking points satisfies that there is no abrupt change in the abscissa direction motion, e.g., (x2-x1)×(x3-x2) >0, the trajectory is considered to coincide with the high-altitude parabolic trajectory.
In an embodiment, the step S5 includes the following sub-steps:
S501, in a post Num frame of the current frame, performing target tracking by using a Hungary algorithm, wherein the Num value can be preset according to a specific scene and is suggested to be 5. If successful tracking is possible, go to step S502;
the target tracking is preferably performed by using the Hungary algorithm, and the specific algorithm flow is the prior art and is not repeated.
S502, obtaining coordinates of candidate targets tracked in a post Num frame, respectively calculating a difference value between the coordinates and a target of a previous frame, and if the difference value gradually becomes larger and the movement in the abscissa direction has no mutation, considering the coordinate as a movement track of a high-altitude parabolic object.
In particular, the method according to the technical solution of the present invention may be implemented by those skilled in the art using computer software technology to implement an automatic operation flow, and a system apparatus for implementing the method, such as a computer readable storage medium storing a corresponding computer program according to the technical solution of the present invention, and a computer device including the operation of the corresponding computer program, should also fall within the protection scope of the present invention.
In some possible embodiments, a high-altitude parabolic detection system based on semantic segmentation is provided, comprising the following modules,
The first module is used for training a lightweight semantic segmentation network through knowledge distillation in advance, taking a frame without camera shake from a monitoring video, inputting the semantic segmentation network to extract a building area as a candidate area for high-altitude parabolic detection;
the second module is used for taking a plurality of camera shake-free frames of the monitoring video, performing binarization processing, and then performing background modeling by using a Gaussian mixture model to obtain a background image of the current scene;
The third module is used for judging camera shake of the current frame image to be detected, if the camera does not shake, the background obtained by the second module is used for detecting the motion of the current frame image in the building area by using a background difference method to obtain a moving object, and then the moving object enters the fourth module;
A fourth module, configured to denoise the moving object obtained by the third module;
and a fifth module, configured to perform target tracking on the denoised candidate object by using a hungarian algorithm, determine a tracking track if the candidate object can be successfully tracked, and consider the candidate object as a high-altitude parabolic object if the tracking track conforms to the track of the high-altitude parabolic object, so as to further obtain a throwing position and a falling point position of the candidate object.
In some possible embodiments, a semantic segmentation based high-altitude parabolic detection system is provided, including a processor and a memory, the memory for storing program instructions, the processor for invoking the store instructions in the memory to perform a semantic segmentation based high-altitude parabolic detection method as described above.
In some possible embodiments, a high-altitude parabolic detection system based on semantic segmentation is provided, which comprises a readable storage medium, wherein the readable storage medium is stored with a computer program, and the computer program realizes the high-altitude parabolic detection method based on semantic segmentation when being executed.
The specific embodiments described herein are offered by way of example only to illustrate the spirit of the invention. Those skilled in the art may make various modifications or additions to the described embodiments or substitutions thereof without departing from the spirit of the invention or exceeding the scope of the invention as defined in the accompanying claims.

Claims (8)

CN202111636002.6A2021-12-292021-12-29High-altitude parabolic detection method and system based on semantic segmentationActiveCN114332163B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202111636002.6ACN114332163B (en)2021-12-292021-12-29High-altitude parabolic detection method and system based on semantic segmentation

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202111636002.6ACN114332163B (en)2021-12-292021-12-29High-altitude parabolic detection method and system based on semantic segmentation

Publications (2)

Publication NumberPublication Date
CN114332163A CN114332163A (en)2022-04-12
CN114332163Btrue CN114332163B (en)2024-10-01

Family

ID=81016454

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202111636002.6AActiveCN114332163B (en)2021-12-292021-12-29High-altitude parabolic detection method and system based on semantic segmentation

Country Status (1)

CountryLink
CN (1)CN114332163B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN114913392A (en)*2022-05-272022-08-16青岛文达通科技股份有限公司 A kind of high-altitude parabolic detection method and system
CN116994201B (en)*2023-07-202024-03-29山东产研鲲云人工智能研究院有限公司Method for tracing and monitoring high-altitude parabolic objects and computing equipment
CN117975373B (en)*2024-03-292024-07-05济南大学Method and system for detecting and tracking high-altitude parabolic target on electric power construction site

Citations (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN109872341A (en)*2019-01-142019-06-11中建三局智能技术有限公司 A method and system for high-altitude parabolic detection based on computer vision
CN110348312A (en)*2019-06-142019-10-18武汉大学A kind of area video human action behavior real-time identification method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
KR102225579B1 (en)*2020-05-142021-03-10아주대학교산학협력단Method for semantic segmentation based on knowledge distillation with improved learning performance
CN111797688A (en)*2020-06-022020-10-20武汉大学Visual SLAM method based on optical flow and semantic segmentation
CN113792606B (en)*2021-08-182024-04-26清华大学 A low-cost self-supervised person re-identification model construction method based on multi-target tracking

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN109872341A (en)*2019-01-142019-06-11中建三局智能技术有限公司 A method and system for high-altitude parabolic detection based on computer vision
CN110348312A (en)*2019-06-142019-10-18武汉大学A kind of area video human action behavior real-time identification method

Also Published As

Publication numberPublication date
CN114332163A (en)2022-04-12

Similar Documents

PublicationPublication DateTitle
US10706285B2 (en)Automatic ship tracking method and system based on deep learning network and mean shift
CN114332163B (en)High-altitude parabolic detection method and system based on semantic segmentation
CN112669349B (en)Passenger flow statistics method, electronic equipment and storage medium
CN111460926B (en) A video pedestrian detection method incorporating multi-target tracking cues
CN101996410B (en)Method and system of detecting moving object under dynamic background
CN112489054A (en)Remote sensing image semantic segmentation method based on deep learning
Nassu et al.A vision-based approach for rail extraction and its application in a camera pan–tilt control system
CN108776974A (en)A kind of real-time modeling method method suitable for public transport scene
CN113689464A (en) A target tracking method based on adaptive multi-layer response fusion based on Siamese network
Pavel et al.Recurrent convolutional neural networks for object-class segmentation of RGB-D video
CN115511920A (en)Detection tracking method and system based on deep sort and deep EMD
Han et al.A method based on multi-convolution layers joint and generative adversarial networks for vehicle detection
CN119763179A (en) A method for detecting students&#39; classroom behavior based on deep learning
CN111986233B (en) Remote sensing video tracking method for extremely small targets in large scenes based on feature self-learning
CN119107582A (en) A security warning method based on YOLOv8 and DeepSORT algorithm
CN115775258A (en)Optical measurement and correlation method for motion trajectory of space target
CN111275733A (en)Method for realizing rapid tracking processing of multiple ships based on deep learning target detection technology
CN116030377A (en)Industrial control network security abnormal behavior identification method and system
CN110111358A (en)A kind of method for tracking target based on multilayer temporal filtering
Allak et al.Siamese neural networks in unmanned aerial vehicle target tracking process
CN112800932A (en)Method for detecting obvious ship target in marine background and electronic equipment
CN113744302A (en)Dynamic target behavior prediction method and system
Li et al.A fast detection method for polynomial fitting lane with self-attention module added
CN114612811B (en) A target behavior classification method, storage medium and terminal
CN119904485B (en)Multi-target tracking method based on weak clue and track prediction

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
GR01Patent grant
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp