Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a black-neck crane monitoring and tracking method based on yolo and deep SORT, and the specific scheme provided by the invention is as follows:
a black-neck crane monitoring and tracking method based on yolo and deep SORT collects image data and video data in a target range,
analyzing the image data and the video data by using a target detection tracking model, detecting and identifying black neck crane images and videos, wherein based on YOLO and DeepsORT, a black neck crane data set is used for learning depth correlation measurement in an offline pre-training stage, a measurement-to-track correlation is established by using a visual appearance space neighbor query method in an online application stage, and the target detection tracking model is trained and completed,
and uploading the detection and identification result to a visualization platform.
Preferably, in the black-neck crane monitoring and tracking method based on yolo and DeepSORT, the acquiring of image data and video data within the target range includes:
and image data and video data in a target range are acquired through a satellite, an unmanned aerial vehicle and a camera terminal.
Preferably, in the black-neck crane monitoring and tracking method based on yolo and deep sort, the analyzing the image data and the video data by using the target detection and tracking model includes:
the image data and video data are analyzed by a target detection tracking model in the edge computing device.
Preferably, in the black-neck crane monitoring and tracking method based on yolo and deep sort, the edge computing device uses multiple threads to perform inference of video stream target detection and tracking.
Preferably, the inference process in the black-neck crane monitoring and tracking method based on yolo and deep sort is as follows:
the method comprises the steps of deploying a data push flow thread which is in charge of receiving and caching video frames, deploying an algorithm reasoning thread, carrying out target detection on the video frames pushed out by the data push flow thread, deploying a target tracking algorithm reasoning thread, tracking targets retreated by the algorithm reasoning thread, deploying a data storage thread, and storing images, videos, recognition results and tracking processes recognized by the algorithm reasoning thread.
The invention also provides a black-neck crane monitoring and tracking system based on yolo and deep SORT, which comprises an acquisition module, an analysis module and a communication module,
the acquisition module acquires image data and video data within a target range,
the analysis module analyzes the image data and the video data by using a target detection tracking model, detects and identifies black neck crane images and videos, learns depth correlation measurement by using a black neck crane data set in an off-line pre-training stage based on YOLO and DeepsORT, establishes measurement-to-track correlation by using a neighbor query method of a visual appearance space in an on-line application stage, and trains and finishes the target detection tracking model,
and the communication module uploads the detection identification result to the visualization platform.
The invention also provides edge computing equipment for monitoring and tracking the black-neck crane based on yolo and deep SORT, which comprises a receiving module, an analysis module and a communication module,
the receiving module receives image data and video data in a target range acquired by the acquisition equipment,
the analysis module analyzes the image data and the video data by using a target detection tracking model, detects and identifies black neck crane images and videos, learns depth correlation measurement by using a black neck crane data set in an off-line pre-training stage based on YOLO and DeepsORT, establishes measurement-to-track correlation by using a neighbor query method of a visual appearance space in an on-line application stage, and trains and finishes the target detection tracking model,
and the communication module uploads the detection identification result to the visualization platform.
Preferably, the black-neck crane monitoring and tracking edge computing device based on yolo and deep sort completes inference of video stream target detection and tracking by utilizing multiple threads.
Preferably, the inference process of the edge computing device based on the black-neck crane monitoring and tracking of yolo and deep sort is as follows:
the method comprises the steps of deploying a data push flow thread which is in charge of receiving and caching video frames, deploying an algorithm reasoning thread, carrying out target detection on the video frames pushed out by the data push flow thread, deploying a target tracking algorithm reasoning thread, tracking targets retreated by the algorithm reasoning thread, deploying a data storage thread, and storing images, videos, recognition results and tracking processes recognized by the algorithm reasoning thread.
The invention has the advantages that:
the invention provides a black-neck crane monitoring and tracking method based on yolo and DeepSORT, which can realize automatic counting and tracking, identify the number of black-neck cranes in a visual field range in real time and reduce the manual workload for identifying the number of the black-neck cranes.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The invention provides a black-neck crane monitoring and tracking method based on yolo and deep SORT, which is used for collecting image data and video data in a target range,
analyzing the image data and the video data by using a target detection tracking model, detecting and identifying black neck crane images and videos, wherein based on YOLO and DeepsORT, a black neck crane data set is used for learning depth correlation measurement in an offline pre-training stage, a measurement-to-track correlation is established by using a visual appearance space neighbor query method in an online application stage, and the target detection tracking model is trained and completed,
and uploading the detection and identification result to a visualization platform.
The method can realize automatic counting and tracking, identify the number of black-neck cranes in a visual field range in real time, reduce the manual workload of identifying the number of the black-neck cranes, has the characteristics of high efficiency, small error and low cost compared with the method for manually identifying the black-neck cranes, and can realize all-weather 24-hour detection and tracking.
In specific application, in some embodiments of the method, when the black-neck crane tracking identification is carried out, an ad hoc network or a special line can be set up for carrying out data transmission,
for example, VPDN private network: it adopts PSTN, ISDN, XDSL, cable or wireless to access the China broadband Internet in dialing mode, adopts special network encryption and communication protocol to construct a virtual special channel without external interference, thereby safely accessing the internal data resource service of the network,
special line: a bridge is provided for access to a public network,
an infinite bridge: bridging communications between two or more networks is accomplished using wireless transmission, and referring to figure 2,
the method of the invention utilizes the camera shooting collection equipment such as the wild protection infrared camera and the fixed camera to collect the image data and the video data in the target range,
receiving the image data and the video data by utilizing an edge computing device, such as an edge computing box, identifying the acquired image data and video data through a target detection tracking model, wherein the target detection tracking model designs a deep convolutional neural network based on YOLO v5 to train and verify a basic data set, perfects the loss and the fitting degree of a monitoring algorithm,
the method comprises the steps that inference of video stream target detection tracking is completed through an edge computing box by means of multithreading, a data stream pushing Thread through which video frames are deployed and used for bearing and caching video frames, an algorithm inference Thread through which detection is performed, video frames pushed out by the Thread through which detection is performed are deployed, a target tracking algorithm inference Thread through which tracking is performed on the basis of targets exited by the Thread through which detection is performed, a data Storage Thread through which images, videos, identification results, tracking processes and the number identified by the Thread through which detection is performed are deployed, and the data such as the identification results, the tracking processes and the number are transmitted back to a server, and the image data and the video data can be backed up into the server slowly and are finally displayed on a visualization platform such as a black-neck crane natural protection area.
In the process, the method of the invention combines the ad hoc network, the special line and the edge computing box, thereby effectively reducing the data transmission cost and greatly improving the monitoring timeliness.
According to the method flow, a black-neck crane monitoring and tracking architecture is also constructed, and referring to fig. 1, a sky-ground integrated network is shown: by means of satellite data, unmanned aerial vehicle monitoring, ground monitoring equipment and personnel management and protection teams, a sky-ground integrated real-time monitoring network is formed.
Monitoring the cloud: and various storage, calculation, network and other resources required by the operation of the big data platform are provided in a cloud mode.
The data center comprises: and summarizing and integrating various data to form a uniform basic data resource for supporting the operation of the platform.
A support platform: and the work identification and data analysis service application supporting the monitoring and tracking of the black-neck crane is realized.
N intelligent applications: and carrying out targeted system development construction according to the properties of different works, including black-neck crane identification, quantity identification, black-neck crane tracking, data analysis and the like.
The invention also provides a black-neck crane monitoring and tracking system based on yolo and deep SORT, which comprises an acquisition module, an analysis module and a communication module,
the acquisition module acquires image data and video data within a target range,
the analysis module analyzes the image data and the video data by using a target detection tracking model, detects and identifies black neck crane images and videos, learns depth correlation measurement by using a black neck crane data set in an off-line pre-training stage based on YOLO and DeepsORT, establishes measurement-to-track correlation by using a neighbor query method of a visual appearance space in an on-line application stage, and trains and finishes the target detection tracking model,
and the communication module uploads the detection identification result to the visualization platform.
The information interaction, execution process and other contents between the modules in the system are based on the same concept as the method embodiment of the present invention, and specific contents can be referred to the description in the method embodiment of the present invention, and are not described herein again.
Similarly, the system can realize automatic counting and tracking, identify the number of black-neck cranes in a visual field range in real time, reduce the manual workload for identifying the number of black-neck cranes, and compared with a method for manually identifying black-neck cranes, the system has the characteristics of high efficiency, small error and low cost, and can realize all-weather 24-hour detection and tracking.
The invention also provides edge computing equipment for black-neck crane monitoring and tracking based on yolo and DeepsORT, which comprises a receiving module, an analysis module and a communication module,
the receiving module receives image data and video data in a target range acquired by the acquisition equipment,
the analysis module analyzes the image data and the video data by using a target detection tracking model, detects and identifies black neck crane images and videos, learns depth correlation measurement by using a black neck crane data set in an off-line pre-training stage based on YOLO and DeepsORT, establishes measurement-to-track correlation by using a neighbor query method of a visual appearance space in an on-line application stage, and trains and finishes the target detection tracking model,
and the communication module uploads the detection identification result to the visualization platform.
Since the contents of information interaction, execution process, and the like between the modules in the edge computing device are based on the same concept as the method embodiment of the present invention, specific contents may refer to the description in the method embodiment of the present invention, and are not described herein again.
Similarly, the edge computing equipment can help to realize automatic counting and tracking, identify the number of black-neck cranes in a visual field range in real time and reduce the manual workload of identifying the number of the black-neck cranes.
It should be noted that not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution sequence of the steps is not fixed and can be adjusted according to the needs. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.