CN109446946A - A kind of multi-cam real-time detection method based on multithreading - Google Patents

Abstract

Translated fromChinese

基于多线程的多摄像头实时检测方法，首先加载基于ResNet50和三重损失函数的行人重识别网络，构建检测人脸库，采用face_recognition库提取人脸库的特征向量，之后开始构建多线程系统，应用multiprocess库中的Queue构建队列，并采用daemon守护进程，之后通过Yolo3将人物定位并用Opencv裁剪出来，之后使用face_recognition库中的识别模块进行识别，若无人脸则采用行人重识别网络进行识别，最后通过多线程并行处理，可以在监控视频中对多个摄像头中的目标进行实时检测。

The real-time detection method of multi-camera based on multi-threading, firstly load the pedestrian re-recognition network based on ResNet50 and triple loss function, build the detection face library, use the face_recognition library to extract the feature vector of the face library, and then start to build a multi-threaded system and apply multiprocess The Queue in the library builds the queue and uses the daemon daemon process, then uses Yolo3 to locate the characters and use Opencv to cut them out, and then use the recognition module in the face_recognition library for recognition. If there is no face, use the pedestrian re-recognition network for recognition, and finally pass Multi-threaded parallel processing enables real-time detection of targets in multiple cameras in surveillance video.

Description

A kind of multi-cam real-time detection method based on multithreading

Technical field

The present invention relates to the methods being measured in real time to camera shooting video.

Background technique

Since safety-security area is quickly grown, camera function is become stronger day by day, and existing camera generally has communicationAgreement may be implemented wired and wireless long-distance video and read.Simultaneously as the increase in demand of safety, more and more to imageHead is installed in building, and monitoring is played the role of in the places such as street.Thus police etc. can carry out personage's by cameraMonitoring, and handle multiple cameras in real time and improve efficiency.

Recognition of face and pedestrian identify it is the key technology for identifying specific pedestrian again.But due to current effective sideFado uses deep learning neural network, its committed memory is big, while calculation amount is also more, and the scene of real time monitoring is difficultIt is handled.So using multithreading, parallel processing carried out using multiple threads for multiple cameras, it is so can be withGuarantee does not have successive influence for the video of multi-source camera, while can guarantee a higher real-time yet.

Summary of the invention

The present invention will overcome the disadvantages mentioned above of the prior art, provide a kind of multi-cam real-time detection side based on multithreadingMethod.

For real-time purpose to be realized, the present invention devises a kind of multi-cam real-time detection side based on multithreadingMethod can effectively improve the requirement of real-time, the precision that reduction recognition of face and pedestrian identify again.This is for real-time meshMark detection, is a very big promotion in efficiency, utilizable because the quantity of same time-triggered protocol camera increasesInformation is also more with time-varying, and it is accurate that user also can judge to detect whether from more information.

The present invention realizes technical solution used by foregoing invention purpose are as follows:

A kind of multi-cam method of real-time of multithreading, comprising the following steps:

Step 1. load pedestrian identifies network again: using the ResNet50 network of pre-training, will connect entirely in ResNet50Output before layer uses triple loss function tectonic networks as pedestrian's feature, and passes through the training of Market1501 data set.

Step 2. establishes face database, loads recognition of face network: selecting the third party library dlib of Python, and face_Recognition carries out the judgement of recognition of face, and the face picture that will test target is added to local library and carries out feature extraction.

Step 3. reads monitoring camera video: monitoring camera mostly uses greatly wired form to be configured, and usually takesIt is loaded with Rstp agreement, carries out video reading using the VideoCapture function in Opencv.

Step 4. personage cuts step: the Yolo3 weight of pre-download is put under specified directory, Yolo3 network is loaded, it willThe picture read from camera is put into Yolo3, obtains the coordinate of pedestrian, and cut out pedestrian's picture and identify.

Step 5. constructs the step of multithreading frame: the multithreading library multiprocess for selecting Python included, and setsSet the picture that multiple queues are used to store multiple cameras (quantity depends on camera quantity).And by main programProcess.start () function starts multithreading service.And daemon finger daemon is used, guarantee that it, in running background, is moreThread comes with being environmentally isolated before operation, guarantees the operation of parent process.

Step 6. person detecting step: in single subprocess, the knowledge of personage is carried out using the target detection network of Yolo3Not, the picture cut is put into the face detection module of face_recoginition and has detected whether clear face, such asFruit can carry out recognition of face if having face, in the case where no face, if someone in pedestrian library, and progressMatch, if there is being matched to people (Euclidean distance is less than threshold value), then identifies successfully and outline personage to add label, if do not hadThe people being matched to can not then judge；If the nobody of pedestrian library, it can not judge

Compared with prior art, have the advantages of technical solution of the present invention:

(1) present invention makes full use of computer memory space, can handle multiple cameras as far as possible, promotes work effectRate, and reduce cost.

(2) third party library carried using Python itself has transplanting convenient, the advantages of should be readily appreciated that.

Detailed description of the invention

Fig. 1: the flow chart of the method for the present invention；

Fig. 2: multithreading setting procedure figure of the invention；

Fig. 3: face characteristic saves flow chart.

Specific embodiment

Understand for the ease of those of ordinary skill in the art and implement the present invention, with reference to the accompanying drawings and examples to this hairIt is bright to be described in further detail.

A kind of multi-cam real-time detection method based on multithreading contains step:

(1) pedestrian that load was trained identifies network again

Step 11: selecting existing pedestrian to identify network again, have on current some open source websites in Market1501 dataThe accuracy for having reached 94% on collection, is substantially able to satisfy demand.The structure of the network of selection is as described below, ResNet50 conductLast full articulamentum is removed, and handled 3 block once by backbone network, and res5a, res5b do pondization processing,And the feature vector of one 1024 dimension is obtained by exporting the full articulamentum tieed up for 1024 after the splicing of left and right, while doing to res5cPondization processing, is directly launched into the feature vector of 2048 dimensions.The two are superimposed and is used as final feature vector.

Step 12: add triple loss functions, be added hard_triplet loss function, softmax loss function andRing loss function is built into last pedestrian and identifies network again.The data set of oneself can be finely adjusted, be had reached more preferableEffect.

(2) face database is established, and loads human face recognition model

Step 21: each 2-3 positive face photos of taking one's hat off of example storage, wherein photo is preferably without any processing, otherwiseIt will affect precision.It is stored in a file then according to naming rule, if Li Ming reference numeral is 0003, then photo is namedFor the photo sequence that 0003_*.jpg, * are this example face.

Step 22: dlib and face_recognition third party library being installed in the environment, and can be first by face databaseFace picture pass through feature vector save.

(3) camera head monitor picture step is read

Step 31: the reading of camera video is carried out using the VideoCapture class in Opencv.It will be assisted with RTSPThe user name of camera is discussed, password and IP address define respectively, and are filled according to specified format, and each company is taken the photographAs head is all different.

Step 32: the picture in VideoCapture class being read out using read () function in Opencv, is providedWe are handled.

(4) person detecting step

Step 41: the Yolo3 weight of pre-download being put under specified directory, Yolo3 is a kind of mesh that occupied space is lessMark detection neural network, accuracy rate is low compared to Faster-Rcnn, but uses enough.

Step 42: the picture read from camera is put into Yolo3, obtains the coordinate of pedestrian by load Yolo3 network,And it cuts out pedestrian's picture and identifies.

(5) the step of constructing multithreading frame

Step 51: the load Python included library Multiprocess, and multiple queues are set, number and desired readingThe quantity of camera video is identical, and capacity is set as 2.

Step 52: camera picture is read and is pressed into queue by two queue operations of setting, operation of bringing up the rear, dequeue operationPicture is extruded from queue, and is handled.

Step 53: each sub thread identifies the pedestrian loaded before again and Yolo3 network reads and carries out independent pointAnalysis carrys out parallel work-flow with this, promotes processing speed.

(6) person detecting step:

Step 61: after the picture of camera is read by RSTP protocol remote first, in single subprocess, usingThe target detection network of Yolo3 carries out the identification of personage, and is cut personage by the coordinate that Yolo3 is exported.

Step 62: the picture cut being put into the face detection module of face_recoginition and is detected whetherThere is clear face, recognition of face can be carried out if there is face, step 3 is jumped into if without face.Carry out face knowledgeIf capableing of the information (reaching under threshold value) of very determining pedestrian when other, current information is charged in pedestrian library.

Step 63: in the case where no face, be first the case where seeing pedestrian library, if someone in pedestrian library,It is then matched, if there is being matched to people (Euclidean distance be less than threshold value), then identifies successfully and outline personage to add label,If the people being not matched to can not judge.If the nobody of pedestrian library, it can not judge.

Content described in this specification embodiment is only enumerating to the way of realization of inventive concept, protection of the inventionRange should not be construed as being limited to the specific forms stated in the embodiments, and protection scope of the present invention is also and in art technologyPersonnel conceive according to the present invention it is conceivable that equivalent technologies mean.

Claims (1)

Translated fromChinese

1.一种基于多线程的多摄像头实时检测方法，包括以下步骤；1. a multi-thread-based multi-camera real-time detection method, comprising the following steps;(1)加载训练过的行人重识别网络；(1) Load the trained pedestrian re-identification network;选择现有行人重识别网络，将ResNet50网络最后部分的结构转换成3个分别的block，并分别将其的输出转化为最后的3072维的特征；最后添加3重损失函数作为行人重识别网络的步骤；Select the existing person re-identification network, convert the structure of the last part of the ResNet50 network into 3 separate blocks, and convert its output into the final 3072-dimensional feature; finally add three loss functions as the pedestrian re-identification network. step;(2)建立人脸库,并加载人脸识别模型；(2) Establish a face database and load the face recognition model;步骤21，每个实例存放2-3张免冠正脸照片，并以编号+排列序号命名图片，保存在指定目录下的步骤；Step 21, each instance stores 2-3 bareheaded face photos, and names the pictures with the serial number + sequence number, and saves them in the specified directory;步骤22，在指定目录下建立图片名与人名相应的文件；Step 22, establishes the corresponding file of picture name and person's name under the designated directory;(3)读取摄像头监控画面步骤；(3) the step of reading the monitoring screen of the camera;步骤31，使用Opencv中的VideoCapture类进行摄像头视频的读取；Step 31, use the VideoCapture class in Opencv to read the camera video;步骤32，使用Opencv中的read()函数将VideoCapture类中的图片图区出来并加入到队列中；Step 32, use the read() function in Opencv to extract the picture area in the VideoCapture class and add it to the queue;(4)人物裁剪步骤；(4) Character cutting steps;步骤41，将预下载的Yolo3权重放入指定目录下，加载Yolo3网络；Step 41, put the pre-downloaded Yolo3 weights into the specified directory, and load the Yolo3 network;步骤42，将从摄像头读取的图片放入Yolo3中，得到行人的坐标，并裁剪出行人图片进行识别的步骤；Step 42, put the picture read from the camera into Yolo3, get the coordinates of the pedestrian, and crop the pedestrian picture for identification;(5)构建多线程框架的步骤；(5) the steps of constructing a multi-threaded framework;步骤51，加载Python自带的Multiprocess库，设定两个队列操作；Step 51, load the Multiprocess library that comes with Python, and set two queue operations;步骤52，每个子线程将之前加载的行人重识别和Yolo3网络读取并进行独立的分析；Step 52, each sub-thread reads the previously loaded pedestrian re-identification and Yolo3 network and performs independent analysis;(6)人物检测步骤；(6) a person detection step;步骤61，在单个子进程中，使用Yolo3的目标检测网络进行人物的识别，将裁剪下来的图片放入face_recoginition的人脸检测模块中检测是否有清晰人脸；Step 61, in a single sub-process, use the target detection network of Yolo3 to identify the person, and put the cropped picture into the face detection module of face_recoginition to detect whether there is a clear face;步骤62，如果有人脸的话可以进行人脸识别，在没有人脸的情况下，如果行人库中有人的话，则进行匹配，如果有匹配到人(欧式距离小于阈值)，则识别成功并将人物框出来加上标签，如果没有匹配到的人，则无法判断；如果行人库没有人的话，则无法判断的。Step 62, if there is a face, face recognition can be performed. In the absence of a face, if there is someone in the pedestrian database, then matching is performed. If there is a matching person (the Euclidean distance is less than the threshold), the recognition is successful and the person is identified. If there is no matching person, it cannot be judged; if there is no one in the pedestrian database, it cannot be judged.