Multi-mode multi-strategy fused stranger identification method and systemTechnical Field
The invention relates to the technical field of face recognition, in particular to a method and a system for recognizing strangers through multi-mode multi-strategy fusion.
Background
Leakage of absolute data (such as production process data) and loss of production materials of modern companies are always difficult points for company management and control. And all personnel entering the company park are managed and controlled and traced, strangers and suspicious personnel are identified in time, corresponding tracks and frequency analysis can be formed, effective early warning is achieved before problems occur, effective tracing is conducted after problems occur, and the problems can be relieved to a great extent. In this process, the core is the identification of strangers and suspicious persons.
In the prior art, a stranger identification method is generally adopted, and people in a black-and-white list library such as an employee library and a family library are directly judged to be strangers according to identification results. According to the method, stranger judgment is carried out only through single piece of single-dimensional information, the technical problem of high false identification rate exists, and the technical problem of low precision and recall rate exists.
Disclosure of Invention
In view of the above, the present invention provides a method and a system for recognizing strangers through multi-modal multi-policy fusion, so as to alleviate the technical problems of high false recognition rate, low precision and low recall rate in the prior art.
In a first aspect, an embodiment of the present invention provides a multi-modal multi-policy fused stranger identification method, which is applied to a server, and includes: extracting a plurality of characteristic vectors of the face image to be recognized through the trained face recognition model to obtain a plurality of characteristic vectors; comparing the plurality of feature vectors with feature vectors in a preset feature vector library respectively to obtain comparison results; the preset feature vector library is a set of feature vectors of a preset face image; identifying each face image to be identified in the plurality of face images to be identified based on the comparison result to obtain a first identification result; the first recognition result includes any one of: a strange face image, a non-strange face image; performing cluster analysis on the plurality of face images to be recognized based on the plurality of feature vectors to obtain a plurality of face categories; wherein one face class corresponds to a face of a person; modifying the first recognition result based on the plurality of face categories to obtain a second recognition result, wherein the second recognition result comprises any one of the following items: a strange face image, a non-strange face image.
Further, comprising: before extracting feature vectors of a plurality of face images to be recognized through the trained face recognition model, the method further comprises the following steps: detecting face images in the target images to obtain a plurality of initial face images; the target images are image frames comprising face images to be recognized; and obtaining a plurality of face images to be recognized based on the plurality of initial face images.
Further, obtaining a plurality of facial images to be recognized based on the plurality of initial facial images includes: performing quality evaluation operation on the plurality of initial face images through the trained face quality evaluation model to obtain face quality scores; and taking the face image of which the face quality score is higher than the preset score in the plurality of initial face images as a face image to be recognized.
Further, modifying the first recognition result based on the plurality of face categories to obtain a second recognition result, including: and correcting the first recognition results of a plurality of face images to be recognized belonging to the same face category into the same recognition result to obtain a second recognition result.
Further, after the first recognition result is modified based on the plurality of face classes to obtain a second recognition result, the method further includes: acquiring snapshot position information of a plurality of face images to be recognized belonging to the same face category; and drawing a moving path of the target face corresponding to the face type based on the snapshot position information.
Further, after the first recognition result is modified based on the plurality of face classes to obtain a second recognition result, the method further includes: and storing the image to be recognized, of which the second recognition result is the strange face image, into a preset strange face bottom library.
In a second aspect, an embodiment of the present invention further provides a system for recognizing strangers in multi-modal multi-policy fusion, which is applied to a server, and includes: the system comprises a feature extraction module, a comparison module, an identification module, a clustering module and a correction module, wherein the feature extraction module is used for extracting feature vectors of a plurality of face images to be identified through a trained face identification model to obtain a plurality of feature vectors; the comparison module is used for comparing the plurality of characteristic vectors with characteristic vectors in a preset characteristic vector library respectively to obtain comparison results; the preset feature vector library is a set of feature vectors of a preset face image; the identification module is used for identifying each face image to be identified in the plurality of face images to be identified based on the comparison result to obtain a first identification result; the first recognition result includes any one of: a strange face image, a non-strange face image; the clustering module is used for clustering and analyzing the plurality of face images to be recognized based on the plurality of feature vectors to obtain a plurality of face categories; wherein one face class corresponds to a face of a person; the correction module is configured to correct the first recognition result based on the plurality of face categories to obtain a second recognition result, where the second recognition result includes any one of: a strange face image, a non-strange face image.
Further, the system further comprises: a path drawing module to: acquiring snapshot position information of a plurality of face images to be recognized belonging to the same face category; and drawing a moving path of the target face corresponding to the face type based on the snapshot position information.
In a third aspect, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method according to the first aspect when executing the computer program.
In a fourth aspect, the present invention further provides a computer-readable medium having non-volatile program code executable by a processor, where the program code causes the processor to execute the method according to the first aspect.
The invention provides a multi-mode multi-strategy fused stranger identification method and system, wherein a plurality of feature vectors of a face image to be identified are extracted through a trained face identification model to obtain a plurality of feature vectors; comparing the plurality of feature vectors with feature vectors in a preset feature vector library respectively to obtain comparison results; identifying each face image to be identified in the plurality of face images to be identified based on the comparison result to obtain a first identification result; performing cluster analysis on a plurality of face images to be recognized based on a plurality of feature vectors to obtain a plurality of face categories; and correcting the first recognition result based on the plurality of face categories to obtain a second recognition result. According to the invention, the face recognition model and the cluster analysis are fused, so that the stranger face recognition accuracy is improved, the recall rate and precision of stranger recognition and grabbing are improved, and the technical problems of high false recognition rate, low precision and low recall rate in the prior art are solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a stranger identification method with multi-modal multi-policy fusion according to an embodiment of the present invention;
FIG. 2 is a flow chart of another multi-modal multi-policy converged stranger identification method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a multi-modal multi-policy converged stranger identification system according to an embodiment of the present invention;
fig. 4 is a schematic diagram of another multi-modal multi-policy converged stranger identification system according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
fig. 1 is a flowchart of a stranger identification method for multi-modal multi-policy fusion, which is applied to a server according to an embodiment of the present invention. As shown in fig. 1, the method specifically includes the following steps:
and S102, extracting a plurality of characteristic vectors of the face image to be recognized through the trained face recognition model to obtain a plurality of characteristic vectors.
Step S104, comparing the plurality of characteristic vectors with characteristic vectors in a preset characteristic vector library respectively to obtain comparison results; the preset feature vector library is a set of feature vectors of a preset face image.
Optionally, the preset vector library includes a feature vector library of a known face image and a feature vector library of a strange face, where the known face image vector library includes: the system comprises a staff base library, a family base library, a stationary outer package base library and a blacklist base library.
Step S106, identifying each face image to be identified in the plurality of face images to be identified based on the comparison result to obtain a first identification result; the first recognition result includes any one of: a strange face image, a non-strange face image.
Step S108, carrying out cluster analysis on a plurality of face images to be recognized based on a plurality of feature vectors to obtain a plurality of face categories; one of the face classes corresponds to a person's face.
Step S110, the first recognition result is corrected based on a plurality of face categories to obtain a second recognition result, and the second recognition result comprises any one of the following items: a strange face image, a non-strange face image.
According to the stranger identification method based on multi-mode multi-strategy fusion, provided by the invention, the face identification model and the cluster analysis are fused, so that the stranger face identification accuracy is improved, the recall rate and precision of stranger identification and grabbing are improved, and the technical problems of high false identification rate, low precision and low recall rate in the prior art are solved.
Optionally, the method provided in the embodiment of the present invention further includes, before step S102, a step of obtaining a plurality of face images to be recognized, specifically, detecting face images in a plurality of target images to obtain a plurality of initial face images; the target images are image frames comprising face images to be recognized; and obtaining a plurality of face images to be recognized based on the plurality of initial face images.
In the embodiment of the invention, firstly, an image frame comprising a face image to be recognized is obtained through a camera, then, the face detection and face alignment operation are carried out on the image frame through a face detection method, the face image after alignment correction is obtained, a plurality of initial face images are obtained, then, the face quality screening is carried out on the initial face images, the face images with unqualified face quality such as blurring, shielding, large posture, strong illumination and the like are filtered, and finally, a plurality of face images to be recognized are obtained.
Specifically, obtaining a plurality of facial images to be recognized based on a plurality of initial facial images includes: performing quality evaluation operation on a plurality of initial face images through the trained face quality evaluation model to obtain face quality scores; and taking the face image with the face quality score higher than the preset score in the plurality of initial face images as the face image to be recognized.
In the embodiment of the invention, a plurality of characteristic vectors are respectively compared with the characteristic vectors in a preset characteristic vector library to obtain comparison results; and then, identifying each face image to be identified in the plurality of face images to be identified based on the comparison result to obtain a first identification result.
Specifically, firstly, calculating the distance between each feature vector and a feature vector in a preset feature vector library, and judging whether the minimum distance is smaller than a preset threshold value or not; if the feature vector is smaller than the preset feature vector, determining whether the feature vector belongs to a feature vector library of a known face image in a preset feature vector library or a feature vector library of a strange face; if the feature vector is larger than or equal to the preset feature vector library, determining that the feature vector is not in the preset feature vector library; and determining the judgment result as a comparison result. According to the comparison result, for example, if the feature vector belongs to a feature vector library of a known face image, identifying the face image to be identified corresponding to the feature vector as a non-unfamiliar face image; and if the feature vector belongs to a feature vector library of the strange face or the feature vector is not in a preset feature vector library, identifying the face image to be identified corresponding to the feature vector as the strange face image.
Optionally, step S110 includes: and correcting the first recognition results of a plurality of face images to be recognized belonging to the same face category into the same recognition result to obtain a second recognition result.
Specifically, the first recognition result of each face image to be recognized in a plurality of face images to be recognized belonging to the same face category is determined, and if the first recognition results of a plurality of face images to be recognized are different, the first recognition result is corrected to be the same recognition result according to the proportion of the face image to be recognized corresponding to each recognition result. For example, the number of the face images to be recognized belonging to the same face category is 10, wherein the first recognition result of 8 face images to be recognized is a strange face image, and the first recognition result of 2 face images to be recognized is a non-strange face image, and the recognition result of 2 face images to be recognized, of which the recognition results are non-strange face images, is modified into a strange face image.
Optionally, fig. 2 is a flowchart of another multi-modal multi-policy fused stranger identification method provided in an embodiment of the present invention, and as shown in fig. 2, the method further includes:
step S112, capturing position information of a plurality of face images to be recognized belonging to the same face category.
Specifically, the snapshot position information of the face image to be recognized can be determined according to the snapshot video corresponding to the image frame where the face image to be recognized is located and the position of the camera shooting the snapshot video.
And step S114, drawing a moving path of the target face corresponding to the face type based on the snapshot position information.
In the embodiment of the invention, whether the face image to be recognized corresponding to each face type belongs to a strange face image or not can be determined according to the face recognition result, and meanwhile, the moving path of the face corresponding to each face type can be drawn according to the snapshot position information of the face image to be recognized. For example, a moving path of the recognized strange face image may be drawn, so that a function of tracking entrance and exit of a stranger entering the recognition area may be realized.
Optionally, as shown in fig. 2, the method further includes:
and step S116, storing the image to be recognized, of which the second recognition result is the strange face image, into a preset strange face bottom library.
Specifically, the face image to be recognized corresponding to the cluster center of each face category obtained in step S108 is stored in a preset strange face base library.
The embodiment of the invention provides a multi-mode multi-strategy fused stranger identification method, which comprises the steps of firstly carrying out face detection alignment on a snap-shot picture after snap-shot data are obtained through a camera to obtain an aligned and corrected face image, then finishing feature vector extraction through face identification, carrying out retrieval comparison on feature vectors of a preset feature vector library to obtain a first identification result, correcting the first identification result through a clustering algorithm to obtain a final second identification result, and finally finishing the storage of strangers in a warehouse on the day according to the second identification result.
The embodiment of the invention integrates various modes and various strategies such as face detection, face quality evaluation, face recognition characteristics, clustering algorithm, trend track analysis and the like, realizes a complete stranger recognition method, greatly improves recall and precision of stranger recognition and grabbing, and improves the recognition accuracy of strangers.
Example two:
fig. 3 is a schematic diagram of a multi-modal multi-policy converged stranger identification system applied to a server according to an embodiment of the invention. As shown in fig. 3, the system includes: the system comprises a feature extraction module 10, a comparison module 20, an identification module 30, a clustering module 40 and a correction module 50.
Specifically, the feature extraction module 10 is configured to extract feature vectors of a plurality of face images to be recognized through the trained face recognition model, so as to obtain a plurality of feature vectors.
The comparison module 20 is configured to perform comparison operations on the plurality of feature vectors and feature vectors in a preset feature vector library respectively to obtain comparison results; the preset feature vector library is a set of feature vectors of a preset face image.
The recognition module 30 is configured to recognize each to-be-recognized face image of the plurality of to-be-recognized face images based on the comparison result to obtain a first recognition result; the first recognition result includes any one of: a strange face image, a non-strange face image.
The clustering module 40 is configured to perform clustering analysis on a plurality of face images to be recognized based on a plurality of feature vectors to obtain a plurality of face categories; one of the face classes corresponds to a person's face.
A modification module 50, configured to modify the first recognition result based on a plurality of face categories to obtain a second recognition result, where the second recognition result includes any one of the following: a strange face image, a non-strange face image.
Specifically, the modification module 50 is configured to modify a first recognition result of a plurality of to-be-recognized face images belonging to the same face category into the same recognition result, so as to obtain a second recognition result.
The invention provides a multi-mode multi-strategy fused stranger identification system, which is characterized in that a feature extraction module is used for extracting feature vectors of a plurality of face images to be identified by using a trained face identification model to obtain a plurality of feature vectors; comparing the plurality of characteristic vectors with characteristic vectors in a preset characteristic vector library respectively through a comparison module to obtain comparison results; identifying each face image to be identified in the plurality of face images to be identified through an identification module based on the comparison result to obtain a first identification result; clustering analysis is carried out on a plurality of face images to be recognized through a clustering module on a plurality of feature vectors to obtain a plurality of face categories; and finally, correcting the first recognition result based on the plurality of face categories through a correction module to obtain a second recognition result. According to the invention, through a mode of fusing the face recognition model and the cluster analysis, the accuracy of stranger face recognition is improved, the recall rate and precision of stranger recognition and grabbing are improved, and the technical problems of high false recognition rate, low precision and low recall rate in the prior art are solved.
Optionally, fig. 4 is a schematic diagram of another multi-modal multi-policy fused stranger identification method and system provided according to an embodiment of the present invention, and as shown in fig. 4, the system further includes: a face detection module 60 and a face quality assessment module 70.
Specifically, the face detection module 60 is configured to detect face images in a plurality of target images to obtain a plurality of initial face images; the target images are image frames including face images to be recognized.
And the face quality evaluation module 70 is configured to obtain a plurality of face images to be recognized based on the plurality of initial face images.
Specifically, the face quality evaluation module 70 is configured to perform quality evaluation operation on a plurality of initial face images through the trained face quality evaluation model to obtain face quality scores; and taking the face image with the face quality score higher than the preset score in the plurality of initial face images as the face image to be recognized.
Optionally, as shown in fig. 4, the system further includes: the path drawing module 80 is configured to obtain snapshot position information of a plurality of to-be-recognized face images belonging to the same face category; and drawing a moving path of the target face corresponding to the face type based on the snapshot position information.
Optionally, as shown in fig. 4, the system further includes: and the storage module 90 is used for storing the image to be recognized, of which the second recognition result is the strange face image, into a preset strange face bottom library.
The embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the steps of the method in the first embodiment are implemented.
The embodiment of the invention also provides a computer readable medium with a non-volatile program code executable by a processor, wherein the program code causes the processor to execute the method in the first embodiment.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.