Disclosure of Invention
The application provides a clothing attribute identification method, a clothing attribute identification device and electronic equipment, which are used for improving the efficiency and accuracy of clothing attribute identification, and the technical scheme adopted by the application is as follows:
in a first aspect, a neural network-based apparel attribute identification method is provided, the method comprising,
identifying and determining at least one target person in the image to be identified in a preset identification mode;
and clothing attribute recognition is carried out on the image to be recognized comprising at least one target character through a pre-trained neural network recognition model, so that clothing attribute recognition results of the at least one target character are obtained.
Specifically, clothing attribute recognition is carried out on an image to be recognized comprising at least one target character through a pre-trained neural network recognition model to obtain a clothing attribute recognition result of the at least one target character, including,
and carrying out body region segmentation on any target person in the image to be recognized through a pre-trained neural network recognition model, and carrying out clothing attribute recognition on each body region to obtain a clothing attribute recognition result of the target person in the image to be recognized.
Specifically, clothing attribute recognition is carried out on an image to be recognized comprising at least one target character through a pre-trained neural network recognition model to obtain a clothing attribute recognition result of the at least one target character, including,
performing segmentation processing on an image to be identified to obtain at least one segmentation image comprising a single target figure;
and (3) clothing attribute recognition is carried out on any segmentation image comprising a single target character through a pre-trained neural network recognition model, so that clothing attribute recognition results of the characters in any segmentation image are obtained.
Wherein, the clothing attribute recognition result of the person comprises at least one of the following items:
a type of apparel; clothing color; the number of clothes;
the apparel includes at least one of:
clothing, hats, shoes, accessories.
Specifically, through a preset identification mode, the image to be identified is identified and determined to include at least one target person, including,
extracting at least one image frame from a video acquired by image acquisition equipment according to a preset extraction frequency, wherein the preset extraction frequency is determined according to the statistical average time length of a pedestrian passing through a control area of an image acquisition device;
and detecting and recognizing at least one image frame through a pre-trained portrait detection and recognition model, and recognizing and determining at least one to-be-recognized image comprising at least one target person.
Further, the image to be identified is segmented to obtain at least one segmented image comprising a single target character, wherein the at least one segmented image comprises at least one item selected from the group consisting of,
performing segmentation processing on an image to be identified based on a region segmentation method to obtain at least one segmentation image comprising a single target figure;
and performing segmentation processing on the image to be identified based on an edge segmentation method to obtain at least one segmentation image comprising a single target character.
Further, clothing attribute recognition is carried out on any segmentation image comprising a single target character through a pre-trained neural network recognition model to obtain clothing attribute recognition results of the characters in any segmentation image, including,
clothing feature extraction is carried out on any segmented image comprising a single target figure to obtain clothing feature information aiming at any target figure;
and inputting the clothing feature information aiming at any target person into a pre-trained neural network recognition model to obtain a clothing attribute recognition result of any person.
Further, the method may further comprise,
aiming at a current image to be recognized, performing segmentation processing on a first preset number of image frames which are in front and a second preset number of image frames which are behind on a video time axis relative to the current image to be recognized to obtain a plurality of segmented images comprising a single target figure;
extracting character features of a plurality of segmentation images comprising a single target character to obtain feature information aiming at each character;
similarity calculation is carried out on the feature information of each character, and duplication removal is carried out on a plurality of segmentation images comprising a single target character according to the similarity calculation result, so that duplicated segmentation images are obtained;
clothing attribute recognition is carried out on any segmentation image comprising a single target character through a pre-trained neural network recognition model to obtain clothing attribute recognition results of the character in any segmentation image, including,
and (4) clothing attribute recognition is carried out on the cut images after the duplication removal through a pre-trained neural network recognition model, and clothing attribute recognition results of figures included in any cut image after the duplication removal are obtained.
Further, the method further comprises:
storing the clothing attribute identification result, the image to be identified and the corresponding relation between the clothing attribute identification result and the image to be identified;
wherein, the method also comprises:
when a person query request including clothing attribute information is received, the image information of the person corresponding to the query request is queried and determined based on the corresponding relation between the clothing attribute recognition result and the image to be recognized.
In a second aspect, a clothing attribute recognition device based on a neural network is provided, and the device comprises a recognition determination module and a recognition module;
the identification determining module is used for identifying and determining at least one target person in the image to be identified in a preset identification mode;
and the recognition module is used for carrying out clothing attribute recognition on the image to be recognized, which is recognized and determined by the recognition determination module and comprises at least one target character, through the pre-trained neural network recognition model to obtain a clothing attribute recognition result of the at least one target character.
Further, the identification module is used for performing body region segmentation on any target person in the image to be identified through the pre-trained neural network identification model, and performing clothing attribute identification on each body region to obtain a clothing attribute identification result of the target person in the image to be identified.
Further, the identification module comprises a first dividing unit and an identification unit;
the first segmentation unit is used for carrying out segmentation processing on the image to be identified to obtain at least one segmentation image comprising a single target figure;
and the identification unit is used for carrying out clothing attribute identification on any segmented image which is obtained by segmenting the first segmentation unit and comprises a single target character through a pre-trained neural network identification model to obtain clothing attribute identification results of the character in any segmented image.
Wherein, the clothing attribute recognition result of the person comprises at least one of the following items:
a type of apparel; clothing color; the number of clothes;
the apparel includes at least one of:
clothing, hats, shoes, accessories.
Further, the identification determination module comprises an extraction unit and an identification determination unit;
the image acquisition device comprises an extraction unit, a processing unit and a control unit, wherein the extraction unit is used for extracting at least one image frame from a video acquired by the image acquisition device according to a preset extraction frequency, and the preset extraction frequency is determined according to the counted average duration of a pedestrian passing through a control area of the image acquisition device;
and the recognition determining unit is used for detecting and recognizing the at least one image frame extracted by the extracting unit through a pre-trained portrait detection and recognition model, and recognizing and determining at least one image to be recognized comprising at least one target person.
Further, the first segmentation unit is used for performing segmentation processing on the image to be recognized based on a region segmentation method to obtain at least one segmentation image comprising a single target character;
and/or the method is used for performing segmentation processing on the image to be identified based on an edge segmentation method to obtain at least one segmentation image comprising a single target person.
Further, the identification unit comprises a feature extraction subunit and an input subunit;
the feature extraction subunit is used for carrying out clothing feature extraction on any segmented image comprising a single target figure to obtain clothing feature information aiming at any target figure;
and the input subunit is used for inputting the clothing feature information, which is extracted by the feature extraction subunit and aims at any target character, into the pre-trained neural network recognition model to obtain the clothing attribute recognition result of any character.
Furthermore, the identification module also comprises a second segmentation unit, a feature extraction unit and a duplication elimination unit;
the second segmentation unit is used for carrying out segmentation processing on a first preset number of image frames which are in front and a second preset number of image frames which are behind on a video time axis and are relative to the current image to be recognized according to the current image to be recognized to obtain a plurality of segmented images comprising a single target figure;
the character extraction unit is used for extracting character characteristics of a plurality of segmentation images including a single target character obtained by segmentation processing of the second segmentation unit to obtain characteristic information aiming at each character;
the duplication removing unit is used for carrying out similarity calculation on the feature information extracted by the feature extracting unit aiming at each figure and carrying out duplication removal on a plurality of split images comprising a single target figure according to the similarity calculation result to obtain duplicated split images;
and the recognition unit is used for carrying out clothing attribute recognition on the de-duplicated segmented image obtained after the de-duplication processing of the de-duplication unit through a pre-trained neural network recognition model to obtain clothing attribute recognition results of persons included in any de-duplicated segmented image.
Further, the device also comprises a storage module;
the storage module is used for storing the clothing attribute identification result, the segmentation image and the corresponding relation between the clothing attribute identification result and the segmentation image;
the apparatus also includes a query determination module;
and the query determining module is used for querying and determining the image information of the person corresponding to the query request through the storage module based on the corresponding relation between the clothing attribute identification result and the segmentation image when the person query request comprising the clothing attribute information is received.
In a third aspect, an electronic device is provided, which includes:
one or more processors;
a memory;
one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to: the neural network-based apparel attribute identification method shown in the first aspect is performed.
In a fourth aspect, a computer-readable storage medium is provided, which is used for storing computer instructions, which when run on a computer, make the computer perform the neural network-based clothing attribute identification method shown in the first aspect.
Compared with the prior art that the target person or the clothing attribute of the target group is identified in a manual mode, the clothing attribute identification method, the clothing attribute identification device and the electronic equipment have the advantages that the preset identification mode is adopted, the target person is identified and determined to be included in the image to be identified, then the clothing attribute identification is carried out on the image to be identified including the target person through the pre-trained neural network identification model, the clothing attribute identification result of the target person is obtained, namely, the clothing attribute identification method, the clothing attribute identification device and the electronic equipment achieve automatic identification of the clothing attribute of the target person included in the image to be identified through the pre-trained neural network identification model, accordingly, the clothing attribute identification efficiency is improved, the problems that manual identification is prone to errors and low in efficiency are solved, and meanwhile the labor cost is reduced.
Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.
Detailed Description
Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
The embodiment of the application provides a clothing attribute identification method based on a neural network, and as shown in fig. 1, the method can comprise the following steps:
s101, identifying and determining that an image to be identified comprises at least one target person in a preset identification mode;
for the embodiment of the application, a corresponding identification mode is preset, at least one target person is determined to be included in the image to be identified, and the image not including the target person is removed.
Step S102, clothing attribute recognition is carried out on the image to be recognized including at least one target character through a pre-trained neural network recognition model, and clothing attribute recognition results of the at least one target character are obtained.
For the embodiment of the application, clothing attribute recognition is carried out on the image to be recognized through the pre-trained neural network recognition model, and clothing attribute recognition results of target characters in the image to be recognized including at least one target character are obtained. The pre-trained Neural Network recognition model may be a Neural Network recognition model based on fast-RCNN (convolutional Neural Network with regions), and the architecture thereof may adopt a Neural Network model using VGG16, ResNet, google net, which is not limited herein, or based on RCNN (region based cnn) or ssd (single Shot multi detector) or YOLO.
Illustratively, the training sample of the neural network recognition model based on the fast-RCNN may include a plurality of images acquired from a video or an image acquired from a camera device and including at least one target person and labeled clothing attributes, the neural network training is performed by using the labeled image sample, which is beneficial to improving the accuracy of the neural network in recognizing image data, in the training process, the training result is compared with the manually labeled information, when the comparison result meets the predetermined accuracy requirement, the training can be considered to be completed, and when the comparison result does not meet the accuracy requirement, the training can be continued by adjusting corresponding parameters (such as parameters in each convolutional neural network layer) until the training result meets the predetermined accuracy requirement; furthermore, the neural network recognition model based on the fast-RCNN can be obtained by performing fine-tuning on the existing model.
Compared with the prior art that the target person or the clothing attribute of the target group is identified in a manual mode, the clothing attribute identification method based on the neural network identifies and determines that the image to be identified comprises at least one target person through a preset identification mode, then clothing attribute identification is carried out on the image to be identified comprising at least one target person through a pre-trained neural network identification model, and the clothing attribute identification result of at least one target person is obtained.
In one possible implementation manner, the step S102 includes,
step 1021 (not shown in the figure), performing body region segmentation on any target person in the image to be recognized through the pre-trained neural network recognition model, and performing clothing attribute recognition on each body region to obtain a clothing attribute recognition result of the target person in the image to be recognized.
In the embodiment of the application, the figure in the image to be recognized is divided into the body regions such as the upper half and the lower half through the pre-trained neural network recognition model, and the clothing attribute of each divided body region is recognized, such as the target figure is recognized to wear red clothes on the upper half and black trousers on the lower half, so that the clothing attribute recognition result of the figure in the segmented image is obtained. The body region can be divided by extracting Gabor features of M directions and N scales of the features of each body region to obtain a body region feature vector, and then the clothing attribute identification result of the person included in the segmented image is identified and determined according to the obtained feature vector.
Illustratively, when the clothing attribute recognition is performed on an image to be recognized including at least one target person, a plurality of detection frames are generated for the image to be processed by using a way of RPN (Region suggested Network layer) in the Fast-RCNN, and the like, the Fast-RCNN detector Network layer in the Fast-RCNN can extract appearance feature information of each detection frame generated by the RPN layer, the Fast-RCNN detector Network layer in the Fast-RCNN judges and processes the appearance feature information of each detection frame in the image to be processed to determine the probability that each detection frame belongs to each class, that is, after extracting the appearance feature information of each detection frame generated by the RPN layer, the Fast-RCNN detector Network layer can respectively predict the appearance feature information of each detection frame extracted by the Fast-RCNN detector Network layer, thus, an N-dimensional vector is formed for each detection frame and output, and one N-dimensional vector is the probability that one detection frame predicted by the N-dimensional vector belongs to N classes respectively, wherein N is the total number of the classes.
According to the embodiment of the application, the figure in the image to be recognized is divided into the body type areas through the pre-trained neural network model, and the clothing attribute of each body type area is recognized and determined, so that the problem of recognizing the clothing attribute (such as the color of clothing worn by the upper half and the lower half) of different body type areas of the figure is solved.
In one possible implementation manner, the step S102 includes,
step 1022 (not shown in the figure), performing segmentation processing on the image to be identified to obtain at least one segmented image including a single target character;
for the embodiment of the application, the image to be recognized is subjected to segmentation processing, so that one or more segmentation images only containing a single target person are obtained, in addition, image parts which are useless for recognizing the target clothing attribute, such as images of blank background parts, can be removed, the processing data amount is reduced, and the image can be subjected to normalized segmentation.
And 1023 (not shown in the figure), clothing attribute recognition is carried out on any segmented image comprising a single target character through a pre-trained neural network recognition model, so that a clothing attribute recognition result of the character in any segmented image is obtained.
For the embodiment of the application, any segmentation image comprising a single target figure is used as input and is input into the pre-trained neural network recognition model, and the pre-trained neural network recognition model outputs the clothing attribute recognition result aiming at the target figure in the input segmentation image. The pre-trained neural network recognition model is obtained by pre-training a training sample, wherein the training sample comprises a plurality of images containing target characters and clothing attributes marked by the target characters in the images, such as clothing colors.
For the embodiment of the application, at least one segmentation image comprising a single target is obtained by segmenting the image to be recognized, and then the clothing attribute recognition result of the target person in the image to be recognized is obtained by recognition of the pre-trained neural network recognition model, so that the clothing attribute of the target person in the image to be recognized is automatically recognized, and the clothing attribute recognition efficiency is improved.
Wherein, the clothing attribute recognition result of the person comprises at least one of the following items:
a type of apparel; clothing color; the number of clothes;
the apparel includes at least one of:
clothing, hats, shoes, accessories.
For the present application embodiment, the apparel attribute identification result includes, but is not limited to, the style of the apparel, the color of the apparel, and the number of the apparel, wherein the apparel includes, but is not limited to, clothing, hats, shoes, and accessories.
For the embodiment of the application, the corresponding clothing attribute information of the target person can be obtained based on different purposes or application scenes.
The embodiment of the present application provides a possible implementation manner, wherein step 101 includes,
step S1011 (not shown), extracting at least one image frame from the video acquired by the image acquisition device according to a preset extraction frequency, where the preset extraction frequency is determined according to the counted average duration of the pedestrian passing through the area controlled by the image acquisition device;
for the embodiment of the present application, any image capturing device has an effective monitoring area, an extraction frequency may be set according to the average time length of the pedestrian entering the effective monitoring area and leaving the effective monitoring area, and an image frame may be extracted from the video captured by the capturing device according to the extraction frequency, for example, the average time length of the pedestrian entering the capturing control range and leaving the capturing control range is 3 seconds, the captured video is 24 frames per second, and one frame of image may be extracted at intervals not greater than 72 frames.
Step S1012 (not shown in the figure), performing detection and recognition on at least one image frame through a pre-trained human image detection and recognition model, and recognizing and determining at least one image to be recognized including at least one target person.
For the embodiment of the application, a portrait detection recognition model may be obtained by training a plurality of positive and negative training samples including a target person and a target person, and then portrait detection recognition is performed on at least one image frame through the pre-trained portrait detection recognition model to obtain at least one to-be-recognized image including at least one target person, where the portrait detection recognition model may also be a portrait detection recognition model based on a background modeling algorithm, and a commonly used background modeling algorithm includes: gaussian Mixture model (Gaussian model), frame difference algorithm (background), gradient direction histogram (HoG), and the like.
For the embodiment of the application, at least one image frame is extracted from the acquired video according to the preset extraction frequency, and then at least one image to be recognized including at least one target person is determined from the at least one image frame through the pre-trained portrait detection recognition model, so that the problem of obtaining the image to be recognized including the at least one target person is solved, and a basis is provided for subsequent clothing attribute recognition of the target person.
The embodiment of the present application provides a possible implementation manner, wherein step S1022 may include but is not limited to at least one of step S10221 (not shown in the figure), step S10222 (not shown in the figure),
step S10221, performing segmentation processing on an image to be recognized based on a region segmentation method to obtain at least one segmentation image comprising a single target person;
step S10222, performing segmentation processing on the image to be recognized based on an edge segmentation method to obtain at least one segmentation image including a single target person.
For the embodiment of the application, the image to be recognized can be segmented by a region segmentation method and/or an edge segmentation method to obtain at least one segmented image comprising a single target person. The edge detection, that is, detecting where the gray level or structure has a sudden change, indicates the ending of one region, and is also where another region starts, such discontinuity is called an edge, the gray levels of different images are different, and the boundary generally has a distinct edge, and the image can be segmented by using this feature. The region segmentation method comprises two types of region production and region splitting and merging, and the basic idea of region growing is to assemble pixels with similar properties to form a region; the region splitting and merging is almost the reverse process of region growing, each sub-region is obtained by continuously splitting starting from the whole image, and then the foreground regions are merged to realize target extraction.
For the embodiment of the application, the segmentation problem of the image to be identified is solved through a region segmentation method and/or an edge segmentation method, and the segmentation of the image to be identified containing a plurality of target characters into the segmented image containing only a single target character is realized.
This embodiment of the present application provides a possible implementation manner, wherein step S1023 includes,
step S10231 (not shown in the figure), clothing feature extraction is carried out on any segmented image comprising a single target character, and clothing feature information for any target character is obtained;
step S10232 (not shown in the figure), the clothing feature information for any target person is input to the pre-trained neural network recognition model, and a clothing attribute recognition result of any person is obtained.
For the embodiment of the application, the clothing features of the target characters in any segmented image can be extracted through the feature extraction model, then the clothing features aiming at any target character are input into the pre-trained neural network recognition model, and the clothing attribute recognition result of any target character is recognized and determined.
For the embodiment of the application, the clothing features of the target person in the extracted segmentation image are input to the pre-trained neural network recognition model for clothing attribute recognition, so that the clothing attribute recognition problem of the target person is solved, and in addition, the data processing amount of the pre-trained neural network recognition model is reduced.
In another possible implementation manner, the embodiment of the present application further provides that step S102 further includes,
step S1024 (not shown in the figure), for the current image to be recognized, performing segmentation processing on a first predetermined number of preceding image frames and a second predetermined number of following image frames on the video time axis relative to the current image to be recognized to obtain a plurality of segmented images including a single target person;
for the embodiment of the application, a first predetermined number of quantity values and a second predetermined number of quantity values are preset, wherein the first predetermined number of quantity values and the second predetermined number of quantity values may be determined based on a predetermined monitoring time length, and segmentation is performed on a first predetermined number of image frames before and a second predetermined number of image frames after the determined current image to be recognized, wherein the segmentation may be implemented based on an image segmentation method such as a region segmentation method, an edge segmentation method, and the like, so as to obtain a plurality of segmented images including a single target person.
Step S1025 (not shown in the figure) of extracting character features of a plurality of segmented images including a single target character to obtain feature information for each character;
for the embodiment of the application, the character features of the target characters in the multiple segmentation images comprising the single target character are extracted through the feature extraction model, so that the feature information of each character is obtained, wherein the feature information can be represented through the feature vectors.
Step S1026 (not shown in the figure), which is to perform similarity calculation on the feature information of each character, and perform deduplication on a plurality of segmented images including a single target character according to the similarity calculation result, to obtain a deduplicated segmented image;
for example, the similarity between the character feature information is determined by calculating the euclidean distance or the cosine similarity between the character feature information, the same character with the similarity reaching a certain threshold is determined according to the similarity calculation result, and the cut image corresponding to the character determined as the same character is subjected to corresponding duplication removal to obtain the duplicated cut image.
Wherein, the step S1023 specifically comprises,
and 10233 (not shown in the figure), clothing attribute recognition is carried out on the cut images after the duplication removal through a pre-trained neural network recognition model, and clothing attribute recognition results of people included in any cut images after the duplication removal are obtained.
For the embodiment of the application, the cut images after the duplication removal are input to a pre-trained neural network recognition model, and clothing attribute recognition results of people included in any cut image after the duplication removal are obtained.
For the embodiment of the application, the duplication removal is carried out on the plurality of the segmented images comprising the single target character, and the clothing attribute identification of the target character is carried out on the duplicated segmented images, so that the repeated identification is avoided, the accuracy of subsequent clothing attribute information statistics can be improved, and in addition, the data processing amount of the pre-trained neural network identification model is reduced.
The embodiment of the present application also provides another possible implementation manner, and the method further includes,
step S103 (not shown in the figure), storing the clothing attribute recognition result, the image to be recognized and the corresponding relation between the clothing attribute recognition result and the image to be recognized;
for the embodiment of the application, the clothing attribute identification result, the direction to be identified and the corresponding relation between the clothing attribute identification result and the direction to be identified are stored through the corresponding storage device.
Wherein, the method also comprises:
step S104 (not shown in the figure), when a person query request including clothing attribute information is received, querying and determining image information of a person corresponding to the query request based on a correspondence between the clothing attribute identification result and the image to be identified.
For the embodiment of the application, when a person query request including clothing attribute information input by a user is received, the image information of a person corresponding to the query request is queried and determined through a corresponding storage device based on the index relationship between the clothing attribute identification result and the image to be identified.
According to the embodiment of the application, through the corresponding relation between the clothing attribute identification result and the image to be identified, the person image information corresponding to the clothing attribute information is inquired and determined when the inquiry request comprising the clothing attribute information is received.
Fig. 2 is a device for identifying clothing attribute based on neural network provided in an embodiment of the present application, where thedevice 20 includes: anidentification determination module 201 and anidentification module 202;
theidentification determining module 201 is configured to identify and determine that the image to be identified includes at least one target person in a preset identification manner;
therecognition module 202 performs clothing attribute recognition on the image to be recognized, which is recognized and determined by the recognition anddetermination module 201 and includes at least one target person, through a pre-trained neural network recognition model, so as to obtain a clothing attribute recognition result of the at least one target person.
The embodiment of the application provides a clothing attribute recognition device based on a neural network, compare with the clothing attribute of prior art through artifical mode identification target personage or target crowd, this application embodiment is through predetermined identification mode, the discernment confirms that including at least one target personage in the image of waiting to discern, then through the neural network recognition model of training in advance to waiting to discern the image including at least one target personage and carry out clothing attribute recognition, obtain the clothing attribute recognition result of at least one target personage, this application embodiment is through the neural network recognition model of training in advance promptly, the automatic identification of the clothing attribute of the target personage in the image of waiting to discern has been realized, thereby clothing attribute recognition's efficiency has been promoted, the problem that artifical recognition is easy to make mistakes has been avoided, the cost of labor has been reduced simultaneously.
The clothing attribute recognition device based on the neural network channel of the embodiment can execute the clothing attribute recognition method based on the neural network provided in the above embodiments of the present application, and the implementation principles are similar, and are not described herein again.
As shown in fig. 3, theapparatus 30 of this embodiment may include an identification determining module 301 and anidentification module 302, wherein,
the identification determining module 301 is configured to identify and determine that the image to be identified includes at least one target person in a preset identification manner;
here, the identification determination module 301 in fig. 3 has the same or similar function as theidentification determination module 201 in fig. 2.
Therecognition module 302 performs clothing attribute recognition on the image to be recognized, which is recognized and determined by the recognition and determination module 301 and includes at least one target person, through a pre-trained neural network recognition model, so as to obtain a clothing attribute recognition result of the at least one target person.
Wherein theidentification module 302 of fig. 3 has the same or similar function as theidentification module 202 of fig. 2.
Specifically, the identifyingmodule 302 is configured to perform body region segmentation on any target person in the image to be identified through a pre-trained neural network identification model, and perform clothing attribute identification on each body region to obtain a clothing attribute identification result of the target person in the image to be identified.
According to the embodiment of the application, the figure in the image to be recognized is divided into the body type areas through the pre-trained neural network model, and the clothing attribute of each body type area is recognized and determined, so that the problem of recognizing the clothing attribute (such as the color of clothing worn by the upper half and the lower half) of different body type areas of the figure is solved.
Specifically, theidentification module 302 includes afirst scoring unit 3021, anidentification unit 3022;
thefirst segmentation unit 3021 is configured to perform segmentation processing on an image to be identified to obtain at least one segmentation image including a single target person;
the identifyingunit 3022 is configured to perform clothing attribute identification on any segmented image including a single target character obtained through segmentation processing by thefirst segmenting unit 3021 through a pre-trained neural network identification model, so as to obtain a clothing attribute identification result of the character included in any segmented image.
For the embodiment of the application, at least one segmentation image comprising a single target is obtained by segmenting the image to be recognized, and then the clothing attribute recognition result of the target person in the image to be recognized is obtained by recognition of the pre-trained neural network recognition model, so that the clothing attribute of the target person in the image to be recognized is automatically recognized, and the clothing attribute recognition efficiency is improved.
Wherein, the clothing attribute recognition result of the person comprises at least one of the following items:
a type of apparel; clothing color; the number of clothes;
the apparel includes at least one of:
clothing, hats, shoes, accessories.
For the embodiment of the application, the corresponding clothing attribute information of the target person can be obtained based on different purposes or application scenes.
Specifically, the recognition determination module 301 includes an extraction unit 3011, a recognition determination unit 3012;
an extracting unit 3011, configured to extract at least one image frame from a video captured by an image capturing device according to a preset extracting frequency, where the preset extracting frequency is determined according to a counted average duration of a pedestrian passing through a region controlled by the image capturing device;
and the recognition determining unit 3012 is configured to perform detection and recognition on the at least one image frame extracted by the extracting unit 3012 through a pre-trained human image detection and recognition model, and recognize and determine at least one to-be-recognized image including at least one target person.
For the embodiment of the application, at least one image frame is extracted from the acquired video according to the preset extraction frequency, and then at least one image to be recognized including at least one target person is determined from the at least one image frame through the pre-trained portrait detection recognition model, so that the problem of obtaining the image to be recognized including the at least one target person is solved, and a basis is provided for subsequent clothing attribute recognition of the target person.
Specifically, thefirst segmentation unit 3021 is configured to perform segmentation processing on an image to be recognized based on a region segmentation method to obtain at least one segmentation image including a single target person;
and/or the method is used for performing segmentation processing on the image to be identified based on an edge segmentation method to obtain at least one segmentation image comprising a single target person.
For the embodiment of the application, the segmentation problem of the image to be identified is solved through a region segmentation method and/or an edge segmentation method, and the segmentation of the image to be identified containing a plurality of target characters into the segmented image containing only a single target character is realized.
Further, therecognition unit 3022 includes a feature extraction subunit (not shown in the figure) and an input subunit (not shown in the figure);
the feature extraction subunit is used for carrying out clothing feature extraction on any segmented image comprising a single target figure to obtain clothing feature information aiming at any target figure;
and the input subunit is used for inputting the clothing feature information, which is extracted by the feature extraction subunit and aims at any target character, into the pre-trained neural network recognition model to obtain the clothing attribute recognition result of any character.
For the embodiment of the application, the clothing features of the target person in the extracted segmentation image are input to the pre-trained neural network recognition model for clothing attribute recognition, so that the clothing attribute recognition problem of the target person is solved, and in addition, the data processing amount of the pre-trained neural network recognition model is reduced.
Further, theidentification module 302 further includes asecond segmentation unit 3023, afeature extraction unit 3024, and adeduplication unit 3025;
asecond segmentation unit 3023, configured to perform segmentation processing on a first predetermined number of preceding image frames and a second predetermined number of following image frames on a video time axis, which are relative to a current image to be recognized, for the current image to be recognized, to obtain a plurality of segmented images including a single target person;
afeature extraction unit 3024, configured to perform person feature extraction on a plurality of segmented images including a single target person obtained by the segmentation processing by thesecond segmentation unit 3023 to obtain feature information for each person;
aduplication removing unit 3025 configured to perform similarity calculation on the feature information for each person extracted by thefeature extracting unit 3024, and perform duplication removal on a plurality of split images including a single target person according to a result of the similarity calculation to obtain a duplication removed split image;
the identifyingunit 3022 is configured to perform clothing attribute identification on the deduplicated segmented image obtained after the deduplication processing by thededuplication unit 3025 through a pre-trained neural network identification model, so as to obtain a clothing attribute identification result of a person included in any deduplicated segmented image.
For the embodiment of the application, the duplication removal is carried out on the plurality of the segmented images comprising the single target character, and the clothing attribute identification of the target character is carried out on the duplicated segmented images, so that the repeated identification is avoided, the accuracy of subsequent clothing attribute information statistics can be improved, and in addition, the data processing amount of the pre-trained neural network identification model is reduced.
Specifically, the device further comprises astorage module 303 and aquery module 304;
thestorage module 303 is configured to store the clothing attribute identification result, the image to be identified, and the correspondence between the clothing attribute identification result and the image to be identified;
thequery determining module 304 is configured to, when a person query request including clothing attribute information is received, query and determine, through the storage module, image information of a person corresponding to the query request based on a correspondence between the clothing attribute identification result and the segmented image.
According to the embodiment of the application, through the corresponding relation between the clothing attribute identification result and the image to be identified, the person image information corresponding to the clothing attribute information is inquired and determined when the inquiry request comprising the clothing attribute information is received.
The embodiment of the application provides a clothing attribute recognition device based on a neural network, compare with the clothing attribute of prior art through artifical mode identification target personage or target crowd, this application embodiment is through predetermined identification mode, the discernment confirms that including at least one target personage in the image of waiting to discern, then through the neural network recognition model of training in advance to waiting to discern the image including at least one target personage and carry out clothing attribute recognition, obtain the clothing attribute recognition result of at least one target personage, this application embodiment is through the neural network recognition model of training in advance promptly, the automatic identification of the clothing attribute of the target personage in the image of waiting to discern has been realized, thereby clothing attribute recognition's efficiency has been promoted, the problem that artifical recognition is easy to make mistakes has been avoided, the cost of labor has been reduced simultaneously.
The embodiment of the application provides a clothing attribute identification device based on a neural network, which is suitable for the method shown in the embodiment and is not described herein again.
An embodiment of the present application provides an electronic device, as shown in fig. 4, anelectronic device 40 shown in fig. 4 includes: aprocessor 4001 and amemory 4003.Processor 4001 is coupled tomemory 4003, such as viabus 4002. Further, theelectronic device 40 may also include atransceiver 4004. In addition, thetransceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 400 is not limited to the embodiment of the present application.
Theprocessor 4001 is applied in the embodiment of the present application, and is configured to implement the functions of the identification determining module and the identification module shown in fig. 2 or fig. 3, and to implement the functions of thestorage module 303 and thequery determining module 304 shown in fig. 3. Thetransceiver 4004 includes a receiver and a transmitter.
Processor 4001 may be a CPU, general purpose processor, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. Theprocessor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.
Bus 4002 may include a path that carries information between the aforementioned components.Bus 4002 may be a PCI bus, EISA bus, or the like. Thebus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 4, but this does not indicate only one bus or one type of bus.
Memory 4003 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
Thememory 4003 is used for storing application codes for executing the scheme of the present application, and the execution is controlled by theprocessor 4001.Processor 4001 is configured to execute application code stored inmemory 4003 to implement the actions of the neural network-based apparel attribute identification apparatus provided by the embodiments shown in fig. 2 or fig. 3.
The embodiment of the application provides an electronic device suitable for the method embodiment. And will not be described in detail herein.
The embodiment of the application provides an electronic device, compare with prior art through the clothing attribute of artifical mode identification target personage or target crowd, this application embodiment is through predetermined identification mode, the discernment confirms to include at least one target personage in the image of waiting to discern, then carry out clothing attribute discernment to the image of waiting to discern including at least one target personage through the neural network recognition model of training in advance, obtain the clothing attribute recognition result of at least one target personage, this application embodiment is through the neural network recognition model of training in advance, the automatic identification of the clothing attribute of the target personage that has realized waiting to discern in the image, thereby clothing attribute recognition's efficiency has been promoted, the problem of easily makeing mistakes of manual identification has been avoided, the cost of labor has been reduced simultaneously.
The present application provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method shown in the above embodiments is implemented.
The embodiment of the application provides a computer-readable storage medium, compared with the prior art that the target person or the clothing attribute of the target group is identified in a manual mode, the embodiment of the application identifies and determines that the image to be identified comprises at least one target person through a preset identification mode, then clothing attribute identification is carried out on the image to be identified comprising at least one target person through a pre-trained neural network identification model, and the clothing attribute identification result of at least one target person is obtained.
The embodiment of the application provides a computer-readable storage medium which is suitable for the method embodiment. And will not be described in detail herein.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.