CN111144167A - Gait information identification optimization method, system and storage medium - Google Patents
Gait information identification optimization method, system and storage mediumDownload PDFInfo
- Publication number
- CN111144167A CN111144167ACN201811302325.XACN201811302325ACN111144167ACN 111144167 ACN111144167 ACN 111144167ACN 201811302325 ACN201811302325 ACN 201811302325ACN 111144167 ACN111144167 ACN 111144167A
- Authority
- CN
- China
- Prior art keywords
- gait
- identification
- energy
- recognition
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
- G06V40/25—Recognition of walking or running movements, e.g. gait recognition
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
Description
Technical Field
The invention relates to the technical field of biological identification, in particular to a gait information identification optimization method, a gait information identification optimization system and a storage medium.
Background
With the need for security level improvement in public places in various countries and the wide spread of video monitoring technologies, intelligent monitoring becomes a very active field in computer vision. In intelligent monitoring, the remote identification of human identity in a monitoring scene is a direction which is full of challenges and has a good application prospect, so that the system has scientific research and commercial values, and has theoretical and practical significance for deep research on the system.
Currently existing gait recognition algorithms can be classified into two broad categories, namely model-based methods and non-model-based methods. Model-based methods typically rely on expensive hardware sensors to extract gait information, and are therefore difficult to apply in conventional security monitoring scenarios. In recent years, non-model based approaches have shown superior performance. The non-model-based method is to directly analyze the human gait video image sequence without presupposing any specific model, and the main methods include a hidden Markov model, Radon transformation, a dynamic and static silhouette contour template and a gait energy image identification method.
However, the conventional gait energy diagram needs to add silhouette images and then average the silhouette images, so that a lot of gait time sequence information is lost; on the other hand, because each frame is extracted by the binary silhouette image, the image can only capture the boundary information of the human body outline, and the internal information of the silhouette is completely discarded, the existing gait energy diagram is difficult to completely reflect the walking characteristics of the human body, thereby influencing the accuracy of identifying the human body identity based on the gait energy diagram.
Disclosure of Invention
In order to solve the problems in the prior art, at least one embodiment of the present invention provides a gait information recognition optimization method, system and storage medium.
In a first aspect, an embodiment of the present invention provides a gait information identification and optimization method, where the gait information identification and optimization method includes:
s1, acquiring first gait preprocessing data corresponding to the video image;
s2, acquiring second gait preprocessing data of the video image by using the generating model;
s3, inputting the second gait preprocessing data and the first gait preprocessing data into a recognition model for identity recognition and data recognition;
s4, judging whether the second gait preprocessing data, the identity recognition result and the data recognition result all meet preset conditions;
if yes, outputting the generation model and the recognition model; and if not, updating the preset parameters of the generated model and the recognition model, and carrying out S2-S4.
Based on the above technical solutions, the embodiments of the present invention may be further improved as follows.
With reference to the first aspect, in a first embodiment of the first aspect, the inputting the second gait preprocessing data and the first gait preprocessing data into a recognition model for identity recognition and data recognition specifically includes:
generating a second step energy map according to the second gait preprocessing data, and generating a first step energy map according to the first gait preprocessing data;
mixing the second step state energy diagram and the first step state energy diagram to obtain a plurality of identification gait energy diagrams of different action postures, and inputting the identification gait energy diagrams into the identification model for identity identification and data identification;
respectively obtaining the identity recognition probability distribution of each recognition gait energy image as the identity recognition result;
and determining whether the identification of each identification gait energy image belonging to the second step energy image or the first step energy image by the identification model is accurate or not as the data identification result.
With reference to the first embodiment of the first aspect, in a second embodiment of the first aspect, the second step state energy diagram and the first step state energy diagram are expressed as a ratio of 1: 1 to obtain a plurality of gait energy recognition graphs with different action postures.
With reference to the first embodiment of the first aspect, in a third embodiment of the first aspect, the determining whether the second gait preprocessing data, the identity recognition result, and the data recognition result all satisfy preset conditions specifically includes:
calculating the identification loss according to the identification result of each identification gait energy graph;
judging whether the identity recognition loss meets a first preset threshold range or not;
calculating data true and false discrimination loss according to the data identification result of each identification gait energy graph;
judging whether the data true and false judging loss meets a second preset threshold range or not;
calculating the mean square error loss of the second step state energy diagram and the first step state energy diagram;
and judging whether the mean square error loss meets a third preset threshold range.
With reference to the third embodiment of the first aspect, in a fourth embodiment of the first aspect, the calculating an identification loss according to the identification result of each identification gait energy map specifically includes:
calculating the identification loss by the following calculation mode:
lIfor the identification loss, n represents the number of the identification gait energy profiles,
for the identification loss of the identification gait energy diagram of the ith, c represents the number of comparison identities, exp is an exponential function with a natural constant as the base, xi[j]The identity recognition probability, x, for the jth comparison identity is the identity recognition of the gait energy mapi[yi]Identifying as the y-th for the identity of the identified gait energy mapiIdentity recognition probability of individual contrasted identities, yiNumbering the real identity corresponding to the video image;
the calculating data true and false discrimination loss according to the data identification result of each identification gait energy diagram specifically comprises:
and calculating the data true and false discrimination loss by the following calculation formula:
lD＝-(mn×logkn+(1-mn)×log(1-kn))；
lDfor judging loss of the data true and false, when the nth gait energy recognition graph is the second step state energy graph, m isnIs 1, when the nth identified gait energy map is the first step state energy map, m isnIs 0;
knfor the nth identified gait energy map, the confidence level of the second step energy map is as knWhen the gait energy is larger than the preset threshold value, the identification gait energy graph is a second step state energy graph, and when k is larger than the preset threshold valuenWhen the gait energy is smaller than or equal to a preset threshold value, the identification gait energy map is a first step state energy map;
calculating the mean square error loss of the second-step energy map and the first-step energy map, specifically comprising:
calculating the mean square error loss by:
lgfor the mean square error loss, t is the number of pixels of the second-step state energy diagram or the number of pixels of the first-step state energy diagram, pxiThe gray value, py, of the 0-mean normalization of the ith pixel point in the second-step energy diagramiAnd normalizing the gray value of the 0 mean value of the ith pixel point in the first step state energy diagram.
With reference to the first aspect or the first, second, third, or fourth embodiment of the first aspect, in a fifth embodiment of the first aspect, the obtaining a verification gait feature number of the video image by using a generative model specifically includes:
acquiring continuous frame images in a video image;
detecting an image area containing human figures from the continuous frame images by using a detection algorithm;
extracting a human-shaped outline in the human-shaped image area to generate a human-shaped outline segmentation graph;
aligning the human shape center and the image center of each frame of the human shape contour segmentation image to generate a primary gait silhouette image;
scaling the human-shaped contour in each frame of primary gait silhouette image to a uniform resolution ratio to generate a gait silhouette image sequence;
respectively acquiring silhouette gait features of each frame of gait silhouette image in the gait silhouette image sequence to form silhouette gait preprocessing data;
calculating the correlation similarity of each frame of silhouette gait features in the silhouette gait preprocessing data;
and performing weighted fusion on the silhouette gait features according to the correlation similarity to obtain the second-step preprocessing data.
With reference to the fifth embodiment of the first aspect, in a sixth embodiment of the first aspect, the performing weighted fusion on the gait features according to the correlation similarity to obtain the second-step preprocessing data specifically includes:
calculating the equal error rate of the corresponding silhouette gait features used in the feature recognition process according to each correlation similarity;
respectively calculating the weight value of the corresponding silhouette gait feature according to each equal error rate;
and performing weighted fusion on all the silhouette gait features according to the corresponding weight values to obtain the second-step preprocessing data.
With reference to the sixth embodiment of the first aspect, in a seventh embodiment of the first aspect, the calculating the weight value of the corresponding silhouette gait feature according to each equal error rate specifically includes:
calculating the weight value according to the following calculation formula:
wherein, wnIs as followsn weight values of said silhouette gait features, enThe nth equal error rate of the silhouette gait features, and M is the total number of the silhouette gait features.
With reference to the fifth embodiment of the first aspect, in an eighth embodiment of the first aspect, the extracting a human-shaped contour in the human-shaped image region to generate a human-shaped contour segmentation map specifically includes:
acquiring a background frame of the frame image based on a method of an intermediate value;
detecting a moving target in a frame image by using a background difference method according to the background frame, and performing binary segmentation on the frame image;
processing the frame image after binary segmentation by using a corrosion operator and an expansion filtering operator, and filling holes in the frame image according to connectivity analysis;
and carrying out boundary tracking on the frame image to obtain a human-shaped outline, and generating a human-shaped outline segmentation graph.
In a second aspect, an embodiment of the present invention provides a gait information identification and optimization system, where the gait information identification and optimization system includes a processor and a memory; the processor is configured to execute the gait information identification optimization program stored in the memory to implement the gait information identification optimization method described in any one of the embodiments of the first aspect.
In a third aspect, an embodiment of the present invention provides a storage medium, where the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the gait information identification optimization method in any one of the first aspects.
Compared with the prior art, the technical scheme of the invention has the following advantages: the method comprises the steps of acquiring first gait preprocessing data of a video, generating second gait preprocessing data through a generating model, identifying the first gait preprocessing data and the second gait preprocessing data through an identification model, confirming whether parameters of the generating model and the identification model need to be adjusted according to an identification result, and carrying out generation and identification processes of a gait energy diagram again, and completing optimization of the generating model and the identification model when the identification result meets a preset condition.
Drawings
Fig. 1 is a schematic flow chart of a gait information identification optimization method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a gait information identification optimization method according to another embodiment of the invention;
fig. 3 is a schematic flow chart of a gait information identification optimization method according to another embodiment of the present invention;
fig. 4 is a schematic flow chart of a gait information identification optimization method according to another embodiment of the invention;
fig. 5 is a schematic structural diagram of a gait information recognition optimizing system according to another embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
As shown in fig. 1, a gait information identification and optimization method provided in an embodiment of the present invention includes:
and S1, acquiring first gait preprocessing data corresponding to the video image.
In this embodiment, the first gait preprocessing data may be obtained by user input, or obtained by processing a video image by using another trained generative model, or obtained by scanning according to the walking characteristics of a person in the video image; the method can also be used for acquiring first-step preprocessing data from a video image through a gait feature extraction technology in the prior art. The first gait preprocessing data refers to the posture and behavior characteristics of the human body when walking, and the human body moves along a certain direction through a series of continuous activities of the hip, the knee, the ankle and the toes. Gait involves factors such as behavioral habits, occupation, education, age and sex, and is also affected by various diseases. Control of walking is complex, including central commands, body balance and coordinated control, involving coordinated movements of the joints and muscles of the lower extremities, as well as associated with the posture of the upper extremities and the trunk. Misadjustment of any link may affect gait, and abnormalities may be compensated or masked. Normal gait has stability, periodicity and rhythmicity, directionality, coordination, and individual variability, however, these gait characteristics will change significantly when a person is ill. Gait analysis is an inspection method for studying walking rules, and aims to disclose key links and influencing factors of gait abnormalities through biomechanical and kinematic means so as to guide rehabilitation assessment and treatment and contribute to clinical diagnosis, curative effect assessment, mechanism research and the like. In gait analysis, whether gait is normal or not is often described by some special parameters, which generally include the following categories: gait cycle, kinematic parameters, kinetic parameters, electromyographic activity parameters, energy metabolism parameters and the like.
And S2, acquiring second gait preprocessing data of the video image by using the generating model.
In this embodiment, the generation model is used to obtain second gait preprocessing data of the video image, such as a gait silhouette image obtained according to a human body contour line, and verification gait data in the gait silhouette image obtained from each frame of image in the video image is respectively obtained.
As shown in fig. 2, the method for generating the second-step preprocessed data includes:
and S21, acquiring continuous frame images in the video images.
A single frame image is a still picture, and a frame is a single picture of the smallest unit in a motion picture, which is equivalent to each frame of a shot on a motion picture film. A single frame is a still picture, and consecutive frames form a moving picture, such as a television image. In this step, successive frame images in the video image are acquired.
S22, an image region including a human figure is detected from the continuous frame image by a detection algorithm.
For example, a moving object in an image may be detected by a background subtraction method, and a contour of the moving object may be detected to confirm a human-shaped image region, or another image detection method may be used to detect a continuous frame image to confirm a human-shaped image region in the continuous frame image.
And S23, extracting the human-shaped outline in the human-shaped image area to generate a human-shaped outline segmentation graph.
In this embodiment, the human-shaped contour in each human-shaped image region in each frame of image is extracted to obtain a continuous human-shaped contour segmentation map for the video image, the image is subjected to binary segmentation, the binary-segmented image is processed through an expansion filtering operator and a corrosion operator, noise in the image is filtered, a hole in the binary-segmented image is filled, and boundary tracking is performed on the processed image, so that the corresponding human-shaped contour can be obtained.
In this embodiment, a human-shaped contour in a human-shaped image region may also be extracted through a semantic analysis algorithm to generate a human-shaped contour segmentation map, specifically, each pixel point in the picture is subjected to secondary classification through a neural network, and then adjusted by some post-processing methods, such as edge smoothing, threshold filtering, and the like, to finally generate a segmentation image.
Specifically, the method of S23 specifically includes: the background frame of the frame image is obtained based on the intermediate value method. And detecting a moving target in the frame image by using a background difference method according to the background frame, and performing binary segmentation on the frame image. Processing the frame image after binary segmentation by using a corrosion operator and an expansion filtering operator, and filling holes in the frame image according to connectivity analysis; and carrying out boundary tracking on the frame image to obtain a human-shaped contour, and generating a human-shaped contour segmentation graph.
And S24, aligning the human figure center and the image center of each human figure outline segmentation image to generate a primary gait silhouette image.
And aligning the human figure center and the image center of the human figure contour segmentation graph to ensure that each human figure contour segmentation graph is adjusted according to the walking video image of the human figure contour segmentation graph to obtain a primary gait silhouette image.
And S25, scaling the human-shaped contour in each frame of primary gait silhouette image to a uniform resolution ratio to generate a gait silhouette image sequence.
And adjusting the resolution of the primary gait silhouette images to enable the resolution of each primary gait silhouette image to be consistent, so as to obtain a gait silhouette image sequence.
And S26, respectively acquiring silhouette gait features of each frame of gait silhouette image in the gait silhouette image sequence to form silhouette gait preprocessing data.
Acquiring silhouette gait features of each frame of gait silhouette image, wherein the gait features refer to features of the force magnitude, direction and action point of the person during walking reflected in the footprints. Is the reflection of the walking habit of the person in the steps of falling feet, rising feet and supporting swing. Generally comprising: the mark can be selected from the group consisting of a bump mark, a tread mark, a push mark, a traveling mark, a sitting mark, a pressing mark, an indentation mark, a twisting mark, a lifting mark, a kicking mark, a digging mark, a scratching mark, a picking mark, an slit mark, a scratch mark, a sweeping mark, a scratch mark and the like.
And S27, calculating the relevance similarity of the silhouette gait features of each frame of silhouette gait preprocessing data.
In this embodiment, the correlation similarity may be a distance measure of two gait features, for example, the euclidean distance of each silhouette gait feature is smaller, and the similarity of each silhouette gait feature is larger.
And S28, performing weighted fusion on the silhouette gait features according to the correlation similarity to obtain second-step preprocessing data.
As shown in fig. 3, the method for obtaining the corresponding second gait preprocessing data by fusing the gait features of each silhouette according to the correlation similarity includes:
and S31, calculating the equal error rate of the corresponding silhouette gait characteristics used for the characteristic identification process according to each correlation similarity.
The credibility of different gait features in the fusion process is inconsistent, and the accuracy of feature identification can be generally expressed by Equal Error Rate (EER). The higher the EER value is, the worse the performance of the feature is, the higher the error rate in the step is, the lower the reliability of the feature in the fusion process is, in the step, the greater the correlation similarity is, the more similar the silhouette gait feature and the real gait feature is, the lower the equal error rate corresponding to the gait feature with the greater correlation similarity is, in the embodiment, the correlation similarity is calculated by the Euclidean distance between the silhouette gait feature and the real gait feature, that is, the lower the Euclidean distance is, the higher the correlation similarity is, and the lower the corresponding equal error rate is.
And S32, respectively calculating the weight values of the corresponding silhouette gait characteristics according to each equal error rate.
The method for respectively calculating the weight values of the corresponding silhouette gait features according to the equal error rates of the different silhouette gait features comprises the following steps:
calculating the weight value according to the following calculation formula:
wherein, wnWeight value of nth silhouette gait feature, enThe equal error rate of the nth silhouette gait feature is shown, and M is the total number of the silhouette gait features.
And S33, performing weighted fusion on all silhouette gait characteristics according to corresponding weight values to obtain second-step preprocessing data.
And performing weighted fusion on all silhouette gait features according to corresponding weight values, so that the weighting proportion of the silhouette gait features with smaller weight values in the fusion process is smaller, the error of the final second gait preprocessing data is reduced, and the accuracy of the second step status preprocessing data is improved.
And S3, inputting the second-step-state preprocessed data and the first-step-state preprocessed data into a recognition model for identity recognition and data recognition.
And the second-step preprocessing data and the first-step preprocessing data are mixed and input into a recognition training mode to carry out identity recognition and data recognition, wherein the identity recognition is carried out by the second gait preprocessing data and the first-step preprocessing data, and the data recognition is carried out by recognizing the difference between the second gait preprocessing data and the first-step preprocessing data.
As shown in fig. 4, in this step, the method for performing identity recognition and data recognition specifically includes:
s41, generating a second-step energy map according to the second-step preprocessing data, and generating a first-step energy map according to the first-step preprocessing data;
and S42, mixing the second step state energy diagram and the first step state energy diagram to obtain a plurality of identification gait energy diagrams of different action postures, and inputting the identification gait energy diagrams into an identification model for identity identification and data identification.
In the step, the second step state energy diagram and the first step state energy diagram are respectively split to obtain a plurality of identification gait energy diagrams of different action postures in each gait energy diagram, and all the identification gait energy diagrams are input into an identification model to respectively carry out identity identification and data identification.
For example, the second step energy map and the first step energy map are expressed as a ratio of 1: 1 to obtain a plurality of gait energy recognition graphs with different action postures.
And S43, respectively obtaining the identification probability distribution of each identification gait energy graph as the identification result.
And obtaining the recognition probability of each recognition gait energy image corresponding to different people, wherein the person with the highest recognition probability is the recognition result of the recognition gait energy image, if the recognition gait energy image which is finally larger than the preset threshold value can accurately recognize the identity, the parameters of the recognition model are trained accurately, the model parameters of the recognition model can not be adjusted, otherwise, the parameters of the recognition model need to be adjusted.
And S44, confirming whether the identification of each identification gait energy image belongs to the second step energy image or the first step energy image by the identification model is accurate or not, and taking the identification result as a data identification result.
And identifying the attribution of each identified gait energy image, determining whether the identification model can accurately identify the second step state energy image or the first step state energy image in the gait energy image, if the identification model can be accurately distinguished, adjusting the parameters of the generated model to regenerate the second step state preprocessing data, otherwise, not adjusting the generated model quantity image.
And S4, judging whether the second gait preprocessing data, the identity recognition result and the data recognition result all meet preset conditions.
Specifically, the method for judging whether the second gait preprocessing data, the identity recognition result and the data recognition result meet the preset conditions includes:
and calculating the identification loss according to the identification result of each identification gait energy graph.
Based on the above embodiment, the identification is performed by identifying the gait energy maps, and the identification loss of all the identification gait energy maps is calculated according to the identification result, for example, the accuracy of identification can be calculated as the identification loss, and the greater the identification loss, the identification model needs to adjust the parameters to identify again, for example, when the accuracy of identification is greater than or equal to 75 percent, the identity recognition by the recognition model with the second gait preprocessing data can be confirmed, the identity of the person in the video can be accurately recognized, when the identity recognition accuracy is lower than 75 percent, the parameters of the recognition model and the generated model need to be adjusted, and the generation of the second-step preprocessing data, the data recognition and the identity recognition are carried out again, wherein the lower the accuracy rate of the identity recognition is, the larger the parameter adjustment amplitude of the recognition model and the generated model is.
For example, the identification loss is calculated by the following calculation method:
lIfor identification loss, n represents the number of identification gait energy patterns,
identifying the identity loss of the ith gait energy image, c represents the number of comparison identities, exp is an exponential function with a natural constant as the base, xi[j]Identification probability, x, for identifying the identity of the gait energy profile as the jth comparison identityi[yi]Identification as the y-th for identifying the gait energy profileiIdentity recognition probability of individual contrasted identities, yiThe number of the corresponding real identity of the video image is obtained.
And calculating the data true and false discrimination loss according to the data identification result of each gait energy identification graph.
Judging whether each recognition gait energy image is first-step preprocessing data or second gait preprocessing data, calculating true and false discrimination loss according to the recognition result, confirming whether the recognition model can accurately distinguish whether the recognition gait energy image is second-step preprocessing data or first gait preprocessing data or not through the true and false discrimination loss, and further confirming whether parameters of the generation model need to be adjusted to regenerate the second-step preprocessing data or not.
For example, the data true and false discrimination loss is calculated by the following calculation formula:
lD＝-(mn×logkn+(1-mn)×log(1-kn))；
lDfor judging loss of data true and false, when the nth gait energy recognition graph is the second step state preprocessing data, mnIs 1, when the nth recognition gait energy map is the first step state preprocessing data, mnIs 0;
knfor the nth identified gait energy map is the confidence of the second step state preprocessed data, when k isnWhen the gait energy is larger than the preset threshold value, recognizing the gait energy map as second-step state preprocessing data, and when k is larger than the preset threshold valuenAnd when the gait energy is less than or equal to the preset threshold, recognizing the gait energy map as first-step preprocessing data.
And calculating the mean square error loss of the second gait preprocessing data and the first step preprocessing data.
In this step, the mean square error loss of the second gait preprocessing data and the first step preprocessing data is calculated, the mean square error is the average of the sum of squared distances of all data from the true value, the deviation degree of the second step preprocessing data and the first step preprocessing data can be obtained by calculating the mean square error loss of the second gait preprocessing data and the first step preprocessing data, if the deviation degree of the second step preprocessing data and the first step preprocessing data is too large, the second step preprocessing data obtained by generating the model can be determined to be unqualified, and therefore if the mean square error loss is too large, the parameters of the generated model need to be adjusted, and new second gait preprocessing data are generated again.
For example, the mean square error loss is calculated by:
lgfor mean square error loss, t is the number of pixels in the second-step gait preprocessing data or the number of pixels in the first gait preprocessing data, pxiThe gray value, py, of the 0-mean normalization of the ith pixel point in the second-step preprocessing dataiAnd normalizing the gray value of the 0 mean value of the ith pixel point in the first-step preprocessing data.
If yes, outputting a generation model and an identification model; if not, updating the preset parameters of the generation model and the identification model, and carrying out S2-S4.
If the accuracy of the identification result is greater than the preset threshold value and the accuracy of the data identification result is less than the preset threshold value, it can be determined that the identification model cannot accurately identify the second-step preprocessing data and the first-step preprocessing data, and the identity of the person corresponding to the video can be accurately identified through the second-step preprocessing data, at this time, the parameters of the generation model and the identification model are trained, and corresponding parameters can be output to obtain the generation model and the identification model, otherwise, the preset parameters of the generation model and the identification model need to be updated, the parameters in the generation model and the identification model can be updated through a random gradient descent algorithm, or the parameters in the generation model and the identification model can be updated through a loss back propagation method, and S2-S4 is performed again.
In a specific embodiment, an embodiment of the present invention provides a gait information identification optimization method, which specifically includes:
s1: and sequentially inputting the gait silhouette image sequences (multi-frame images) p1, p2, … and pn into a basic feature extraction part for generating the network to respectively obtain feature sequences f1, f2, … and fn.
And S2, inputting the characteristics F1, F2, … and fn of the multi-frame input image into the encoder network part of the generation network, and carrying out characteristic weighting fusion to obtain the characteristic F of the whole gait silhouette image sequence.
S3, inputting the characteristics F of the whole gait silhouette image sequence into the decoding network part of the generation network to generate a frame of gait energy image GEI with the same resolution as the input silhouette imagef。
S4 GEIfAnd true GEI in the training data setrAccording to the following steps: the proportional mix of 1 is then input to the recognition network. Wherein the true GEIrIs a GEI generated by selecting those sums from a set of training datafData belonging to the same target and having a walking angle of 54 degrees and being close-fitting to the body.
S5: identifying GEI of network pair inputfAnd GEIrAnd identifying which data in the input data belong to the generated data and which data belong to the real data.
S6: calculating loss of identity recognition based on the result of identity recognition and the result of discrimination of true and false data_I and loss of discrimination between true and falseD. Based on total loss LD＝α×loss_I+β×loss_DAnd updating the parameters of the judgment network by using a random gradient descent algorithm.
S7, calculating GEI of each group of same identity objectsfAnd GEIrLoss of mean square error (loss)gBased on the total loss LG＝λ×lossg+μ×lossDAnd updating the parameters of the generated network by using a random gradient descent algorithm.
S8, repeating S1-S7 until reaching the model overall convergence condition, namely that the judgment network can hardly distinguish the GEI any morefAnd GEIrWhile correctly recognizing GEIfAnd GEIrThe identity of the corresponding target, so as to obtain the well-learned generative confrontation model M.
As shown in fig. 5, an embodiment of the present invention provides a gait information recognition optimization system, which includes a processor, a memory; the processor is configured to execute the gait information identification optimization program stored in the memory to implement the gait information identification optimization method according to any one of the above embodiments.
The storage medium for recording the program code of the software program that can realize the functions of the above-described embodiments is provided to the system or apparatus in the above-described embodiments, and the program code stored in the storage medium is read and executed by the computer (or CPU or MPU) of the system or apparatus.
In this case, the program code itself read out from the storage medium performs the functions of the above-described embodiments, and the storage medium storing the program code constitutes an embodiment of the present invention.
As a storage medium for supplying the program code, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, and the like can be used.
The functions of the above-described embodiments may be realized not only by executing the readout program code by the computer, but also by some or all of actual processing operations executed by an OS (operating system) running on the computer according to instructions of the program code.
Further, the embodiments of the present invention also include a case where after the program code read out from the storage medium is written into a function expansion card inserted into the computer or into a memory provided in a function expansion unit connected to the computer, a CPU or the like included in the function expansion card or the function expansion unit performs a part of or the whole of the processing in accordance with the command of the program code, thereby realizing the functions of the above-described embodiments.
An embodiment of the present invention provides a storage medium, where the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the gait information identification optimization method according to any one of the above embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (11)
1. A gait information identification optimization method is characterized by comprising the following steps:
s1, acquiring first gait preprocessing data corresponding to the video image;
s2, acquiring second gait preprocessing data of the video image by using the generating model;
s3, inputting the second gait preprocessing data and the first gait preprocessing data into a recognition model for identity recognition and data recognition;
s4, judging whether the second gait preprocessing data, the identity recognition result and the data recognition result all meet preset conditions;
if yes, outputting the generation model and the recognition model; and if not, updating the preset parameters of the generated model and the recognition model, and carrying out S2-S4.
2. The gait information recognition optimization method according to claim 1, wherein the inputting the second gait preprocessing data and the first gait preprocessing data into a recognition model for identification and data recognition specifically comprises:
generating a second step energy map according to the second gait preprocessing data, and generating a first step energy map according to the first gait preprocessing data;
mixing the second step state energy diagram and the first step state energy diagram to obtain a plurality of identification gait energy diagrams of different action postures, and inputting the identification gait energy diagrams into the identification model for identity identification and data identification;
respectively obtaining the identity recognition probability distribution of each recognition gait energy image as the identity recognition result;
and determining whether the identification of each identification gait energy image belonging to the second step energy image or the first step energy image by the identification model is accurate or not as the data identification result.
3. The gait information recognition optimization method according to claim 2, characterized in that the second step energy map and the first step energy map are calculated by a method in which 1: 1 to obtain a plurality of gait energy recognition graphs with different action postures.
4. The gait information recognition optimization method according to claim 2, wherein the step of determining whether the second gait preprocessing data, the identification result and the data recognition result all satisfy a preset condition specifically includes:
calculating the identification loss according to the identification result of each identification gait energy graph;
judging whether the identity recognition loss meets a first preset threshold range or not;
calculating data true and false discrimination loss according to the data identification result of each identification gait energy graph;
judging whether the data true and false judging loss meets a second preset threshold range or not;
calculating the mean square error loss of the second step state energy diagram and the first step state energy diagram;
and judging whether the mean square error loss meets a third preset threshold range.
5. The gait information recognition optimization method according to claim 4, wherein the calculating of the recognition loss from the recognition result of each recognition gait energy map specifically comprises:
calculating the identification loss by the following calculation mode:
lIfor the identification loss, n represents the number of the identification gait energy profiles,
for the identification loss of the identification gait energy diagram of the ith, c represents the number of comparison identities, exp is an exponential function with a natural constant as the base, xi[j]The identity recognition probability, x, for the jth comparison identity is the identity recognition of the gait energy mapi[yi]Identifying as the y-th for the identity of the identified gait energy mapiIdentity recognition probability of individual contrasted identities, yiNumbering the real identity corresponding to the video image;
the calculating data true and false discrimination loss according to the data identification result of each identification gait energy diagram specifically comprises:
and calculating the data true and false discrimination loss by the following calculation formula:
lD＝-(mn×logkn+(1-mn)×log(1-kn))；
lDfor judging loss of the data true and false, when the nth gait energy recognition graph is the second step state energy graph, m isnIs 1, when the nth identified gait energy map is the first step state energy map, m isnIs 0;
knfor the nth identified gait energy map, the confidence level of the second step energy map is as knWhen the gait energy is larger than the preset threshold value, the identification gait energy graph is a second step state energy graph, and when k is larger than the preset threshold valuenWhen the value is less than or equal to a preset threshold value, the identification is carried outThe gait energy map is a first step energy map;
calculating the mean square error loss of the second-step energy map and the first-step energy map, specifically comprising:
calculating the mean square error loss by:
lgfor the mean square error loss, t is the number of pixels of the second-step state energy diagram or the number of pixels of the first-step state energy diagram, pxiThe gray value, py, of the 0-mean normalization of the ith pixel point in the second-step energy diagramiAnd normalizing the gray value of the 0 mean value of the ith pixel point in the first step state energy diagram.
6. The gait information identification and optimization method according to any one of claims 1 to 5, wherein the obtaining of the second gait preprocessing data of the video image by using the generative model specifically comprises:
acquiring continuous frame images in a video image;
detecting an image area containing human figures from the continuous frame images by using a detection algorithm;
extracting a human-shaped outline in the human-shaped image area to generate a human-shaped outline segmentation graph;
aligning the human shape center and the image center of each frame of the human shape contour segmentation image to generate a primary gait silhouette image;
scaling the human-shaped contour in each frame of primary gait silhouette image to a uniform resolution ratio to generate a gait silhouette image sequence;
respectively acquiring silhouette gait features of each frame of gait silhouette image in the gait silhouette image sequence to form silhouette gait preprocessing data;
calculating the correlation similarity of each frame of silhouette gait features in the silhouette gait preprocessing data;
and performing weighted fusion on the silhouette gait features according to the correlation similarity to obtain the second-step preprocessing data.
7. The gait information identification and optimization method according to claim 6, wherein the weighting and fusion of the gait features according to the correlation similarity to obtain the second-step preprocessing data specifically comprises:
calculating the equal error rate of the corresponding silhouette gait features used in the feature recognition process according to each correlation similarity;
respectively calculating the weight value of the corresponding silhouette gait feature according to each equal error rate;
and performing weighted fusion on all the silhouette gait features according to the weight values to obtain the second-step preprocessing data.
8. The gait information recognition optimization method according to claim 7, wherein the calculating the weight value of the silhouette gait feature according to each of the equal error rates includes:
calculating the weight value according to the following calculation formula:
wherein, wnWeight value of the nth silhouette gait feature, enThe nth equal error rate of the silhouette gait features, and M is the total number of the silhouette gait features.
9. The gait information identification optimization method according to claim 6, wherein the extracting a human-shaped contour in a human-shaped image region to generate a human-shaped contour segmentation map specifically comprises:
acquiring a background frame of the frame image based on a method of an intermediate value;
detecting a moving target in a frame image by using a background difference method according to the background frame, and performing binary segmentation on the frame image;
processing the frame image after binary segmentation by using a corrosion operator and an expansion filtering operator, and filling holes in the frame image according to connectivity analysis;
and carrying out boundary tracking on the frame image to obtain a human-shaped outline, and generating a human-shaped outline segmentation graph.
10. A gait information identification and optimization system is characterized by comprising a processor and a memory; the processor is used for executing the gait information identification optimization program stored in the memory to realize the gait information identification optimization method of any one of claims 1-9.
11. A storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the gait information recognition optimization method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811302325.XACN111144167A (en) | 2018-11-02 | 2018-11-02 | Gait information identification optimization method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811302325.XACN111144167A (en) | 2018-11-02 | 2018-11-02 | Gait information identification optimization method, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111144167Atrue CN111144167A (en) | 2020-05-12 |
Family
ID=70516284
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811302325.XAPendingCN111144167A (en) | 2018-11-02 | 2018-11-02 | Gait information identification optimization method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111144167A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914762A (en)* | 2020-08-04 | 2020-11-10 | 浙江大华技术股份有限公司 | Gait information-based identity recognition method and device |
| CN112487903A (en)* | 2020-11-23 | 2021-03-12 | 中国信息通信科技集团有限公司 | Gait data generation method and device based on confrontation network |
| CN113869109A (en)* | 2021-08-20 | 2021-12-31 | 江汉大学 | Gait recognition method and device and related equipment |
| CN113887316A (en)* | 2021-09-06 | 2022-01-04 | 浙江大华技术股份有限公司 | Identity recognition method and device, electronic equipment and storage medium |
| CN113951607A (en)* | 2021-09-14 | 2022-01-21 | 天津科技大学 | Design method of diabetic foot prevention insole based on multi-step time |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1758263A (en)* | 2005-10-31 | 2006-04-12 | 浙江大学 | Multi-model ID recognition method based on scoring difference weight compromised |
| CN103839081A (en)* | 2014-02-25 | 2014-06-04 | 中国科学院自动化研究所 | Across-viewing-angle gait recognition method based on topology expression |
| CN104299012A (en)* | 2014-10-28 | 2015-01-21 | 中国科学院自动化研究所 | Gait recognition method based on deep learning |
| CN107085716A (en)* | 2017-05-24 | 2017-08-22 | 复旦大学 | Cross-view gait recognition method based on multi-task generative adversarial network |
| CN108009526A (en)* | 2017-12-25 | 2018-05-08 | 西北工业大学 | A kind of vehicle identification and detection method based on convolutional neural networks |
- 2018
- 2018-11-02CNCN201811302325.XApatent/CN111144167A/enactivePending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1758263A (en)* | 2005-10-31 | 2006-04-12 | 浙江大学 | Multi-model ID recognition method based on scoring difference weight compromised |
| CN103839081A (en)* | 2014-02-25 | 2014-06-04 | 中国科学院自动化研究所 | Across-viewing-angle gait recognition method based on topology expression |
| CN104299012A (en)* | 2014-10-28 | 2015-01-21 | 中国科学院自动化研究所 | Gait recognition method based on deep learning |
| CN107085716A (en)* | 2017-05-24 | 2017-08-22 | 复旦大学 | Cross-view gait recognition method based on multi-task generative adversarial network |
| CN108009526A (en)* | 2017-12-25 | 2018-05-08 | 西北工业大学 | A kind of vehicle identification and detection method based on convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 杨路明等: "基于特征融合的步态识别算法研究", 《计算机应用研究》* |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914762A (en)* | 2020-08-04 | 2020-11-10 | 浙江大华技术股份有限公司 | Gait information-based identity recognition method and device |
| CN112487903A (en)* | 2020-11-23 | 2021-03-12 | 中国信息通信科技集团有限公司 | Gait data generation method and device based on confrontation network |
| CN113869109A (en)* | 2021-08-20 | 2021-12-31 | 江汉大学 | Gait recognition method and device and related equipment |
| CN113869109B (en)* | 2021-08-20 | 2024-12-03 | 江汉大学 | A method, device and related equipment for gait recognition |
| CN113887316A (en)* | 2021-09-06 | 2022-01-04 | 浙江大华技术股份有限公司 | Identity recognition method and device, electronic equipment and storage medium |
| CN113951607A (en)* | 2021-09-14 | 2022-01-21 | 天津科技大学 | Design method of diabetic foot prevention insole based on multi-step time |
| CN113951607B (en)* | 2021-09-14 | 2023-08-04 | 天津科技大学 | Design method of diabetic foot prevention insole based on multi-gait moments |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111144165B (en) | A gait information recognition method, system and storage medium | |
| CN114067358B (en) | Human body posture recognition method and system based on key point detection technology | |
| Singh et al. | Vision-based gait recognition: A survey | |
| CN111144167A (en) | Gait information identification optimization method, system and storage medium | |
| KR101815975B1 (en) | Apparatus and Method for Detecting Object Pose | |
| CN111753747B (en) | Violent motion detection method based on monocular camera and three-dimensional attitude estimation | |
| US7330566B2 (en) | Video-based gait recognition | |
| US20060269145A1 (en) | Method and system for determining object pose from images | |
| US11854306B1 (en) | Fitness action recognition model, method of training model, and method of recognizing fitness action | |
| CN109902565B (en) | Multi-feature fusion human behavior recognition method | |
| CN110472489A (en) | A kind of face beauty grade prediction technique, device and storage medium | |
| US20230237694A1 (en) | Method and system for detecting children's sitting posture based on face recognition of children | |
| CN114863471B (en) | A multi-feature fusion human fall detection method and system | |
| CN108171201A (en) | Eyelashes rapid detection method based on gray scale morphology | |
| Vajdi et al. | Human gait database for normal walk collected by smartphone accelerometer | |
| CN115346272B (en) | Real-time fall detection method based on depth image sequences | |
| CN113516005A (en) | Dance action evaluation system based on deep learning and attitude estimation | |
| Yeoh et al. | On the effectiveness of feature selection methods for gait classification under different covariate factors | |
| Jinnovart et al. | Abnormal gait recognition in real-time using recurrent neural networks | |
| CN120234654B (en) | Spatial interaction precise recognition method and system based on multimodal fusion | |
| Krzeszowski et al. | Gait recognition based on marker-less 3D motion capture | |
| CN111144166A (en) | Method, system and storage medium for establishing abnormal crowd information base | |
| Zhen et al. | Hybrid Deep‐Learning Framework Based on Gaussian Fusion of Multiple Spatiotemporal Networks for Walking Gait Phase Recognition | |
| CN112990090A (en) | Face living body detection method and device | |
| CN113378691A (en) | Intelligent home management system and method based on real-time user behavior analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | Effective date of registration:20210202 Address after:315016 9-3, building 91, 16 Buzheng lane, Haishu District, Ningbo City, Zhejiang Province Applicant after:Yinhe shuidi Technology (Ningbo) Co.,Ltd. Address before:Room 501-1753, Development Zone office building, 8 Xingsheng South Road, Miyun District Economic Development Zone, Beijing Applicant before:Watrix Technology (Beijing) Co.,Ltd. | |
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20200512 | |
| RJ01 | Rejection of invention patent application after publication |