CN103325122A - Pedestrian retrieval method based on bidirectional sequencing - Google Patents

Abstract

Translated fromChinese

本发明公开了一种基于双向排序的行人检索方法，属于监控视频检索技术领域。本发明首先通过行人对象的特征提取和尺度学习对待测行人集进行初始排序；然后，反向查询待测行人集中各行人，并计算待查询行人和待测行人的双向内容相似性和近邻相似性；最后，依据双向内容相似性和近邻相似性重排待测行人集中各待测行人。本发明引入双向匹配的思想，通过内容和近邻相似性重排待测行人，可获得更准确的行人检索结果，且对于环境变化导致的行人外貌变化更鲁棒。

The invention discloses a pedestrian retrieval method based on bidirectional sorting, and belongs to the technical field of surveillance video retrieval. The present invention first performs initial sorting on the set of pedestrians to be measured through feature extraction and scale learning of pedestrian objects; then, reversely inquires each pedestrian in the set of pedestrians to be measured, and calculates the two-way content similarity and neighbor similarity between the pedestrians to be queried and the pedestrians to be measured ;Finally, according to the two-way content similarity and neighbor similarity, each pedestrian in the pedestrian set to be tested is rearranged. The present invention introduces the idea of two-way matching, rearranges the pedestrians to be tested by content and neighbor similarity, can obtain more accurate pedestrian retrieval results, and is more robust to pedestrian appearance changes caused by environmental changes.

Description

Translated fromChinese

基于双向排序的行人检索方法Pedestrian Retrieval Method Based on Bidirectional Sorting

技术领域technical field

本发明属于监控视频检索技术领域，尤其涉及一种基于双向排序的行人检索方法。The invention belongs to the technical field of surveillance video retrieval, in particular to a pedestrian retrieval method based on bidirectional sorting.

背景技术Background technique

监控视频行人检索是在照射区域无重叠的多摄像头下匹配特定行人对象的技术。在实际视频侦查中，侦查员主要根据同一行人对象的活动画面和轨迹来快速锁定、排查和追踪嫌疑目标。传统人工浏览的视频侦查模式需要耗费大量的人力和时间，容易贻误破案时机。行人重识别技术便于视频侦查员快速、准确地发现嫌疑目标活动画面和轨迹，对公安部门提高破案率、维护人民群众生命财产安全具有重要意义。Surveillance video pedestrian retrieval is a technique for matching specific pedestrian objects under multiple cameras with non-overlapping illuminated areas. In the actual video investigation, investigators mainly use the moving pictures and trajectories of the same pedestrian object to quickly lock, check and track the suspected target. The traditional video investigation mode of manual browsing requires a lot of manpower and time, and it is easy to delay the time to solve the case. Pedestrian re-identification technology facilitates video investigators to quickly and accurately discover the moving pictures and trajectories of suspected targets, which is of great significance for the public security department to improve the detection rate and maintain the safety of people's lives and property.

现有行人检索（又称行人重识别）方法可以分为两类：第一类方法主要构造鲁棒的视觉特征，然后使用标准的距离函数（如欧式距离等）进行相似性度量。例如文献1（参见：M.Farenzena,L.Bazzani,A.Perina,V.Murino,and M.Cristani,“Person re-identification by symmetry-driven accumulation of local features”,IEEEConf.on Computer Vision and Pattern Recognition(CVPR),PP.2360–2367,2010.）提出基于对称分割的多局部特征匹配的行人重识别方法。首先，利用颜色特征线索对身体进行水平和垂直分割；其次，提取各分割区域的多种颜色特征和纹理特征，并基于水平中轴加权提取的颜色特征和纹理特征得到综合视觉特征；最后，利用上述综合视觉特征进行对象的表示和匹配。Existing pedestrian retrieval (also known as pedestrian re-identification) methods can be divided into two categories: the first category mainly constructs robust visual features, and then uses standard distance functions (such as Euclidean distance, etc.) for similarity measurement. For example, literature 1 (see: M.Farenzena, L.Bazzani, A.Perina, V.Murino, and M.Cristani, "Person re-identification by symmetry-driven accumulation of local features", IEEEConf.on Computer Vision and Pattern Recognition (CVPR), PP.2360–2367, 2010.) proposed a pedestrian re-identification method based on multi-local feature matching with symmetric segmentation. Firstly, the body is segmented horizontally and vertically by using color feature clues; secondly, various color features and texture features of each segmented area are extracted, and the comprehensive visual features are obtained based on the color features and texture features extracted by weighting the horizontal central axis; finally, using The above-mentioned comprehensive visual features perform object representation and matching.

第二类方法对于特征构造无严格要求，主要通过学习一个合适的尺度进行更准确的距离度量。例如文献2（参见：M.Kostinger,M.Hirzer,P.Wohlhart,P.M.Roth,and H.Bischof,“Large scale metric learning from equivalence constraints”,inComputer Vision and Pattern Recognition(CVPR),PP.2288-2295,2012.）将同类样本的差向量和不同样本的差向量分别表示成不同的高斯分布；然后，采用概率的比值来度量样本之间的距离；最终，将高斯分布的比值转换成马氏距离的形式，从而学习一个合适的马氏距离函数。The second type of method has no strict requirements for feature construction, mainly by learning an appropriate scale for more accurate distance measurement. For example, literature 2 (see: M. Kostinger, M. Hirzer, P. Wohlhart, P. M. Roth, and H. Bischof, "Large scale metric learning from equivalence constraints", in Computer Vision and Pattern Recognition (CVPR), PP.2288-2295 , 2012.) Express the difference vector of the same sample and the difference vector of different samples as different Gaussian distributions; then, use the ratio of probability to measure the distance between samples; finally, convert the ratio of Gaussian distribution into Mahalanobis distance In order to learn a suitable Mahalanobis distance function.

上述行人检索方法都是根据查询行人对象和所有待测行人对象外貌特征的距离对待测集进行排序。然而实际视频监控环境下，不同摄像头的视角、光照、色差等因素不同，导致同一行人在多摄像头下的外貌特征往往存在显著差异，从而使得检索结果不准确。The above pedestrian retrieval methods all sort the test set according to the distance between the query pedestrian object and the appearance features of all pedestrian objects to be tested. However, in the actual video surveillance environment, different cameras have different viewing angles, lighting, color differences and other factors, resulting in significant differences in the appearance characteristics of the same pedestrian under multiple cameras, which makes the retrieval results inaccurate.

发明内容Contents of the invention

针对现有技术存在的不足，本发明提供了一种基于双向排序的行人检索方法，该方法可提升多摄像头下同一行人匹配的准确性。Aiming at the deficiencies in the prior art, the present invention provides a pedestrian retrieval method based on bidirectional sorting, which can improve the matching accuracy of the same pedestrian under multiple cameras.

为解决上述技术问题，本发明采用如下的技术方案：In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

一种基于双向排序的行人检索方法，包括步骤：A pedestrian retrieval method based on bidirectional sorting, comprising steps:

步骤一，度量待查询行人p和待测行人集G＝{g_i|i＝1,...,n}中各待测行人g_i的相似性，获取待测行人集G中各待测行人g_i的正向排序结果，根据正向排序结果获得待查询行人p和待测行人g_i的正向内容相似度

n为待测行人集G中待测行人数量；Step 1: Measure the similarity between the pedestrian p to be queried and the pedestrian g_i to be tested in the set of pedestrians to be tested G={g_i |i=1,...,n}, and obtain the The forward sorting result of the pedestrian g_i , according to the forward sorting result, the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested is obtained

n is the number of pedestrians to be measured in the pedestrian set G to be measured;

步骤二，构建待测行人g_i的待测行人集

度量待测行人g_i与待测行人集

中各行人的相似性，获取待测行人集中各行人的反向排序结果，并根据反向排序结果获得待查询行人p和待测行人g_i的反向内容相似度

Step 2, construct the pedestrian set to be tested for the pedestrian g_i to be tested

Measuring the pedestrian g_i to be tested and the set of pedestrians to be tested

The similarity of each pedestrian in the , to obtain the set of pedestrians to be tested The reverse sorting results of the pedestrians in , and according to the reverse sorting results, the reverse content similarity of the pedestrian p to be queried and the pedestrian g_i to be tested is obtained

步骤三，根据正向排序结果获得待查询行人p的近邻集和远邻集，根据反向排序结果获得待测行人g_i的近邻集和远邻集，基于待查询行人p和待测行人g_i的近邻集和远邻集获得待查询行人p和待测行人g_i的近邻相似性，所述的近邻相似性包括近邻相似度S_near(p,g_i)和远邻相似度S_far(p,g_i)；Step 3: Obtain the nearest neighbor set and far neighbor set of the pedestrian p to be queried according to the forward sorting result, and obtain the nearest neighbor set and far neighbor set of the pedestrian g_i to be tested according to the reverse sorting result, based on the pedestrian p to be queried and the pedestrian g to be tested The neighbor set and far neighbor set of_i obtain the neighbor similarity of the pedestrian p to be queried and the pedestrian g_i to be tested, and the neighbor similarity includes the near neighbor similarity S_near (p, g_i ) and the far neighbor similarity S_far ( p, g_i );

步骤四，综合考虑待查询行人p和待测行人g_i的正向内容相似度反向内容相似度

和近邻相似性，对待测行人集G中各待测行人g_i重新排序。Step 4, comprehensively consider the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested reverse content similarity

and neighbor similarity, each pedestrian g_i to be tested in the pedestrian set G to be tested is reordered.

上述步骤一和步骤二中均基于外貌特征度量相似性。In the above step 1 and step 2, the similarity is measured based on appearance features.

上述待查询行人p和待测行人g_i的正向内容相似度

根据待测行人g_i在正向排序结果中的序号获得。The positive content similarity between the above-mentioned pedestrian p to be queried and the pedestrian g_i to be tested

Obtained according to the serial number of the pedestrian g_i to be tested in the forward sorting result.

上述待查询行人p和待测行人g_i的反向内容相似度

根据待查询行人p在反向排序结果中的序号获得。The reverse content similarity between the above-mentioned pedestrian p to be queried and the pedestrian g_i to be tested

Obtained according to the serial number of the pedestrian p to be queried in the reverse sorting result.

上述待查询行人p的近邻集为：正向排序结果中与待查询行人p相似性最小的k个待测行人的集合n_k(p)，其中，k根据经验取值。The neighbor set of the above-mentioned pedestrian p to be queried is: a set n_k (p) of k pedestrians to be tested with the least similarity to the pedestrian p to be queried in the forward sorting result, where k is selected according to experience.

上述待查询行人p的远邻集为：正向排序结果中与待查询行人p相似性最大的k'个待测行人的集合n_k'(p)，其中，k'根据经验取值。The distant neighbor set of the above-mentioned pedestrian p to be queried is: a set n_k' (p) of the k' pedestrians to be tested with the greatest similarity to the pedestrian p to be queried in the forward sorting results, where k' is selected according to experience.

上述待测行人g_i的近邻集为：反向排序结果中与待测行人g_i相似性最小的k个待测行人的集合n_k(g_i)，其中，k根据经验取值。The neighbor set of the above-mentioned pedestrian g_i to be tested is: a set n_k (g_i ) of k pedestrians to be tested that have the least similarity with the pedestrian g_i to be tested in the reverse sorting result, where k is selected according to experience.

上述待测行人g_i的远邻集为：反向排序结果中与待测行人g_i相似性最大的k'个待测行人的集合n_k'(g_i)，其中，k'根据经验取值。The distant neighbor set of the above-mentioned pedestrian g_i to be tested is: the set n_k' (g_i ) of the k' pedestrians to be tested that have the greatest similarity with the pedestrian g_i to be tested in the reverse sorting results, where k' is selected according to experience value.

上述待查询行人p和待测行人g_i的近邻相似度S_near(p,g_i)根据待查询行人p的近邻集n_k(p)和待测行人g_i的近邻集n_k(g_i)获得，具体为n_k(p)和n_k(g_i)的交集中元素的个数。The above neighbor similarity S_near (p,_gi ) between the pedestrian p to be queried and the pedestrian g_i to be tested is based on the neighbor set_nk (p) of the pedestrian p to be queried and the pedestrian g_i to be tested The neighbor set_nk (g_i ), specifically the number of elements in the intersection of n_k (p) and n_k (g_i ).

上述待查询行人p和待测行人g_i的远邻相似度S_far(p,g_i)根据待查询行人p的远邻集n_k'(p)和待测行人g_i的远邻集n_k'(g_i)获得，具体为n_k'(p)和n_k'(g_i)的交集中元素的个数。The far neighbor similarity S_far (p,_gi ) of the pedestrian p to be queried and the pedestrian g_i to be tested is based on the far neighbor set n_k' (p) of the pedestrian p to be queried and the far neighbor set n of the pedestrian g_i to be tested_k' (g_i ), specifically the number of elements in the intersection of n_k' (p) and n_k' (g_i ).

上述步骤四的一种具体实施方式为：A specific implementation of the above step 4 is:

根据待查询行人p和待测行人g_i的正向内容相似度

反向内容相似度

和近邻相似性获得综合相似性S_*(g_i)，基于综合相似性S_*(g_i)对待测行人集G＝{g_i|i＝1,...,n}中各待测行人g_i进行重新排序，所述的综合相似性$S_{*}\left(g_i\right)=\frac{S_p^{g_i}\×S_{g_i}^p}{\mathrm{\α}+S_{\mathrm{near}}(p,g_i)}+\frac{S_p^{g_i}\×S_{g_i}^p}{\mathrm{\α}+S_{\mathrm{far}}(p,g_i)},$其中，S_near(p,g_i)和S_far(p,g_i)分别为待查询行人p和待测行人g_i的近邻相似度和远邻相似度，

和

分别为待查询行人p和待测行人g_i的正向内容相似度和反向内容相似度，α是一个防止分母为0的常数，0≤α≤1。According to the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested

reverse content similarity

Get the comprehensive similarity S_* (g_i ) with the similarity of the neighbors, and based on the comprehensive similarity S_* (g_i ), each pedestrian to be tested in the pedestrian set G={g_i |i=1,...,n} g_i is reordered, the combined similarity$S_{*}\left(g_i\right)=\frac{S_p^{g_i}\×S_{g_i}^p}{\mathrm{\α}+S_{\mathrm{near}}(p,g_i)}+\frac{S_p^{g_i}\×S_{g_i}^p}{\mathrm{\α}+S_{\mathrm{far\; away}}(p,g_i)},$ Among them, S_near (p, g_i ) and S_far (p, g_i ) are the near-neighbor similarity and far-neighbor similarity of the pedestrian p to be queried and the pedestrian g_i to be tested, respectively,

and

are the forward content similarity and reverse content similarity of the pedestrian p to be queried and the pedestrian g_i to be tested, respectively, α is a constant to prevent the denominator from being 0, 0≤α≤1.

与现有技术相比，本发明具有以下优点和有益效果：Compared with the prior art, the present invention has the following advantages and beneficial effects:

1）本发明引入双向匹配的思想，通过内容和近邻相似性重排待测行人，可获得更准确的行人检索结果；1) The present invention introduces the idea of two-way matching, and rearranges the pedestrians to be tested by content and neighbor similarity, so that more accurate pedestrian retrieval results can be obtained;

2）本发明不仅考虑了行人对象特征空间的相似性，还考虑了近邻空间的相似性，对于环境变化导致的行人外貌变化更鲁棒。2) The present invention not only considers the similarity of pedestrian object feature space, but also considers the similarity of neighboring space, which is more robust to the change of pedestrian appearance caused by environmental changes.

附图说明Description of drawings

图1为本发明方法流程图。Fig. 1 is a flow chart of the method of the present invention.

具体实施方式Detailed ways

本发明基于双向排序的行人检索方法，首先，通过行人对象的特征提取和尺度学习对待测行人集进行初始排序；然后，反向查询待测行人集中各行人，并计算待查询行人和待测行人的双向内容相似性和近邻相似性；最后，依据双向内容相似性和近邻相似性重排待测行人集中各待测行人。The pedestrian retrieval method based on the two-way sorting of the present invention, firstly, perform initial sorting on the set of pedestrians to be measured through feature extraction and scale learning of pedestrian objects; The two-way content similarity and neighbor similarity of the two-way content similarity; finally, according to the two-way content similarity and neighbor similarity rearrange each pedestrian in the test pedestrian set.

本发明中待查询行人p为摄像头A拍摄的行人对象，待测行人是摄像头B拍摄的行人对象，摄像头B拍摄的行人对象构成待测行人集。本发明就是要从摄像头B拍摄的行人对象中找到和待查询行人p为同一人的行人对象。In the present invention, the pedestrian p to be queried is the pedestrian object captured by the camera A, the pedestrian object to be tested is the pedestrian object captured by the camera B, and the pedestrian objects captured by the camera B constitute the pedestrian object to be measured. The present invention is to find the same pedestrian object as the pedestrian p to be queried from the pedestrian objects photographed by the camera B.

图1为本发明方法流程图，下面将结合图1对本发明方法的具体实施作具体说明。Fig. 1 is a flow chart of the method of the present invention, and the specific implementation of the method of the present invention will be described in detail below in conjunction with Fig. 1 .

下述具体实施采用MATLAB7作为仿真实验平台，在常用的行人检索数据集VIPeR上进行测试。VIPeR数据集有两个摄像头下的632个行人图像对，两个摄像头之间存在明显的视角、光照等差异。下面结合各步骤详细描述本具体实施。The following specific implementation uses MATLAB7 as the simulation experiment platform, and is tested on the commonly used pedestrian retrieval dataset VIPeR. The VIPeR dataset has 632 pedestrian image pairs under two cameras, and there are obvious differences in viewing angle and illumination between the two cameras. The specific implementation will be described in detail below in conjunction with each step.

1、对待测行人集中各待测行人进行初始排序，并获取待查询行人和待测行人集中各待测行人的正向内容相似性。1. Initially sort the pedestrians to be tested in the pedestrian set to be tested, and obtain the positive content similarity between the pedestrians to be queried and the pedestrians to be tested in the pedestrian set to be tested.

①特征提取① Feature extraction

待查询行人表示为p，待测行人集G＝{g_i|i＝1,...,n}，n是待测行人集的大小，即待测行人集中元素个数。本具体实施中，随机选择行人检索数据集VIPeR中一半数据做样本训练，另一半做测试，那么，n=316。Pedestrians to be queried are denoted as p, pedestrian set to be tested G={g_i |i=1,...,n}, n is the size of pedestrian set to be tested, that is, the number of elements in pedestrian set to be tested. In this specific implementation, half of the data in the pedestrian retrieval dataset VIPeR is randomly selected for sample training and the other half for testing, then, n=316.

行人图像由8×16像素的窗口以8×8的步长进行分块，提取各块上的颜色特征和纹理特征。颜色特征使用RGB颜色特征（其中，R表示红，G表示绿，B表示蓝）和HSV特征（其中，H表示色度，S表示饱和度，V表示亮度），每种特征表示成24维。纹理特征使用LBP（局部二值模式），表示为59维。所有块的特征加总并PCA（主成分分析）至400维用于表示行人对象的外貌特征。The pedestrian image is divided into blocks by a window of 8×16 pixels with a step size of 8×8, and the color features and texture features on each block are extracted. The color features use RGB color features (where R means red, G means green, and B means blue) and HSV features (where H means chroma, S means saturation, and V means brightness), and each feature is represented as 24 dimensions. Texture features use LBP (Local Binary Pattern), expressed as 59 dimensions. The features of all blocks are summed and PCA (Principal Component Analysis) is used to represent the appearance characteristics of pedestrian objects to 400 dimensions.

②尺度学习②Scale learning

利用步骤①1中特征提取方法表示训练样本，然后，通过尺度学习方法学习马氏距离函数。尺度学习方法一般在马氏距离的基础上进行，根据样本训练得到的距离度量矩阵代替马氏距离中的协方差矩阵M。Use the feature extraction method in step ①1 to represent the training samples, and then learn the Mahalanobis distance function through the scale learning method. The scale learning method is generally carried out on the basis of the Mahalanobis distance, and the distance measurement matrix obtained according to the sample training replaces the covariance matrix M in the Mahalanobis distance.

给定两个行人对象图像O_a和O_b，两者的距离D(O_a,O_b)定义为：Given two pedestrian object images O_a and O_b , the distance D(O_a , O_b ) between them is defined as:

D(O_a,O_b)＝(O_a-O_b)^TM(O_a-O_b)           （1）D(O_a ,O_b )＝(O_a -O_b )^T M(O_a -O_b ) (1)

其中，M是一个半定矩阵，(O_a-O_b)^T是(O_a-O_b)的转置。在极小上述距离D(O_a,O_b)求解M时，使用随机梯度下降进行算法学习。Among them, M is a semidefinite matrix, and (O_a -O_b )^T is the transpose of (O_a -O_b ). When M is solved for the minimum distance D(O_a , O_b ), stochastic gradient descent is used for algorithm learning.

③初始正向排序③Initial forward sorting

利用步骤②中距离学习方法获得的距离度量函数D(O_a,O_b)度量待查询行人p和待测行人g_i的相似性，分别计算待查询行人p和各待测行人g_i的距离d(p,g_i)。根据d(p,g_i)大小对待测行人集G中各待测行人g_i进行排序，获得初始的正向排序结果列表

是正向排序结果列表中第i个待测行人，其中，$d(p,g_1^0)\≤...\≤d(p,g_i^0)\≤...\≤d(p,g_n^0).$Use the distance measurement function D(O_a , O_b ) obtained by the distance learning method in step ② to measure the similarity between the pedestrian p to be queried and the pedestrian g_i to be measured, and calculate the distance between the pedestrian p to be queried and the pedestrian g_i to be measured d(p,g_i ). According to the size of d(p, g_i ), sort the pedestrians g_i to be measured in the pedestrian set G to be measured, and obtain the initial list of forward sorted results

is the i-th pedestrian to be tested in the forward sorting result list, where,$d(p,g_1^0)\≤...\≤d(p,g_i^0)\≤...\≤d(p,g_{\mathrm{no}}^0).$

采用分值S(p,g₁)表示待查询行人p和待测行人g_i间的正向内容相似度，其值等于待测行人g_i在正向排序结果列表中的序号。分值S(p,g₁)被简写成Score S(p,g₁ ) is used to represent the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested, and its value is equal to the serial number of the pedestrian g_i to be tested in the forward sorting result list. The score S(p,g₁ ) is abbreviated as

2、获取待查询行人和待测行人集中各待测行人的反向内容相似度。2. Obtain the reverse content similarity of the pedestrian to be queried and each pedestrian to be tested in the pedestrian set to be tested.

对待测行人集G中待测行人g_i构造对应的待测行人集

待测行人集

包括待查询行人p和待测行人集G中除g_i外的行人对象。Construct the corresponding set of pedestrians to be tested in the set of pedestrians to be tested g_i

Pedestrian set to be tested

Including the pedestrian p to be queried and the pedestrian objects except g_i in the pedestrian set G to be tested.

利用步骤1子步骤②中距离学习方法获得的距离度量函数D(O_a,O_b)度量待测行人g_i与待测行人集

中各行人对象的相似性，按照相似性从小到大，获取待测行人集

中各行人对象的反向排序结果列表，并将待查询行人p在反向排序结果列表上的序号表示待查询行人p和待测行人g_i的反向内容相似度，记为S(g₁,p)，简写为

Use the distance measurement function D(O_a , O_b ) obtained by the distance learning method in step 1 sub-step ② to measure the pedestrian g_i to be measured and the pedestrian set to be measured

The similarity of each pedestrian object in , according to the similarity from small to large, obtain the pedestrian set to be tested

The reverse sorting result list of each pedestrian object in , and the serial number of the pedestrian p to be queried on the reverse sorting result list represents the reverse content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested, which is denoted as S(g₁ ,p), abbreviated as

结合正向内容相似度和反向内容相似度获取待查询行人p和待测行人g_i的内容相似度S_cn(g_i)：Combine the forward content similarity and reverse content similarity to obtain the content similarity S_cn (g_i ) of the pedestrian p to be queried and the pedestrian g_i to be tested:

${\mathrm{<span><span>S</span>S</span>}}_{\mathrm{<span><span>cn</span>cn</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span><span><span>=</span>=</span>{\mathrm{<span><span>S</span>S</span>}}_{\mathrm{<span><span>P</span>P</span>}}^{{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}}<span><span>\&times;</span>\&times;</span>{\mathrm{<span><span>S</span>S</span>}}_{{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}}^{\mathrm{<span><span>p</span>p</span>}}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>2</span>2</span>}<span><span>)</span>)</span>$

3、获取待查询行人和待测行人集中各待测行人的近邻相似性。3. Obtain the neighbor similarity of the pedestrians to be queried and the pedestrians to be tested in the pedestrian set to be tested.

根据步骤1生成的正向排序结果列表，取正向排序结果列表前k个待测行人构成待查询行人p的近邻集n_k(p)，取正向排序结果列表后k'个待测行人构成待查询行人p的远邻集n_k′(p)。According to the forward sorting result list generated in step 1, take the first k pedestrians to be tested in the forward sorting result list to form the neighbor set n_k (p) of the query pedestrian p, and take the k' pedestrians to be tested in the rearward sorting result list Constitute the distant neighbor set n_k′ (p) of the pedestrian p to be queried.

根据步骤2生成的反向排序结果列表，取反向排序结果列表前k个行人对象构成待测行人g_i的近邻集n_k(g_i)，取反向排序结果列表后k'个行人对象构成待测行人g_i的远邻集n_k'(g_i)。According to the reverse sorting result list generated in step 2, take the first k pedestrian objects in the reverse sorting result list to form the neighbor set n_k (g_i ) of the pedestrian g_i to be tested, and take the k' pedestrian objects in the reverse sorting result list constitute the distant neighbor set_nk' (g_i ) of the pedestrian g_i to be tested.

k和k'根据经验取值，本具体实施中，k取值优选为20～40，k'取值范围优选为80～120。令k＝30，k'＝100。计算待查询行人p和待测行人g_i的k近邻空间和k'远邻空间的相似性。待查询行人p和待测行人g_i的近邻相似度S_near(p,g_i)为n_k(p)和n_k(g_i)的交集中元素的个数，如下：The values of k and k' are determined according to experience. In this specific implementation, the value of k is preferably 20-40, and the value range of k' is preferably 80-120. Let k=30, k'=100. Calculate the similarity between the k-nearest neighbor space and the k'far-neighbor space of the pedestrian p to be queried and the pedestrian g_i to be tested. The neighbor similarity S_near (p,_gi ) between the pedestrian p to be queried and the pedestrian g_i to be tested is the number of elements in the intersection of n_k (p) and n_k (g_i ), as follows:

S_near(p,g_i)＝|n_k(p)∩n_k(g_i)|        （3）S_near (p,g_i )＝|n_k (p)∩n_k (g_i )| (3)

待查询行人p和待测行人g_i的远邻相似度S_far(p,g_i)为n_k'(p)和n_k'(g_i)的交集中元素的个数，如下：The far neighbor similarity S_far (p, g_i ) of the pedestrian p to be queried and the pedestrian g_i to be tested is the number of elements in the intersection of n_k' (p) and n_k' (g_i ), as follows:

S_far(p,g_i)＝|n_k'(p)∩n_k'(g_i)|       （4）S_far (p,g_i )＝|n_k' (p)∩n_k' (g_i )| (4)

4、综合待查询行人p和待测行人g_i的内容相似度和近邻相似性，对待测行人集G中各待测行人g_i重新排序。4. Based on the content similarity and neighbor similarity of the pedestrian p to be queried and the pedestrian_gi to be tested, the pedestrians_gi to be tested in the pedestrian set G to be tested are reordered.

综合待查询行人p和待测行人g_i的内容相似度和近邻相似性获得待查询行人p和待测行人g_i的最终相似度S_*(g_i)：Combining the content similarity and neighbor similarity of the pedestrian p to be queried and the pedestrian g_i to be tested to obtain the final similarity S_* (g_i ) of the pedestrian p to be queried and the pedestrian g_i to be tested:

${\mathrm{<span><span>S</span>S</span>}}_{<span><span>*</span>*</span>}<span><span>(</span>(</span>{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span><span><span>=</span>=</span>\frac{{\mathrm{<span><span>S</span>S</span>}}_{\mathrm{<span><span>cn</span>cn</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span>}{{\mathrm{<span><span>S</span>S</span>}}_{\mathrm{<span><span>cx</span>cx</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span>}<span><span>=</span>=</span>\frac{{\mathrm{<span><span>S</span>S</span>}}_{\mathrm{<span><span>p</span>p</span>}}^{{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}}<span><span>\&times;</span>\&times;</span>{\mathrm{<span><span>S</span>S</span>}}_{{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}}^{\mathrm{<span><span>p</span>p</span>}}}{\mathrm{<span><span>\&alpha;</span>\&alpha;</span>}<span><span>+</span>+</span><span><span>|</span>|</span>{\mathrm{<span><span>n</span>no</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>(</span>(</span>\mathrm{<span><span>p</span>p</span>}<span><span>)</span>)</span><span><span>\&cap;</span>\&cap;</span>{\mathrm{<span><span>n</span>no</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span><span><span>|</span>|</span>}<span><span>+</span>+</span>\frac{{\mathrm{<span><span>S</span>S</span>}}_{\mathrm{<span><span>p</span>p</span>}}^{{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}}<span><span>\&times;</span>\&times;</span>{\mathrm{<span><span>S</span>S</span>}}_{{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}}^{\mathrm{<span><span>p</span>p</span>}}}{\mathrm{<span><span>\&alpha;</span>\&alpha;</span>}<span><span>+</span>+</span><span><span>|</span>|</span>{\mathrm{<span><span>n</span>no</span>}}_{{\mathrm{<span><span>k</span>k</span>}}^{<span><span>\&prime;</span>\&prime;</span>}}<span><span>(</span>(</span>\mathrm{<span><span>p</span>p</span>}<span><span>)</span>)</span><span><span>\&cap;</span>\&cap;</span>{\mathrm{<span><span>n</span>no</span>}}_{{\mathrm{<span><span>k</span>k</span>}}^{<span><span>\&prime;</span>\&prime;</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>g</span>g</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span><span><span>|</span>|</span>}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>5</span>5</span>}<span><span>)</span>)</span>$

其中：in:

S_cn(g_i)用来描述待查询行人p和待测行人g_i的内容相似性；S_cn (g_i ) is used to describe the content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested;

S_cx(g_i)用来描述待查询行人p和待测行人g_i的近邻相似性；S_cx (g_i ) is used to describe the neighbor similarity between the pedestrian p to be queried and the pedestrian g_i to be tested;

α是一个防止分母为0的常数，0≤α≤1，本具体实施中令α＝0.01。α is a constant preventing the denominator from being 0, 0≤α≤1, and α=0.01 in this specific implementation.

根据最终相似度S_*(g_i)大小重排待测行人集G中各待测行人g_i，最终相似度S_*(g_i)越小，其对应的待测行人g_i排序越靠前。According to the size of the final similarity S_* (g_i ), each pedestrian g_i to be tested in the pedestrian set G to be tested is rearranged. The smaller the final similarity S_* (gi₎ , the higher the ranking of the corresponding pedestrian g_i to be tested .

采用CMC行人检索评价指标评价本发明方法。CMC值是指N次查询中，返回前r个结果中有正确行人对象的概率。当返回前r个结果时，CMC值越高，表示行人检索性能越好。重复本具体实施的测试过程10次，计算其10次重复的平均CMC值，对比马氏距离尺度学习和重排序的平均CMC值，见表1。从表1中可以发现，本发明的行人重识别方法的检索性能明显优于对比算法。The method of the present invention is evaluated by using the CMC pedestrian retrieval evaluation index. The CMC value refers to the probability that there are correct pedestrian objects in the first r results returned in N queries. When the top r results are returned, a higher CMC value indicates better pedestrian retrieval performance. Repeat the test process of this specific implementation 10 times, calculate the average CMC value of the 10 repetitions, and compare the average CMC value of Mahalanobis distance scale learning and reordering, see Table 1. It can be found from Table 1 that the retrieval performance of the pedestrian re-identification method of the present invention is obviously better than that of the comparison algorithm.

表1在VIPeR上分别返回前1、10、25、50个结果时的平均CMC值Table 1 The average CMC values when returning the first 1, 10, 25, and 50 results on VIPeR respectively

Claims (7)

Translated fromChinese

1.一种基于双向排序的行人检索方法，其特征在于，包括步骤：1. A pedestrian retrieval method based on bidirectional sorting, characterized in that, comprising steps:步骤一，度量待查询行人p和待测行人集G＝{g_i|i＝1,...,n}中各待测行人g_i的相似性，获取待测行人集G中各待测行人g_i的正向排序结果，根据正向排序结果获得待查询行人p和待测行人g_i的正向内容相似度

n为待测行人集G中待测行人数量；Step 1: Measure the similarity between the pedestrian p to be queried and the pedestrian g_i to be tested in the set of pedestrians to be tested G={g_i |i=1,...,n}, and obtain the The forward sorting result of the pedestrian g_i , according to the forward sorting result, the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested is obtained

n is the number of pedestrians to be measured in the pedestrian set G to be measured;步骤二，构建待测行人g_i的待测行人集度量待测行人g_i与待测行人集

中各行人的相似性，获取待测行人集

中各行人的反向排序结果，并根据反向排序结果获得待查询行人p和待测行人g_i的反向内容相似度

Step 2, construct the pedestrian set to be tested for the pedestrian g_i to be tested Measuring the pedestrian g_i to be tested and the set of pedestrians to be tested

The similarity of each pedestrian in the , to obtain the set of pedestrians to be tested

The reverse sorting results of the pedestrians in , and according to the reverse sorting results, the reverse content similarity of the pedestrian p to be queried and the pedestrian g_i to be tested is obtained

步骤三，根据正向排序结果获得待查询行人p的近邻集和远邻集，根据反向排序结果获得待测行人g_i的近邻集和远邻集，基于待查询行人p和待测行人g_i的近邻集和远邻集获得待查询行人p和待测行人g_i的近邻相似性，所述的近邻相似性包括近邻相似度S_near(p,g_i)和远邻相似度S_far(p,g_i)；Step 3: Obtain the nearest neighbor set and far neighbor set of the pedestrian p to be queried according to the forward sorting result, and obtain the nearest neighbor set and far neighbor set of the pedestrian g_i to be tested according to the reverse sorting result, based on the pedestrian p to be queried and the pedestrian g to be tested The neighbor set and far neighbor set of_i obtain the neighbor similarity of the pedestrian p to be queried and the pedestrian g_i to be tested, and the neighbor similarity includes the near neighbor similarity S_near (p, g_i ) and the far neighbor similarity S_far ( p, g_i );步骤四，综合考虑待查询行人p和待测行人g_i的正向内容相似度反向内容相似度和近邻相似性，对待测行人集G中各待测行人g_i重新排序。Step 4, comprehensively consider the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested reverse content similarity and neighbor similarity, each pedestrian g_i to be tested in the pedestrian set G to be tested is reordered.2.如权利要求1所述的基于双向排序的行人检索方法，其特征在于：2. the pedestrian retrieval method based on two-way sorting as claimed in claim 1, is characterized in that:步骤一和步骤二中均基于外貌特征度量相似性。Both step 1 and step 2 measure similarity based on appearance features.3.如权利要求1所述的基于双向排序的行人检索方法，其特征在于：3. the pedestrian retrieval method based on two-way sorting as claimed in claim 1, is characterized in that:所述的待查询行人p和待测行人g_i的正向内容相似度

根据待测行人g_i在正向排序结果中的序号获得。The positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested

Obtained according to the serial number of the pedestrian g_i to be tested in the forward sorting result.4.如权利要求1所述的基于双向排序的行人检索方法，其特征在于：4. the pedestrian retrieval method based on two-way sorting as claimed in claim 1, is characterized in that:所述的待查询行人p和待测行人g_i的反向内容相似度

根据待查询行人p在反向排序结果中的序号获得。The reverse content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested

Obtained according to the serial number of the pedestrian p to be queried in the reverse sorting result.5.如权利要求1所述的基于双向排序的行人检索方法，其特征在于：5. the pedestrian retrieval method based on two-way sorting as claimed in claim 1, is characterized in that:所述的待查询行人p的近邻集为：正向排序结果中与待查询行人p相似性最小的k个待测行人的集合n_k(p)；The neighbor set of the pedestrian p to be queried is: a set n_k (p) of k pedestrians to be measured with the smallest similarity to the pedestrian p to be queried in the forward sorting result;所述的待查询行人p的远邻集为：正向排序结果中与待查询行人p相似性最大的k'个待测行人的集合；The distant neighbor set of the pedestrian p to be queried is: a set of k' pedestrians to be tested that have the greatest similarity with the pedestrian p to be queried in the forward sorting results;所述的待测行人g_i的近邻集为：反向排序结果中与待测行人g_i相似性最小的k个待测行人的集合n_k(g_i)；The neighbor set of the pedestrian g_i to be measured is: a set n_k (g_i ) of k pedestrians to be measured with the smallest similarity to the pedestrian g_i to be measured in the reverse sorting result;所述的待测行人g_i的远邻集为：反向排序结果中与待测行人g_i相似性最大的k'个待测行人的集合n_k'(g_i)；The distant neighbor set of the pedestrian g_i to be measured is: a set n k_' (g_i ) of the k' pedestrians to be measured that have the greatest similarity with the pedestrian g_i to be measured in the reverse sorting result;上述k和k'为经验值。The above k and k' are empirical values.6.如权利要求1所述的基于双向排序的行人检索方法，其特征在于：6. the pedestrian retrieval method based on two-way sorting as claimed in claim 1, is characterized in that:所述的待查询行人p和待测行人g_i的近邻相似度S_near(p,g_i)根据待查询行人p的近邻集n_k(p)和待测行人g_i的近邻集n_k(g_i)获得，具体为n_k(p)和n_k(g_i)的交集中元素的个数；The neighbor similarity S_near (p,gi₎ of the pedestrian p to be queried and the pedestrian g_i to be tested is based on the neighbor set_nk (p) of the pedestrian p to be queried and_the neighbor set_nk ( g_i ), specifically the number of elements in the intersection of n_k (p) and n_k (g_i );所述的待查询行人p和待测行人g_i的远邻相似度S_far(p,g_i)根据待查询行人p的远邻集n_k'(p)和待测行人g_i的远邻集n_k'(g_i)获得，具体为n_k'(p)和n_k'(g_i)的交集中元素的个数。The far neighbor similarity S_far (p,gi₎ of the pedestrian p to be queried and the pedestrian g_i to be tested is based on the far neighbor set_nk' (p) of the pedestrian p to be queried and the far neighbors of the pedestrian g_i to be tested Set n_k' (g_i ), specifically the number of elements in the intersection of n_k' (p) and n_k' (g_i ).7.如权利要求1所述的基于双向排序的行人检索方法，其特征在于：7. the pedestrian retrieval method based on two-way sorting as claimed in claim 1, is characterized in that:步骤四具体为：Step four is specifically:根据待查询行人p和待测行人g_i的正向内容相似度

反向内容相似度

和近邻相似性获得综合相似性S_*(g_i)，基于综合相似性S_*(g_i)对待测行人集G＝{g_i|i＝1,...,n}中各待测行人g_i进行重新排序，所述的综合相似性$S_{*}\left(g_i\right)=\frac{S_p^{g_i}\&times;S_{g_i}^p}{\mathrm{\&alpha;}+S_{\mathrm{near}}(p,g_i)}+\frac{S_p^{g_i}\&times;S_{g_i}^p}{\mathrm{\&alpha;}+S_{\mathrm{far}}(p,g_i)},$其中，S_near(p,g_i)和S_far(p,g_i)分别为待查询行人p和待测行人g_i的近邻相似度和远邻相似度，

和

分别为待查询行人p和待测行人g_i的正向内容相似度和反向内容相似度，α是一个防止分母为0的常数，0≤α≤1。According to the positive content similarity between the pedestrian p to be queried and the pedestrian g_i to be tested

reverse content similarity

Get the comprehensive similarity S_* (g_i ) with the similarity of the neighbors, and based on the comprehensive similarity S_* (g_i ), each pedestrian to be tested in the pedestrian set G={g_i |i=1,...,n} g_i is reordered, the combined similarity$S_{*}\left(g_i\right)=\frac{S_p^{g_i}\&times;S_{g_i}^p}{\mathrm{\&alpha;}+S_{\mathrm{near}}(p,g_i)}+\frac{S_p^{g_i}\&times;S_{g_i}^p}{\mathrm{\&alpha;}+S_{\mathrm{far\; away}}(p,g_i)},$ Among them, S_near (p, g_i ) and S_far (p, g_i ) are the near-neighbor similarity and far-neighbor similarity of the pedestrian p to be queried and the pedestrian g_i to be tested, respectively,

and

are the forward content similarity and reverse content similarity of the pedestrian p to be queried and the pedestrian g_i to be tested, respectively, α is a constant to prevent the denominator from being 0, 0≤α≤1.

Images