CN113066102A - A Correlation Filter Tracking Method Combining Adaptive Spatial Weights and Distortion Suppression - Google Patents

Abstract

The invention discloses a correlation filtering tracking method combining adaptive spatial weight and distortion suppression. Firstly, improving the expression capacity of an algorithm model to a target by extracting FHOG characteristics, CN characteristics and gray characteristics; secondly, adding a distortion suppression term into the target function to restrict the change rate of the current frame response image and enhance the discrimination capability of the filter so as to relieve the degradation problem of the filter model; finally, an adaptive spatial weight term is added into the target function, so that the spatial regularization weight can be updated along with the change of the target, and the filter can fully utilize the diversity information of the target. The method realizes effective improvement of distance precision and success rate, has good robustness in complex scenes, and can be applied to the fields of motion analysis, man-machine interaction, behavior analysis, unmanned aerial vehicle tracking, intelligent video monitoring and the like.

Description

Translated fromChinese

一种结合自适应空间权重和畸变抑制的相关滤波跟踪方法A Correlation Filter Tracking Method Combining Adaptive Spatial Weights and Distortion Suppression

技术领域technical field

本发明涉及一种结合自适应空间权重和畸变抑制的相关滤波跟踪方法，属于计算机视觉与智能信息处理领域。The invention relates to a correlation filtering tracking method combining adaptive space weight and distortion suppression, belonging to the field of computer vision and intelligent information processing.

背景技术Background technique

目标跟踪是计算机视觉领域研究中的基础且关键的问题，在智能视频监控、运动分析、人机交互、行为分析、无人机追踪等领域有着广泛的应用。虽然目标跟踪技术在过去几十年取得了较大进展，但在运动目标出现形变、遮挡、尺度变化、背景杂斑等情形下准确且鲁棒的对目标进行跟踪依然是个具有挑战性的任务。Object tracking is a basic and key problem in the field of computer vision research, and has a wide range of applications in intelligent video surveillance, motion analysis, human-computer interaction, behavior analysis, UAV tracking and other fields. Although object tracking technology has made great progress in the past few decades, it is still a challenging task to accurately and robustly track objects under the conditions of deformation, occlusion, scale change, background noise, etc. of moving objects.

近年来，由于相关滤波类跟踪算法性能优异，能够较好的平衡速度和精度，成为主流跟踪算法，受到了国内外学者的广泛关注。2010年提出的MOSSE跟踪算法创新性地将相关滤波的思想引入到目标跟踪领域，奠定了相关滤波类跟踪算法的基础。之后学者们在MOSSE跟踪算法的基础上，针对特征、尺度等进行改进，提出了CSK、KCF、SAMF、Staple、fDSST等一系列经典跟踪算法。In recent years, due to the excellent performance of correlation filtering tracking algorithms, which can balance speed and accuracy well, they have become mainstream tracking algorithms and have received extensive attention from scholars at home and abroad. The MOSSE tracking algorithm proposed in 2010 innovatively introduced the idea of correlation filtering into the field of target tracking, laying the foundation for correlation filtering-based tracking algorithms. Later, on the basis of the MOSSE tracking algorithm, scholars improved features and scales, and proposed a series of classic tracking algorithms such as CSK, KCF, SAMF, Staple, and fDSST.

边界效应是一个值得关注的问题，由于传统相关滤波类跟踪方法利用循环矩阵的性质将计算转换到频域，在提高计算速度的同时也产生了部分非真实样本，从而导致不期望的边界效应，降低滤波器的判别能力，影响跟踪性能。为减小边界效应带来的影响，学者们提出了BACF、ARCF、ASRCF等算法。基于背景感知的BACF算法，使用了真实移位产生的负样本并增大搜索区域，但较大的搜索区域容易引入背景噪声，在背景复杂时容易造成跟踪漂移。基于畸变抑制的ARCF算法，能够抑制响应图的畸变，但是空间正则化权重没有学习能力，不能适应目标外观变化。基于自适应空间正则化的ASRCF算法，能够高效地学习得到一个空间权重以适应目标外观变化，但在目标出现运动模糊及较大形变时滤波器对不准确目标的学习容易过拟合。Boundary effects are a concern. Since the traditional correlation filtering-based tracking method uses the properties of circulant matrices to convert the calculation to the frequency domain, while improving the calculation speed, it also generates some unreal samples, resulting in undesired boundary effects. Reduce the discriminative ability of the filter and affect the tracking performance. In order to reduce the impact of boundary effects, scholars have proposed algorithms such as BACF, ARCF, and ASRCF. The BACF algorithm based on background perception uses the negative samples generated by the real shift and enlarges the search area, but the larger search area is easy to introduce background noise, and it is easy to cause tracking drift when the background is complex. The ARCF algorithm based on distortion suppression can suppress the distortion of the response map, but the spatial regularization weight has no learning ability and cannot adapt to the change of target appearance. The ASRCF algorithm based on adaptive spatial regularization can efficiently learn a spatial weight to adapt to changes in the appearance of the target, but when the target has motion blur and large deformation, the filter is prone to overfitting to inaccurate targets.

发明内容SUMMARY OF THE INVENTION

本发明提出了一种结合自适应空间权重和畸变抑制的相关滤波跟踪方法，目的在于保证足够搜索区域的同时，能有效缓解边界效应和模型退化问题而提供一种有着较高精度的目标跟踪方法。The present invention proposes a correlation filtering tracking method combining adaptive space weight and distortion suppression. The purpose is to ensure sufficient search area, while effectively alleviating boundary effects and model degradation problems and providing a target tracking method with higher accuracy .

本发明通过以下技术方案来实现上述目的：The present invention realizes above-mentioned purpose through following technical scheme:

一种结合自适应空间权重和畸变抑制的相关滤波跟踪方法，包括以下步骤：(1)提取手工特征A correlation filtering tracking method combining adaptive spatial weights and distortion suppression, comprising the following steps: (1) extracting handcrafted features

本发明通过提取FHOG特征，CN特征和灰度特征以增强算法模型对目标的表达能力。FHOG特征对光照变化不敏感，CN特征对运动模糊、低分辨率表征能力强，灰度特征运算速度快，特征互补提升算法模型对目标表达能力。The present invention enhances the expression ability of the algorithm model to the target by extracting the FHOG feature, the CN feature and the gray level feature. The FHOG feature is insensitive to illumination changes, the CN feature has a strong ability to represent motion blur and low resolution, the grayscale feature has a fast computing speed, and the feature complementarity improves the algorithm model's ability to express the target.

(2)在目标函数中结合自适应空间权重项和畸变抑制项(2) Combine the adaptive spatial weight term and the distortion suppression term in the objective function

在目标函数中加入畸变抑制项来约束当前帧响应图的变化率，增强滤波器的判别能力，以缓解滤波器模型退化问题；在目标函数中加入自适应空间权重项使空间正则化权重能够随着目标的变化而更新，使得滤波器能充分利用目标的多样性信息。结合自适应空间权重项和畸变抑制项后，目标函数为：A distortion suppression term is added to the objective function to constrain the rate of change of the response map of the current frame, and the discriminative ability of the filter is enhanced to alleviate the degradation problem of the filter model; adding an adaptive spatial weight term to the objective function enables the spatial regularization weight to change with time. It is updated as the target changes, so that the filter can make full use of the diversity information of the target. After combining the adaptive space weight term and the distortion suppression term, the objective function is:

式中，D表示总通道数，k和k-1分别代表第k帧和第k-1帧，d代表d通道，

代表帧第通道的特征，C是从背景感知相关滤波器(BACF)中保留的裁剪矩阵用以确保足够的搜索区域，λ₁和λ₂是空间正则化参数，β是畸变惩罚参数，p,q表示二维空间中两幅响应图中两个峰值的位置差异，ψ_p,q表示使两幅响应图两个峰值重合的移位操作，w^r是空间正则化参考权重，w是空间正则化权重，第二项代表畸变抑制项，第三项和第四项代表自适应空间权重项。In the formula, D represents the total number of channels, k and k-1 represent the kth frame and the k-1th frame respectively, d represents the d channel,

represents the features of the frame channel, C is the clipping matrix retained from the background-aware correlation filter (BACF) to ensure sufficient search area, λ₁ and λ₂ are the spatial regularization parameters, β is the distortion penalty parameter, p, q represents the position difference of the two peaks in the two response maps in the two-dimensional space, ψ_p,q represents the shift operation to make the two peaks of the two response maps coincide, w^r is the spatial regularization reference weight, w is the spatial regularity The second term represents the distortion suppression term, and the third and fourth terms represent the adaptive spatial weight term.

(3)采用交替方向乘子法(ADMM)对相关滤波器进行优化迭代求解(3) Using the alternating direction multiplier method (ADMM) to optimize the iterative solution of the correlation filter

考虑到计算的方便，首先将式(1)转换为如下形式：Considering the convenience of calculation, formula (1) is first converted into the following form:

X_k是x_k的向量形式，I_D是D×D的单位矩阵，符号

代表克罗内克积，上标T代表共轭转置运算，M_k-1代表上一帧的响应图，其值为

为了减小运算量，将式(2)转换到频域：X_k is the vector form of x_k , ID is the_D ×D identity matrix, notation

Represents the Kronecker product, the superscript T represents the conjugate transpose operation, M_k-1 represents the response map of the previous frame, and its value is

In order to reduce the computational complexity, formula (2) is converted to the frequency domain:

式(3)中，

代表X_k的离散傅里叶变换，

代表M_k-1[ψ_p,q]的离散傅里叶变换，为了方便后续的优化求解，在式(3)中引入一个新参数

In formula (3),

represents the discrete Fourier transform of X_k ,

Represents the discrete Fourier transform of M_k-1 [ψ_p,q ]. In order to facilitate the subsequent optimization solution, a new parameter is introduced in Eq. (3).

由于式(3)是凸函数，将式(3)写成如下的增广拉格朗日形式：Since formula (3) is a convex function, formula (3) can be written in the augmented Lagrangian form as follows:

式(4)中μ是惩罚因子，并引入傅立叶域中的拉格朗日向量

作为辅助变量。In formula (4), μ is the penalty factor, and the Lagrangian vector in the Fourier domain is introduced

as an auxiliary variable.

对第k帧运用ADMM算法意味着可将式(4)分解为两个子问题求解，即求解h^*_k+1和

Applying the ADMM algorithm to the kth frame means that equation (4) can be decomposed into two sub-problems to solve, namely solving h^*_k+1 and

子问题1：求解h^*_k+1。Subproblem 1: Solve for h^*_k+1 .

容易求得：Easy to get:

其中，g_k和

以及α和

的转换关系如下：where g_k and

and α and

The conversion relationship is as follows:

子问题2：求解

Subproblem 2: Solving

为了求解方便，将式(8)分解为N个子问题的求解，n＝[1,2,...N]。For the convenience of solving, formula (8) is decomposed into the solving of N sub-problems, n=[1,2,...N].

其中，

conj(·)表示复共轭运算。

是

的离散傅里叶变换，即

每个子问题的解如下：in,

conj(·) represents a complex conjugate operation.

Yes

The discrete Fourier transform of

The solution to each subproblem is as follows:

由于式(10)中含有求逆运算，计算量较大，采用Sherman-Morrison公式对其继续优化，可得出式(10)的等价形式如下：Since Equation (10) contains an inversion operation, which requires a large amount of calculation, the Sherman-Morrison formula is used to continue to optimize it, and the equivalent form of Equation (10) can be obtained as follows:

其中

至此，子问题h^*_k+1和

解毕。in

So far, the subproblems h^*_k+1 and

Solved.

拉格朗日乘子

的更新方案为：Lagrange Multipliers

The update plan is:

(4)采用交替方向乘子法(ADMM)对自适应空间权重参数进行迭代求解(4) Iteratively solve the adaptive spatial weight parameters by using the alternating direction multiplier method (ADMM).

为了减少计算量，对自适应空间权重参数w的求解依然采用ADMM算法求解。引入辅助变量f构造限制等式w＝f，则可将目标函数写成：In order to reduce the amount of calculation, the ADMM algorithm is still used to solve the adaptive spatial weight parameter w. Introducing the auxiliary variable f to construct the restriction equation w=f, the objective function can be written as:

接着将式(13)写成增广拉格朗日形式：Then formula (13) can be written in augmented Lagrangian form:

δ是惩罚参数，s是拉格朗日乘子。引入参数

式(14)可写成如下等价形式：δ is the penalty parameter and s is the Lagrange multiplier. Import parameters

Equation (14) can be written in the following equivalent form:

则对式(15)的求解可以转换为两个子问题的求解。Then the solution of equation (15) can be transformed into the solution of two sub-problems.

子问题1：w^*的求解。Subproblem 1: Solving for w^* .

w^*的解为：The solution for w^* is:

其中，

in,

子问题2：f^*的求解。Subproblem 2: Solving for f^* .

f^*的解为：The solution for f^* is:

拉格朗日乘子的更新方案为：The update scheme of Lagrange multipliers is:

m⁽ⁱ⁺¹⁾＝m⁽ⁱ⁾+w⁽ⁱ⁺¹⁾-f⁽ⁱ⁺¹⁾ (20)m⁽ⁱ⁺¹⁾ = m⁽ⁱ⁾ + w⁽ⁱ⁺¹⁾ - f⁽ⁱ⁺¹⁾ (20)

(5)模型更新(5) Model update

目标表观模型按照下式更新：The target appearance model is updated as follows:

式(21)中，k和k-1分别表示k帧和k-1帧，η表示表观模型学习率。In Equation (21), k and k-1 represent k frames and k-1 frames, respectively, and η represents the apparent model learning rate.

附图说明Description of drawings

图1为本发明整体框架图。Fig. 1 is the overall frame diagram of the present invention.

具体实施方式Detailed ways

下面结合附图对本发明作进一步说明：The present invention will be further described below in conjunction with the accompanying drawings:

如图1所示，一种结合自适应空间权重和畸变抑制的相关滤波跟踪方法包括以下步骤：As shown in Figure 1, a correlation filtering tracking method combining adaptive spatial weights and distortion suppression includes the following steps:

(1)输入视频帧，提取当前帧的FHOG特征、CN特征和灰度特征用以描述目标。(1) Input the video frame, extract the FHOG feature, CN feature and gray level feature of the current frame to describe the target.

(2)自适应空间权重参数随着目标表观模型的变化而更新。(2) The adaptive spatial weight parameter is updated with the change of the target appearance model.

(3)根据上一帧的响应图约束当前帧响应图的变化率，以得到质量更高的响应图。(3) Constrain the change rate of the response map of the current frame according to the response map of the previous frame, so as to obtain a response map of higher quality.

(4)把当前帧响应图的最高点作为最终的跟踪结果。(4) Take the highest point of the response graph of the current frame as the final tracking result.

为了验证本发明所述结合自适应空间权重和畸变抑制的相关滤波跟踪方法的合理性和有效性，选取OTB-2013和OTB-2015两个标准数据集进行实验，并采用距离精度和成功率作为评价指标。本发明实现所用编程软件为MATLAB R2017a，操作系统为Windows10，计算机配置为CPU：Intel(R)Celeron(R)G1820，主频2.70GHz，内存8G。In order to verify the rationality and effectiveness of the correlation filtering tracking method combining adaptive spatial weight and distortion suppression, two standard data sets, OTB-2013 and OTB-2015, were selected for experiments, and the distance accuracy and success rate were used as evaluation indicators. The programming software used in the implementation of the present invention is MATLAB R2017a, the operating system is Windows 10, and the computer is configured as CPU: Intel(R) Celeron(R) G1820, the main frequency is 2.70GHz, and the memory is 8G.

表1列出了本发明在OTB-2013数据集上的实验结果，距离精度达到88.9％，成功率达到67.8％，跟踪性能优于对比算法，实现了更加准确的目标跟踪。Table 1 lists the experimental results of the present invention on the OTB-2013 data set. The distance accuracy reaches 88.9%, and the success rate reaches 67.8%. The tracking performance is better than the comparison algorithm, and more accurate target tracking is achieved.

本发明也使用了更加具有挑战性的OTB-2015数据库进行实验，实验结果如表2所示，本发明的距离精度达到87.9％，成功率达到67.5％，与其他方法相比，跟踪精度和成功率均有不同程度的提升，因而本发明的合理性和有效性得到充分证实。The present invention also uses the more challenging OTB-2015 database for experiments. The experimental results are shown in Table 2. The distance accuracy of the present invention reaches 87.9%, and the success rate reaches 67.5%. Compared with other methods, the tracking accuracy and success rate Therefore, the rationality and effectiveness of the present invention have been fully confirmed.

表1 OTB-2013数据集实验结果Table 1 Experimental results of OTB-2013 dataset

表2 OTB-2015数据集实验结果Table 2 Experimental results of OTB-2015 dataset

Claims (5)

Translated fromChinese

1.一种结合自适应空间权重和畸变抑制的相关滤波跟踪方法，其特征在于包括以下步骤：1. a correlation filter tracking method combining adaptive space weight and distortion suppression, is characterized in that comprising the following steps:(1)通过提取FHOG特征、CN特征和灰度特征，提升算法模型对目标的表达能力；(1) By extracting FHOG features, CN features and grayscale features, the expression ability of the algorithm model to the target is improved;(2)在目标函数中结合自适应空间权重项和畸变抑制项；(2) Combine the adaptive space weight term and the distortion suppression term in the objective function;(3)采用交替方向乘子法(ADMM)对相关滤波器进行优化迭代求解；(3) Using the alternating direction multiplier method (ADMM) to optimize the iterative solution of the correlation filter;(4)采用交替方向乘子法(ADMM)对自适应空间权重参数进行优化迭代求解。(4) Alternating direction multiplier method (ADMM) is used to optimize and iteratively solve the adaptive spatial weight parameters.2.根据权利要求1所述的方法，其特征在于(1)中提取FHOG特征、CN特征和灰度特征，FHOG特征对光照变化不敏感，CN特征对运动模糊、低分辨率表征能力强，灰度特征运算速度快，特征互补提升算法模型对目标的表达能力。2. method according to claim 1 is characterized in that extracting FHOG feature, CN feature and grayscale feature in (1), FHOG feature is insensitive to illumination change, CN feature is strong to motion blur, low-resolution representation ability, The grayscale feature operation speed is fast, and the feature complementarity improves the expression ability of the algorithm model to the target.3.根据权利要求1所述的方法，其特征在于(2)中在目标函数中结合自适应空间权重项和畸变抑制项，目标函数为：3. method according to claim 1, it is characterized in that in (2), combine adaptive space weight term and distortion suppression term in objective function, objective function is:

式中，D表示总通道数，k和k-1分别代表第k帧和第k-1帧，d代表d通道，

代表k帧第d通道的特征，C是从背景感知相关滤波器(BACF)中保留的裁剪矩阵用以确保足够的搜索区域，λ₁和λ₂是空间正则化参数，β是畸变惩罚参数，p,q表示二维空间中两幅响应图中两个峰值的位置差异，ψ_p,q表示使两幅响应图两个峰值重合的移位操作，w^r是空间正则化参考权重，w是空间正则化权重，第二项代表畸变抑制项，第三项和第四项代表自适应空间权重项。In the formula, D represents the total number of channels, k and k-1 represent the kth frame and the k-1th frame respectively, d represents the d channel,

represents the features of the d-th channel of k frames, C is the clipping matrix retained from the background-aware correlation filter (BACF) to ensure sufficient search area, λ₁ and λ₂ are the spatial regularization parameters, β is the distortion penalty parameter, p,q represents the position difference of the two peaks in the two response maps in two-dimensional space, ψ_p,q represents the shift operation to make the two peaks of the two response maps coincide, w^r is the spatial regularization reference weight, w is The spatial regularization weight, the second term represents the distortion suppression term, and the third and fourth terms represent the adaptive spatial weight term.4.根据权利要求1所述的方法，其特征在于(3)中采用交替方向乘子法(ADMM)对滤波器进行优化迭代求解，采用交替方向乘子法(ADMM)对滤波器的求解可转化为对

和

两个子问题的求解，对子问题h^*_k+1的求解如下：4. method according to claim 1 is characterized in that adopting alternate direction multiplier method (ADMM) in (3) to carry out optimization iterative solution to filter, adopting alternate direction multiplier method (ADMM) to solve filter can be solved. convert to

and

The solution of the two sub-problems, the solution to the sub-problem h^*_k+1 is as follows:

求得：Get:

对子问题

的求解如下：pair question

The solution is as follows:

求得：Get:

至此，两个子问题解毕。So far, the two sub-problems have been solved.5.根据权利要求1所述的方法，其特征在于(4)中采用交替方向乘子法(ADMM)对自适应空间权重参数进行优化迭代求解，目标函数为：5. method according to claim 1 is characterized in that adopting alternate direction multiplier method (ADMM) in (4) to carry out optimization iterative solution to adaptive space weight parameter, and objective function is:

采用交替方向乘子法(ADMM)对自适应空间权重参数进行求解可转化为对w^*和f^*的求解，对子问题w^*的求解如下：Using the Alternating Direction Multiplier Method (ADMM) to solve the adaptive spatial weight parameters can be transformed into the solution of w^* and f^* , and the solution of the sub-problem w^* is as follows:

w^*的解为：The solution for w^* is:

对子问题f^*的求解如下：The solution to the subproblem f^* is as follows:

f^*的解为：The solution for f^* is:

至此，两个子问题解毕。So far, the two sub-problems have been solved.

Images