CN106203429A - Based on the shelter target detection method under binocular stereo vision complex background - Google Patents

Abstract

Translated fromChinese

本发明公开了一种基于双目立体视觉复杂背景下的遮挡目标检测方法，用于解决现有遮挡目标检测方法检测精度差的技术问题。技术方案是首先标定双目相机得到行对准的校正图像，然后通过立体匹配得到视差图进行背景建模，计算场景三维坐标生成俯视投影图，最后用MeanShift方法对俯视投影图进行聚类得到检测结果。本发明利用空间三维信息，有效地解决了目标遮挡、场景光线变化、阴影以及复杂背景的干扰等单目视觉中的技术问题，提高了检测精度。The invention discloses an occlusion target detection method based on binocular stereo vision in a complex background, which is used to solve the technical problem of poor detection accuracy of the existing occlusion target detection method. The technical solution is to first calibrate the binocular camera to obtain the corrected image for line alignment, then obtain the disparity map through stereo matching for background modeling, calculate the three-dimensional coordinates of the scene to generate a top-view projection map, and finally use the MeanShift method to cluster the top-view projection map to obtain detection result. The invention utilizes spatial three-dimensional information to effectively solve technical problems in monocular vision such as object occlusion, scene light changes, shadows, and interference from complex backgrounds, and improves detection accuracy.

Description

Translated fromChinese

基于双目立体视觉复杂背景下的遮挡目标检测方法Occluded object detection method in complex background based on binocular stereo vision

技术领域technical field

本发明涉及一种遮挡目标检测方法，特别是涉及一种基于双目立体视觉复杂背景下的遮挡目标检测方法。The invention relates to a method for detecting an occluded target, in particular to a method for detecting an occluded target based on a complex background of binocular stereo vision.

背景技术Background technique

传统的运动目标检测大多基于单目视觉的方法，相对于立体视觉而言单目视觉有其优点，但也存在很大缺陷。单目视觉的信息量小，每次只需处理一幅图像，运算速度相对较快，但是图像在投影过程中丢失了实际场景的三维信息，因此有着不可弥补的缺陷。在采用基于单目视觉的方法进行运动目标检测中，如文献“Effective Gaussian mixturelearning for video background subtraction.Pattern Analysis and MachineIntelligence,IEEE Transactions on,2005,27(5):827-832”常常存在着目标遮挡以及周围场景光线的变化和阴影的干扰等问题，如何解决这些问题一直是研究的难点。针对这些问题许多学者做了大量研究，针对遮挡问题提出来基于目标特征匹配的算法，多子模板匹配的算法，针对光线变化和阴影干扰等问题，提出多高斯背景模型和阴影消除算法等，但这些方法受环境因素的影响较大，在实际应用中很容易出现目标检测失败的现象。Most of the traditional moving target detection methods are based on monocular vision. Compared with stereo vision, monocular vision has its advantages, but it also has great defects. Monocular vision has a small amount of information, and only needs to process one image at a time, and the calculation speed is relatively fast, but the image loses the three-dimensional information of the actual scene during the projection process, so it has irreparable defects. In the detection of moving targets based on monocular vision, such as the document "Effective Gaussian mixture learning for video background subtraction. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2005, 27(5): 827-832", there is often target occlusion As well as the change of light in the surrounding scene and the interference of shadows, how to solve these problems has always been a difficult point in research. Many scholars have done a lot of research on these problems, proposed algorithms based on target feature matching, multi-sub-template matching algorithms for occlusion problems, proposed multi-Gaussian background models and shadow elimination algorithms for problems such as light changes and shadow interference, etc., but These methods are greatly affected by environmental factors, and it is easy to fail in target detection in practical applications.

发明内容Contents of the invention

为了克服现有遮挡目标检测方法检测精度差的不足，本发明提供一种基于双目立体视觉复杂背景下的遮挡目标检测方法。该方法首先标定双目相机得到行对准的校正图像，然后通过立体匹配得到视差图进行背景建模，计算场景三维坐标生成俯视投影图，最后用MeanShift方法对俯视投影图进行聚类得到检测结果。本发明利用空间三维信息，有效地解决了目标遮挡、场景光线变化、阴影以及复杂背景的干扰等单目视觉中的技术问题，提高了检测精度。In order to overcome the deficiency of poor detection accuracy of existing occlusion target detection methods, the present invention provides a occlusion target detection method based on binocular stereo vision in a complex background. This method firstly calibrates the binocular camera to obtain the corrected image for line alignment, then obtains the disparity map through stereo matching for background modeling, calculates the three-dimensional coordinates of the scene to generate the top view projection map, and finally uses the MeanShift method to cluster the top view projection map to obtain the detection results . The invention utilizes spatial three-dimensional information to effectively solve technical problems in monocular vision such as object occlusion, scene light changes, shadows, and interference from complex backgrounds, and improves detection accuracy.

本发明解决其技术问题所采用的技术方案是：一种基于双目立体视觉复杂背景下的遮挡目标检测方法，其特点是包括以下步骤：The technical solution adopted by the present invention to solve the technical problem is: a method for detecting an occluded target based on a complex background of binocular stereo vision, which is characterized in that it comprises the following steps:

步骤一、双目相机标定。Step 1, binocular camera calibration.

首先采用张正友棋盘格标定方法，分别拍摄多张棋盘格图像，用以标定两个相机的内部参数M₁，外部参数M₂通过拍摄放置在地面的标定板图像解算。图像坐标(u,v)与世界坐标(X_w,Y_w,Z_w)的齐次转换关系如下：Firstly, Zhang Zhengyou’s checkerboard calibration method is used to shoot multiple checkerboard images to calibrate the internal parameter M₁ of the two cameras, and the external parameter M₂ is calculated by taking images of the calibration board placed on the ground. The homogeneous transformation relationship between image coordinates (u, v) and world coordinates (X_w , Y_w , Z_w ) is as follows:

$\begin{array}{c}{\mathrm{<span><span>z</span>z</span>}}_{\mathrm{<span><span>c</span>c</span>}}\left[\begin{array}{c}\mathrm{<span><span>u</span>u</span>}\\ \mathrm{<span><span>v</span>v</span>}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{ccc}\frac{\mathrm{<span><span>1</span>1</span>}}{{\mathrm{<span><span>d</span>d</span>}}_{\mathrm{<span><span>x</span>x</span>}}}& \mathrm{<span><span>0</span>0</span>}& {\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>0</span>0</span>}}\\ \mathrm{<span><span>0</span>0</span>}& \frac{\mathrm{<span><span>1</span>1</span>}}{{\mathrm{<span><span>d</span>d</span>}}_{\mathrm{<span><span>y</span>the\; y</span>}}}& {\mathrm{<span><span>v</span>v</span>}}_{\mathrm{<span><span>0</span>0</span>}}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{cccc}\mathrm{<span><span>f</span>f</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>f</span>f</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>1</span>1</span>}& \mathrm{<span><span>0</span>0</span>}\end{array}\right]\left[\begin{array}{cc}{\mathrm{<span><span>R</span>R</span>}}_{\mathrm{<span><span>C</span>C</span>}}& {\mathrm{<span><span>T</span>T</span>}}_{\mathrm{<span><span>C</span>C</span>}}\\ \stackrel{<span><span>\&RightArrow;</span>\&Right\; Arrow;</span>}{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>X</span>x</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Y</span>Y</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Z</span>Z</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]<span><span>=</span>=</span>\\ <span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>u</span>u</span>}}& \mathrm{<span><span>0</span>0</span>}& {\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& {\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>v</span>v</span>}}& {\mathrm{<span><span>v</span>v</span>}}_{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>1</span>1</span>}& \mathrm{<span><span>0</span>0</span>}\end{array}\right]\left[\begin{array}{cc}{\mathrm{<span><span>R</span>R</span>}}_{\mathrm{<span><span>C</span>C</span>}}& {\mathrm{<span><span>T</span>T</span>}}_{\mathrm{<span><span>C</span>C</span>}}\\ \stackrel{<span><span>\&RightArrow;</span>\&Right\; Arrow;</span>}{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>X</span>x</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Y</span>Y</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Z</span>Z</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]<span><span>=</span>=</span>{\mathrm{<span><span>M</span>m</span>}}_{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}}_{\mathrm{<span><span>2</span>2</span>}}\left[\begin{array}{c}{\mathrm{<span><span>X</span>x</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Y</span>Y</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Z</span>Z</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]\end{array}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>$

其中，f为相机焦距，(u₀,v₀)为图像主点坐标，d_x和d_y分别表示每个像素在横轴x和纵轴y上的物理坐标。Among them, f is the focal length of the camera, (u₀ , v₀ ) is the coordinates of the principal point of the image, d_x and d_y represent the physical coordinates of each pixel on the horizontal axis x and vertical axis y, respectively.

然后立体标定计算空间上两台摄像机P₁和P₂的几何关系，即两个相机之间的旋转矩阵R和平移矩阵T。选择右相机作为参考相机。关系如下：Then the stereo calibration calculates the geometric relationship of the two cameras P₁ and P₂ in space, that is, the rotation matrix R and the translation matrix T between the two cameras. Select the right camera as the reference camera. The relationship is as follows:

P₁＝R*(P₂-T) (2)P₁ =R*(P₂ −T) (2)

最后通过非标定立体校正HartLey算法得到行对准的校正图像。要求双目相机采集图像同步。Finally, the line-aligned corrected image is obtained by uncalibrated stereo correction HartLey algorithm. It is required that the binocular camera acquires images synchronously.

步骤二、立体匹配获取视差。Step 2: Stereo matching to obtain disparity.

通过双目立体匹配计算左右相机视图之间的匹配点，得到视差图，选择高斯混合建模方法对视差图进行背景建模，消除复杂背景对目标检测的干扰。根据视差，基线和内参，采用三角测量计算场景三维坐标。选取以地面为XOY面的世界坐标系，将三维点投影到地面，投影到某个像素点的三维点的个数作为该像素点的颜色值，生成俯视投影图。The matching points between the left and right camera views are calculated through binocular stereo matching to obtain a disparity map, and the Gaussian mixture modeling method is selected to model the background of the disparity map to eliminate the interference of complex backgrounds on target detection. According to the disparity, baseline and internal parameters, triangulation is used to calculate the three-dimensional coordinates of the scene. Select the world coordinate system with the ground as the XOY plane, project the 3D points onto the ground, and use the number of 3D points projected to a certain pixel as the color value of the pixel to generate a top view projection map.

步骤三、俯视投影图聚类。Step 3: Clustering the top view projection map.

x处的概率密度为f_h,k(x)：The probability density at x is f_h,k (x):

${\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>h</span>h</span>}<span><span>,</span>,</span>\mathrm{<span><span>k</span>k</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>K</span>K</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span><span><span>|</span>|</span><span><span>)</span>)</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>3</span>3</span>}<span><span>)</span>)</span>$

其中，K(x)为核函数，h为半径Among them, K(x) is the kernel function, h is the radius

要使得f_h,k(x)最大，对f_h,k(x)求导得其中g(s)＝-k'(s)，To maximize f_h,k (x), derive f h,k (x) where g(s)=-k'(s),

$\begin{array}{c}<span><span>\&dtri;</span>\&dtri;</span>{\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>h</span>h</span>}<span><span>,</span>,</span>\mathrm{<span><span>k</span>k</span>}}<span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span>\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>\\ <span><span>=</span>=</span><span><span>\&lsqb;</span>\&lsqb;</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span><span><span>\&rsqb;</span>\&rsqb;</span><span><span>\&lsqb;</span>\&lsqb;</span>\frac{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}<span><span>-</span>-</span>\mathrm{<span><span>x</span>x</span>}<span><span>\&rsqb;</span>\&rsqb;</span>\end{array}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>4</span>4</span>}<span><span>)</span>)</span>$

令：make:

${\mathrm{<span><span>m</span>m</span>}}_{\mathrm{<span><span>h</span>h</span>}<span><span>,</span>,</span>\mathrm{<span><span>g</span>g</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\frac{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}<span><span>-</span>-</span>\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>5</span>5</span>}<span><span>)</span>)</span>$

要使得当且仅当m_h,g(x)＝0，得出新的圆心坐标：to make If and only if m_h,g (x)=0, the new coordinates of the center of the circle are obtained:

$\mathrm{<span><span>x</span>x</span>}<span><span>=</span>=</span>\frac{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>6</span>6</span>}<span><span>)</span>)</span>$

由于投影图的特殊性，仅考虑像素点的距离无法得到准确的聚类结果，计算概率密度时，需要满足：(a)像素点的颜色值与中心像素点的颜色值越相近，概率密度越高；(b)离中心点的位置越近的像素点，概率密度越高。因此，选择核函数K_h(x)：Due to the particularity of the projection map, accurate clustering results cannot be obtained only by considering the distance of the pixels. When calculating the probability density, it needs to satisfy: (a) the closer the color value of the pixel point is to the color value of the central pixel point, the higher the probability density is. High; (b) The closer the pixel is to the center point, the higher the probability density. Therefore, the kernel function K_h (x) is chosen:

${\mathrm{<span><span>K</span>K</span>}}_{\mathrm{<span><span>h</span>h</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\mathrm{<span><span>K</span>K</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{{\mathrm{<span><span>x</span>x</span>}}^{\mathrm{<span><span>s</span>the\; s</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}^{\mathrm{<span><span>s</span>the\; s</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span><span><span>|</span>|</span><span><span>)</span>)</span><span><span>*</span>*</span>\mathrm{<span><span>K</span>K</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{{\mathrm{<span><span>x</span>x</span>}}^{\mathrm{<span><span>r</span>r</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}^{\mathrm{<span><span>r</span>r</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span><span><span>|</span>|</span><span><span>)</span>)</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>7</span>7</span>}<span><span>)</span>)</span>$

MeanShift聚类后，每一类代表一个目标。将这一结果投影到原右图像中显示最终的检测结果。After MeanShift clustering, each class represents a target. Projecting this result into the original right image shows the final detection result.

本发明的有益效果是：该方法首先标定双目相机得到行对准的校正图像，然后通过立体匹配得到视差图进行背景建模，计算场景三维坐标生成俯视投影图，最后用MeanShift方法对俯视投影图进行聚类得到检测结果。本发明利用空间三维信息，有效地解决了目标遮挡、场景光线变化、阴影以及复杂背景的干扰等单目视觉中的技术问题，提高了检测精度。The beneficial effects of the present invention are: the method first calibrates the binocular camera to obtain the corrected image for line alignment, then obtains the disparity map through stereo matching for background modeling, calculates the three-dimensional coordinates of the scene to generate a top view projection map, and finally uses the MeanShift method to perform the top view projection The graph is clustered to get the detection results. The invention utilizes spatial three-dimensional information to effectively solve technical problems in monocular vision such as object occlusion, scene light changes, shadows, and interference from complex backgrounds, and improves detection accuracy.

下面结合具体实施方式对本发明作详细说明。The present invention will be described in detail below in combination with specific embodiments.

具体实施方式detailed description

本发明基于双目立体视觉复杂背景下的遮挡目标检测方法具体步骤如下：The present invention is based on binocular stereo vision occlusion target detection method under complex background specific steps are as follows:

步骤一、双目相机标定。Step 1, binocular camera calibration.

首先采用张正友的棋盘格标定方法，分别拍摄20张左右棋盘格图像，用以标定两个相机的内部参数M₁，外部参数M₂通过拍摄放置在地面的标定板图像解算。图像坐标(u,v)与世界坐标(X_w,Y_w,Z_w)的齐次转换关系如下：Firstly, Zhang Zhengyou’s checkerboard calibration method is adopted, and about 20 checkerboard images are taken respectively to calibrate the internal parameter M₁ of the two cameras, and the external parameter M₂ is calculated by taking images of the calibration board placed on the ground. The homogeneous transformation relationship between image coordinates (u, v) and world coordinates (X_w , Y_w , Z_w ) is as follows:

$\begin{array}{c}{\mathrm{<span><span>z</span>z</span>}}_{\mathrm{<span><span>c</span>c</span>}}\left[\begin{array}{c}\mathrm{<span><span>u</span>u</span>}\\ \mathrm{<span><span>v</span>v</span>}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]<span><span>=</span>=</span>\left[\begin{array}{ccc}\frac{\mathrm{<span><span>1</span>1</span>}}{{\mathrm{<span><span>d</span>d</span>}}_{\mathrm{<span><span>x</span>x</span>}}}& \mathrm{<span><span>0</span>0</span>}& {\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>0</span>0</span>}}\\ \mathrm{<span><span>0</span>0</span>}& \frac{\mathrm{<span><span>1</span>1</span>}}{{\mathrm{<span><span>d</span>d</span>}}_{\mathrm{<span><span>y</span>the\; y</span>}}}& {\mathrm{<span><span>v</span>v</span>}}_{\mathrm{<span><span>0</span>0</span>}}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{cccc}\mathrm{<span><span>f</span>f</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>f</span>f</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>1</span>1</span>}& \mathrm{<span><span>0</span>0</span>}\end{array}\right]\left[\begin{array}{cc}{\mathrm{<span><span>R</span>R</span>}}_{\mathrm{<span><span>C</span>C</span>}}& {\mathrm{<span><span>T</span>T</span>}}_{\mathrm{<span><span>C</span>C</span>}}\\ \stackrel{<span><span>\&RightArrow;</span>\&Right\; Arrow;</span>}{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>X</span>x</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Y</span>Y</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Z</span>Z</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]\\ <span><span>=</span>=</span>\left[\begin{array}{cccc}{\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>u</span>u</span>}}& \mathrm{<span><span>0</span>0</span>}& {\mathrm{<span><span>u</span>u</span>}}_{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& {\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>v</span>v</span>}}& {\mathrm{<span><span>v</span>v</span>}}_{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>0</span>0</span>}\\ \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>0</span>0</span>}& \mathrm{<span><span>1</span>1</span>}& \mathrm{<span><span>0</span>0</span>}\end{array}\right]\left[\begin{array}{cc}{\mathrm{<span><span>R</span>R</span>}}_{\mathrm{<span><span>C</span>C</span>}}& {\mathrm{<span><span>T</span>T</span>}}_{\mathrm{<span><span>C</span>C</span>}}\\ \stackrel{<span><span>\&RightArrow;</span>\&Right\; Arrow;</span>}{\mathrm{<span><span>0</span>0</span>}}& \mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>X</span>x</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Y</span>Y</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Z</span>Z</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]<span><span>=</span>=</span>{\mathrm{<span><span>M</span>m</span>}}_{\mathrm{<span><span>1</span>1</span>}}{\mathrm{<span><span>M</span>m</span>}}_{\mathrm{<span><span>2</span>2</span>}}\left[\begin{array}{c}{\mathrm{<span><span>X</span>x</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Y</span>Y</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ {\mathrm{<span><span>Z</span>Z</span>}}_{\mathrm{<span><span>w</span>w</span>}}\\ \mathrm{<span><span>1</span>1</span>}\end{array}\right]\end{array}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span>$

其中f为相机焦距，(u₀,v₀)为图像主点坐标，d_x和d_y分别表示每个像素在横轴x和纵轴y上的物理坐标，这些参数均可通过相机标定获取。Where f is the focal length of the camera, (u₀ , v₀ ) is the coordinates of the principal point of the image, d_x and d_y represent the physical coordinates of each pixel on the horizontal axis x and vertical axis y, respectively, and these parameters can be obtained through camera calibration .

然后立体标定计算空间上两台摄像机P₁和P₂的几何关系，即两个相机之间的旋转矩阵R和平移矩阵T。选择右相机作为参考相机。关系如下：Then the stereo calibration calculates the geometric relationship of the two cameras P₁ and P₂ in space, that is, the rotation matrix R and the translation matrix T between the two cameras. Select the right camera as the reference camera. The relationship is as follows:

P₁＝R*(P₂-T) (2)P₁ =R*(P₂ −T) (2)

最后通过非标定立体校正HartLey算法得到行对准的校正图像。要求双目相机采集图像同步。Finally, the line-aligned corrected image is obtained by uncalibrated stereo correction HartLey algorithm. It is required that the binocular camera acquires images synchronously.

步骤二、立体匹配获取视差。Step 2: Stereo matching to obtain disparity.

通过双目立体匹配计算左右相机视图之间的匹配点，得到视差图，选择高斯混合建模方法对视差图进行背景建模，以消除复杂背景对目标检测的干扰。根据视差，基线和内参，采用三角测量计算场景三维坐标。选取以地面为XOY面的世界坐标系，将三维点投影到地面，投影到某个像素点的三维点的个数作为该像素点的颜色值，生成俯视投影图。The matching points between the left and right camera views are calculated by binocular stereo matching to obtain a disparity map, and the Gaussian mixture modeling method is selected to model the background of the disparity map to eliminate the interference of complex backgrounds on target detection. According to the disparity, baseline and intrinsic parameters, triangulation is used to calculate the three-dimensional coordinates of the scene. Select the world coordinate system with the ground as the XOY plane, project the 3D points onto the ground, and use the number of 3D points projected to a certain pixel as the color value of the pixel to generate a top view projection map.

步骤三、俯视投影图聚类。Step 3: Clustering the top view projection map.

x处的概率密度为f_h,k(x)：The probability density at x is f_h,k (x):

${\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>h</span>h</span>}<span><span>,</span>,</span>\mathrm{<span><span>k</span>k</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>K</span>K</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span><span><span>|</span>|</span><span><span>)</span>)</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>3</span>3</span>}<span><span>)</span>)</span>$

其中K(x)为核函数，h为半径Where K(x) is the kernel function and h is the radius

要使得f_h,k(x)最大，对f_h,k(x)求导得其中g(s)＝-k'(s)，To maximize f_h,k (x), derive f h,k (x) where g(s)=-k'(s),

$\begin{array}{c}<span><span>\&dtri;</span>\&dtri;</span>{\mathrm{<span><span>f</span>f</span>}}_{\mathrm{<span><span>h</span>h</span>}<span><span>,</span>,</span>\mathrm{<span><span>k</span>k</span>}}<span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}<span><span>)</span>)</span>\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>\\ <span><span>=</span>=</span><span><span>\&lsqb;</span>\&lsqb;</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span><span><span>\&rsqb;</span>\&rsqb;</span><span><span>\&lsqb;</span>\&lsqb;</span>\frac{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}<span><span>-</span>-</span>\mathrm{<span><span>x</span>x</span>}<span><span>\&rsqb;</span>\&rsqb;</span>\end{array}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>4</span>4</span>}<span><span>)</span>)</span>$

令：make:

${\mathrm{<span><span>m</span>m</span>}}_{\mathrm{<span><span>h</span>h</span>}<span><span>,</span>,</span>\mathrm{<span><span>g</span>g</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\frac{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}<span><span>-</span>-</span>\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>5</span>5</span>}<span><span>)</span>)</span>$

要使得当且仅当m_h,g(x)＝0，可以得出新的圆心坐标：to make If and only if m_h,g (x)=0, the new coordinates of the center of the circle can be obtained:

$\mathrm{<span><span>x</span>x</span>}<span><span>=</span>=</span>\frac{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}{\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>n</span>no</span>}}{<span><span>\&Sigma;</span>\&Sigma;</span>}}\mathrm{<span><span>g</span>g</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{\mathrm{<span><span>x</span>x</span>}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span>{<span><span>|</span>|</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>)</span>)</span>}<span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>6</span>6</span>}<span><span>)</span>)</span>$

由于投影图的特殊性，仅考虑像素点的距离无法得到准确的聚类结果，计算概率密度时，需要满足：(a)像素点的颜色值与中心像素点的颜色值越相近，概率密度越高；(b)离中心点的位置越近的像素点，概率密度越高。因此，选择核函数K_h(x)：Due to the particularity of the projection map, accurate clustering results cannot be obtained only by considering the distance of the pixels. When calculating the probability density, it needs to satisfy: (a) the closer the color value of the pixel point is to the color value of the central pixel point, the higher the probability density is. High; (b) The closer the pixel is to the center point, the higher the probability density. Therefore, the kernel function K_h (x) is chosen:

${\mathrm{<span><span>K</span>K</span>}}_{\mathrm{<span><span>h</span>h</span>}}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\mathrm{<span><span>K</span>K</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{{\mathrm{<span><span>x</span>x</span>}}^{\mathrm{<span><span>s</span>the\; s</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}^{\mathrm{<span><span>s</span>the\; s</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span><span><span>|</span>|</span><span><span>)</span>)</span><span><span>*</span>*</span>\mathrm{<span><span>K</span>K</span>}<span><span>(</span>(</span><span><span>|</span>|</span><span><span>|</span>|</span>\frac{{\mathrm{<span><span>x</span>x</span>}}^{\mathrm{<span><span>r</span>r</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>x</span>x</span>}}_{\mathrm{<span><span>i</span>i</span>}}^{\mathrm{<span><span>r</span>r</span>}}}{\mathrm{<span><span>h</span>h</span>}}<span><span>|</span>|</span><span><span>|</span>|</span><span><span>)</span>)</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>-</span>-</span><span><span>(</span>(</span>\mathrm{<span><span>7</span>7</span>}<span><span>)</span>)</span>$

MeanShift聚类后，每一类代表一个目标。将这一结果投影到原右图像中显示最终的检测结果。After MeanShift clustering, each class represents a target. Projecting this result into the original right image shows the final detection result.

Claims (1)

1. one kind based on the shelter target detection method under binocular stereo vision complex background, it is characterised in that include following stepRapid:

Step one, binocular camera are demarcated；

Initially with Zhang Zhengyou gridiron pattern scaling method, shoot multiple gridiron pattern images respectively, in order to demarcate in two camerasPortion's parameter M₁, external parameter M₂The scaling board image calculation on ground it is placed on by shooting；Image coordinate (u, v) and world coordinates(X_w,Y_w,Z_w) homogeneous transformational relation as follows:

$\begin{array}{c}{z}_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{d_x}& 0& u_0\\ 0& \frac{1}{d_y}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}_C& T_C\\ \stackrel{\&RightArrow;}{0}& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]\\ =\left[\begin{array}{cccc}{f}_u& 0& u_0& 0\\ 0& f_v& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{R}_C& T_C\\ \stackrel{\&RightArrow;}{0}& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=M_1{M}_2\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]\end{array}---\left(1\right)$

Wherein, f is camera focus, (u₀,v₀) it is figure principal point coordinate, d_xAnd d_yRepresent that each pixel is at transverse axis x and longitudinal axis y respectivelyOn physical coordinates；

Then stereo calibration calculates spatially two video camera P₁And P₂Geometrical relationship, the spin matrix R between i.e. two camerasWith translation matrix T；Select right camera as reference camera；Relation is as follows:

P₁=R* (P₂-T) (2)

The correction chart picture of row alignment is obtained finally by non-demarcation three-dimensional correction HartLey algorithm；Require binocular camera collection figureAs synchronizing；

Step 2, Stereo matching obtain parallax；

By the match point between the camera view of binocular solid matching primitives left and right, obtain disparity map, select Gaussian Mixture modelingMethod carries out background modeling to disparity map, eliminates the complex background interference to target detection；According to parallax, baseline and internal reference, adoptWith trigonometric calculations scene three-dimensional coordinate；Choose the world coordinate system for XOY face with ground, three-dimensional point projected to ground,Project to the number of three-dimensional point of certain pixel as the color value of this pixel, generate downward projection figure；

Step 3, downward projection figure cluster；

Probability density at x is f_h,k(x):

$f_{h,k}\left(x\right)=\underset{i=1}{\overset{n}{\&Sigma;}}K\left(\right|\left|\frac{x-x_i}{h}\right|\left|\right)---\left(3\right)$

Wherein, K (x) is kernel function, and h is radius

F to be made_h,kX () is maximum, to f_h,kX () derivation obtainsWherein g (s)=-k'(s),

$\begin{array}{c}\&dtri;f_{h,k}=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}\left(x-x_i\right)g\left(\left|\right|\frac{x-x_i}{h}|{|}^2\right)\\ =\&lsqb;\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}g\left(\left|\right|\frac{x-x_i}{h}|{|}^2\right)\&rsqb;\&lsqb;\frac{\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{x}_{i}g\left(\left|\right|\frac{x-x_i}{h}|{|}^2\right)}{\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}g\left(\left|\right|\frac{x-x_i}{h}|{|}^2\right)}-x\&rsqb;\end{array}---\left(4\right)$

Order:

$m_{h,g}\left(x\right)=\frac{\underset{i=1}{\overset{n}{\&Sigma;}}{x}_{i}g\left(\right|\left|\frac{x-x_i}{h}\right|{|}^2)}{\underset{i=1}{\overset{n}{\&Sigma;}}g\left(\right|\left|\frac{x-x_i}{h}\right|{|}^2)}-x---\left(5\right)$

MakeAnd if only if m_h,g(x)=0, the central coordinate of circle that must make new advances:

$x=\frac{\underset{i=1}{\overset{n}{\&Sigma;}}{x}_{i}g\left(\right|\left|\frac{x-x_i}{h}\right|{|}^2)}{\underset{i=1}{\overset{n}{\&Sigma;}}g\left(\right|\left|\frac{x-x_i}{h}\right|{|}^2)}---\left(6\right)$

Due to the particularity of projection, only consider that the distance of pixel cannot obtain cluster result accurately, calculating probability densityTime, need to meet: the color value of (a) pixel is the most close with the color value of central pixel point, and probability density is the highest；(b) fromThe pixel that the position of heart point is the nearest, probability density is the highest；Therefore, kernel function K is selected_h(x):

$K_h\left(x\right)=K\left(\right|\left|\frac{x^s-x_i^s}{h}\right|\left|\right)*K\left(\right|\left|\frac{x^r-x_i^r}{h}\right|\left|\right)---\left(7\right)$

After MeanShift cluster, each class represents a target；Project to this result former right image shows final inspectionSurvey result.