CN115131421A - Depth image processing method and device, terminal and storage medium - Google Patents
Depth image processing method and device, terminal and storage mediumDownload PDFInfo
- Publication number
- CN115131421A CN115131421ACN202110326429.XACN202110326429ACN115131421ACN 115131421 ACN115131421 ACN 115131421ACN 202110326429 ACN202110326429 ACN 202110326429ACN 115131421 ACN115131421 ACN 115131421A
- Authority
- CN
- China
- Prior art keywords
- pixel point
- target
- value
- pixel
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及图像处理领域，尤其涉及的是一种深度图像处理方法、装置、终端及存储介质。The present invention relates to the field of image processing, and in particular, to a depth image processing method, device, terminal and storage medium.
背景技术Background technique
高斯滤波是一种线性平滑滤波，适用于消除高斯噪声，广泛应用于图像处理的减噪过程。通俗的讲，高斯滤波就是对整幅图像进行加权平均的过程，每一个像素点的深度值，都由其本身和邻域内的其他像素点的深度值经过加权平均后得到。然而，在现有技术中，高斯滤波会将靠近中心像素点的像素点的权重值加大，远离中心像素点的像素点的权重值减小，即仅依靠距离的远近确定周围像素点的权值。因此，有可能出现与中心像素点相距较近，但颜色差异过大的像素点对中心像素点的深度值造成干扰的问题。Gaussian filtering is a linear smoothing filter, suitable for removing Gaussian noise, and is widely used in the noise reduction process of image processing. In layman's terms, Gaussian filtering is a process of weighted averaging of the entire image. The depth value of each pixel is obtained by weighted averaging of the depth values of itself and other pixels in the neighborhood. However, in the prior art, Gaussian filtering increases the weight of pixels close to the center pixel, and decreases the weight of pixels far from the center, that is, the weight of surrounding pixels is determined only by the distance. value. Therefore, there may be a problem that the pixel points that are close to the center pixel point, but the color difference is too large, interfere with the depth value of the center pixel point.
因此，现有技术还有待改进和发展。Therefore, the existing technology still needs to be improved and developed.
发明内容SUMMARY OF THE INVENTION
本发明要解决的技术问题在于，针对现有技术的上述缺陷，提供一种深度图像处理方法、装置、终端及存储介质，旨在解决现有的高斯滤波方法在确定中心像素点周围的像素点的权重值时，仅考虑到周围的像素点与中心像素点之间的距离远近，导致与中心像素点相距较近，但颜色差异过大的像素点对中心像素点的深度值造成干扰的问题。The technical problem to be solved by the present invention is to provide a depth image processing method, device, terminal and storage medium in view of the above-mentioned defects of the prior art, aiming at solving the problem of determining the pixel points around the central pixel point in the existing Gaussian filtering method When the weight value is , only the distance between the surrounding pixels and the center pixel is considered, which leads to the problem that the pixels that are close to the center pixel but the color difference is too large will interfere with the depth value of the center pixel. .
本发明解决问题所采用的技术方案如下：The technical scheme adopted by the present invention to solve the problem is as follows:
第一方面，本发明实施例提供一种深度图像处理方法，其中，所述方法包括：In a first aspect, an embodiment of the present invention provides a depth image processing method, wherein the method includes:
获取深度图像中各个像素点对应的颜色信息，根据所述颜色信息确定所述各个像素点对应的目标像素点，并获取所述各个像素点与各自对应的目标像素点之间的颜色差值；Acquiring color information corresponding to each pixel in the depth image, determining the target pixel corresponding to each pixel according to the color information, and acquiring the color difference between each pixel and the corresponding target pixel;
获取所述各个像素点对应的目标像素点的原始深度值，根据所述各个像素点与各自对应的目标像素点之间的颜色差值、所述各个像素点对应的目标像素点的原始深度值，确定所述各个像素点对应的目标深度值；Obtain the original depth value of the target pixel point corresponding to each pixel point, according to the color difference value between each pixel point and the corresponding target pixel point, the original depth value of the target pixel point corresponding to each pixel point , determine the target depth value corresponding to each pixel point;
根据所述各个像素点对应的目标深度值生成目标深度图像。A target depth image is generated according to the target depth value corresponding to each pixel point.
第二方面，本发明实施例还提供一种深度图像处理装置，其中，所述装置包括：In a second aspect, an embodiment of the present invention further provides a depth image processing apparatus, wherein the apparatus includes:
确定模块，用于获取深度图像中各个像素点对应的颜色信息，根据所述颜色信息确定所述各个像素点对应的目标像素点，并获取所述各个像素点与各自对应的目标像素点之间的颜色差值；A determination module, configured to obtain the color information corresponding to each pixel in the depth image, determine the target pixel corresponding to each pixel according to the color information, and obtain the difference between each pixel and the corresponding target pixel color difference;
计算模块，用于获取所述各个像素点对应的目标像素点的原始深度值，根据所述各个像素点与各自对应的目标像素点之间的颜色差值、所述各个像素点对应的目标像素点的原始深度值，确定所述各个像素点对应的目标深度值；The calculation module is used to obtain the original depth value of the target pixel point corresponding to each pixel point, according to the color difference value between the each pixel point and the corresponding target pixel point, the target pixel corresponding to each pixel point The original depth value of the point, to determine the target depth value corresponding to each pixel point;
生成模块，用于根据所述各个像素点对应的目标深度值生成目标深度图像。A generating module is configured to generate a target depth image according to the target depth value corresponding to each pixel point.
第三方面，本发明实施例还提供一种终端，其中，所述终端包括有存储器，以及一个或者一个以上的程序，其中一个或者一个以上程序存储于存储器中，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于执行如上述任一所述的一种深度图像处理方法。In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more programs. Executing the one or more programs by the above processor includes performing a depth image processing method as described in any of the above.
第四方面，本发明实施例还提供一种计算机可读存储介质，其上存储有多条指令，所述指令适用于由处理器加载并执行，以实现上述任一所述的一种深度图像处理方法的步骤。In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a plurality of instructions are stored, and the instructions are suitable for being loaded and executed by a processor to realize any one of the above-mentioned depth images The steps of the processing method.
本发明的有益效果：本发明通过将深度图像中每一个像素点与周围像素点之间的颜色差异信息与周围像素点的深度值相结合来重新确定深度图像中的像素点的深度值，解决了现有的高斯滤波方法在确定各个像素点周围的像素点的权重值时，仅依靠距离的远近来确定各个像素点周围的像素点的权重值，因此有可能出现赋予颜色差异过大的像素点更高的权重值的情况，从而导致最终生成的各个像素点的深度值不准确的问题。Beneficial effects of the present invention: the present invention re-determines the depth value of the pixel points in the depth image by combining the color difference information between each pixel point in the depth image and the surrounding pixel points with the depth value of the surrounding pixel points, so as to solve the problem. When the existing Gaussian filtering method determines the weight value of the pixels around each pixel, it only depends on the distance to determine the weight value of the pixels around each pixel, so there may be pixels with too large color difference. In the case of a higher weight value, the resulting depth value of each pixel is inaccurate.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.
图1是本发明实施例提供的深度图像处理方法的流程示意图。FIG. 1 is a schematic flowchart of a depth image processing method provided by an embodiment of the present invention.
图2是本发明实施例提供的对深度图像进行处理前和处理后的效果图。FIG. 2 is an effect diagram before and after processing a depth image according to an embodiment of the present invention.
图3是本发明实施例提供的一个像素点对应的所有目标像素点组成的十字形交叉区域的示意图。FIG. 3 is a schematic diagram of a cross-shaped intersection area formed by all target pixel points corresponding to one pixel point according to an embodiment of the present invention.
图4是本发明实施例提供的深度图像处理装置的内部模块连接图。FIG. 4 is a connection diagram of internal modules of a depth image processing apparatus provided by an embodiment of the present invention.
图5是本发明实施例提供的终端的原理框图。FIG. 5 is a principle block diagram of a terminal provided by an embodiment of the present invention.
具体实施方式Detailed ways
为使本发明的目的、技术方案及优点更加清楚、明确，以下参照附图并举实施例对本发明进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the objectives, technical solutions and advantages of the present invention clearer and clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.
需要说明，若本发明实施例中有涉及方向性指示(诸如上、下、左、右、前、后……)，则该方向性指示仅用于解释在某一特定姿态(如附图所示)下各部件之间的相对位置关系、运动情况等，如果该特定姿态发生改变时，则该方向性指示也相应地随之改变。It should be noted that if there are directional indications (such as up, down, left, right, front, back, etc.) involved in the embodiments of the present invention, the directional indications are only used to explain a certain posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly.
随着科学技术和人类认识世界需求的不断发展，传统的机器视觉已经不能满足人们对于三维物体识别的要求。与灰度图像相比，深度图像具有物体三维特征信息，即深度信息。由于深度图像不受光源照射方向及物体表面的发射特性的影响，而且不存在阴影，所以可以更准确地表现物体目标表面的三维深度信息。With the continuous development of science and technology and the needs of human beings to understand the world, traditional machine vision can no longer meet people's requirements for three-dimensional object recognition. Compared with grayscale images, depth images have three-dimensional feature information of objects, that is, depth information. Since the depth image is not affected by the illumination direction of the light source and the emission characteristics of the surface of the object, and there is no shadow, the three-dimensional depth information of the target surface of the object can be more accurately represented.
然而直接采集到的深度图像通常都存在噪声的影响，从而导致深度图像边缘的准确性降低。现有技术中采集到深度图像以后，会对深度图像进行滤波处理，从而消除部分噪声点，提高深度图像边缘的准确性。现有的滤波方法包括均值滤波方法、方框滤波方法和高斯滤波方法，其中高斯滤波方法相比于前两种滤波方法来说对图像的模糊程度较小，更能够保持图像的整体细节，因此被广泛应用在深度图像的处理中。However, the directly acquired depth images usually have the influence of noise, which reduces the accuracy of the edge of the depth image. In the prior art, after the depth image is collected, the depth image is filtered, so as to eliminate some noise points and improve the accuracy of the edge of the depth image. The existing filtering methods include mean filtering method, box filtering method and Gaussian filtering method. Compared with the first two filtering methods, the Gaussian filtering method has less blur on the image and can better maintain the overall details of the image. It is widely used in the processing of depth images.
高斯滤波是一种线性平滑滤波，适用于消除高斯噪声，广泛应用于图像处理的减噪过程。通俗的讲，高斯滤波就是对整幅图像进行加权平均的过程，每一个像素点的深度值，都由其本身和邻域内的其他像素点的深度值经过加权平均后得到。然而，在现有技术中，高斯滤波仅依靠距离的远近来确定各个像素点周围的像素点的权重值。举例说明，假设像素点A对应的目标像素点包括：像素点B、C，其中像素点B与像素点A之间的距离较近，而像素点C与像素点A之间的距离较远，则在重新确定像素点A的深度值时，赋予像素点B较高的权重值，赋予像素点C较低的权重值。由于高斯滤波仅依靠距离的远近来确定各个像素点周围的像素点的权重值，因此有可能出现赋予颜色差异过大的像素点更高的权重值的情况，从而导致最终生成的各个像素点的深度值不准确的问题。Gaussian filtering is a linear smoothing filter, suitable for removing Gaussian noise, and is widely used in the noise reduction process of image processing. In layman's terms, Gaussian filtering is a process of weighted averaging of the entire image. The depth value of each pixel is obtained by weighted averaging of the depth values of itself and other pixels in the neighborhood. However, in the prior art, Gaussian filtering only relies on the distance to determine the weight value of the pixel points around each pixel point. For example, it is assumed that the target pixel points corresponding to the pixel point A include: pixel points B and C, wherein the distance between the pixel point B and the pixel point A is relatively short, and the distance between the pixel point C and the pixel point A is relatively long, Then, when the depth value of pixel A is re-determined, a higher weight value is assigned to pixel B, and a lower weight value is assigned to pixel C. Since Gaussian filtering only relies on the distance to determine the weight value of the pixels around each pixel, it is possible to assign a higher weight value to the pixel with too large color difference, resulting in the final generation of each pixel. The problem of inaccurate depth values.
针对现有技术的上述缺陷，本发明提供了一种深度图像处理方法，通过获取深度图像中各个像素点对应的颜色信息，根据所述颜色信息确定所述各个像素点对应的目标像素点，并获取所述各个像素点与各自对应的目标像素点之间的颜色差值。然后，获取所述各个像素点对应的目标像素点的原始深度值，根据所述各个像素点与各自对应的目标像素点之间的颜色差值、所述各个像素点对应的目标像素点的原始深度值，确定所述各个像素点对应的目标深度值。最后，根据所述各个像素点对应的目标深度值生成目标深度图像。本发明中会将深度图像中每一个像素点与周围像素点之间的颜色差异信息与周围像素点的深度值相结合来重新确定深度图像中的像素点的深度值，解决了现有的高斯滤波方法在确定各个像素点周围的像素点的权重值时，仅依靠距离的远近来确定各个像素点周围的像素点的权重值，因此有可能出现赋予颜色差异过大的像素点更高的权重值的情况，从而导致最终生成的各个像素点的深度值不准确的问题。In view of the above-mentioned defects of the prior art, the present invention provides a depth image processing method, by acquiring the color information corresponding to each pixel point in the depth image, determining the target pixel point corresponding to each pixel point according to the color information, and The color difference values between the respective pixel points and the respective corresponding target pixel points are acquired. Then, obtain the original depth value of the target pixel point corresponding to each pixel point, according to the color difference value between each pixel point and the corresponding target pixel point, the original depth value of the target pixel point corresponding to each pixel point Depth value, to determine the target depth value corresponding to each pixel point. Finally, a target depth image is generated according to the target depth value corresponding to each pixel point. In the present invention, the color difference information between each pixel in the depth image and the surrounding pixels is combined with the depth value of the surrounding pixels to re-determine the depth value of the pixel in the depth image, which solves the problem of the existing Gaussian When the filtering method determines the weight value of the pixels around each pixel, it only depends on the distance to determine the weight value of the pixels around each pixel, so it may appear that the pixels with too large color difference are given a higher weight. value, resulting in an inaccurate depth value of each pixel that is finally generated.
如图1所示，所述方法包括如下步骤：As shown in Figure 1, the method includes the following steps:
步骤S100、获取深度图像中各个像素点对应的颜色信息，根据所述颜色信息确定所述各个像素点对应的目标像素点，并获取所述各个像素点与各自对应的目标像素点之间的颜色差值。Step S100: Obtain color information corresponding to each pixel in the depth image, determine the target pixel corresponding to each pixel according to the color information, and obtain the color between each pixel and the corresponding target pixel. difference.
本实施例的处理对象是深度图像，在对所述深度图像进行处理之前，本实施例需要先采用具有双摄技术移动终端获取深度图像以及所述深度图像对应的RGB图像。即所述终端设备采用的是彩色相机(主相机)+黑白相机(副相机)或彩色相机(主相机)+彩色相机(副相机)的配置，其中基于主副相机获取的图像，利用双目视差算法获取所述深度图像，进而得到图像的景深信息，而主相机可以用于获取所述RGB图像，进而得到图像的颜色信息。在一种实现方式中，获取到RGB图像和其对应的深度图像以后，可以首先对两者进行去噪平滑处理，例如采用窗口大小为3x3的中值滤波对RGB图像和其对应的深度图像进行滤波处理，从而减少两幅图像数据中的噪声点对后续步骤的影响。The processing object in this embodiment is a depth image. Before processing the depth image, this embodiment needs to use a mobile terminal with dual-camera technology to acquire the depth image and the RGB image corresponding to the depth image. That is, the terminal device adopts the configuration of color camera (main camera) + black and white camera (secondary camera) or color camera (main camera) + color camera (secondary camera). The parallax algorithm obtains the depth image, and then obtains the depth information of the image, and the main camera can be used to obtain the RGB image, and then obtains the color information of the image. In an implementation manner, after the RGB image and its corresponding depth image are obtained, denoising and smoothing can be performed on the two first, for example, the RGB image and its corresponding depth image are processed by median filtering with a window size of 3×3. Filter processing to reduce the influence of noise points in the two image data on subsequent steps.
本实施例需要确定所述深度图像中各个像素点周围与其深度值密切相关的目标像素点，进而通过各个像素点分别对应的目标像素点与各自的颜色差值重新确定各个像素点的深度值，从而减少深度图像的噪声点。举例说明，假设深度图像中像素点A的目标像素点为像素点B、C，则需要获取像素点A与像素点B的颜色差值，以及像素点A与像素点C的颜色差值，然后根据这两个颜色差值重新确定像素点A的深度值。In this embodiment, it is necessary to determine the target pixel points around each pixel point in the depth image that are closely related to the depth value thereof, and then re-determine the depth value of each pixel point according to the target pixel point corresponding to each pixel point and the respective color difference value, Thereby reducing the noise points of the depth image. For example, assuming that the target pixels of pixel A in the depth image are pixels B and C, it is necessary to obtain the color difference between pixel A and pixel B, as well as the color difference between pixel A and pixel C, and then The depth value of the pixel point A is re-determined according to the two color difference values.
在一种实现方式中，所述步骤S100具体包括如下步骤：In an implementation manner, the step S100 specifically includes the following steps:
步骤S110、获取所述深度图像对应的RGB图像，根据所述RGB图像获取所述深度图像中各个像素点对应的颜色信息；Step S110, obtaining the RGB image corresponding to the depth image, and obtaining color information corresponding to each pixel in the depth image according to the RGB image;
步骤S120、将所述各个像素点中的任意一个作为中心像素点，将在所述深度图像中与所述中心像素点位于同一列以及同一行的像素点作为所述中心像素点对应的邻域像素点；Step S120, take any one of the respective pixel points as the center pixel point, and take the pixel points located in the same column and the same row as the center pixel point in the depth image as the neighborhood corresponding to the center pixel point. pixel;
步骤S120、根据所述中心像素点的颜色信息、所述邻域像素点的颜色信息确定颜色阈值，根据所述中心像素点的颜色信息、所述邻域像素点的颜色信息以及所述颜色阈值对所述邻域像素点进行筛选操作，并将筛选出的邻域像素点作为所述中心像素点对应的目标像素点；Step S120: Determine a color threshold according to the color information of the center pixel point and the color information of the neighbor pixel point, and determine the color threshold value according to the color information of the center pixel point, the color information of the neighbor pixel point, and the color threshold Perform a screening operation on the neighborhood pixels, and use the filtered neighborhood pixels as the target pixel corresponding to the center pixel;
步骤S130、获取所述中心像素点对应的目标像素点的颜色信息，根据所述中心像素点的颜色信息和所述中心像素点对应的目标像素点的颜色信息，确定所述中心像素点与对应的目标像素点之间的颜色差值，并确定所述各个像素点与各自对应的目标像素点之间的颜色差值。Step S130: Obtain the color information of the target pixel corresponding to the central pixel, and determine the central pixel and the corresponding target pixel according to the color information of the central pixel and the target pixel corresponding to the central pixel. The color difference value between the target pixel points is determined, and the color difference value between each pixel point and the corresponding target pixel point is determined.
具体地，本实施例中需要重新确定深度图像中的每一个像素点的深度值，确定每一个像素点深度值的方法均相同，因此本实施例以确定一个像素点的深度值的方法为例。本实施例将当前需要处理的像素点作为中心像素点，首先需要在该中心像素点的周围确定该像素点所对应的目标像素点。具体地，本实施例将与该中心像素点位于同一列以及同一行的像素点作为该中心像素点所对应的邻域像素点。然后再在这些邻域像素点中筛选出该中心像素点对应的目标像素点。具体地，本实施例预先设定了一个颜色阈值作为筛选条件，然后本实施例需要获取中心像素点的颜色信息以及该中心像素点对应的邻域像素点的颜色信息，并根据该中心像素点的颜色信息、该邻域像素点的颜色信息以及颜色阈值在该中心像素点对应的邻域像素点中进行筛选，筛选出的邻域像素点即为该中心像素点对应的目标像素点，通过该方法得到深度图像中每一个像素点对应的目标像素点，可以理解的是，由于每一个像素点的目标像素点均为该像素点同一行或者同一列上的像素点，因此每一个像素点对应的所有目标像素点所构成的区域是一个十字形的交叉区域(如图3所示)。Specifically, in this embodiment, the depth value of each pixel in the depth image needs to be re-determined, and the method for determining the depth value of each pixel is the same, so the method for determining the depth value of one pixel in this embodiment is taken as an example . In this embodiment, the pixel point to be processed currently is used as the center pixel point. First, the target pixel point corresponding to the pixel point needs to be determined around the center pixel point. Specifically, in this embodiment, the pixel points located in the same column and the same row as the center pixel point are used as the neighborhood pixel points corresponding to the center pixel point. Then, the target pixel corresponding to the central pixel is filtered out from these neighborhood pixels. Specifically, this embodiment pre-sets a color threshold as a screening condition, and then this embodiment needs to obtain the color information of the center pixel and the color information of the neighboring pixels corresponding to the center pixel, and according to the center pixel The color information, the color information of the neighborhood pixel point and the color threshold are screened in the neighborhood pixel points corresponding to the center pixel point, and the filtered neighborhood pixel point is the target pixel point corresponding to the center pixel point. This method obtains the target pixel corresponding to each pixel in the depth image. It can be understood that since the target pixel of each pixel is a pixel on the same row or column of the pixel, each pixel is The area formed by all the corresponding target pixels is a cross-shaped intersection area (as shown in Figure 3).
在一种实现方式中，为了确定进行筛选时采用的颜色阈值，本实施例首先根据该中心像素点的颜色信息和该中心像素点对应的邻域像素点的颜色信息，计算出该中心像素点和其对应的邻域像素点之间的颜色差值，然后获取该中心像素点与其对应的邻域像素点之间的距离值，根据所述距离值确定进行筛选时使用的颜色阈值。简言之，本实施例中预先设定的颜色阈值可以不止一个，且根据不同邻域像素点与该中心像素点的距离远近可以采用不同的颜色阈值对该中心像素点对应的邻域像素点进行筛选。In an implementation manner, in order to determine the color threshold used for screening, this embodiment first calculates the center pixel point according to the color information of the center pixel point and the color information of the neighboring pixel point corresponding to the center pixel point and the color difference value between the corresponding neighboring pixel points, and then obtain the distance value between the central pixel point and its corresponding neighboring pixel points, and determine the color threshold used for screening according to the distance value. In short, there can be more than one preset color threshold in this embodiment, and different color thresholds can be used according to the distance between different neighborhood pixels and the center pixel. to filter.
具体地，本实施例预先设定了两个颜色阈值：第一颜色阈值和第二颜色阈值，其中所述第一颜色阈值的数值大于所述第二颜色阈值的数值。为了确定每一个邻域像素点在筛选过程中应该使用何种颜色阈值，本实施例预先设定了第一距离阈值。具体地，在获取到该中心像素点与其对应的邻域像素点之间的距离值以后，将该距离值与该第一距离阈值进行比较，当所述距离值小于或者等于所述第一距离阈值时，将第一颜色阈值作为当前使用的颜色阈值；当所述距离值大于所述第一距离阈值时，将第二颜色阈值作为当前使用的颜色阈值。简言之，本实施例中对与中心像素点相距较近的邻域像素点采用更大的颜色阈值进行筛选，对于中心像素点相距较远的邻域像素点采用更小的颜色阈值进行筛选。Specifically, this embodiment presets two color thresholds: a first color threshold and a second color threshold, wherein the value of the first color threshold is greater than the value of the second color threshold. In order to determine which color threshold should be used for each neighborhood pixel in the screening process, this embodiment presets a first distance threshold. Specifically, after obtaining the distance value between the central pixel point and its corresponding neighboring pixel points, the distance value is compared with the first distance threshold, and when the distance value is less than or equal to the first distance When the threshold is used, the first color threshold is used as the currently used color threshold; when the distance value is greater than the first distance threshold, the second color threshold is used as the currently used color threshold. In short, in this embodiment, a larger color threshold is used to filter the neighboring pixels that are closer to the center pixel, and a smaller color threshold is used to filter the neighboring pixels that are far away from the center pixel. .
然后，将该中心像素点与其对应的邻域像素点之间的颜色差值与确定好的颜色阈值进行比较，当中心像素点与其对应的邻域像素点之间的颜色差值小于或者等于该颜色阈值的时候，表示该邻域像素点与该中心像素点的颜色差异在可接受范围内，因此可以将所述邻域像素点作为目标像素点；当中心像素点与邻域像素点的颜色差值大于该颜色阈值的时候，表示该邻域像素点与中心像素点的颜色差异过大，因此将该邻域像素点排除，并停止继续筛选。Then, compare the color difference between the center pixel and its corresponding neighborhood pixels with the determined color threshold, when the color difference between the center pixel and its corresponding neighborhood pixels is less than or equal to the color difference When the color threshold is set, it means that the color difference between the neighboring pixel and the central pixel is within an acceptable range, so the neighboring pixel can be used as the target pixel; when the color of the central pixel and the neighboring pixel is When the difference is greater than the color threshold, it means that the color difference between the neighborhood pixel and the center pixel is too large, so the neighborhood pixel is excluded, and the screening is stopped.
举例说明，假设深度图像上存在像素点A，并将与像素点A位于同一行的像素点B、C、D作为像素点A的邻域像素点，且这四个像素的排列顺序从左到右依次为A、B、C、D。假设像素点A作为中心像素，首先将与像素点A相邻的像素点B作为第一个进行筛选的邻域像素点。获取像素点A与像素点B的颜色信息(如RGB值)来计算像素点A与像素点B的颜色差值，并计算像素点A与像素点B的距离值，并将该距离值与第一距离阈值进行比较，判断出像素点B与像素点A的距离值小于第一距离阈值，则将像素点A与像素点B的颜色差值与第一颜色阈值进行比较，判断出像素点A与像素点B的颜色差值小于第一颜色阈值，则将像素点B作为一个目标像素点。然后将与像素点B相邻的像素点C作为下一个进行筛选的邻域像素点，当像素点C不满足成为目标像素点的时候，则将像素点C排除并停止筛选，即不会继续对像素点D进行筛选，因此像素点A的目标像素点只有像素点B；只有当像素点C筛选后也可以作为目标像素点的时候，才能继续筛选，并将像素点D作为下一个进行筛选的邻域像素点。For example, it is assumed that pixel A exists in the depth image, and pixels B, C, and D located in the same row as pixel A are used as the neighboring pixels of pixel A, and the arrangement order of these four pixels is from left to On the right are A, B, C, D. Assuming that pixel A is used as the center pixel, firstly, the pixel B adjacent to pixel A is used as the first neighborhood pixel to be screened. Obtain the color information (such as RGB value) of pixel point A and pixel point B to calculate the color difference between pixel point A and pixel point B, and calculate the distance value between pixel point A and pixel point B, and compare the distance value with the first A distance threshold is compared, and it is determined that the distance between pixel B and pixel A is less than the first distance threshold, then the color difference between pixel A and pixel B is compared with the first color threshold, and pixel A is determined. If the color difference value from the pixel point B is less than the first color threshold, the pixel point B is used as a target pixel point. Then the pixel point C adjacent to the pixel point B is used as the next neighborhood pixel point for screening. When the pixel point C does not meet the target pixel point, the pixel point C is excluded and the screening is stopped, that is, it will not continue. Pixel point D is screened, so the target pixel point of pixel point A is only pixel point B; only when pixel point C can also be used as a target pixel point after screening, can continue to screen, and pixel point D is selected as the next one the neighborhood pixels.
在一种实现方式中，本实施例还设置了第二距离阈值以实现对每个像素点筛选目标像素点的范围进行限定，其中所述第二距离阈值大于所述第一距离阈值。具体地，当获取到中心像素点与其对应的邻域像素点之间的距离值，且判断出该距离值大于所述第一距离阈值的时候，还需要将该距离值与该第二距离阈值进行比较，当该距离值小于或者等于该第二距离阈值的时候，表示该邻域像素点与该中心像素点相隔的距离并不远，可以进一步根据该邻域像素点与该中心像素点之间的颜色差值来判断该邻域像素点是否可以成为该中心像素点对应的目标像素点；当该距离值大于该第二距离阈值的时候，表示该邻域像素点与其对应的中心像素点相隔太远了，则将该邻域像素点不能作为该中心像素点对应的目标像素点，并且停止筛选，即对该邻域像素点之后的其他邻域像素点不再进行筛选。In an implementation manner, this embodiment further sets a second distance threshold to limit the range of screening target pixels for each pixel, wherein the second distance threshold is greater than the first distance threshold. Specifically, when the distance value between the central pixel point and its corresponding neighboring pixel points is obtained, and it is determined that the distance value is greater than the first distance threshold, the distance value and the second distance threshold also need to be For comparison, when the distance value is less than or equal to the second distance threshold, it means that the distance between the neighborhood pixel point and the center pixel point is not far, and the distance between the neighborhood pixel point and the center pixel point can be further based on the distance between the neighborhood pixel point and the center pixel point. When the distance value is greater than the second distance threshold, it means that the neighborhood pixel and its corresponding center pixel If the distance is too far, the neighborhood pixel point cannot be used as the target pixel point corresponding to the center pixel point, and the screening is stopped, that is, the other neighborhood pixel points after the neighborhood pixel point are no longer screened.
在一种实现方式中，由于本实施例中必须是对深度图像中的像素点进行筛选，因此深度图像的尺寸数据对筛选各个像素点分别对应的目标像素点的范围也具有一定的局限性，换言之，每一个像素点在其对应的邻域像素点中筛选目标像素点的时候，筛选范围会受到深度图像的高度和宽度的影响。因此在确定每一个像素点对应的目标像素点时还需要考虑到原始深度图像的高度数据和宽度数据。为了便于理解本实施例中筛选目标像素点的过程，本实施例还提供了一个详细步骤图：In an implementation manner, since the pixel points in the depth image must be screened in this embodiment, the size data of the depth image also has certain limitations for screening the range of the target pixel points corresponding to each pixel point, In other words, when each pixel selects the target pixel in its corresponding neighborhood pixels, the filtering range will be affected by the height and width of the depth image. Therefore, the height data and width data of the original depth image also need to be considered when determining the target pixel corresponding to each pixel. In order to facilitate the understanding of the process of screening target pixels in this embodiment, this embodiment also provides a detailed step diagram:
首先，本实施例预先设置了颜色阈值Cth1,Cth2及距离阈值Sth1,Sth2，其中，Cth1＞Cth2，Sth1＞Sth2。First, the present embodiment presets color thresholds Cth1 , Cth2 and distance thresholds Sth1 , Sth2 , where Cth1 >Cth2 and Sth1 >Sth2 .
由于本实施例是将与中心像素点位于同一列和同一行的像素点作为该中心像素点的邻域像素点，因此本实施例可以分别确定中心像素点上、下、左、右方向上的目标像素点，进而得到该中心像素点对应的所有目标像素点。需要注意的是，在实际应用中深度图像对应的坐标系中的竖轴与常用坐标系中的竖轴的方向相反。Since this embodiment uses the pixel points located in the same column and the same row as the central pixel point as the adjacent pixel points of the central pixel point, this embodiment can respectively determine the up, down, left and right directions of the central pixel point. The target pixel point is obtained, and then all the target pixel points corresponding to the central pixel point are obtained. It should be noted that, in practical applications, the vertical axis in the coordinate system corresponding to the depth image is in the opposite direction to the vertical axis in the common coordinate system.
1)确定与中心像素点位于同一列，且位于该中心像素点的上方的目标像素点的详细步骤如下：1) The detailed steps for determining the target pixel point located in the same column as the center pixel point and located above the center pixel point are as follows:
步骤1：设置偏移量d＝1，中心像素点为V(x,y)。Step 1: Set the offset d=1, and the center pixel is V(x,y).
步骤2：将中心像素点的纵坐标值y与偏移量相减得到进行筛选的邻域像素点的纵坐标y-d，判断y-d与0的大小关系，若y-d<0，表示该邻域像素点位于深度图像对应的范围外，则跳至步骤5；若y-d>＝0，表示该邻域像素点位于深度图像对应的范围内，则进入步骤3。Step 2: Subtract the ordinate value y of the center pixel point and the offset to obtain the ordinate y-d of the neighborhood pixel point to be screened, and determine the size relationship between y-d and 0. If y-d<0, it means the neighborhood pixel point If it is outside the range corresponding to the depth image, then skip to step 5; if y-d>=0, it means that the neighborhood pixel is located in the range corresponding to the depth image, then go to step 3.
步骤3：获取该邻域像素点与中心像素点的距离值D以及颜色差值diff，其中diff＝|V(x,y)-V(x,y-d)|，V(x,y)表示与深度图像对应的RGB图像中坐标为(x,y)的像素点的RGB值，V(x,y-d)表示RGB图像中坐标为(x,y-d)的像素点的RGB值。然后判断D与Sth1、Sth2之间的大小关系，当D≤Sth1时，判断diff与Cth1的大小关系，当diff≤Cth1，进入步骤4，当diff>Cth1，跳至步骤5；当Sth1<d<Sth2时，判断diff与Cth2的大小关系，若diff≤Cth2，进入步骤4，若diff>Cth2，跳至步骤5。Step 3: Obtain the distance value D between the neighborhood pixel point and the center pixel point and the color difference value diff, where diff=|V(x,y)-V(x,yd)|, V(x,y) represents the The RGB value of the pixel with coordinates (x, y) in the RGB image corresponding to the depth image, and V(x, yd) represents the RGB value of the pixel with coordinates (x, yd) in the RGB image. Then judge the size relationship between D and Sth1 and Sth2 , when D≤Sth1 , judge the size relationship between diff and Cth1 , when diff≤Cth1 , go to step 4, when diff>Cth1 , skip to step 5; When Sth1 <d<Sth2 , determine the size relationship between diff and Cth2 , if diff≤Cth2 , go to step 4, if diff>Cth2 , skip to step 5.
步骤4：d＝d+1，跳至步骤2。Step 4: d=d+1, skip to step 2.
步骤5：d＝d-1，将位于中心像素点上方且纵坐标值与中心像素点的纵坐标值的差值小于或者等于d的邻域像素点作为该中心像素点的目标像素点。Step 5: d=d-1, take the neighborhood pixel point located above the center pixel point and the difference between the ordinate value and the ordinate value of the center pixel point is less than or equal to d as the target pixel point of the center pixel point.
确定与中心像素点位于同一列，且位于该中心像素点的下方的目标像素点的详细步骤如下：The detailed steps for determining the target pixel located in the same column as the center pixel and below the center pixel are as follows:
步骤1：设置偏移量d＝1，中心像素点为V(x,y)。Step 1: Set the offset d=1, and the center pixel is V(x,y).
步骤2：将中心像素点的纵坐标值y与偏移量相减得到进行筛选的邻域像素点的纵坐标y+d，判断y与原始深度图像的高度h的大小关系，若y+d>＝h，表示该邻域像素点位于深度图像对应的范围外，则跳至步骤5；若y+d>＝h，表示该邻域像素点位于深度图像对应的范围内，则进入步骤3。Step 2: Subtract the ordinate value y of the central pixel point and the offset to obtain the ordinate y+d of the neighborhood pixel point to be screened, and determine the size relationship between y and the height h of the original depth image, if y+d >=h, it means that the neighborhood pixel is outside the corresponding range of the depth image, then skip to step 5; if y+d>=h, it means that the neighborhood pixel is located in the corresponding range of the depth image, then go to step 3 .
步骤3：获取该邻域像素点与中心像素点的距离值D以及颜色差值diff，其中diff＝|V(x,y)-V(x,y+d)|，其中V(x,y)表示与深度图像对应的RGB图像中坐标为(x,y)的像素点的RGB值，V(x,y+d)表示RGB图像中坐标为(x,y+d)的像素点的RGB值。然后判断D与Sth1、Sth2之间的大小关系，当D≤Sth1时，判断diff与Cth1的大小关系，当diff≤Cth1，进入步骤4，当diff>Cth1，跳至步骤5；当Sth1<d<Sth2时，判断diff与Cth2的大小关系，若diff≤Cth2，进入步骤4，若diff>Cth2，跳至步骤5。Step 3: Obtain the distance value D between the neighborhood pixel point and the center pixel point and the color difference value diff, where diff=|V(x,y)-V(x,y+d)|, where V(x,y ) represents the RGB value of the pixel with coordinates (x, y) in the RGB image corresponding to the depth image, and V(x, y+d) represents the RGB value of the pixel with coordinates (x, y+d) in the RGB image value. Then judge the size relationship between D and Sth1 and Sth2 , when D≤Sth1 , judge the size relationship between diff and Cth1 , when diff≤Cth1 , go to step 4, when diff>Cth1 , skip to step 5; When Sth1 <d<Sth2 , determine the size relationship between diff and Cth2 , if diff≤Cth2 , go to step 4, if diff>Cth2 , skip to step 5.
步骤4：d＝d+1，跳至步骤2。Step 4: d=d+1, skip to step 2.
步骤5：d＝d-1，将位于中心像素点下方且纵坐标值与中心像素点的纵坐标值的差值小于或者等于d的邻域像素点作为该中心像素点的目标像素点。Step 5: d=d-1, take the neighborhood pixel point below the center pixel point and the difference between the ordinate value and the center pixel point's ordinate value less than or equal to d as the target pixel point of the center pixel point.
确定与中心像素点位于同一行，且位于该中心像素点的左侧的目标像素点的详细步骤如下：The detailed steps for determining the target pixel on the same row as the central pixel and to the left of the central pixel are as follows:
步骤1：设置偏移量d＝1，中心像素点为V(x,y)。Step 1: Set the offset d=1, and the center pixel is V(x,y).
步骤2：将中心像素点的横坐标值x与偏移量相减得到进行筛选的邻域像素点的纵坐标x-d，判断x-d与0的大小关系，若x-d<0，表示该邻域像素点位于深度图像对应的范围外，则跳至步骤5；若x-d>＝0，表示该邻域像素点位于深度图像对应的范围内，则进入步骤3。Step 2: Subtract the abscissa value x of the center pixel point and the offset to obtain the ordinate x-d of the neighborhood pixel point to be screened, and judge the size relationship between x-d and 0. If x-d<0, it means the neighborhood pixel point If it is outside the range corresponding to the depth image, then skip to step 5; if x-d>=0, it means that the neighborhood pixel is located in the range corresponding to the depth image, and then go to step 3.
步骤3：获取该邻域像素点与中心像素点的距离值D以及颜色差值diff其中diff＝|V(x,y)-V(x-d,y)|，其中V(x,y)表示与深度图像对应的RGB图像中坐标为(x,y)的像素点的RGB值，V(x-d,y)表示RGB图像中坐标为(x-d,y)的像素点的RGB值。然后，判断D与Sth1、Sth2之间的大小关系，当D≤Sth1时，判断diff与Cth1的大小关系，当diff≤Cth1，进入步骤4，当diff>Cth1，跳至步骤5；当Sth1<d<Sth2时，判断diff与Cth2的大小关系，若diff≤Cth2，进入步骤4，若diff>Cth2，跳至步骤5。Step 3: Obtain the distance value D between the neighborhood pixel point and the center pixel point and the color difference value diff where diff=|V(x,y)-V(xd,y)|, where V(x,y) represents the difference between The RGB value of the pixel with coordinates (x, y) in the RGB image corresponding to the depth image, and V(xd, y) represents the RGB value of the pixel with coordinates (xd, y) in the RGB image. Then, judge the size relationship between D and Sth1 and Sth2 , when D≤Sth1 , judge the size relationship between diff and Cth1 , when diff≤Cth1 , go to step 4, when diff>Cth1 , skip to Step 5: When Sth1 <d<Sth2 , determine the size relationship between diff and Cth2 , if diff≤Cth2 , go to step 4, if diff>Cth2 , skip to step 5.
步骤4：d＝d+1，跳至步骤2。Step 4: d=d+1, skip to step 2.
步骤5：d＝d-1，将位于中心像素点左侧且横坐标值与中心像素点的横坐标值的差值小于或者等于d的邻域像素点作为该中心像素点的目标像素点。Step 5: d=d-1, take the neighbor pixel point located on the left side of the center pixel point and the difference between the abscissa value and the abscissa value of the center pixel point is less than or equal to d as the target pixel point of the center pixel point.
确定与中心像素点位于同一行，且位于该中心像素点的右侧的目标像素点的详细步骤如下：The detailed steps for determining the target pixel located on the same row as the center pixel and to the right of the center pixel are as follows:
步骤1：设置偏移量d＝1，中心像素点为V(x,y)。Step 1: Set the offset d=1, and the center pixel is V(x,y).
步骤2：将中心像素点的横坐标值x与偏移量相加得到进行筛选的邻域像素点的纵坐标x+d，判断x+d与原始深度图像的宽度w的大小关系，若x+d>＝w，表示该邻域像素点位于深度图像对应的范围外，则跳至步骤5；若x+d<w，表示该邻域像素点位于深度图像对应的范围内，则进入步骤3。Step 2: Add the abscissa value x of the center pixel and the offset to get the ordinate x+d of the neighborhood pixel to be screened, and determine the relationship between x+d and the width w of the original depth image, if x +d>=w, indicating that the neighborhood pixel is outside the corresponding range of the depth image, then skip to step 5; if x+d<w, indicating that the neighborhood pixel is located within the corresponding range of the depth image, then go to step 5 3.
步骤3：获取该邻域像素点与中心像素点的距离值D以及颜色差值diff其中diff＝|V(x,y)-V(x+d,y)|，其中V(x,y)表示与深度图像对应的RGB图像中坐标为(x,y)的像素点的RGB值，V(x+d,y)表示RGB图像中坐标为(x+d,y)的像素点的RGB值。判断D与Sth1、Sth2之间的大小关系，当D≤Sth1时，判断diff与Cth1的大小关系，当diff≤Cth1，进入步骤4，当diff>Cth1，跳至步骤5；当Sth1<d<Sth2时，判断diff与Cth2的大小关系，若diff≤Cth2，进入步骤4，若diff>Cth2，跳至步骤5。Step 3: Obtain the distance value D of the neighborhood pixel point and the center pixel point and the color difference value diff where diff=|V(x,y)-V(x+d,y)|, where V(x,y) Represents the RGB value of the pixel with coordinates (x, y) in the RGB image corresponding to the depth image, and V(x+d, y) represents the RGB value of the pixel with coordinates (x+d, y) in the RGB image . Determine the size relationship between D and Sth1 and Sth2 , when D≤Sth1 , determine the size relationship between diff and Cth1 , when diff≤Cth1 , go to step 4, when diff>Cth1 , skip to step 5 ; When Sth1 <d<Sth2 , judge the size relationship between diff and Cth2 , if diff≤Cth2 , go to step 4, if diff>Cth2 , skip to step 5.
步骤4：d＝d+1，跳至步骤2。Step 4: d=d+1, skip to step 2.
步骤5：d＝d-1，将位于中心像素点右侧且横坐标值与中心像素点的横坐标值的差值小于或者等于d的邻域像素点作为该中心像素点的目标像素点。Step 5: d=d-1, take the neighbor pixel point located on the right side of the center pixel point and the difference between the abscissa value and the abscissa value of the center pixel point is less than or equal to d as the target pixel point of the center pixel point.
获取到深度图像中各个像素点上、下、左、右四个方向上分别对应的目标像素点以后，即相当于确定了各个像素点对应的所有目标像素点，为了重新确定各个像素点的深度值，如图1所示，该方法还包括如下步骤：After obtaining the corresponding target pixels in the up, down, left, and right directions of each pixel in the depth image, it is equivalent to determining all the target pixels corresponding to each pixel. In order to re-determine the depth of each pixel value, as shown in Figure 1, the method also includes the following steps:
步骤S200、获取所述各个像素点对应的目标像素点的原始深度值，根据所述各个像素点与各自对应的目标像素点之间的颜色差值、所述各个像素点对应的目标像素点的原始深度值，确定所述各个像素点对应的目标深度值。Step S200: Obtain the original depth value of the target pixel point corresponding to each pixel point, according to the color difference value between the each pixel point and the corresponding target pixel point, and the target pixel point corresponding to each pixel point. The original depth value is used to determine the target depth value corresponding to each pixel point.
简单来说，本实施例中在重新确定深度图像中每一个像素点的深度值的时候，会参考每一个像素点对应的所有目标像素点的原始深度值，以及每一个像素点和其对应的各个目标像素点之间的颜色差异。To put it simply, in this embodiment, when re-determining the depth value of each pixel in the depth image, the original depth values of all target pixels corresponding to each pixel, as well as each pixel and its corresponding The color difference between each target pixel.
在一种实现方式中，所述步骤S200具体地包括如下步骤：In an implementation manner, the step S200 specifically includes the following steps:
步骤S210根据所述各个像素点与各自对应的目标像素点之间的颜色差值，确定所述各个像素点对应的目标像素点的权重值；Step S210 determines the weight value of the target pixel corresponding to each pixel according to the color difference between each pixel and the corresponding target pixel;
步骤S220、根据所述各个像素点对应的目标像素点的原始深度值和所述各个像素点对应的目标像素点的权重值，确定所述各个像素点对应的目标深度值。Step S220: Determine the target depth value corresponding to each pixel point according to the original depth value of the target pixel point corresponding to each pixel point and the weight value of the target pixel point corresponding to each pixel point.
具体地，本实施例在重新确定某一个像素点的深度值时，需要获取该像素点对应的每一个目标像素点的原始深度值，并根据该像素点对应的每一个目标像素点与该像素点之间的颜色差值，确定该像素点对应的每一个目标像素点的权重值wi，计算权重值wi的公式如下所示：Specifically, when the depth value of a certain pixel is re-determined in this embodiment, the original depth value of each target pixel corresponding to the pixel needs to be acquired, and each target pixel corresponding to the pixel needs to be compared with the pixel. The color difference between the points determines the weight valuewi of each target pixel corresponding to the pixel point, and the formula for calculating the weight valuewi is as follows:
wi＝Gauss(|V(i,y)-V(x,y)|)wi =Gauss(|V(i,y)-V(x,y)|)
其中，V(x,y)表示横坐标为x，纵坐标为y的中心像素点的RGB值，(i,y)表示目标像素点的坐标数据，V(i,y)表示目标像素点的RGB值，Gauss表示高斯函数。Among them, V(x, y) represents the RGB value of the center pixel with the horizontal coordinate of x and the vertical coordinate of y, (i, y) representing the coordinate data of the target pixel point, and V(i, y) representing the target pixel point. RGB value, Gauss represents the Gaussian function.
然后根据该像素点对应的每一个目标像素点的权重值与该像素点对应的每一个目标像素点的原始深度值，重新确定该像素点对应的目标深度值，并通过该方法确定深度图像中的每一个像素点的目标深度值。概括地讲，本实施例中重新确定像素点的深度值的过程实际上相当于对深度图像进行加权滤波处理，其中处理方法与高斯滤波相似，都需要基于每一个像素点周围的其他像素点来确定每一个像素点的新的深度值，并在计算的时候给每一个像素点周围的其他像素点按照一定条件赋予一定的权重值。不同之处则在于，本发明中在确定各个像素点对应的目标像素点的权重值时，会参考颜色差异因素。举例说明，假设像素点A对应的目标像素点包括：像素点B、C，其中像素点B与像素点A之间的颜色差异较小，而像素点C与像素点A之间的颜色差异较大，则在确定像素点A的目标深度值时，赋予像素点B较高的权重值，赋予像素点C较低的权重值。而现有的高斯滤波在确定各个像素点对应的目标像素点的权重值时，仅考虑到距离远近因素，因此本实施例最终生成的目标深度值更加准确。Then, according to the weight value of each target pixel corresponding to the pixel and the original depth value of each target pixel corresponding to the pixel, the target depth value corresponding to the pixel is re-determined, and the depth image is determined by this method. The target depth value of each pixel. In general, the process of re-determining the depth value of a pixel in this embodiment is actually equivalent to performing a weighted filtering process on the depth image, in which the processing method is similar to Gaussian filtering. Determine the new depth value of each pixel, and give certain weights to other pixels around each pixel according to certain conditions during calculation. The difference is that, in the present invention, when determining the weight value of the target pixel point corresponding to each pixel point, the color difference factor is referred to. For example, it is assumed that the target pixel points corresponding to pixel point A include: pixel points B and C, wherein the color difference between pixel point B and pixel point A is small, and the color difference between pixel point C and pixel point A is relatively small. When the target depth value of pixel point A is determined, a higher weight value is assigned to pixel point B, and a lower weight value is assigned to pixel point C. However, the existing Gaussian filter only considers the distance factor when determining the weight value of the target pixel corresponding to each pixel, so the target depth value finally generated in this embodiment is more accurate.
为了计算出每一个像素点的目标深度值，在一种实现方式中，可以将所述各个像素点中的任意一个作为中心像素点，将与该中心像素点位于同一行的目标像素点作为该中心像素点对应的行目标像素点，并将与该中心像素点位于同一列的目标像素点作为该中心像素点对应的列目标像素点。然后，根据该行目标像素点的原始深度值和该行目标像素点的权重值确定该中心像素点的初步深度值，并生成初步深度图像。简言之，本实施例相当于采用行目标像素点对该中心像素点进行一次加权滤波处理：In order to calculate the target depth value of each pixel, in an implementation manner, any one of the pixels may be used as the center pixel, and the target pixel located in the same row as the center pixel may be used as the center pixel. The row target pixel point corresponding to the center pixel point, and the target pixel point located in the same column as the center pixel point is used as the column target pixel point corresponding to the center pixel point. Then, a preliminary depth value of the center pixel is determined according to the original depth value of the row of target pixels and the weight value of the row of target pixels, and a preliminary depth image is generated. In short, this embodiment is equivalent to using the row target pixel to perform a weighted filtering process on the central pixel:
其中，D_表示生成的初步深度图像，D_(x,y)表示初步深度图像中横坐标为x、纵坐标为y的像素点的初步深度值，D(i,y)表示横坐标为i、纵坐标为y的行目标像素点在原始深度图像中的原始深度值，wi表示行目标像素点的权重值。Among them, D_ represents the generated preliminary depth image, D_(x, y) represents the preliminary depth value of the pixel whose abscissa is x and the ordinate is y in the preliminary depth image, and D(i, y) represents that the abscissa is i , the original depth value of the row target pixel whose vertical coordinate is y in the original depth image, and wi represents the weight value of the row target pixel.
然后在该初步深度图像中，获取该中心像素点对应的列目标像素点的初步深度值，根据该列目标像素点的初步深度值和该列目标像素点的权重值，确定该中心像素点对应的目标深度值，并得到深度图像中各个像素点对应的目标深度值。简言之，本实施例通过该中心像素点对应的列目标像对该中心像素点进行二次加权滤波处理后确定该中心像素点对应的最终的深度值(即目标深度值)。Then, in the preliminary depth image, the preliminary depth value of the column target pixel corresponding to the central pixel is obtained, and according to the preliminary depth value of the column target pixel and the weight value of the column target pixel, it is determined that the central pixel corresponds to and obtain the target depth value corresponding to each pixel in the depth image. In short, in this embodiment, the final depth value (ie, target depth value) corresponding to the central pixel is determined by performing a secondary weighted filtering process on the central pixel through the column target image corresponding to the central pixel.
最后本实施例根据深度图像中每一个像素点的目标深度值生成最终的目标深度图像，即如图1所示，该方法还包含步骤S300、根据该目标深度值生成与该目标深度值所对应的目标深度图像：Finally, in this embodiment, a final target depth image is generated according to the target depth value of each pixel in the depth image, that is, as shown in FIG. 1 , the method further includes step S300 , generating a corresponding depth value corresponding to the target depth value according to the target depth value. The target depth image of:
其中，disp表示目标深度图像，disp(x,y)表示目标深度图像中横坐标为x、纵坐标为y的像素点的目标深度值，D_(x,j)表示横坐标为x、纵坐标为j的列目标像素点在初步深度图像中的初步深度值，wj表示列目标像素点的权重值。Among them, disp represents the target depth image, disp(x, y) represents the target depth value of the pixel whose abscissa is x and ordinate is y in the target depth image, and D_(x, j) represents that the abscissa is x and the ordinate is y. is the preliminary depth value of the column target pixel of j in the preliminary depth image, andwj represents the weight value of the column target pixel.
在一种实现方式中，为了减少计算机的计算开销以及优化耗时，本实施例可以在生成初步深度图像和目标深度图像的时候，对行目标像素点与列目标像素点采取间隔采样处理的方式，从而将总的计算开销降低为原来的1/4，达到减少计算机的计算开销以及优化耗时的目的。In an implementation manner, in order to reduce the computational overhead of the computer and the time-consuming optimization, in this embodiment, when the preliminary depth image and the target depth image are generated, the row target pixel points and the column target pixel points can be processed by interval sampling. , so as to reduce the total computational cost to 1/4 of the original, to achieve the purpose of reducing the computational cost of the computer and optimizing the time-consuming.
在一种实现方式中，为了获取到边缘更为平滑的深度图像，可以在获得目标深度图像以后，再对目标深度图像进行均值滤波处理。例如，可以对目标深度图像进行窗口大小为3*3的均值滤波处理，得到平滑的目标深度图像(如图2所示)。In an implementation manner, in order to obtain a depth image with a smoother edge, after the target depth image is obtained, mean filtering may be performed on the target depth image. For example, mean filtering processing with a window size of 3*3 may be performed on the target depth image to obtain a smooth target depth image (as shown in Figure 2).
基于上述实施例，本发明还提供了一种深度图像处理装置，如图4所示，该装置包括：Based on the above embodiments, the present invention also provides a depth image processing device, as shown in FIG. 4 , the device includes:
确定模块01，用于获取深度图像中各个像素点对应的颜色信息，根据所述颜色信息确定所述各个像素点对应的目标像素点，并获取所述各个像素点与各自对应的目标像素点之间的颜色差值；A
计算模块02，用于取所述各个像素点对应的目标像素点的原始深度值，根据所述各个像素点与各自对应的目标像素点之间的颜色差值、所述各个像素点对应的目标像素点的原始深度值，确定所述各个像素点对应的目标深度值；The
生成模块03，用于根据所述各个像素点对应的目标深度值生成目标深度图像。A generating module 03 is configured to generate a target depth image according to the target depth value corresponding to each pixel point.
基于上述实施例，本发明还提供了一种终端，其原理框图可以如图5所示。该终端包括通过系统总线连接的处理器、存储器、网络接口、显示屏。其中，该终端的处理器用于提供计算和控制能力。该终端的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该终端的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种深度图像处理方法。该终端的显示屏可以是液晶显示屏或者电子墨水显示屏。Based on the above embodiments, the present invention further provides a terminal, the principle block diagram of which may be shown in FIG. 5 . The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The nonvolatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The network interface of the terminal is used to communicate with external terminals through a network connection. The computer program, when executed by a processor, implements a depth image processing method. The display screen of the terminal may be a liquid crystal display screen or an electronic ink display screen.
本领域技术人员可以理解，图5中示出的原理框图，仅仅是与本发明方案相关的部分结构的框图，并不构成对本发明方案所应用于其上的终端的限定，具体的终端可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。Those skilled in the art can understand that the principle block diagram shown in FIG. 5 is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include There are more or fewer components than shown in the figures, or some components are combined, or have a different arrangement of components.
在一种实现方式中，所述终端的存储器中存储有一个或者一个以上的程序，且经配置以由一个或者一个以上处理器执行所述一个或者一个以上程序包含用于进行一种深度图像处理方法的指令。In one implementation, one or more programs are stored in a memory of the terminal and are configured to be executed by one or more processors, including for performing a kind of depth image processing method directive.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本发明所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided by the present invention may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
综上所述，本发明公开了一种深度图像处理方法、装置、终端及存储介质，通过将深度图像中每一个像素点与周围像素点之间的颜色差异信息与周围像素点的深度值相结合来重新确定深度图像中的像素点的深度值，解决了现有的高斯滤波方法在确定各个像素点周围的像素点的权重值时，仅依靠距离的远近来确定各个像素点周围的像素点的权重值，因此有可能出现赋予颜色差异过大的像素点更高的权重值的情况，从而导致最终生成的各个像素点的深度值不准确的问题。To sum up, the present invention discloses a depth image processing method, device, terminal and storage medium. Combined to re-determine the depth value of the pixel in the depth image, it solves the problem that the existing Gaussian filtering method only relies on the distance to determine the pixel around each pixel when determining the weight value of each pixel. Therefore, it is possible to assign a higher weight value to the pixels with too large color difference, resulting in the inaccurate depth value of each pixel that is finally generated.
应当理解的是，本发明的应用不限于上述的举例，对本领域普通技术人员来说，可以根据上述说明加以改进或变换，所有这些改进和变换都应属于本发明所附权利要求的保护范围。It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above descriptions, and all these improvements and transformations should belong to the protection scope of the appended claims of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110326429.XACN115131421A (en) | 2021-03-26 | 2021-03-26 | Depth image processing method and device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110326429.XACN115131421A (en) | 2021-03-26 | 2021-03-26 | Depth image processing method and device, terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115131421Atrue CN115131421A (en) | 2022-09-30 |
Family
ID=83374246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110326429.XAPendingCN115131421A (en) | 2021-03-26 | 2021-03-26 | Depth image processing method and device, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115131421A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104680496A (en)* | 2015-03-17 | 2015-06-03 | 山东大学 | Kinect deep image remediation method based on colorful image segmentation |
| CN109345484A (en)* | 2018-09-30 | 2019-02-15 | 北京邮电大学 | A depth map repair method and device |
| CN112102386A (en)* | 2019-01-22 | 2020-12-18 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
- 2021
- 2021-03-26CNCN202110326429.XApatent/CN115131421A/enactivePending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104680496A (en)* | 2015-03-17 | 2015-06-03 | 山东大学 | Kinect deep image remediation method based on colorful image segmentation |
| CN109345484A (en)* | 2018-09-30 | 2019-02-15 | 北京邮电大学 | A depth map repair method and device |
| CN112102386A (en)* | 2019-01-22 | 2020-12-18 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 韩紫婷;崔志玲: "基于Kinect的三维数据修复和融合算法", 电子世界, no. 18, 23 September 2018 (2018-09-23), pages 19 - 21* |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109359602B (en) | Lane line detection method and device | |
| CN113486797B (en) | Unmanned vehicle position detection method, unmanned vehicle position detection device, unmanned vehicle position detection equipment, storage medium and vehicle | |
| CN109785291B (en) | Lane line self-adaptive detection method | |
| US11314979B2 (en) | Method and apparatus for evaluating image acquisition accuracy, electronic device and storage medium | |
| JP4416039B2 (en) | Striped pattern detection system, striped pattern detection method, and striped pattern detection program | |
| US12039767B2 (en) | Subject detection method and apparatus, electronic device, and computer-readable storage medium | |
| WO2021175180A1 (en) | Line of sight determination method and apparatus, and electronic device and computer-readable storage medium | |
| CN115170669A (en) | Identification and positioning method and system based on edge feature point set registration and storage medium | |
| CN109035287B (en) | Foreground image extraction method and device and moving vehicle identification method and device | |
| CN115457063A (en) | Method, device and equipment for extracting edge of circular hole of PCB (printed Circuit Board) and storage medium | |
| CN111915541B (en) | Image enhancement processing method, device, equipment and medium based on artificial intelligence | |
| CN113744200B (en) | Camera dirt detection method, device and equipment | |
| JP6177541B2 (en) | Character recognition device, character recognition method and program | |
| US20240020846A1 (en) | Corner point detection method and apparatus | |
| CN116385415A (en) | Edge defect detection method, device, equipment and storage medium | |
| CN111105452A (en) | High-low resolution fusion stereo matching method based on binocular vision | |
| CN110736610A (en) | Method, device, storage medium and depth camera for measuring optical center offset | |
| CN114674826A (en) | Cloth-based visual inspection method and inspection system | |
| JP2016197377A (en) | Computer program for image correction, image correction device, and image correction method | |
| CN109584165A (en) | A kind of antidote of digital picture, device, medium and electronic equipment | |
| US10970592B2 (en) | Adhering substance detection apparatus and adhering substance detection method | |
| CN115131421A (en) | Depth image processing method and device, terminal and storage medium | |
| US11995156B2 (en) | Confidence aided upsampling of categorical maps | |
| CN115760693A (en) | PCB patch component positioning method, device, equipment and medium | |
| CN114549551A (en) | Image cropping method and device, terminal and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |