CN109472741B - Three-dimensional splicing method and device - Google Patents
Three-dimensional splicing method and deviceDownload PDFInfo
- Publication number
- CN109472741B CN109472741BCN201811163271.3ACN201811163271ACN109472741BCN 109472741 BCN109472741 BCN 109472741BCN 201811163271 ACN201811163271 ACN 201811163271ACN 109472741 BCN109472741 BCN 109472741B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- mentioned
- frame depth
- relationship
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Generation (AREA)
- Image Analysis (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及三维扫描领域，具体而言，涉及一种三维拼接方法及装置。The present invention relates to the field of three-dimensional scanning, in particular to a three-dimensional splicing method and device.
背景技术Background technique
现有手持式三维扫描技术方案中，基本的拼接原理都是基于前后帧重建的三维点之间的重叠面进行ICP拼接，现有方案对前后帧之间的初始相对位置的初始值要求比较高，同时在平面特征、无明显特征或者对称性物体容易造成拼接失败，导致扫描无法进行。In the existing handheld 3D scanning technology solutions, the basic stitching principle is to perform ICP stitching based on the overlapping surfaces between the 3D points reconstructed from the front and back frames. The existing solutions require relatively high initial values for the initial relative positions between the front and back frames. , At the same time, flat features, no obvious features, or symmetrical objects are likely to cause splicing failures, resulting in the inability to scan.
针对上述的问题，目前尚未提出有效的解决方案。For the above problems, no effective solution has been proposed yet.
发明内容Contents of the invention
本发明实施例提供了一种三维拼接方法及装置，以至少解决现有技术中所采用的三维拼接方案的拼接失败率较高的技术问题。Embodiments of the present invention provide a three-dimensional splicing method and device to at least solve the technical problem of high splicing failure rate of the three-dimensional splicing scheme adopted in the prior art.
根据本发明实施例的一个方面，提供了一种三维拼接方法，包括：确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。According to an aspect of an embodiment of the present invention, a three-dimensional mosaic method is provided, including: determining the first mapping relationship between the three-dimensional points in the current frame depth map and the pixels in the texture map, and determining the first mapping relationship between the identification points in different texture maps The first corresponding relationship between; according to the first mapping relationship and the first corresponding relationship, determine the first relative positional relationship between the current frame depth map and other frame depth maps, wherein the other frame depth maps are related to the above Depth maps different from the depth map of the current frame; based on the first relative positional relationship, performing three-dimensional splicing of the three-dimensional points in the depth map of the current frame and the depth maps of other frames.
进一步地，确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，包括：通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。Further, determining the first mapping relationship between the three-dimensional points of the depth map of the current frame and the pixel points of the texture map includes: determining the second relative position between the depth camera and the texture camera by calibrating the depth camera and the texture camera. positional relationship; perform 3D reconstruction based on the aforementioned depth camera, and obtain the 3D point of the aforementioned current frame depth map; based on the aforementioned second relative positional relationship, project the aforementioned 3D point onto the aforementioned texture map collected by the aforementioned texture camera, and obtain the aforementioned first Mapping relations.
进一步地，依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，包括：确定上述标识点与上述像素点之间的第二映射关系；依据上述第一映射关系和上述第二映射关系，确定上述标识点与上述三维点之间的第三映射关系；依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系。Further, according to the above-mentioned first mapping relationship and the above-mentioned first corresponding relationship, determining a first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps includes: determining the first relative position relationship between the above-mentioned identification point and the above-mentioned pixel point Two mapping relationships; according to the above-mentioned first mapping relationship and the above-mentioned second mapping relationship, determine the third mapping relationship between the above-mentioned identification point and the above-mentioned three-dimensional point; according to the above-mentioned third mapping relationship and the above-mentioned first correspondence relationship, determine the above-mentioned first Relative positional relationship.
进一步地，依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系，包括：依据上述第三映射关系和上述第一对应关系，确定上述当前帧深度图与上述其他帧深度图中的三维点之间的第二对应关系；基于上述第二对应关系确定上述第一相对位置关系。Further, determining the above-mentioned first relative positional relationship according to the above-mentioned third mapping relationship and the above-mentioned first correspondence relationship includes: determining the above-mentioned current frame depth map and the above-mentioned other frame depths according to the above-mentioned third mapping relationship and the above-mentioned first correspondence relationship. A second correspondence between the three-dimensional points in the figure; determining the first relative positional relationship based on the second correspondence.
进一步地，基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接，包括：基于上述第一相对位置关系对上述当前帧深度图与上述其他帧深度图中的三维点进行初始三维拼接；在上述初始三维拼接的基础上，采用迭代最近点算法对上述当前帧深度图与上述其他帧深度图中的三维点进行精准三维拼接。Further, based on the above-mentioned first relative positional relationship, performing three-dimensional stitching on the above-mentioned current frame depth map and the above-mentioned three-dimensional points in the above-mentioned other frame depth map includes: based on the above-mentioned first relative positional relationship Perform initial 3D stitching on the 3D points in the depth map; on the basis of the above initial 3D stitching, use the iterative closest point algorithm to perform precise 3D stitching on the 3D points in the above-mentioned current frame depth map and the above-mentioned other frame depth maps.
进一步地，上述标识点包括如下至少之一：角点、标志点，在上述标识点为上述角点的情况下，在确定上述标识点与上述像素点之间的第二映射关系之前，上述方法还包括：通过目标提取算法提取上述纹理图中的每个上述角点，其中，上述目标提取算法包括如下至少之一：Harris角点提取算法、SIFT算法、SURF特征提取算法、FAST角点检测算法、AGAST角点检测算法、BRISK特征提取算法、FREAK特征提取算法、ORB特征提取算法。Further, the above-mentioned identification point includes at least one of the following: a corner point and a marker point. In the case that the above-mentioned identification point is the above-mentioned corner point, before determining the second mapping relationship between the above-mentioned identification point and the above-mentioned pixel point, the above-mentioned method It also includes: extracting each of the above-mentioned corner points in the above-mentioned texture map through a target extraction algorithm, wherein the above-mentioned target extraction algorithm includes at least one of the following: Harris corner point extraction algorithm, SIFT algorithm, SURF feature extraction algorithm, FAST corner point detection algorithm , AGAST corner detection algorithm, BRISK feature extraction algorithm, FREAK feature extraction algorithm, ORB feature extraction algorithm.
进一步地，通过RANSAC算法将不同纹理图中的标识点进行点对点匹配，得到上述第一对应关系。Further, the identification points in different texture maps are matched point-to-point by using the RANSAC algorithm to obtain the above-mentioned first corresponding relationship.
根据本发明实施例的另一方面，还提供了一种三维拼接装置，包括：第一确定模块，用于确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；第二确定模块，用于依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；拼接模块，用于基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。According to another aspect of the embodiments of the present invention, there is also provided a three-dimensional splicing device, including: a first determination module, configured to determine a first mapping relationship between the three-dimensional points of the depth map of the current frame and the pixels of the texture map, and the first corresponding relationship between the identification points in different texture maps; the second determining module is configured to determine the relationship between the current frame depth map and other frame depth maps according to the first mapping relationship and the first corresponding relationship The first relative positional relationship, wherein, the above-mentioned other frame depth map is a depth map different from the above-mentioned current frame depth map; the splicing module is used to compare the above-mentioned current frame depth map and the above-mentioned other frame depth map based on the above-mentioned first relative positional relationship 3D points in 3D stitching.
进一步地，上述第一确定模块，包括：确定单元，用于通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；获取单元，用于基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；映射单元，用于基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。Further, the above-mentioned first determination module includes: a determination unit, configured to determine the second relative positional relationship between the above-mentioned depth camera and the above-mentioned texture camera by calibrating the depth camera and the texture camera; The camera performs 3D reconstruction to obtain the 3D points of the depth map of the current frame; the mapping unit is configured to project the 3D points onto the texture map collected by the texture camera based on the second relative positional relationship to obtain the first mapping relation.
根据本发明实施例的另一方面，还提供了一种存储介质，上述存储介质包括存储的程序，其中，在上述程序运行时控制上述存储介质所在设备执行任意一项上述的三维拼接方法。According to another aspect of the embodiments of the present invention, there is also provided a storage medium, the above-mentioned storage medium includes a stored program, wherein when the above-mentioned program is running, the device where the above-mentioned storage medium is located is controlled to execute any one of the above-mentioned three-dimensional mosaic methods.
根据本发明实施例的另一方面，还提供了一种处理器，上述处理器用于运行程序，其中，上述程序运行时执行任意一项上述的三维拼接方法。According to another aspect of the embodiments of the present invention, there is also provided a processor, the above-mentioned processor is used to run a program, wherein, when the above-mentioned program is running, any one of the three-dimensional stitching methods mentioned above is executed.
在本发明实施例中，通过确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接，达到了提高三维拼接的准确性的目的，从而实现了减少三维拼接失败的次数，增加扫描的流畅性的技术效果，进而解决了现有技术中所采用的三维拼接方案的拼接失败率较高的技术问题。In the embodiment of the present invention, by determining the first mapping relationship between the three-dimensional points of the current frame depth map and the pixel points of the texture map, and the first corresponding relationship between the identification points in different texture maps; according to the above first The mapping relationship and the above-mentioned first corresponding relationship determine the first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps, wherein the above-mentioned other frame depth maps are depth maps different from the above-mentioned current frame depth map; based on the above The first relative positional relationship is to perform three-dimensional stitching on the above-mentioned current frame depth map and the above-mentioned three-dimensional points in other frame depth maps to achieve the purpose of improving the accuracy of three-dimensional stitching, thereby reducing the number of three-dimensional stitching failures and increasing the scanning. The technical effect of fluency further solves the technical problem of the high splicing failure rate of the three-dimensional splicing scheme adopted in the prior art.
附图说明Description of drawings
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：The accompanying drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations to the present invention. In the attached picture:
图1是根据本发明实施例的一种三维拼接方法的流程图；FIG. 1 is a flowchart of a three-dimensional splicing method according to an embodiment of the present invention;
图2是根据本发明实施例的一种可选的三维拼接方法的流程图；Fig. 2 is a flow chart of an optional three-dimensional mosaic method according to an embodiment of the present invention;
图3是根据本发明实施例的一种可选的三维拼接方法的流程图；Fig. 3 is a flow chart of an optional three-dimensional mosaic method according to an embodiment of the present invention;
图4是根据本发明实施例的一种可选的三维拼接方法的流程图；以及FIG. 4 is a flow chart of an optional three-dimensional mosaic method according to an embodiment of the present invention; and
图5是根据本发明实施例的一种三维拼接装置的结构示意图。Fig. 5 is a schematic structural diagram of a three-dimensional splicing device according to an embodiment of the present invention.
具体实施方式Detailed ways
为了使本技术领域的人员更好地理解本发明方案，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分的实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本发明保护的范围。In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only It is an embodiment of a part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or device comprising a sequence of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.
实施例1Example 1
根据本发明实施例，提供了一种三维拼接方法的实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。According to an embodiment of the present invention, an embodiment of a three-dimensional stitching method is provided. It should be noted that the steps shown in the flow charts of the drawings can be executed in a computer system such as a set of computer-executable instructions, and, although In the flowcharts, a logical order is shown, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.
图1是根据本发明实施例的一种三维拼接方法的流程图，如图1所示，该方法包括如下步骤：Fig. 1 is a flowchart of a three-dimensional mosaic method according to an embodiment of the present invention. As shown in Fig. 1, the method includes the following steps:
步骤S102，确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；Step S102, determining the first mapping relationship between the 3D points in the depth map of the current frame and the pixels in the texture map, and the first corresponding relationship between the identification points in different texture maps;
步骤S104，依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；Step S104, according to the above-mentioned first mapping relationship and the above-mentioned first corresponding relationship, determine a first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps, wherein the above-mentioned other frame depth map is related to the above-mentioned current frame depth map different depth maps;
步骤S106，基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。Step S106, based on the first relative positional relationship, three-dimensional stitching is performed on the three-dimensional points in the depth map of the current frame and the depth maps of other frames.
需要说明的是，本申请实施例可以但不限于适用于基于结构光的三维扫描领域，尤其是手持式扫描仪领域，例如，多用于消费扫描、齿科入口式扫描等。并且，在手持式扫描仪领域，可以但不限于用于标志点、非标志点的三维扫描方案中，通过利用纹理图的纹理信息进行二维粗匹配，可以最大程度上减少单纯依靠三维ICP拼接对于对称物体造成的二义性，提高三维拼接的准确性。It should be noted that the embodiments of the present application may be applicable to but not limited to the field of three-dimensional scanning based on structured light, especially the field of hand-held scanners, for example, it is mostly used in consumer scanning and dental entrance scanning. Moreover, in the field of handheld scanners, it can be used, but not limited to, in the 3D scanning scheme for marker points and non-marker points. By using the texture information of the texture map for 2D rough matching, it is possible to minimize the number of 3D ICP stitching alone. For the ambiguity caused by symmetrical objects, the accuracy of 3D stitching is improved.
可选的，上述三维点为基于深度相机进行三维重建获取到的当前帧深度图的三维点；上述标识点包括如下至少之一：角点、标志点，上述角点的位置位于纹理图中在x和y方向上，变化均很大的交点处；上述标志点可以为人为进行标志的点；上述其他帧深度图可以但不限于为当前帧深度图的前一帧深度图，当前帧深度图的后一帧深度图，等等。Optionally, the above-mentioned three-dimensional point is a three-dimensional point of the current frame depth map obtained based on the three-dimensional reconstruction of the depth camera; the above-mentioned identification point includes at least one of the following: a corner point, a marker point, and the position of the above-mentioned corner point is located in the texture map. In the x and y directions, the intersection points with large changes; the above-mentioned mark points can be artificially marked points; the above-mentioned other frame depth maps can be but not limited to the previous frame depth map of the current frame depth map, and the current frame depth map The depth map of the next frame, and so on.
在一种可选的实施例中，可以基于预先对深度相机和纹理相机进行标定，其中，上述深度相机和纹理相机均为手持式扫描仪中的相机，基于标定好的深度相机和纹理相机之间的相对位置关系，将三维点投影到纹理相机采集的纹理图上，通过将三维点与纹理图上的像素点一一映射，得到三维点与像素点之间的第一映射关系。需要说明的是，三维点和像素点均可以以坐标的形式进行体现，例如，三维点与像素点之间的映射关系，可以但不限于理解为三维坐标和像素坐标之间的映射关系。In an optional embodiment, the depth camera and the texture camera can be calibrated in advance, wherein the above-mentioned depth camera and the texture camera are all cameras in the hand-held scanner, and based on the calibrated depth camera and the texture camera The relative positional relationship among them is projected onto the texture map collected by the texture camera, and the first mapping relationship between the 3D point and the pixel is obtained by mapping the 3D point with the pixels on the texture map one by one. It should be noted that both 3D points and pixel points can be represented in the form of coordinates. For example, the mapping relationship between 3D points and pixel points can be understood as, but not limited to, the mapping relationship between 3D coordinates and pixel coordinates.
在上述可选的实施例中，在上述标识点为角点的情况下，对于通过将不同纹理图中所提取的角点，需要进行点对点匹配，以确定上述不同纹理图中的角点之间的第一对应关系。In the above optional embodiment, in the case that the above-mentioned identification points are corner points, point-to-point matching needs to be performed for the corner points extracted in different texture maps to determine the difference between the corner points in the above-mentioned different texture maps. the first corresponding relationship.
在本申请实施例中，每个纹理图中的每个角点均对应一个深度值，其中，上述深度值是指每个角点对应的深度相机所测量的三维点(x,y,z)中的z值。In the embodiment of the present application, each corner point in each texture map corresponds to a depth value, wherein the above-mentioned depth value refers to the three-dimensional point (x, y, z) measured by the depth camera corresponding to each corner point The z value in .
其中，在上述可选的实施例中，由于每个纹理图中的每个角点均对应一个深度值，进而可以依据每个角点的深度值以及三维点的深度值，确定角点和三维点之间的第三映射关系。Among them, in the above optional embodiment, since each corner point in each texture map corresponds to a depth value, the corner point and the three-dimensional point can be determined according to the depth value of each corner point and the depth value of the three-dimensional point. The third mapping relationship between points.
在另一种可选的实施例中，在本申请实施例中，可以但不限于通过RANSAC算法将不同纹理图中的标识点进行点对点匹配，得到上述第一对应关系。In another optional embodiment, in the embodiment of the present application, it is possible, but not limited to, to perform point-to-point matching on the identification points in different texture maps by using the RANSAC algorithm to obtain the above-mentioned first corresponding relationship.
需要说明的是，作为一种可选的实施例，本申请可以但不限于可以采用RANSAC匹配算法，针对两幅或多幅不同的纹理图上的无序角点进行准确匹配。其中，RANSAC匹配算法的基本思路即是随机选择一个小的数据点子集，然后对其进行拟合，查看有多少其他点匹配到拟合的模型上，迭代这个过程直至有较大的概率找到需要拟合的模型。经过RANSAC算法的匹配运算，可以得到不同纹理图中的角点之间的第一对应关系，则此时即可基于第一映射关系和上述第一对应关系，获取上述当前帧深度图与其他帧深度图之间的第一相对位置关系，进而可以基于上述第一相对位置关系对上述当前帧深度图与其他帧深度图中的三维点进行三维拼接。It should be noted that, as an optional embodiment, the present application may, but is not limited to, use a RANSAC matching algorithm to perform accurate matching on disordered corner points on two or more different texture maps. Among them, the basic idea of the RANSAC matching algorithm is to randomly select a small subset of data points, and then fit it, check how many other points match the fitted model, iterate this process until there is a greater probability to find the required the fitted model. After the matching operation of the RANSAC algorithm, the first corresponding relationship between the corner points in different texture maps can be obtained, and then based on the first mapping relationship and the first corresponding relationship, the depth map of the current frame and other frames can be obtained The first relative positional relationship between the depth maps, and then based on the first relative positional relationship, three-dimensional splicing can be performed on the above-mentioned current frame depth map and three-dimensional points in other frame depth maps.
在本申请实施例中，通过在使用手持扫描仪对不贴标志点的物体进行三维扫描过程中，通过纹理图中的纹理信息与深度相机采集到的帧图像结合，基于被扫描物体的纹理信息辅助进行三维拼接，可以解决由于被扫描物体为平面特征、无明显特征或者对称性物体容易造成拼接失败的问题，可以显著提高手持式扫描仪的准确性及流畅性。In the embodiment of the present application, by combining the texture information in the texture map with the frame images collected by the depth camera during the three-dimensional scanning of objects without marker points using a handheld scanner, based on the texture information of the scanned object Auxiliary three-dimensional stitching can solve the problem of stitching failure due to the fact that the scanned object is a flat feature, no obvious feature, or a symmetrical object, and can significantly improve the accuracy and fluency of the handheld scanner.
在本发明实施例中，通过确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接，达到了提高三维拼接的准确性的目的，从而实现了减少三维拼接失败的次数，增加扫描的流畅性的技术效果，进而解决了现有技术中所采用的三维拼接方案的拼接失败率较高的技术问题。In the embodiment of the present invention, by determining the first mapping relationship between the three-dimensional points of the current frame depth map and the pixel points of the texture map, and the first corresponding relationship between the identification points in different texture maps; according to the above first The mapping relationship and the above-mentioned first corresponding relationship determine the first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps, wherein the above-mentioned other frame depth maps are depth maps different from the above-mentioned current frame depth map; based on the above The first relative positional relationship is to perform three-dimensional stitching on the above-mentioned current frame depth map and the above-mentioned three-dimensional points in other frame depth maps to achieve the purpose of improving the accuracy of three-dimensional stitching, thereby reducing the number of three-dimensional stitching failures and increasing the scanning. The technical effect of fluency further solves the technical problem of the high splicing failure rate of the three-dimensional splicing scheme adopted in the prior art.
在一种可选的实施例中，在上述标识点为上述角点的情况下，在确定上述标识点与上述像素点之间的第二映射关系之前，上述方法还包括：通过目标提取算法提取上述纹理图中的每个上述角点，其中，上述目标提取算法包括如下至少之一：Harris角点提取算法、SIFT算法、SURF特征提取算法、FAST角点检测算法、AGAST角点检测算法、BRISK特征提取算法、FREAK特征提取算法、ORB特征提取算法。In an optional embodiment, when the above-mentioned identification point is the above-mentioned corner point, before determining the second mapping relationship between the above-mentioned identification point and the above-mentioned pixel point, the above-mentioned method further includes: using the target extraction algorithm to extract Each of the above-mentioned corner points in the above-mentioned texture map, wherein the above-mentioned target extraction algorithm includes at least one of the following: Harris corner point extraction algorithm, SIFT algorithm, SURF feature extraction algorithm, FAST corner point detection algorithm, AGAST corner point detection algorithm, BRISK Feature extraction algorithm, FREAK feature extraction algorithm, ORB feature extraction algorithm.
在一种可选的实施例中，图2是根据本发明实施例的一种可选的三维拼接方法的流程图，如图2所示，确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，包括：In an optional embodiment, FIG. 2 is a flow chart of an optional three-dimensional mosaic method according to an embodiment of the present invention. As shown in FIG. 2, determine the three-dimensional points of the current frame depth map and the pixels of the texture map The first mapping relationship between points, including:
步骤S202，通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；Step S202, determining a second relative positional relationship between the depth camera and the texture camera by calibrating the depth camera and the texture camera;
步骤S204，基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；Step S204, performing 3D reconstruction based on the above-mentioned depth camera, and obtaining the 3D points of the above-mentioned current frame depth map;
步骤S206，基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。Step S206: Based on the second relative positional relationship, the first mapping relationship is obtained by projecting the three-dimensional point onto the texture map collected by the texture camera.
在上述可选的实施例中，通过对深度相机与纹理相机进行标定，标定出深度相机与纹理相机之间的第二相对位置关系，并基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点，基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，进而可以得到上述第一映射关系。In the above optional embodiment, by calibrating the depth camera and the texture camera, the second relative positional relationship between the depth camera and the texture camera is calibrated, and three-dimensional reconstruction is performed based on the above depth camera to obtain the above current frame depth map Based on the above-mentioned second relative positional relationship, the above-mentioned first mapping relationship can be obtained by projecting the above-mentioned three-dimensional point onto the above-mentioned texture map collected by the above-mentioned texture camera.
在一种可选的实施例中，图3是根据本发明实施例的一种可选的三维拼接方法的流程图，如图3所示，依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，包括：In an optional embodiment, FIG. 3 is a flowchart of an optional three-dimensional mosaic method according to an embodiment of the present invention. As shown in FIG. 3 , according to the first mapping relationship and the first correspondence relationship above, Determining the first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps, including:
步骤S302，确定上述标识点与上述像素点之间的第二映射关系；Step S302, determining a second mapping relationship between the above-mentioned identification point and the above-mentioned pixel point;
步骤S304，依据上述第一映射关系和上述第二映射关系，确定上述标识点与上述三维点之间的第三映射关系；Step S304, according to the above-mentioned first mapping relationship and the above-mentioned second mapping relationship, determine a third mapping relationship between the above-mentioned identification point and the above-mentioned three-dimensional point;
步骤S306，依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系。Step S306: Determine the first relative positional relationship according to the third mapping relationship and the first corresponding relationship.
可选的，上述标识点可以为纹理图中的角点、标志点，与纹理图中的像素点之间存在坐标数据之间的映射关系，在本申请实施例中，由于标识点与像素点之间存在第二映射关系，三维点与像素点之间存在第一映射关系，因而，依据上述第一映射关系和第二映射关系，可以唯一确定标识点与三维点之间的第三映射关系，依据上述第三映射关系和第一对应关系，可以确定当前帧深度图与其他帧深度图之间的第一相对位置关系。Optionally, the above-mentioned identification points may be corner points and marker points in the texture map, and there is a mapping relationship between coordinate data and pixel points in the texture map. In the embodiment of the present application, since the identification points and pixel points There is a second mapping relationship between them, and there is a first mapping relationship between three-dimensional points and pixel points. Therefore, according to the first mapping relationship and the second mapping relationship above, the third mapping relationship between the identification point and the three-dimensional point can be uniquely determined , according to the third mapping relationship and the first corresponding relationship, the first relative positional relationship between the current frame depth map and other frame depth maps can be determined.
作为一种可选的实施例，图4是根据本发明实施例的一种可选的三维拼接方法的流程图，如图4所示，依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系，包括：As an optional embodiment, FIG. 4 is a flowchart of an optional three-dimensional mosaic method according to an embodiment of the present invention. As shown in FIG. 4, according to the above-mentioned third mapping relationship and the above-mentioned first correspondence relationship, determine The first relative positional relationship mentioned above includes:
步骤S402，依据上述第三映射关系和上述第一对应关系，确定上述当前帧深度图与上述其他帧深度图中的三维点之间的第二对应关系；Step S402, according to the above-mentioned third mapping relationship and the above-mentioned first correspondence relationship, determine the second correspondence relationship between the above-mentioned current frame depth map and the three-dimensional points in the above-mentioned other frame depth maps;
步骤S404，基于上述第二对应关系确定上述第一相对位置关系。Step S404, determining the first relative positional relationship based on the second corresponding relationship.
在本申请实施例中，通过上述第三映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图中的三维点之间的第二对应关系，并且，在本申请实施例中，还可以但不限于基于四元数法求取上述当前帧深度图与其他帧深度图之间的第一相对位置关系。In the embodiment of the present application, the second corresponding relationship between the above-mentioned current frame depth map and the 3D points in other frame depth maps is determined through the above-mentioned third mapping relationship and the above-mentioned first corresponding relationship, and, in the embodiment of the present application In this method, the first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps may also be obtained, but not limited to, based on the quaternion method.
在本申请实施例中，还存在一种可选的实施例，基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接，包括：In the embodiment of the present application, there is also an optional embodiment, based on the above-mentioned first relative position relationship, performing three-dimensional stitching on the above-mentioned current frame depth map and the above-mentioned three-dimensional points in other frame depth maps, including:
步骤S502，基于上述第一相对位置关系对上述当前帧深度图与上述其他帧深度图中的三维点进行初始三维拼接；Step S502, based on the first relative positional relationship, perform initial three-dimensional stitching on the above-mentioned current frame depth map and the above-mentioned three-dimensional points in other frame depth maps;
步骤S504，在上述初始三维拼接的基础上，采用迭代最近点算法对上述当前帧深度图与上述其他帧深度图中的三维点进行精准三维拼接。Step S504, on the basis of the initial 3D stitching, use an iterative closest point algorithm to perform precise 3D stitching on the 3D points in the depth map of the current frame and the depth maps of other frames.
可选的，上述第一相对位置关系可以但不限于为一个初始位置关系，在基于上述初始位置关系对上述当前帧深度图与其他帧深度图中的三维点进行初始三维拼接之后，还可以在初始三维拼接的基础上，后续进行多轮迭代最近点算法ICP拼接，以实现对上述当前帧深度图与其他帧深度图中的三维点进行精准三维拼接。Optionally, the above-mentioned first relative position relationship may be, but not limited to, an initial position relationship. After initial three-dimensional splicing of the above-mentioned current frame depth map and three-dimensional points in other frame depth maps based on the above-mentioned initial position relationship, it may also be On the basis of the initial 3D stitching, subsequent rounds of iterative closest point algorithm ICP stitching are performed to achieve accurate 3D stitching of the above-mentioned current frame depth map and 3D points in other frame depth maps.
可选的，在本申请实施例中，可以但不限于将上述第一相关位置关系理解为一个旋转平移矩阵，通过上述旋转平移矩阵将后帧图像转到前帧图像的三维点的坐标系下，实现统一的坐标，得到一个可能存在误差的初始三维拼接的相对位置结果，通过ICP算法迭代进行更精确的位置修正，使得相对位置求得更加精准。Optionally, in the embodiment of the present application, the above-mentioned first relative positional relationship can be understood as a rotation-translation matrix, but not limited to, and the subsequent frame image is transferred to the coordinate system of the three-dimensional point of the previous frame image through the above-mentioned rotation-translation matrix , to achieve unified coordinates, to obtain a relative position result of the initial three-dimensional mosaic that may have errors, and to perform more accurate position correction through ICP algorithm iterations, so that the relative position can be obtained more accurately.
实施例2Example 2
根据本发明实施例，还提供了一种用于实施上述三维拼接方法的装置实施例，图5是根据本发明实施例的一种三维拼接装置的结构示意图，如图5所示，上述三维拼接装置，包括：第一确定模块50、第二确定模块52和拼接模块54，其中，According to an embodiment of the present invention, an embodiment of a device for implementing the above three-dimensional splicing method is also provided. FIG. 5 is a schematic structural diagram of a three-dimensional splicing device according to an embodiment of the present invention. As shown in FIG. 5, the above three-dimensional splicing The device includes: a first determination module 50, a second determination module 52 and a splicing module 54, wherein,
第一确定模块50，用于确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；第二确定模块52，用于依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；拼接模块54，用于基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。The first determining module 50 is used to determine the first mapping relationship between the three-dimensional points of the depth map of the current frame and the pixels of the texture map, and the first corresponding relationship between the identification points in different texture maps; the second determining module 52. Determine a first relative positional relationship between the above-mentioned current frame depth map and other frame depth maps according to the above-mentioned first mapping relationship and the above-mentioned first correspondence relationship, wherein the above-mentioned other frame depth maps are the same as the above-mentioned current frame depth map different depth maps; a stitching module 54 configured to, based on the first relative positional relationship, perform three-dimensional stitching on the depth map of the current frame and the three-dimensional points in the depth maps of other frames.
需要说明的是，上述各个模块是可以通过软件或硬件来实现的，例如，对于后者，可以通过以下方式实现：上述各个模块可以位于同一个处理器中；或者，上述各个模块以任意组合的方式位于不同的处理器中。It should be noted that each of the above-mentioned modules can be realized by software or hardware. For example, for the latter, it can be realized in the following manner: each of the above-mentioned modules can be located in the same processor; or, the above-mentioned modules can be implemented in any combination way resides in a different processor.
此处需要说明的是，上述第一确定模块50、第二确定模块52和拼接模块54对应于实施例1中的步骤S102至步骤S106，上述模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块作为装置的一部分可以运行在计算机终端中。It should be noted here that the above-mentioned first determination module 50, second determination module 52, and splicing module 54 correspond to steps S102 to S106 in Embodiment 1, and the examples and application scenarios implemented by the above-mentioned modules are the same as those of the corresponding steps , but not limited to the content disclosed in Embodiment 1 above. It should be noted that, as a part of the device, the above modules can run in the computer terminal.
需要说明的是，本实施例的可选或优选实施方式可以参见实施例1中的相关描述，此处不再赘述。It should be noted that, for optional or preferred implementation manners of this embodiment, reference may be made to relevant descriptions in Embodiment 1, and details are not repeated here.
在一种可选的实施例中，上述第一确定模块，包括：确定单元，用于通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；获取单元，用于基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；映射单元，用于基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。In an optional embodiment, the above-mentioned first determining module includes: a determining unit configured to determine a second relative positional relationship between the above-mentioned depth camera and the above-mentioned texture camera by calibrating the depth camera and the above-mentioned texture camera; A unit, configured to perform 3D reconstruction based on the depth camera, and obtain the 3D points of the depth map of the current frame; a mapping unit, configured to project the 3D points to the texture map collected by the texture camera based on the second relative positional relationship , the above-mentioned first mapping relationship is obtained.
上述的三维拼接装置还可以包括处理器和存储器，上述第一确定模块50、第二确定模块52和拼接模块54等均作为程序单元存储在存储器中，由处理器执行存储在存储器中的上述程序单元来实现相应的功能。The above-mentioned three-dimensional splicing device can also include a processor and a memory, and the above-mentioned first determination module 50, the second determination module 52 and the splicing module 54, etc. are all stored in the memory as program units, and the above-mentioned program stored in the memory is executed by the processor unit to achieve the corresponding function.
处理器中包含内核，由内核去存储器中调取相应的程序单元，上述内核可以设置一个或以上。存储器可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)，存储器包括至少一个存储芯片。The processor includes a kernel, and the kernel fetches corresponding program units from the memory, and one or more kernels can be set. Memory may include non-permanent memory in computer-readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), memory includes at least one memory chip.
根据本申请实施例，还提供了一种存储介质实施例。可选地，在本实施例中，上述存储介质包括存储的程序，其中，在上述程序运行时控制上述存储介质所在设备执行上述任意一种三维拼接方法。According to an embodiment of the present application, an embodiment of a storage medium is also provided. Optionally, in this embodiment, the above-mentioned storage medium includes a stored program, wherein when the above-mentioned program is running, the device where the above-mentioned storage medium is located is controlled to perform any one of the above-mentioned three-dimensional mosaic methods.
可选地，在本实施例中，上述存储介质可以位于计算机网络中计算机终端群中的任意一个计算机终端中，或者位于移动终端群中的任意一个移动终端中，上述存储介质包括存储的程序。Optionally, in this embodiment, the above-mentioned storage medium may be located in any computer terminal in the computer terminal group in the computer network, or in any mobile terminal in the mobile terminal group, and the above-mentioned storage medium includes stored programs.
可选地，在程序运行时控制存储介质所在设备执行以下功能：确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。Optionally, when the program is running, the device where the storage medium is located is controlled to perform the following functions: determine the first mapping relationship between the three-dimensional points in the depth map of the current frame and the pixels in the texture map, and determine the first mapping relationship between the identification points in different texture maps The first corresponding relationship: according to the first mapping relationship and the first corresponding relationship, determine the first relative positional relationship between the depth map of the current frame and other frame depth maps, wherein the depth map of other frames is related to the current frame Depth maps with different depth maps; based on the first relative positional relationship, three-dimensional splicing is performed on the three-dimensional points in the depth map of the current frame and the depth maps of other frames.
可选地，在程序运行时控制存储介质所在设备执行以下功能：通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。Optionally, when the program is running, the device where the storage medium is located is controlled to perform the following functions: determine the second relative positional relationship between the above-mentioned depth camera and the above-mentioned texture camera by calibrating the depth camera and the above-mentioned texture camera; perform three-dimensional reconstruction based on the above-mentioned depth camera , acquiring the 3D point of the depth map of the current frame; based on the second relative positional relationship, the first mapping relationship is obtained by projecting the 3D point onto the texture map collected by the texture camera.
可选地，在程序运行时控制存储介质所在设备执行以下功能：确定上述标识点与上述像素点之间的第二映射关系；依据上述第一映射关系和上述第二映射关系，确定上述标识点与上述三维点之间的第三映射关系；依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系。Optionally, when the program is running, the device where the storage medium is located is controlled to perform the following functions: determine the second mapping relationship between the above-mentioned identification point and the above-mentioned pixel point; determine the above-mentioned identification point according to the above-mentioned first mapping relationship and the above-mentioned second mapping relationship A third mapping relationship with the above-mentioned three-dimensional point; according to the above-mentioned third mapping relationship and the above-mentioned first corresponding relationship, the above-mentioned first relative position relationship is determined.
可选地，在程序运行时控制存储介质所在设备执行以下功能：依据上述第三映射关系和上述第一对应关系，确定上述当前帧深度图与上述其他帧深度图中的三维点之间的第二对应关系；基于上述第二对应关系确定上述第一相对位置关系。Optionally, when the program is running, the device where the storage medium is located is controlled to perform the following function: according to the above-mentioned third mapping relationship and the above-mentioned first correspondence relationship, determine the first three-dimensional point between the above-mentioned current frame depth map and the above-mentioned other frame depth map Two correspondences: determining the first relative positional relationship based on the second correspondence.
可选地，在程序运行时控制存储介质所在设备执行以下功能：基于上述第一相对位置关系对上述当前帧深度图与上述其他帧深度图中的三维点进行初始三维拼接；在上述初始三维拼接的基础上，采用迭代最近点算法对上述当前帧深度图与上述其他帧深度图中的三维点进行精准三维拼接。Optionally, when the program is running, the device where the storage medium is located is controlled to perform the following functions: perform initial three-dimensional splicing of the three-dimensional points in the above-mentioned current frame depth map and the above-mentioned other frame depth maps based on the above-mentioned first relative position relationship; On the basis of , the iterative closest point algorithm is used to perform precise three-dimensional splicing of the three-dimensional points in the above-mentioned current frame depth map and the above-mentioned other frame depth maps.
可选地，在程序运行时控制存储介质所在设备执行以下功能：通过目标提取算法提取上述纹理图中的每个上述角点，其中，上述目标提取算法包括如下至少之一：Harris角点提取算法、SIFT算法、SURF特征提取算法、FAST角点检测算法、AGAST角点检测算法、BRISK特征提取算法、FREAK特征提取算法、ORB特征提取算法。Optionally, when the program is running, the device where the storage medium is located is controlled to perform the following function: each of the above-mentioned corner points in the above-mentioned texture map is extracted through a target extraction algorithm, wherein the above-mentioned target extraction algorithm includes at least one of the following: Harris corner point extraction algorithm , SIFT algorithm, SURF feature extraction algorithm, FAST corner detection algorithm, AGAST corner detection algorithm, BRISK feature extraction algorithm, FREAK feature extraction algorithm, ORB feature extraction algorithm.
根据本申请实施例，还提供了一种处理器实施例。可选地，在本实施例中，上述处理器用于运行程序，其中，上述程序运行时执行上述任意一种三维拼接方法。According to an embodiment of the present application, a processor embodiment is also provided. Optionally, in this embodiment, the above-mentioned processor is configured to run a program, wherein any one of the above-mentioned three-dimensional stitching methods is executed when the above-mentioned program is running.
本申请实施例提供了一种设备，设备包括处理器、存储器及存储在存储器上并可在处理器上运行的程序，处理器执行程序时实现以下步骤：确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored on the memory and operable on the processor. When the processor executes the program, the following steps are implemented: determine the three-dimensional point and texture of the depth map of the current frame The first mapping relationship between the pixels of the map, and the first corresponding relationship between the identification points in different texture maps; according to the first mapping relationship and the first corresponding relationship, determine the depth map of the current frame and other frames A first relative positional relationship between the depth maps, wherein the other frame depth maps are depth maps different from the current frame depth map; based on the first relative positional relationship, the current frame depth map and the other frame depth maps 3D points in 3D stitching.
可选地，上述处理器执行程序时，还可以通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。Optionally, when the above-mentioned processor executes the program, it may also determine the second relative positional relationship between the above-mentioned depth camera and the above-mentioned texture camera by calibrating the depth camera and the above-mentioned texture camera; perform three-dimensional reconstruction based on the above-mentioned depth camera, and obtain the above-mentioned current A three-dimensional point of the frame depth map: based on the second relative positional relationship, the first mapping relationship is obtained by projecting the three-dimensional point onto the texture map collected by the texture camera.
可选地，上述计算机程序产品执行程序时，还可以确定上述标识点与上述像素点之间的第二映射关系；依据上述第一映射关系和上述第二映射关系，确定上述标识点与上述三维点之间的第三映射关系；依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系。Optionally, when the computer program product executes the program, it may also determine the second mapping relationship between the above-mentioned identification point and the above-mentioned pixel point; according to the above-mentioned first mapping relationship and the above-mentioned second mapping relationship, determine the above-mentioned identification point and the above-mentioned three-dimensional A third mapping relationship between points; according to the third mapping relationship and the first corresponding relationship, determine the first relative position relationship.
可选地，上述处理器执行程序时，还可以依据上述第三映射关系和上述第一对应关系，确定上述当前帧深度图与上述其他帧深度图中的三维点之间的第二对应关系；基于上述第二对应关系确定上述第一相对位置关系。Optionally, when the above-mentioned processor executes the program, it may also determine a second corresponding relationship between the above-mentioned current frame depth map and the above-mentioned three-dimensional points in other frame depth maps according to the above-mentioned third mapping relationship and the above-mentioned first corresponding relationship; The first relative positional relationship is determined based on the second corresponding relationship.
可选地，上述处理器执行程序时，还可以基于上述第一相对位置关系对上述当前帧深度图与上述其他帧深度图中的三维点进行初始三维拼接；在上述初始三维拼接的基础上，采用迭代最近点算法对上述当前帧深度图与上述其他帧深度图中的三维点进行精准三维拼接。Optionally, when the above-mentioned processor executes the program, it may also perform initial three-dimensional splicing of the three-dimensional points in the above-mentioned current frame depth map and the above-mentioned other frame depth maps based on the above-mentioned first relative position relationship; on the basis of the above-mentioned initial three-dimensional splicing, The iterative closest point algorithm is used to perform precise three-dimensional splicing of the three-dimensional points in the above-mentioned current frame depth map and the above-mentioned other frame depth maps.
可选地，上述处理器执行程序时，还可以通过目标提取算法提取上述纹理图中的每个上述角点，其中，上述目标提取算法包括如下至少之一：Harris角点提取算法、SIFT算法、SURF特征提取算法、FAST角点检测算法、AGAST角点检测算法、BRISK特征提取算法、FREAK特征提取算法、ORB特征提取算法。Optionally, when the above-mentioned processor executes the program, each of the above-mentioned corner points in the above-mentioned texture map may also be extracted through a target extraction algorithm, wherein the above-mentioned target extraction algorithm includes at least one of the following: Harris corner point extraction algorithm, SIFT algorithm, SURF feature extraction algorithm, FAST corner detection algorithm, AGAST corner detection algorithm, BRISK feature extraction algorithm, FREAK feature extraction algorithm, ORB feature extraction algorithm.
本申请还提供了一种计算机程序产品，当在数据处理设备上执行时，适于执行初始化有如下方法步骤的程序：确定当前帧深度图的三维点与纹理图的像素点之间的第一映射关系，以及不同纹理图中的标识点之间的第一对应关系；依据上述第一映射关系和上述第一对应关系，确定上述当前帧深度图与其他帧深度图之间的第一相对位置关系，其中，上述其他帧深度图为与上述当前帧深度图不同的深度图；基于上述第一相对位置关系，对上述当前帧深度图与上述其他帧深度图中的三维点进行三维拼接。The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: determining the first distance between the three-dimensional point of the depth map of the current frame and the pixel point of the texture map A mapping relationship, and a first corresponding relationship between identification points in different texture maps; according to the first mapping relationship and the first corresponding relationship, determine the first relative position between the current frame depth map and other frame depth maps relationship, wherein, the above-mentioned other frame depth map is a depth map different from the above-mentioned current frame depth map; based on the above-mentioned first relative position relationship, three-dimensional splicing is performed on the above-mentioned current frame depth map and the three-dimensional points in the above-mentioned other frame depth map.
可选地，上述计算机程序产品执行程序时，还可以通过标定深度相机和纹理相机，确定出上述深度相机和上述纹理相机之间的第二相对位置关系；基于上述深度相机进行三维重建，获取上述当前帧深度图的三维点；基于上述第二相对位置关系，通过将上述三维点投影至上述纹理相机采集的上述纹理图上，得到上述第一映射关系。Optionally, when the above-mentioned computer program product executes the program, the second relative positional relationship between the above-mentioned depth camera and the above-mentioned texture camera can also be determined by calibrating the depth camera and the above-mentioned texture camera; three-dimensional reconstruction is performed based on the above-mentioned depth camera, and the above-mentioned The 3D point of the depth map of the current frame: based on the second relative positional relationship, the first mapping relationship is obtained by projecting the 3D point onto the texture map collected by the texture camera.
可选地，上述计算机程序产品执行程序时，还可以确定上述标识点与上述像素点之间的第二映射关系；依据上述第一映射关系和上述第二映射关系，确定上述标识点与上述三维点之间的第三映射关系；依据上述第三映射关系和上述第一对应关系，确定上述第一相对位置关系。Optionally, when the computer program product executes the program, it may also determine the second mapping relationship between the above-mentioned identification point and the above-mentioned pixel point; according to the above-mentioned first mapping relationship and the above-mentioned second mapping relationship, determine the above-mentioned identification point and the above-mentioned three-dimensional A third mapping relationship between points; according to the third mapping relationship and the first corresponding relationship, determine the first relative position relationship.
可选地，上述计算机程序产品执行程序时，还可以依据上述第三映射关系和上述第一对应关系，确定上述当前帧深度图与上述其他帧深度图中的三维点之间的第二对应关系；基于上述第二对应关系确定上述第一相对位置关系。Optionally, when the above computer program product executes the program, it may also determine the second corresponding relationship between the above-mentioned current frame depth map and the three-dimensional points in the above-mentioned other frame depth maps according to the above-mentioned third mapping relationship and the above-mentioned first corresponding relationship. ; Determine the first relative positional relationship based on the second corresponding relationship.
可选地，上述计算机程序产品执行程序时，还可以基于上述第一相对位置关系对上述当前帧深度图与上述其他帧深度图中的三维点进行初始三维拼接；在上述初始三维拼接的基础上，采用迭代最近点算法对上述当前帧深度图与上述其他帧深度图中的三维点进行精准三维拼接。Optionally, when the above-mentioned computer program product executes the program, an initial three-dimensional splicing may be performed on the above-mentioned current frame depth map and the three-dimensional points in the above-mentioned other frame depth maps based on the above-mentioned first relative positional relationship; on the basis of the above-mentioned initial three-dimensional splicing , using an iterative closest point algorithm to perform precise three-dimensional stitching on the above-mentioned current frame depth map and the above-mentioned three-dimensional points in other frame depth maps.
可选地，上述计算机程序产品执行程序时，还可以通过目标提取算法提取上述纹理图中的每个上述角点，其中，上述目标提取算法包括如下至少之一：Harr is角点提取算法、SIFT算法、SURF特征提取算法、FAST角点检测算法、AGAST角点检测算法、BRISK特征提取算法、FREAK特征提取算法、ORB特征提取算法。Optionally, when the above-mentioned computer program product executes the program, each of the above-mentioned corner points in the above-mentioned texture map may also be extracted by a target extraction algorithm, wherein the above-mentioned target extraction algorithm includes at least one of the following: Harris corner point extraction algorithm, SIFT Algorithm, SURF feature extraction algorithm, FAST corner detection algorithm, AGAST corner detection algorithm, BRISK feature extraction algorithm, FREAK feature extraction algorithm, ORB feature extraction algorithm.
上述本发明实施例序号仅仅为了描述，不代表实施例的优劣。The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.
在本发明的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。In the above-mentioned embodiments of the present invention, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，可以为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed technical content can be realized in other ways. Wherein, the device embodiments described above are only illustrative. For example, the division of the units may be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or may be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of units or modules may be in electrical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-OnlyMemory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media capable of storing program codes such as U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk.
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811163271.3ACN109472741B (en) | 2018-09-30 | 2018-09-30 | Three-dimensional splicing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811163271.3ACN109472741B (en) | 2018-09-30 | 2018-09-30 | Three-dimensional splicing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109472741A CN109472741A (en) | 2019-03-15 |
| CN109472741Btrue CN109472741B (en) | 2023-05-30 |
Family
ID=65663294
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811163271.3AActiveCN109472741B (en) | 2018-09-30 | 2018-09-30 | Three-dimensional splicing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109472741B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110096922B (en)* | 2019-05-08 | 2022-07-12 | 深圳市易尚展示股份有限公司 | Method and device for processing coding points, computer equipment and storage medium |
| CN118379469B (en)* | 2024-05-28 | 2024-11-29 | 先临三维科技股份有限公司 | Scanning method, electronic device and computer readable storage medium |
| CN119624768B (en)* | 2025-02-14 | 2025-06-13 | 先临三维科技股份有限公司 | Three-dimensional data splicing method and device, storage medium and electronic device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102945565A (en)* | 2012-10-18 | 2013-02-27 | 深圳大学 | Three-dimensional photorealistic reconstruction method and system for objects and electronic device |
| CN103886593A (en)* | 2014-03-07 | 2014-06-25 | 华侨大学 | Method for detecting hook face circular hole based on three-dimensional point cloud |
| CN104299211A (en)* | 2014-09-25 | 2015-01-21 | 周翔 | Free-moving type three-dimensional scanning method |
| CN107680039A (en)* | 2017-09-22 | 2018-02-09 | 武汉中观自动化科技有限公司 | A kind of point cloud method and system based on white light scanning instrument |
| CN108267097A (en)* | 2017-07-17 | 2018-07-10 | 杭州先临三维科技股份有限公司 | Three-dimensional reconstruction method and device based on binocular three-dimensional scanning system |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000348199A (en)* | 1999-06-04 | 2000-12-15 | Minolta Co Ltd | Method and device for texture mapping |
| JP4680104B2 (en)* | 2006-03-22 | 2011-05-11 | 日本電信電話株式会社 | Panorama image creation method |
| US8463073B2 (en)* | 2010-11-29 | 2013-06-11 | Microsoft Corporation | Robust recovery of transform invariant low-rank textures |
| CN103198523B (en)* | 2013-04-26 | 2016-09-21 | 清华大学 | A kind of three-dimensional non-rigid body reconstruction method based on many depth maps and system |
| CN103761765B (en)* | 2013-12-25 | 2017-12-19 | 浙江慧谷信息技术有限公司 | Three-dimensional object model texture mapping method based on mapped boundaries optimization |
| CN104008571B (en)* | 2014-06-12 | 2017-01-18 | 深圳奥比中光科技有限公司 | Human body model obtaining method and network virtual fitting system based on depth camera |
| CN105279508A (en)* | 2015-09-08 | 2016-01-27 | 哈尔滨工程大学 | Medical image classification method based on KAP digraph model |
| CN106568394A (en)* | 2015-10-09 | 2017-04-19 | 西安知象光电科技有限公司 | Hand-held three-dimensional real-time scanning method |
| CN106952331B (en)* | 2017-02-28 | 2020-12-08 | 深圳信息职业技术学院 | A three-dimensional model-based texture mapping method and device |
| CN108694740A (en)* | 2017-03-06 | 2018-10-23 | 索尼公司 | Information processing equipment, information processing method and user equipment |
| CN107784687A (en)* | 2017-09-22 | 2018-03-09 | 武汉中观自动化科技有限公司 | A kind of three-dimensional rebuilding method and system based on white light scanning instrument |
| CN108470323B (en)* | 2018-03-13 | 2020-07-31 | 京东方科技集团股份有限公司 | An image stitching method, computer equipment and display device |
- 2018
- 2018-09-30CNCN201811163271.3Apatent/CN109472741B/enactiveActive
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102945565A (en)* | 2012-10-18 | 2013-02-27 | 深圳大学 | Three-dimensional photorealistic reconstruction method and system for objects and electronic device |
| CN103886593A (en)* | 2014-03-07 | 2014-06-25 | 华侨大学 | Method for detecting hook face circular hole based on three-dimensional point cloud |
| CN104299211A (en)* | 2014-09-25 | 2015-01-21 | 周翔 | Free-moving type three-dimensional scanning method |
| CN108267097A (en)* | 2017-07-17 | 2018-07-10 | 杭州先临三维科技股份有限公司 | Three-dimensional reconstruction method and device based on binocular three-dimensional scanning system |
| CN107680039A (en)* | 2017-09-22 | 2018-02-09 | 武汉中观自动化科技有限公司 | A kind of point cloud method and system based on white light scanning instrument |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109472741A (en) | 2019-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11830141B2 (en) | Systems and methods for 3D facial modeling | |
| CN109389665B (en) | Texture acquisition method, device, device and storage medium for three-dimensional model | |
| US8781161B2 (en) | Image processing method and apparatus for generating a 3D model of a target object | |
| US9898856B2 (en) | Systems and methods for depth-assisted perspective distortion correction | |
| US9420265B2 (en) | Tracking poses of 3D camera using points and planes | |
| WO2021115071A1 (en) | Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device | |
| CN107563304B (en) | Terminal device unlocking method and device, and terminal device | |
| JP5015126B2 (en) | Image generation method, image authentication method, image generation apparatus, image authentication apparatus, program, and recording medium | |
| CN111480342B (en) | Encoding device, encoding method, decoding device, decoding method, and storage medium | |
| CN111091063A (en) | Living body detection method, device and system | |
| CN107483845B (en) | Photographing method and device | |
| CN107480613A (en) | Face identification method, device, mobile terminal and computer-readable recording medium | |
| CN108230395A (en) | Stereoscopic image is calibrated and image processing method, device, storage medium and electronic equipment | |
| WO2021136386A1 (en) | Data processing method, terminal, and server | |
| WO2019169884A1 (en) | Image saliency detection method and device based on depth information | |
| CN109472741B (en) | Three-dimensional splicing method and device | |
| CN109712230B (en) | Three-dimensional model supplement method, device, storage medium and processor | |
| JP2015133101A (en) | How to configure scene image descriptors | |
| CN110798677A (en) | Three-dimensional scene modeling method and device, electronic device, readable storage medium and computer equipment | |
| CN111882655A (en) | Method, apparatus, system, computer device and storage medium for three-dimensional reconstruction | |
| CN111833441A (en) | A method and device for face 3D reconstruction based on multi-camera system | |
| CN107370952A (en) | Image capturing method and device | |
| CN116167925A (en) | Image restoration method, device, electronic equipment and computer readable storage medium | |
| JP2006113832A (en) | Stereo image processing apparatus and program | |
| CN116704111B (en) | Image processing method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |