技术领域technical field
本申请涉及计算机技术领域，尤其涉及一种相机标定方法、装置、电子设备和存储介质。The present application relates to the field of computer technology, and in particular to a camera calibration method, device, electronic equipment and storage medium.
背景技术Background technique
近年来，基于计算机视觉的手势识别技术发展迅速。这种技术不需要借助可穿戴设备和外部传感器，只需要通过摄像头等视频捕捉设备，再利用机器学习和深度学习等技术，即可实现对人体手势的识别。手势识别技术又可以分为静态手势识别和动态手势识别(包括二维动态手势识别和三维动态手势识别)，其中静态手势识别和二维动态手势识别因为只需要二维摄像系统即可实现，获得了广泛的应用。In recent years, gesture recognition technology based on computer vision has developed rapidly. This technology does not require the use of wearable devices and external sensors. It only needs to use video capture devices such as cameras, and then use technologies such as machine learning and deep learning to realize the recognition of human gestures. Gesture recognition technology can be divided into static gesture recognition and dynamic gesture recognition (including two-dimensional dynamic gesture recognition and three-dimensional dynamic gesture recognition), of which static gesture recognition and two-dimensional dynamic gesture recognition can be realized because only two-dimensional camera system is needed, and the obtained a wide range of applications.
目前，视频会议成为了人们工作和生活中不可或缺少的一种交互方式。人们在视频会议时可能有画符号、图形等特殊文本的演示需求，现有的视频会议交互方式难以高效直接地传递这种信息。At present, video conferencing has become an indispensable interactive mode in people's work and life. People may have special text presentation requirements such as drawing symbols and graphics during video conferences, and the existing video conference interaction methods are difficult to efficiently and directly transmit such information.
发明内容Contents of the invention
本公开的实施例提供了一种相机标定方法、装置、电子设备和存储介质。Embodiments of the present disclosure provide a camera calibration method, device, electronic equipment and storage medium.
第一方面，本公开的实施例提供了一种相机标定方法，包括：根据已初始标定的至少两个相机之间的相对位姿，构建包括至少两个相机的三维场景，其中，三维场景中包括目标空间；响应于检测到目标空间在三维场景中的位置发生改变，获取标定物位于目标空间内不同位置处的标定物的第一图像；响应于确定图像采集终止条件满足，根据所获取的第一图像对至少两个相机的初始标定结果进行优化。In a first aspect, embodiments of the present disclosure provide a camera calibration method, including: constructing a three-dimensional scene including at least two cameras according to the relative poses between at least two cameras that have been initially calibrated, wherein, in the three-dimensional scene Including the target space; in response to detecting that the position of the target space in the three-dimensional scene changes, acquiring the first image of the calibration object at a different position in the target space; in response to determining that the image acquisition termination condition is satisfied, according to the acquired The first image optimizes the initial calibration results of at least two cameras.
第二方面，本公开的实施例提供了一种相机标定装置，包括：三维场景构建单元，被配置成根据已初始标定的至少两个相机之间的相对位姿，构建包括至少两个相机的三维场景，其中，三维场景中包括目标空间；第一图像获取单元，被配置成响应于检测到目标空间在三维场景中的位置发生改变，获取标定物位于目标空间内不同位置处的标定物的第一图像；标定结果优化单元，被配置成响应于确定图像采集终止条件满足，根据所获取的第一图像对至少两个相机的初始标定结果进行优化。In a second aspect, an embodiment of the present disclosure provides a camera calibration device, including: a 3D scene construction unit configured to construct a scene including at least two cameras according to the relative poses between at least two cameras that have been initially calibrated. A three-dimensional scene, wherein the three-dimensional scene includes a target space; the first image acquisition unit is configured to acquire images of calibration objects located at different positions in the target space in response to detecting a change in the position of the target space in the three-dimensional scene The first image; a calibration result optimization unit configured to optimize the initial calibration results of at least two cameras according to the acquired first image in response to determining that the image acquisition termination condition is satisfied.
第三方面，本公开的实施例提供了一种电子设备，包括存储器、处理器、总线及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现如第一方面所描述的相机标定方法。In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory, a processor, a bus, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the The camera calibration method described in the first aspect.
第四方面，本公开的实施例提供了一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如第一方面所描述的相机标定方法。In a fourth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the camera calibration method as described in the first aspect is implemented.
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.
附图说明Description of drawings
附图用于更好地理解本方案，不构成对本公开的限定。其中：The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:
图1为本公开的相机标定方法的一个实施例可以应用于其中的示例性系统架构图；FIG. 1 is an exemplary system architecture diagram to which an embodiment of the camera calibration method of the present disclosure can be applied;
图2为本公开的相机标定方法的一个实施例的流程示意图；FIG. 2 is a schematic flow diagram of an embodiment of the camera calibration method of the present disclosure;
图3为本公开的相机标定方法的另一个实施例的流程示意图；FIG. 3 is a schematic flowchart of another embodiment of the camera calibration method of the present disclosure;
图4为本公开的相机标定方法的一个应用场景的示意图；FIG. 4 is a schematic diagram of an application scenario of the camera calibration method of the present disclosure;
图5为本公开的相机标定装置的一个实施例的结构示意图；FIG. 5 is a schematic structural diagram of an embodiment of the camera calibration device of the present disclosure;
图6为本公开的电子设备的一个实施例的结构示意图。FIG. 6 is a schematic structural diagram of an embodiment of the electronic device of the present disclosure.
具体实施方式Detailed ways
应该指出，以下详细说明都是示例性的，旨在对本公开提供进一步的说明。除非另有指明，本文中使用的所有技术和科学术语具有与本公开所属技术领域的普通技术人员通常理解的相同含义。It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
需要注意的是，这里所使用的术语仅是为了描述具体实施方式，而非意图限制根据本公开的示例性实施方式。如在这里所使用的，除非上下文另外明确指出，否则单数形式也意图包括复数形式，此外，还应当理解的是，当在本说明书中使用术语“包含”和/或“包括”时，其指明存在特征、步骤、操作、器件、组件和/或它们的组合。It should be noted that the terminology used herein is only for describing specific embodiments, and is not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and/or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and/or combinations thereof.
在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互组合。In the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.
为使本公开的技术方案及优点更加清楚明白，以下结合附图及具体实施例，对本公开作进一步详细的说明。In order to make the technical solutions and advantages of the present disclosure clearer, the present disclosure will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.
图1示出了可以应用本公开的相机标定方法或相机标定装置的实施例的示例性系统架构100。FIG. 1 shows an exemplary system architecture 100 to which embodiments of the camera calibration method or camera calibration apparatus of the present disclosure can be applied.
如图1所示，系统架构100可以包括终端设备101，网络102和相机103、104、105。网络102用以在终端设备101和相机103、104、105之间提供通信链路的介质。网络102可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。As shown in FIG. 1 , a system architecture 100 may include a terminal device 101 , a network 102 and cameras 103 , 104 , and 105 . The network 102 serves as a medium for providing communication links between the terminal device 101 and the cameras 103 , 104 , 105 . Network 102 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.
用户可以使用终端设备101通过网络102接收相机103、104、105采集的图像。终端设备101上可以安装有各种通讯客户端应用，例如图像处理类应用等。The user can use the terminal device 101 to receive images collected by the cameras 103 , 104 , and 105 through the network 102 . Various communication client applications, such as image processing applications, can be installed on the terminal device 101 .
终端设备101可以是硬件，也可以是软件。当终端设备101为硬件时，可以是各种电子设备，包括但不限于智能手机、平板电脑、车载电脑、膝上型便携计算机和台式计算机等等。当终端设备101为软件时，可以安装在上述所列举的电子设备中。其可以实现成多个软件或软件模块(例如用来提供分布式服务)，也可以实现成单个软件或软件模块。在此不做具体限定。The terminal device 101 may be hardware or software. When the terminal device 101 is hardware, it may be various electronic devices, including but not limited to smart phones, tablet computers, vehicle-mounted computers, laptop computers, desktop computers, and the like. When the terminal device 101 is software, it can be installed in the electronic devices listed above. It can be implemented as a plurality of software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.
相机103、104、105可以是各种可以用于采集图像的图像采集装置。相机103、104、105可以放置在不同的位置，朝向不同的角度，以从各个角度采集目标对象的图像。相机103、104、105具有公共视野。The cameras 103, 104, 105 can be various image acquisition devices that can be used to acquire images. The cameras 103, 104, 105 can be placed at different positions and facing different angles, so as to collect images of the target object from various angles. The cameras 103, 104, 105 have a common field of view.
需要说明的是，本公开实施例所提供的相机标定方法一般由终端设备101执行。相应地，相机标定装置一般设置于终端设备101中。It should be noted that the camera calibration method provided by the embodiment of the present disclosure is generally executed by the terminal device 101 . Correspondingly, the camera calibration device is generally set in the terminal device 101 .
应该理解，图1中的终端设备、网络和相机的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和相机。It should be understood that the numbers of terminal devices, networks and cameras in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and cameras.
图2示出了本公开的相机标定方法的一个实施例的流程200。如图2所示，本实施例的相机标定方法可以包括以下步骤：FIG. 2 shows a flow 200 of an embodiment of the camera calibration method of the present disclosure. As shown in Figure 2, the camera calibration method of this embodiment may include the following steps:
步骤201，根据已初始标定的至少两个相机之间的相对位姿，构建包括上述至少两个相机的三维场景。Step 201, construct a 3D scene including the at least two cameras based on the initially calibrated relative poses between the at least two cameras.
本实施例中，相机标定方法的执行主体(例如图1所示的终端设备101)可以预先对至少两个相机进行初始标定。具体的，执行主体可以采用现有的标定算法(例如直接线性变换法、Tsai两步法、张正友平面标定法等)对各相机进行初始标定。可以理解的是，经初始标定后的各相机之间具有公共视野。经初始标定后的各相机之间存在相对位姿，执行主体可以根据各相机之间的相对位置，构件包括各相机的三维场景。具体的，执行主体可以通过预设的三维场景构建应用，通过控制命令，首先固定其中一个相机，然后根据各相机之间的相对位姿，确定其它相机相对于已固定的相机的位置和角度，从而得到三维场景。上述三维场景中可以包括目标空间。这里，目标空间可以是一个尺寸固定的立体空间，其可以在三维场景中任意位置处。具体的，上述目标空间的位置可变，从而实现对三维场景的扫场。In this embodiment, the subject executing the camera calibration method (for example, the terminal device 101 shown in FIG. 1 ) may perform initial calibration on at least two cameras in advance. Specifically, the execution subject may use existing calibration algorithms (such as direct linear transformation method, Tsai two-step method, Zhang Zhengyou plane calibration method, etc.) to perform initial calibration on each camera. It can be understood that the initially calibrated cameras have a common field of view. After the initial calibration, there is a relative pose between the cameras, and the execution subject can construct a 3D scene including the cameras according to the relative positions of the cameras. Specifically, the execution subject can build an application through a preset 3D scene, firstly fix one of the cameras through control commands, and then determine the positions and angles of other cameras relative to the fixed cameras according to the relative poses between the cameras. Thus a three-dimensional scene is obtained. The above three-dimensional scene may include a target space. Here, the target space may be a three-dimensional space with a fixed size, which may be at any position in the three-dimensional scene. Specifically, the position of the above-mentioned target space is variable, so as to realize sweeping of the three-dimensional scene.
步骤202，响应于检测到目标空间在三维场景中的位置发生改变，获取位于目标空间内不同位置处的标定物的第一图像。Step 202, in response to detecting a change in the position of the target space in the three-dimensional scene, acquire first images of calibration objects at different positions in the target space.
执行主体可以实时监测目标空间在三维场景中的位置，如果确定目标空间在三维场景中的位置发生改变，说明目标空间移动到了一个新的位置。此时需要采集此位置处的标定物的图像。标定物可以是各种可以用于标定的物品，例如可以是手指、直杆等等。标定物可以在目标空间中移动，各相机可以在标定物在目标空间中改变位置时采集其图像，这里称为第一图像。The execution subject can monitor the position of the target space in the 3D scene in real time, and if it is determined that the position of the target space in the 3D scene changes, it means that the target space has moved to a new position. At this time, an image of the calibration object at this position needs to be collected. The calibration object can be various items that can be used for calibration, for example, it can be a finger, a straight rod, and the like. The calibration object can move in the target space, and each camera can collect its image when the calibration object changes position in the target space, which is referred to as the first image here.
步骤203，响应于确定图像采集终止条件满足，根据所获取的第一图像对至少两个相机的初始标定结果进行优化。Step 203, in response to determining that the image acquisition termination condition is met, optimize the initial calibration results of at least two cameras according to the acquired first image.
执行主体可以实时判断图像采集终止条件是否满足，如果不满足，说明对各相机的优化标定仍然需要更多的图像。如果满足，则说明不需要再采集标定物的图像。则可以根据所获取的各第一图像对各相机的初始标定结果进行优化。具体的，执行主体可以根据各第一图像，通过现有的标定算法再次确定出各相机之间的相对位姿。然后，利用此相对位姿对初始位姿进行优化。例如，可以调整初始位姿中的某些参数，或者将两次确定的相对位姿输入预先训练的迭代优化模型进行迭代优化。The execution subject can judge in real time whether the image acquisition termination condition is satisfied. If not, it means that more images are still needed for the optimal calibration of each camera. If it is satisfied, it means that there is no need to collect images of calibration objects. Then the initial calibration results of the cameras can be optimized according to the acquired first images. Specifically, the execution subject may re-determine the relative poses between the cameras through the existing calibration algorithm according to the first images. Then, the initial pose is optimized using this relative pose. For example, some parameters in the initial pose can be adjusted, or the relative pose determined twice can be input into a pre-trained iterative optimization model for iterative optimization.
本公开的上述实施例提供的相机标定方法，可以对已初始标定的各相机之间的相对位姿进行优化，以提高各相机的标定准确度。The camera calibration method provided by the above-mentioned embodiments of the present disclosure can optimize the relative poses of the initially calibrated cameras, so as to improve the calibration accuracy of each camera.
继续参见图3，其示出了根据本公开的相机标定方法的另一个实施例的流程300。如图3所示，本实施例中可以首先通过步骤301～302对各相机进行初始标定，然后利用步骤303～305对各相机的初始标定结果进行优化。本实施例的方法可以包括以下步骤：Continue referring to FIG. 3 , which shows a flow 300 of another embodiment of the camera calibration method according to the present disclosure. As shown in FIG. 3 , in this embodiment, the initial calibration of each camera can be performed first through steps 301-302, and then the initial calibration results of each camera can be optimized by using steps 303-305. The method of this embodiment may include the following steps:
步骤301，获取由至少两个相机采集的、在公共视野内不同位置处的标定物的第二图像。Step 301 , acquiring second images of a calibration object at different positions in a common field of view collected by at least two cameras.
本实施例中，执行主体可以首先获取由至少两个相机采集的、在公共视野内不同位置处的标定物的第二图像。这里的标定物可以与图2所示实施例中的标定物相同。执行主体可以检测各第二图像，如果标定物在公共视野内的位置发生变化，则将第二图像作为有效图像存储起来。In this embodiment, the execution subject may first acquire the second images of the calibration object at different positions in the common field of view captured by at least two cameras. The calibration object here can be the same as the calibration object in the embodiment shown in FIG. 2 . The execution subject may detect each second image, and store the second image as a valid image if the position of the calibration object in the public field of view changes.
步骤302，根据标定物在各第二图像中的位置，对至少至少两个相机进行初始标定。Step 302: Initially calibrate at least two cameras according to the positions of the calibration objects in each second image.
执行主体可以对各第二图像进行目标识别，确定出标定物在第二图像中的位置。具体的，执行主体可以利用预先训练的神经网络确定出标定物所在区域。然后利用最小二乘法确定出标定物的特定区域的位置。在一些具体的实践中，标定物为人的食指，执行主体可以利用预先训练的神经网络确定出食指所占的区域。然后利用最小二乘法确定食指指尖的位置。The execution subject may perform target recognition on each second image, and determine the position of the calibration object in the second image. Specifically, the execution subject can use the pre-trained neural network to determine the area where the calibration object is located. Then use the least squares method to determine the position of the specific area of the calibration object. In some specific practices, the calibration object is a human index finger, and the execution subject can use a pre-trained neural network to determine the area occupied by the index finger. Then the position of the fingertip of the index finger is determined by the method of least squares.
在确定了食指指尖在各第二图像中的位置，可以利用现有的标定算法对各相机进行初始标定，得到初始标定结果。After determining the position of the fingertip of the index finger in each second image, an existing calibration algorithm can be used to perform initial calibration on each camera to obtain an initial calibration result.
步骤303，根据已初始标定的至少两个相机之间的相对位姿，确定每个相机相对于其它相机的位置、朝向和视野；根据所确定的位置、朝向和视野，将上述至少两个相机绘制在三维场景中。Step 303, according to the relative pose between at least two cameras that have been initially calibrated, determine the position, orientation and field of view of each camera relative to other cameras; Draw in a 3D scene.
在对各相机进行初始标定后，可以根据各相机之间的相对位姿，确定出每个相机相对于其它相机的位置、朝向和视野。然后，执行主体可以首先将其中一个相机作为坐标系原点，然后根据其它相机相对于该相机的位置、朝向和视野，确定出其它相机在三维场景中的位置、朝向和视野。构建的三维场景中可以包括目标空间，这里，目标空间可以是一个半透明的立方体，这样也方便用户通过终端设备查看。目标空间可以在三维场景中不断改变位置。After the initial calibration of each camera, the position, orientation and field of view of each camera relative to other cameras can be determined according to the relative poses between the cameras. Then, the execution subject can first use one of the cameras as the origin of the coordinate system, and then determine the position, orientation, and field of view of other cameras in the 3D scene according to the positions, orientations, and fields of view of other cameras relative to the camera. The constructed 3D scene may include a target space. Here, the target space may be a translucent cube, which is also convenient for users to view through a terminal device. The object space can continuously change position in the 3D scene.
步骤304，响应于检测到所述目标空间在所述三维场景中的位置发生改变，获取标定物位于所述目标空间内不同位置处的标定物的第一图像。Step 304, in response to detecting that the position of the target space in the three-dimensional scene changes, acquire first images of the calibration objects at different positions in the target space.
步骤305，确定在目标空间位置固定的过程中，所获取的第一图像中有效图像的数量；响应于确定有效图像的数量大于第一预设阈值，控制改变目标空间在三维场景中的位置。Step 305, determine the number of effective images in the acquired first image during the process of fixing the object space position; in response to determining that the number of effective images is greater than a first preset threshold, control changing the position of the object space in the 3D scene.
当目标空间在三维场景中的位置固定时，执行主体可以控制各相机不断采集标志物的图像。同时可以控制标定物在目标空间中不断移动以改变位置。执行主体可以对采集的每张图像进行评估，确定是否为有效图像。这里有效图像可以理解为清晰不模糊、边界清楚的图像。如果有效图像的数量大于第一预设阈值，则可以认定在该位置处的目标空间中标志物的数量足够，则可以控制改变目标空间在三维场景中的位置。When the position of the target space in the 3D scene is fixed, the execution subject can control the cameras to continuously collect images of the landmarks. At the same time, the calibration object can be controlled to move continuously in the target space to change its position. The execution subject can evaluate each image collected to determine whether it is a valid image. Here, an effective image can be understood as an image that is clear, not fuzzy, and has clear boundaries. If the number of valid images is greater than the first preset threshold, it can be determined that the number of markers in the target space at this position is sufficient, and the position of the target space in the three-dimensional scene can be controlled to be changed.
步骤306，确定不同位置的目标空间占三维场景的比例；响应于确定上述比例大于第二预设阈值，确定图像采集终止条件满足。Step 306, determine the proportion of the target space at different positions in the three-dimensional scene; in response to determining that the above proportion is greater than a second preset threshold, determine that the image acquisition termination condition is met.
执行主体可以实时计算不同位置的目标空间站三维场景的比例。具体的，执行主体可以将目标空间向三维场景的地面进行投影，确定投影区域的面积。目标空间每次移动后，可以将叠加投影区域的面积。同时执行主体还可以计算各相机的公共视野向三维场景的底面的投影面积。将二者相除即可得到比例。如果上述比例大于第二预设阈值，则说明扫场完毕，不需要继续采集图像，即图像采集终止条件满足。如果小于，说明目标空间在三维场景中的覆盖度不够，需要继续移动目标空间。The execution subject can calculate the ratio of the 3D scene of the target space station in different positions in real time. Specifically, the execution subject may project the target space onto the ground of the three-dimensional scene, and determine the area of the projection area. After each movement of the target space, the area of the projection area can be superimposed. At the same time, the execution subject may also calculate the projected area of the common field of view of each camera to the bottom surface of the 3D scene. Divide the two to get the ratio. If the above-mentioned ratio is greater than the second preset threshold, it means that the scanning is completed and there is no need to continue to collect images, that is, the image collection termination condition is satisfied. If it is smaller, it means that the coverage of the target space in the 3D scene is not enough, and it is necessary to continue to move the target space.
步骤307，响应于确定图像采集终止条件满足，根据第一图像以及第二图像，确定至少两个相机之间的相对位姿；对所确定的相对位姿进行合并迭代优化。Step 307, in response to determining that the image acquisition termination condition is met, determine the relative pose between the at least two cameras according to the first image and the second image; perform combined iterative optimization on the determined relative pose.
如果图像采集终止条件满足，则可以根据第一图像以及第二图像，确定至少两个相机之间的相对位姿。这里，执行主体可以将第一图像和第二图像输入预先训练的位姿确定模型中，模型的输出即为各相机之间的相对位姿。执行主体还可以将输出的相对位置继续输入到预先设置的迭代优化模型中，直至迭代优化模型的损失函数值收敛，从而得到优化后的相对位姿。If the image acquisition termination condition is satisfied, the relative pose between at least two cameras may be determined according to the first image and the second image. Here, the execution subject may input the first image and the second image into the pre-trained pose determination model, and the output of the model is the relative pose between the cameras. The execution subject can also continue to input the output relative position into the preset iterative optimization model until the loss function value of the iterative optimization model converges, so as to obtain the optimized relative pose.
继续参加图4，其示出了本实施例的相机标定方法的一个应用场景的示意图。在图4的应用场景中，用户可以将两个相机位置固定后，将食指在两个相机的公共视野中移动，通过利用两个相机不断采集用户食指的图像。终端中安装的图像处理应用通过对图像进行不断处理，完成对上述两个相机的初始标定。并在初始标定后利用两相机之间的相对位姿，构建出包括两个相机的三维场景。在上述三维场景中生成一个半透明立方体标识目标区域，并提示用户将食指指尖移动到该立方体内并缓慢移动，控制各相机不断采集食指指尖的图像。在完成扫场后，利用所采集的图像实现对各相机的标定。Continue referring to FIG. 4 , which shows a schematic diagram of an application scenario of the camera calibration method of this embodiment. In the application scenario shown in FIG. 4 , the user can fix the positions of the two cameras, and then move the index finger in the common field of view of the two cameras, and continuously collect images of the user's index finger by using the two cameras. The image processing application installed in the terminal completes the initial calibration of the above two cameras by continuously processing the images. And after the initial calibration, the relative pose between the two cameras is used to construct a three-dimensional scene including the two cameras. In the above three-dimensional scene, a translucent cube is generated to mark the target area, and the user is prompted to move the tip of the index finger into the cube and move slowly, and the cameras are controlled to continuously collect images of the tip of the index finger. After sweeping the field, use the collected images to calibrate each camera.
本公开的上述实施例提供的相机标定方法，不需要购买特定的标定物道具，只需要利用人的手指就可以完成相机的标定。从而不需要具有专业的技术和经验门槛就可以完成相机的标定，方便用户利用多个相机实现动态捕捉。The camera calibration method provided by the above-mentioned embodiments of the present disclosure does not need to purchase specific calibration objects and props, and only needs to use human fingers to complete the camera calibration. In this way, camera calibration can be completed without professional technical and experience thresholds, and it is convenient for users to use multiple cameras to realize dynamic capture.
进一步参考图5，作为对上述各图所示方法的实现，本公开提供了一种相机标定装置的一个实施例，该装置实施例与图2所示的方法实施例相对应，该装置具体可以应用于各种电子设备中。Further referring to FIG. 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a camera calibration device. The device embodiment corresponds to the method embodiment shown in FIG. 2 , and the device can specifically Used in various electronic equipment.
如图5所示，本实施例的相机标定装置500包括：三维场景构建单元501、第一图像获取单元502和标定结果优化单元503。As shown in FIG. 5 , the camera calibration apparatus 500 of this embodiment includes: a three-dimensional scene construction unit 501 , a first image acquisition unit 502 and a calibration result optimization unit 503 .
三维场景构建单元501，被配置成根据已初始标定的至少两个相机之间的相对位姿，构建包括至少两个相机的三维场景。其中，三维场景中包括目标空间。The 3D scene construction unit 501 is configured to construct a 3D scene including at least two cameras according to the initially calibrated relative poses between the at least two cameras. Wherein, the 3D scene includes the target space.
第一图像获取单元502，被配置成响应于检测到目标空间在三维场景中的位置发生改变，获取标定物位于目标空间内不同位置处的标定物的第一图像。The first image acquisition unit 502 is configured to, in response to detecting that the position of the target space changes in the three-dimensional scene, acquire first images of the calibration objects at different positions in the target space.
标定结果优化单元503，被配置成响应于确定图像采集终止条件满足，根据所获取的第一图像对至少两个相机的初始标定结果进行优化。The calibration result optimization unit 503 is configured to optimize the initial calibration results of at least two cameras according to the acquired first image in response to determining that the image acquisition termination condition is met.
在本实施例的一些可选的实现方式中，上述装置500还可以包括图5中未示出的第二图像获取单元和初始标定单元。In some optional implementation manners of this embodiment, the foregoing apparatus 500 may further include a second image acquisition unit and an initial calibration unit not shown in FIG. 5 .
第二图像获取单元，被配置成获取由至少两个相机采集的、在公共视野内不同位置处的标定物的第二图像。The second image acquisition unit is configured to acquire second images of the calibration object at different positions within the common field of view, acquired by at least two cameras.
初始标定单元，被配置成根据标定物在各第二图像中的位置，对至少至少两个相机进行初始标定。The initial calibration unit is configured to perform initial calibration on at least two cameras according to the position of the calibration object in each second image.
另外，在本申请的技术方案中，还提出了一种电子设备。In addition, in the technical solution of the present application, an electronic device is also proposed.
图6示出了本公开一实施例提供的一种电子设备的结构示意图。Fig. 6 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
如图6所示，该电子设备可以包括处理器601、存储器602、总线603以及存储在存储器602上并可在处理器601上运行的计算机程序，其中，处理器601和存储器602通过总线603完成相互间的通信。所述处理器601执行所述计算机程序时实现上述方法的步骤，例如包括：根据已初始标定的至少两个相机之间的相对位姿，构建包括至少两个相机的三维场景，其中，三维场景中包括目标空间；响应于检测到目标空间在三维场景中的位置发生改变，获取标定物位于目标空间内不同位置处的标定物的第一图像；响应于确定图像采集终止条件满足，根据所获取的第一图像对至少两个相机的初始标定结果进行优化。As shown in FIG. 6, the electronic device may include a processor 601, a memory 602, a bus 603, and a computer program stored in the memory 602 and operable on the processor 601, wherein the processor 601 and the memory 602 complete the process through the bus 603. mutual communication. When the processor 601 executes the computer program, the steps of the above method are implemented, for example, including: constructing a three-dimensional scene including at least two cameras according to the initially calibrated relative poses between the at least two cameras, wherein the three-dimensional scene Including the target space; in response to detecting that the position of the target space in the three-dimensional scene changes, obtain the first image of the calibration object at a different position in the target space; in response to determining that the image acquisition termination condition is satisfied, according to the acquired The first image of is optimized for the initial calibration results of at least two cameras.
另外，本公开一实施例中还提供了一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述方法的步骤，例如包括：根据已初始标定的至少两个相机之间的相对位姿，构建包括至少两个相机的三维场景，其中，三维场景中包括目标空间；响应于检测到目标空间在三维场景中的位置发生改变，获取标定物位于目标空间内不同位置处的标定物的第一图像；响应于确定图像采集终止条件满足，根据所获取的第一图像对至少两个相机的初始标定结果进行优化。In addition, an embodiment of the present disclosure also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented, for example, including: The relative poses between at least two cameras are constructed to construct a three-dimensional scene including at least two cameras, wherein the three-dimensional scene includes the target space; in response to detecting that the position of the target space in the three-dimensional scene changes, the location of the calibration object is acquired First images of calibration objects at different positions in the target space; in response to determining that the image acquisition termination condition is met, optimizing initial calibration results of at least two cameras according to the acquired first images.
综上所述，在本公开的技术方案中，不需要购买特定的标定物道具，只需要利用人的手指就可以完成相机的标定。从而不需要具有专业的技术和经验门槛就可以完成相机的标定，方便用户利用多个相机实现动态捕捉。To sum up, in the technical solution of the present disclosure, there is no need to purchase specific calibration object props, and the calibration of the camera can be completed only by using human fingers. In this way, camera calibration can be completed without professional technical and experience thresholds, and it is convenient for users to use multiple cameras to realize dynamic capture.
以上所述仅为本公开的较佳实施例而已，并不用以限制本公开，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开保护的范围之内。The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure within the scope of protection.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310375792.XACN116503484A (en) | 2023-04-10 | 2023-04-10 | Camera calibration method, device, electronic equipment and storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310375792.XACN116503484A (en) | 2023-04-10 | 2023-04-10 | Camera calibration method, device, electronic equipment and storage medium |
| Publication Number | Publication Date |
|---|---|
| CN116503484Atrue CN116503484A (en) | 2023-07-28 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310375792.XAPendingCN116503484A (en) | 2023-04-10 | 2023-04-10 | Camera calibration method, device, electronic equipment and storage medium |
| Country | Link |
|---|---|
| CN (1) | CN116503484A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114531560A (en)* | 2022-02-18 | 2022-05-24 | 深圳地平线机器人科技有限公司 | Video call method and device |
| CN114663518A (en)* | 2022-02-15 | 2022-06-24 | 深圳市如本科技有限公司 | Camera calibration method, system, terminal device and computer readable storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663518A (en)* | 2022-02-15 | 2022-06-24 | 深圳市如本科技有限公司 | Camera calibration method, system, terminal device and computer readable storage medium |
| CN114531560A (en)* | 2022-02-18 | 2022-05-24 | 深圳地平线机器人科技有限公司 | Video call method and device |
| Title |
|---|
| 傅子秋;张晓龙;余成;梁丹;梁冬泰;: "多场景下基于快速相机标定的柱面图像拼接方法", 光电工程, no. 04, 15 April 2020 (2020-04-15)* |
| 赫美琳;高明慧;李爽;杨开伟;: "一种单目相机标定算法研究", 数字通信世界, no. 05, 1 May 2018 (2018-05-01)* |
| Publication | Publication Date | Title |
|---|---|---|
| US11270460B2 (en) | Method and apparatus for determining pose of image capturing device, and storage medium | |
| US10043308B2 (en) | Image processing method and apparatus for three-dimensional reconstruction | |
| CN108200334B (en) | Image capturing method, device, storage medium and electronic device | |
| CN104317391B (en) | A kind of three-dimensional palm gesture recognition exchange method and system based on stereoscopic vision | |
| CN108520552A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN109144252B (en) | Object determination method, device, equipment and storage medium | |
| CN111527468A (en) | A method, device and device for remote interaction | |
| CN114690900B (en) | Input recognition method, device and storage medium in a virtual scene | |
| CN117372657B (en) | Key point rotation model training method and device, electronic device and storage medium | |
| CN113470112B (en) | Image processing method, device, storage medium and terminal | |
| JP6746419B2 (en) | Information processing apparatus, control method thereof, and computer program | |
| CN112083801A (en) | Gesture recognition system and method based on VR virtual office | |
| CN110782412A (en) | Image processing method and device, processor, electronic device and storage medium | |
| US9065972B1 (en) | User face capture in projection-based systems | |
| WO2016165614A1 (en) | Method for expression recognition in instant video and electronic equipment | |
| JP2016139396A (en) | User interface device, method and program | |
| CN114640833A (en) | Projection picture adjusting method and device, electronic equipment and storage medium | |
| CN112270242A (en) | Track display method and device, readable medium and electronic equipment | |
| KR102505951B1 (en) | Apparatus and method for providing image, and computer program recorded on computer readable medium for excuting the method | |
| CN113342157A (en) | Eyeball tracking processing method and related device | |
| CN113780045B (en) | Method and apparatus for training distance prediction model | |
| CN112818733B (en) | Information processing method, device, storage medium and terminal | |
| CN114241127A (en) | Panoramic image generation method and device, electronic equipment and medium | |
| CN116503484A (en) | Camera calibration method, device, electronic equipment and storage medium | |
| CN111258413A (en) | Control method and device of virtual object |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |