技术领域Technical Field
本公开涉及计算机技术领域，尤其是一种图像转换方法、装置、计算机可读存储介质及电子设备。The present disclosure relates to the field of computer technology, and in particular to an image conversion method, device, computer-readable storage medium, and electronic device.
背景技术Background technique
不同摄像头由于在相机内参、畸变系数等方面存在差异，会导致成像质量不同。基于深度学习的图像处理算法对训练数据集有很强的依赖性与适配性，若实际场景中采集数据的摄像头与采集训练数据的摄像头差异(在相机内参/畸变系数等方面)较大，算法模型在实际场景的预测效果通常会比较差。在实际应用中，实际场景使用的摄像头可能是多种多样的，为了适配实际场景，往往需要使用实际场景的摄像头重新采集一批数据进行模型训练。而数据采集的成本(人力/物力/时间等)通常很高。如何有效利用现有数据，生成实际摄像头下的数据是一个非常值得探究的问题。Different cameras have different imaging qualities due to differences in camera intrinsic parameters, distortion coefficients, etc. Image processing algorithms based on deep learning have a strong dependence on and adaptability to training data sets. If the camera that collects data in the actual scene is very different from the camera that collects training data (in terms of camera intrinsic parameters/distortion coefficients, etc.), the prediction effect of the algorithm model in the actual scene will usually be poor. In practical applications, the cameras used in actual scenes may be diverse. In order to adapt to the actual scenes, it is often necessary to use the cameras in the actual scenes to re-collect a batch of data for model training. However, the cost of data collection (manpower/material resources/time, etc.) is usually very high. How to effectively use existing data to generate data under actual cameras is a very worthwhile issue to explore.
目前生成不同摄像头下的数据的方法，通常采用基于深度学习的方法，比如生成对抗网络，进行风格迁移，生成贴合实际场景的数据。The current method of generating data under different camera conditions usually adopts deep learning-based methods, such as generative adversarial networks, to perform style transfer and generate data that fits the actual scene.
发明内容Summary of the invention
本公开的实施例提供了一种图像转换方法、装置、计算机可读存储介质及电子设备。Embodiments of the present disclosure provide an image conversion method, an apparatus, a computer-readable storage medium, and an electronic device.
本公开的实施例提供了一种图像转换方法，该方法包括：基于用于采集原始图像的原始相机的第一畸变参数和第一内参，对原始图像进行畸变矫正处理，得到矫正后图像；基于矫正后图像，确定待映射图像；基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像；基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像。An embodiment of the present disclosure provides an image conversion method, which includes: based on a first distortion parameter and a first internal parameter of an original camera used to acquire the original image, performing distortion correction processing on the original image to obtain a corrected image; based on the corrected image, determining an image to be mapped; based on the first internal parameter of the original camera and a second internal parameter of a target camera, mapping the image to be mapped to a two-dimensional image plane coordinate system of a target camera to obtain a mapped image; based on a second distortion parameter and a second internal parameter of the target camera, performing distortion restoration processing on the mapped image to obtain a target image.
根据本公开实施例的另一个方面，提供了一种图像转换装置，该装置包括：矫正模块，用于基于用于采集原始图像的原始相机的第一畸变参数和第一内参，对原始图像进行畸变矫正处理，得到矫正后图像；确定模块，用于基于矫正后图像，确定待映射图像；映射模块，用于基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像；还原模块，用于基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像。According to another aspect of an embodiment of the present disclosure, an image conversion device is provided, the device comprising: a correction module, used to perform distortion correction processing on an original image based on a first distortion parameter and a first internal parameter of an original camera used to acquire the original image, so as to obtain a corrected image; a determination module, used to determine an image to be mapped based on the corrected image; a mapping module, used to map the image to be mapped to a two-dimensional image plane coordinate system of a target camera based on a first internal parameter of the original camera and a second internal parameter of a target camera, so as to obtain a mapped image; and a restoration module, used to perform distortion restoration processing on the mapped image based on a second distortion parameter and a second internal parameter of the target camera, so as to obtain a target image.
根据本公开实施例的另一个方面，提供了一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，计算机程序用于执行上述图像转换方法。According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is used to execute the above-mentioned image conversion method.
根据本公开实施例的另一个方面，提供了一种电子设备，电子设备包括：处理器；用于存储处理器可执行指令的存储器；处理器，用于从存储器中读取可执行指令，并执行指令以实现上述图像转换方法。According to another aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing processor executable instructions; the processor, configured to read the executable instructions from the memory and execute the instructions to implement the above-mentioned image conversion method.
基于本公开上述实施例提供的图像转换方法、装置、计算机可读存储介质及电子设备，通过对原始图像进行畸变矫正处理，得到矫正后图像，然后基于矫正后图像，确定待映射图像，接着基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像，最后基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像，从而实现了根据相机内参和畸变系数即可得到模拟另一相机拍摄的不同风格的图像，实现方式更加简便高效。由于生成的图像的像素由原始图像的像素转变而来，从而使生成的图像更接近真实场景。Based on the image conversion method, device, computer-readable storage medium and electronic device provided by the above embodiments of the present disclosure, the original image is subjected to distortion correction processing to obtain a corrected image, and then based on the corrected image, the image to be mapped is determined, and then based on the first internal parameter of the original camera and the second internal parameter of the target camera, the image to be mapped is mapped to the two-dimensional image plane coordinate system of the target camera to obtain a mapped image, and finally based on the second distortion parameter and the second internal parameter of the target camera, the mapped image is subjected to distortion restoration processing to obtain a target image, thereby achieving that images of different styles simulated by another camera can be obtained based on the camera internal parameters and distortion coefficients, and the implementation method is simpler and more efficient. Since the pixels of the generated image are converted from the pixels of the original image, the generated image is closer to the real scene.
下面通过附图和实施例，对本公开的技术方案做进一步的详细描述。The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
通过结合附图对本公开实施例进行更详细的描述，本公开的上述以及其他目的、特征和优势将变得更加明显。附图用来提供对本公开实施例的进一步理解，并且构成说明书的一部分，与本公开实施例一起用于解释本公开，并不构成对本公开的限制。在附图中，相同的参考标号通常代表相同部件或步骤。The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.
图1是本公开所适用的系统图。FIG. 1 is a diagram of a system to which the present disclosure is applicable.
图2是本公开一示例性实施例提供的图像转换方法的流程示意图。FIG. 2 is a schematic flow chart of an image conversion method provided by an exemplary embodiment of the present disclosure.
图3是本公开另一示例性实施例提供的图像转换方法的流程示意图。FIG. 3 is a schematic flow chart of an image conversion method provided by another exemplary embodiment of the present disclosure.
图4是本公开另一示例性实施例提供的图像转换方法的流程示意图。FIG. 4 is a schematic flow chart of an image conversion method provided by another exemplary embodiment of the present disclosure.
图5是本公开一示例性实施例提供的图像转换装置的结构示意图。FIG. 5 is a schematic diagram of the structure of an image conversion device provided by an exemplary embodiment of the present disclosure.
图6是本公开另一示例性实施例提供的图像转换装置的结构示意图。FIG. 6 is a schematic structural diagram of an image conversion device provided by another exemplary embodiment of the present disclosure.
图7是本公开一示例性实施例提供的电子设备的结构图。FIG. 7 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure.
具体实施方式Detailed ways
下面，将参考附图详细地描述根据本公开的示例实施例。显然，所描述的实施例仅仅是本公开的一部分实施例，而不是本公开的全部实施例，应理解，本公开不受这里描述的示例实施例的限制。Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.
应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
本领域技术人员可以理解，本公开实施例中的“第一”、“第二”等术语仅用于区别不同步骤、设备或模块等，既不代表任何特定技术含义，也不表示它们之间的必然逻辑顺序。Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.
还应理解，在本公开实施例中，“多个”可以指两个或两个以上，“至少一个”可以指一个、两个或两个以上。It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
还应理解，对于本公开实施例中提及的任一部件、数据或结构，在没有明确限定或者在前后文给出相反启示的情况下，一般可以理解为一个或多个。It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.
另外，本公开中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本公开中字符“/”，一般表示前后关联对象是一种“或”的关系。In addition, the term "and/or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.
还应理解，本公开对各个实施例的描述着重强调各个实施例之间的不同之处，其相同或相似之处可以相互参考，为了简洁，不再一一赘述。It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.
同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，技术、方法和设备应当被视为说明书的一部分。Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
本公开实施例可以应用于终端设备、计算机系统、服务器等电子设备，其可与众多其它通用或专用计算系统环境或配置一起操作。适于与终端设备、计算机系统、服务器等电子设备一起使用的众所周知的终端设备、计算系统、环境和/或配置的例子包括但不限于：个人计算机系统、服务器计算机系统、瘦客户机、厚客户机、手持或膝上设备、基于微处理器的系统、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机系统、大型计算机系统和包括上述任何系统的分布式云计算技术环境，等等。The disclosed embodiments can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and/or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.
终端设备、计算机系统、服务器等电子设备可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机系统/服务器可以在分布式云计算环境中实施，分布式云计算环境中，任务是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems/servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.
申请概述Application Overview
为了生成不同拍摄风格的图像，目前常用的方法是基于深度学习的方法训练模型，比如生成对抗网络，进行风格迁移，生成贴合实际场景的数据。但该方法受到模型精度影响较大，且图像中的像素值是通过网络合成的，所以图像在细节部分比较容易失真。In order to generate images with different shooting styles, the commonly used method is to train models based on deep learning methods, such as generating adversarial networks, to perform style transfer, and generate data that fits the actual scene. However, this method is greatly affected by the accuracy of the model, and the pixel values in the image are synthesized through the network, so the image is more likely to be distorted in details.
示例性系统Exemplary Systems
图1示出了可以应用本公开的实施例的图像转换方法或图像转换装置的示例性系统架构100。FIG. 1 shows an exemplary system architecture 100 to which an image conversion method or an image conversion apparatus according to an embodiment of the present disclosure can be applied.
如图1所示，系统架构100可以包括终端设备101，网络102和服务器103。网络102用于在终端设备101和服务器103之间提供通信链路的介质。网络102可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。As shown in Fig. 1, system architecture 100 may include terminal device 101, network 102 and server 103. Network 102 is used to provide a medium for a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.
用户可以使用终端设备101通过网络102与服务器103交互，以接收或发送消息等。终端设备101上可以安装有各种通讯客户端应用，例如图像处理应用、人机交互应用、搜索类应用、网页浏览器应用、即时通信工具等。The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. The terminal device 101 can be installed with various communication client applications, such as image processing applications, human-computer interaction applications, search applications, web browser applications, instant messaging tools, etc.
终端设备101可以是各种电子设备，包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。The terminal device 101 can be various electronic devices, including but not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
服务器103可以是提供各种服务的服务器，例如对终端设备101上传的图像进行拍摄风格转换的后台图像处理服务器。后台图像处理服务器可以对接收到的图像进行处理，得到处理结果(例如模拟另一相机的拍摄风格的图像)。The server 103 may be a server that provides various services, such as a background image processing server that performs shooting style conversion on the image uploaded by the terminal device 101. The background image processing server may process the received image to obtain a processing result (such as an image simulating the shooting style of another camera).
需要说明的是，本公开的实施例所提供的图像转换方法可以由服务器103执行，也可以由终端设备101执行，相应地，图像转换装置可以设置于服务器103中，也可以设置于终端设备101中。It should be noted that the image conversion method provided in the embodiment of the present disclosure can be executed by the server 103 or by the terminal device 101. Accordingly, the image conversion device can be set in the server 103 or in the terminal device 101.
应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。在用于转换的原始图像不需要从远程获取的情况下，上述系统架构可以不包括网络，只包括服务器或终端设备。It should be understood that the number of terminal devices, networks and servers in FIG1 is merely illustrative. Depending on the implementation requirements, any number of terminal devices, networks and servers may be provided. In the case where the original image for conversion does not need to be acquired remotely, the above system architecture may not include a network, but only include a server or a terminal device.
示例性方法Exemplary Methods
图2是本公开一示例性实施例提供的图像转换方法的流程示意图。本实施例可应用在电子设备(如图1所示的终端设备101或服务器103)上，如图2所示，该方法包括如下步骤：FIG2 is a flowchart of an image conversion method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device (such as the terminal device 101 or the server 103 shown in FIG1 ). As shown in FIG2 , the method includes the following steps:
步骤201，基于用于采集原始图像的原始相机的第一畸变参数和第一内参，对原始图像进行畸变矫正处理，得到矫正后图像。Step 201 : Based on a first distortion parameter and a first internal parameter of an original camera used to acquire the original image, a distortion correction process is performed on the original image to obtain a corrected image.
在本实施例中，电子设备可以基于用于采集原始图像的原始相机的第一畸变参数和第一内参，对原始图像进行畸变矫正处理，得到矫正后图像。其中，第一畸变参数可以是预先对上述原始相机进行畸变参数标定得到的。In this embodiment, the electronic device can perform distortion correction processing on the original image based on the first distortion parameter and the first internal parameter of the original camera used to capture the original image to obtain a corrected image. The first distortion parameter can be obtained by pre-calibrating the distortion parameter of the original camera.
具体地，相机畸变包含径向畸变与切向畸变，整合后的畸变公式：Specifically, camera distortion includes radial distortion and tangential distortion. The integrated distortion formula is:
ud＝u*(1+k1r2+k2r4+k3r6)+2p1u*v*+p2(r2+2u*2) (1)ud ＝u* (1+k1 r2 +k2 r4 +k3 r6 )+2p1 u* v* +p2 (r2 +2u*2 ) (1)
vd＝v*(1+k1r2+k2r4+k3r6)+p1(r2+2v*2)+2p2u*v* (2)vd ＝v* (1+k1 r2 +k2 r4 +k3 r6 )+p1 (r2 +2v*2 )+2p2 u* v* (2)
其中，r2＝u*2+v*2,(k1,k2,p1,p2,k3)为相机的畸变参数，fx、fy为第一内参。/>表示有畸变的图像在二维坐标系下的坐标，/>表示无畸变的图像在二维坐标系下中的坐标。式(1)和式(2)是将图像从有畸变到无畸变的坐标转换公式。电子设备可以利用上述式(1)和式(2)进行逆变换以对原始图像进行畸变矫正处理。in, r2 =u*2 +v*2 , (k1 , k2 , p1 , p2 , k3 ) is the distortion parameter of the camera, fx , fy is the first intrinsic parameter. /> Represents the coordinates of the distorted image in a two-dimensional coordinate system, /> Represents the coordinates of the undistorted image in the two-dimensional coordinate system. Equations (1) and (2) are coordinate transformation formulas for transforming an image from distorted to undistorted. Electronic devices can use the above equations (1) and (2) to perform inverse transformation to perform distortion correction processing on the original image.
步骤202，基于矫正后图像，确定待映射图像。Step 202: Determine an image to be mapped based on the rectified image.
在本实施例中，电子设备可以基于矫正后图像，确定待映射图像。In this embodiment, the electronic device may determine the image to be mapped based on the rectified image.
其中，待映射图像为用于被映射到另一相机的二维像面坐标下的图像。可选的，可以直接将矫正后图像确定为待映射图像，还可对矫正后图像进行其他方式的处理后(例如相机拍摄角度转换)得到待映射图像。The image to be mapped is an image to be mapped to the two-dimensional image plane coordinates of another camera. Optionally, the rectified image can be directly determined as the image to be mapped, or the rectified image can be processed in other ways (such as camera shooting angle conversion) to obtain the image to be mapped.
步骤203，基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像。Step 203 : Based on the first intrinsic parameter of the original camera and the second intrinsic parameter of the target camera, the image to be mapped is mapped to the two-dimensional image plane coordinate system of the target camera to obtain a mapped image.
在本实施例中，电子设备可以基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像。In this embodiment, the electronic device may map the image to be mapped to the two-dimensional image plane coordinate system of the target camera based on the first internal parameter of the original camera and the second internal parameter of the target camera to obtain a mapped image.
其中，第一内参和第二内参可以是基于现有的内参标定方法对原始相机和目标相机进行内参标定得到的。目标相机是与原始相机的参数(包括内参、畸变参数等)不同的相机，由于目标相机与原始相机的参数不同，因此，目标相机拍摄的图像与原始相机拍摄的图像的风格不同。例如，原始相机为普通镜头相机，目标相机为鱼眼镜头相机。The first internal parameter and the second internal parameter can be obtained by performing internal parameter calibration on the original camera and the target camera based on an existing internal parameter calibration method. The target camera is a camera with different parameters (including internal parameters, distortion parameters, etc.) from the original camera. Since the parameters of the target camera and the original camera are different, the style of the image taken by the target camera is different from that of the image taken by the original camera. For example, the original camera is a normal lens camera, and the target camera is a fisheye lens camera.
第一内参用于表征原始相机的三维局部坐标系与原始相机的二维像面坐标系的映射关系，第二内参用于表征目标相机的三维局部坐标系与目标相机的二维像面坐标系的映射关系。本实施例的目的是将原始图像的风格转换为另一风格，可以模拟在待映射图像对应的相机位置和姿态下，使用目标相机拍摄图像。The first internal parameter is used to characterize the mapping relationship between the 3D local coordinate system of the original camera and the 2D image plane coordinate system of the original camera, and the second internal parameter is used to characterize the mapping relationship between the 3D local coordinate system of the target camera and the 2D image plane coordinate system of the target camera. The purpose of this embodiment is to convert the style of the original image into another style, which can simulate the use of the target camera to shoot the image at the camera position and posture corresponding to the image to be mapped.
通常，在目标相机的位置和姿态与原始相机相同时，可以基于第一内参，将待映射图像中的像素点转换到三维局部坐标系下，再基于第二内参，将三维局部坐标系下的像素点转换到二维像面坐标系下，从而得到映射图像。Usually, when the position and posture of the target camera are the same as those of the original camera, the pixel points in the image to be mapped can be converted to a three-dimensional local coordinate system based on the first internal parameter, and then the pixel points in the three-dimensional local coordinate system can be converted to a two-dimensional image plane coordinate system based on the second internal parameter to obtain a mapped image.
步骤204，基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像。Step 204 : Based on the second distortion parameter and the second internal parameter of the target camera, the mapping image is subjected to distortion restoration processing to obtain a target image.
在本实施例中，电子设备可以基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像。具体地，电子设备可以基于上述式(1)和式(2)以实现对映射图像进行畸变还原处理。In this embodiment, the electronic device can perform distortion restoration processing on the mapped image based on the second distortion parameter and the second internal parameter of the target camera to obtain the target image. Specifically, the electronic device can implement distortion restoration processing on the mapped image based on the above formula (1) and formula (2).
本公开的上述实施例提供的方法，通过对原始图像进行畸变矫正处理，得到矫正后图像，然后基于矫正后图像，确定待映射图像，接着基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像，最后基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像，从而实现了根据相机内参和畸变系数即可得到模拟另一相机拍摄的不同风格的图像，实现方式更加简便高效。由于生成的图像的像素由原始图像的像素转变而来，从而使生成的图像更接近真实场景。The method provided by the above-mentioned embodiment of the present disclosure performs distortion correction processing on the original image to obtain a corrected image, then determines the image to be mapped based on the corrected image, and then maps the image to be mapped to the two-dimensional image plane coordinate system of the target camera based on the first internal parameter of the original camera and the second internal parameter of the target camera to obtain a mapped image, and finally performs distortion restoration processing on the mapped image based on the second distortion parameter and the second internal parameter of the target camera to obtain a target image, thereby achieving different styles of images simulated by another camera shot based on the camera internal parameter and the distortion coefficient, and the implementation method is simpler and more efficient. Since the pixels of the generated image are converted from the pixels of the original image, the generated image is closer to the real scene.
在一些可选的实现方式中，步骤203可以如下执行：In some optional implementations, step 203 may be performed as follows:
首先，基于第一内参，确定原始相机的二维像面坐标系与原始相机的局部三维坐标系的第一映射关系。Firstly, based on the first internal parameter, a first mapping relationship between the two-dimensional image plane coordinate system of the original camera and the local three-dimensional coordinate system of the original camera is determined.
具体地，相机成像公式为：Specifically, the camera imaging formula is:
其中是相机的三维局部坐标系下的坐标，/>表示图像的二维坐标系下的坐标，K表示相机内参。in is the coordinate of the camera in the 3D local coordinate system, /> Represents the coordinates of the image in the two-dimensional coordinate system, and K represents the camera intrinsic parameter.
公式(3)是从相机的三维局部坐标系到二维像面坐标系的转换公式，所以由上述式(3)，可以得到从待映射图像的二维像面坐标系到原始相机的三维局部坐标系的转换公式为：Formula (3) is the conversion formula from the camera's 3D local coordinate system to the 2D image plane coordinate system. Therefore, from the above formula (3), the conversion formula from the 2D image plane coordinate system of the image to be mapped to the 3D local coordinate system of the original camera can be obtained as follows:
其中，Ko表示第一内参，Ko-1即可表示上述第一映射关系，为待映射图像中的像素点的坐标。Wherein, Ko represents the first internal parameter, and Ko-1 can represent the above first mapping relationship. is the coordinate of the pixel in the image to be mapped.
然后，基于第二内参，确定原始相机的局部三维坐标系与目标相机的二维像面坐标系的第二映射关系。Then, based on the second internal parameter, a second mapping relationship between the local three-dimensional coordinate system of the original camera and the two-dimensional image plane coordinate system of the target camera is determined.
具体地，根据式(3)，可以得到从目标相机的三维局部坐标系到映射图像的二维像面坐标系的转换公式为：Specifically, according to formula (3), the conversion formula from the three-dimensional local coordinate system of the target camera to the two-dimensional image plane coordinate system of the mapped image can be obtained as follows:
其中，Kd为第二内参，Kd即可表示上述第二映射关系。为映射图像中的像素点的坐标。Wherein, Kd is the second internal parameter, and Kd can represent the above second mapping relationship. is the coordinate of the pixel in the mapped image.
最后，基于第一映射关系和第二映射关系，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像。Finally, based on the first mapping relationship and the second mapping relationship, the image to be mapped is mapped to the two-dimensional image plane coordinate system of the target camera to obtain a mapped image.
具体地，基于上述式(4)和式(5)，可以得到待映射图像和映射图像之间的坐标映射关系：Specifically, based on the above equations (4) and (5), the coordinate mapping relationship between the image to be mapped and the mapping image can be obtained:
将第一内参和第二内参代入式(6)即可得到映射图像。Substituting the first internal parameter and the second internal parameter into equation (6) can obtain the mapping image.
本实现方式通过第一内参和第二内参确定待映射图像和映射图像之间的坐标映射关系，可以将待映射图像中的每个像素点直接映射到映射图像内，从而可以提高生成映射图像的效率，由于映射图像的像素点均来自待映射图像，因此可以降低图像失真的风险。This implementation method determines the coordinate mapping relationship between the image to be mapped and the mapping image through the first internal parameter and the second internal parameter, and can directly map each pixel point in the image to be mapped to the mapping image, thereby improving the efficiency of generating the mapping image. Since the pixel points of the mapping image all come from the image to be mapped, the risk of image distortion can be reduced.
在一些可选的实现方式中，步骤202可以如下执行：In some optional implementations, step 202 may be performed as follows:
在原始相机的三维局部坐标系下，基于第一内参和预设的相机旋转角度，将矫正后图像转换为待映射图像。其中，相机旋转角度可以是预先指定的角度。通常，相机旋转角度可以包括俯仰角(pitch)、偏航角(yaw)、翻滚角(roll)。相机旋转角度是在第二图像的相机拍摄角度的基础上，相机拍摄角度发生的变化。In the three-dimensional local coordinate system of the original camera, based on the first intrinsic parameter and the preset camera rotation angle, the rectified image is converted into the image to be mapped. The camera rotation angle may be a pre-specified angle. Generally, the camera rotation angle may include a pitch angle, a yaw angle, and a roll angle. The camera rotation angle is a change in the camera shooting angle based on the camera shooting angle of the second image.
通常，可以基于第一内参，将矫正后图像中的像素点转换到原始相机的三维局部坐标系下，在三维局部坐标系下进行仿射变换，完成像素点的旋转，然后再映射回二维坐标系下，从而得到待映射图像。Usually, based on the first intrinsic parameter, the pixels in the rectified image can be converted to the three-dimensional local coordinate system of the original camera, and an affine transformation is performed in the three-dimensional local coordinate system to complete the rotation of the pixels, and then mapped back to the two-dimensional coordinate system to obtain the image to be mapped.
本实现方式通过对矫正后图像进行相机拍摄角度转换，可以生成不同相机拍摄角度下的图像，从而可以模拟与原始相机不同参数的目标相机在不同相机拍摄角度下拍摄图像，有助于得到与原始图像相比具有更多不同之处的目标图像，扩大了图像转换的应用范围。This implementation method can generate images at different camera shooting angles by converting the camera shooting angle of the corrected image, thereby simulating a target camera with different parameters from the original camera to shoot images at different camera shooting angles, which helps to obtain a target image that has more differences than the original image and expands the application scope of image conversion.
在一些可选的实现方式中，如图3所示，上述可选实施例中，可以按照如下步骤基于第一内参和预设的相机旋转角度，将矫正后图像转换为待映射图像：In some optional implementations, as shown in FIG3 , in the above optional embodiment, the rectified image may be converted into an image to be mapped based on the first intrinsic parameter and a preset camera rotation angle according to the following steps:
步骤2021，从矫正后图像中提取包含预设数量个点的第一像素点集。Step 2021, extracting a first pixel point set including a preset number of points from the rectified image.
其中，预设数量可以任意设置，例如预设数量为4。电子设备可以基于各种方式提取第一像素点集。例如随机提取、按指定提取等。The preset number can be set arbitrarily, for example, the preset number is 4. The electronic device can extract the first pixel point set based on various methods, such as random extraction, specified extraction, etc.
可选的，电子设备可以从矫正后图像中提取包含预设数量个不共线的点的第一像素点集。不共线的第一像素点集可以使第一像素点集充分表征图像的二维特征，提高后续仿射变换的准确性。Optionally, the electronic device can extract a first pixel point set including a preset number of non-collinear points from the corrected image. The non-collinear first pixel point set can enable the first pixel point set to fully characterize the two-dimensional features of the image and improve the accuracy of subsequent affine transformation.
步骤2022，在矫正后图像的二维像面坐标系下，基于第一内参和相机旋转角度，将第一像素点集转换为第二像素点集。Step 2022: In the two-dimensional image plane coordinate system of the corrected image, based on the first intrinsic parameter and the camera rotation angle, convert the first pixel point set into a second pixel point set.
通常，电子设备可以基于第一内参，将第一像素点集转换到原始相机的三维局部坐标系下，在三维局部坐标系下进行仿射变换，完成像素点的旋转，然后再映射回二维坐标系下，从而得到第二像素点集。Typically, the electronic device can convert the first pixel point set to the three-dimensional local coordinate system of the original camera based on the first internal parameter, perform affine transformation in the three-dimensional local coordinate system to complete the rotation of the pixel points, and then map it back to the two-dimensional coordinate system to obtain a second pixel point set.
步骤2023，确定第一像素点集和第二像素点集在矫正后图像的二维像面坐标系下坐标映射关系。Step 2023, determining the coordinate mapping relationship between the first pixel point set and the second pixel point set in the two-dimensional image plane coordinate system of the corrected image.
具体地，上述映射关系可以基于如下公式得到：Specifically, the above mapping relationship can be obtained based on the following formula:
设f＝K·Rz·Ry·Rx·K-1，则f即表示该坐标映射关系。其中，表示第三像素点集的二维坐标，/>表示第二像素点集的二维坐标，K为第一内参，Rz、Ry、Rx为旋转矩阵。其中，Assume f = K·Rz ·Ry ·Rx ·K-1 , then f represents the coordinate mapping relationship. Represents the two-dimensional coordinates of the third pixel point set,/> represents the two-dimensional coordinates of the second pixel set, K is the first internal parameter, and Rz , Ry , and Rx are rotation matrices.
Rx表示绕x轴的旋转矩阵，Ry表示绕y轴的旋转矩阵，Rz表示绕z轴的旋转矩阵。旋转矩阵中的pitch、yaw、roll为相机旋转角度。Rx represents the rotation matrix around the x-axis, Ry represents the rotation matrix around the y-axis, and Rz represents the rotation matrix around the z-axis. The pitch, yaw, and roll in the rotation matrix are the camera rotation angles.
步骤2024，根据坐标映射关系，将矫正后图像转换为待映射图像。Step 2024: convert the corrected image into an image to be mapped according to the coordinate mapping relationship.
具体地，基于上述f，可以将矫正后图像中的每个像素的坐标进行转换，最终得到待映射图像。Specifically, based on the above f, the coordinates of each pixel in the rectified image can be transformed to finally obtain the image to be mapped.
上述图3对应实施例提供的方法，通过从矫正后图像中提取第一像素点集，基于第一内参和相机旋转角度，得到第二像素点集，从而可以得到像素点集的坐标映射关系，从而实现了使用较少的像素点确定图像间的坐标映射关系，有助于降低计算量，提高图像转换的效率。The method provided in the embodiment corresponding to Figure 3 above extracts a first pixel point set from the corrected image, obtains a second pixel point set based on the first intrinsic parameter and the camera rotation angle, and thus obtains the coordinate mapping relationship of the pixel point set, thereby realizing the use of fewer pixel points to determine the coordinate mapping relationship between images, which helps to reduce the amount of calculation and improve the efficiency of image conversion.
在一些可选的实现方式中，如图4所示，步骤2022可以包括如下步骤：In some optional implementations, as shown in FIG. 4 , step 2022 may include the following steps:
步骤20221，基于第一内参，将第一像素点集转换为三维局部坐标系下的第一空间点集。Step 20221: Based on the first internal parameter, convert the first pixel point set into a first spatial point set in a three-dimensional local coordinate system.
具体地，基于下式(8)，可以将第一像素点集转换为三维局部坐标系下的第一空间点集。Specifically, based on the following formula (8), the first pixel point set can be converted into a first space point set in a three-dimensional local coordinate system.
其中，Ko为第一内参，表示第一像素点集的二维坐标。Among them, Ko is the first internal reference, Represents the two-dimensional coordinates of the first pixel point set.
步骤20222，基于相机旋转角度，对第一空间点集的坐标进行仿射变换，得到第二空间点集。Step 20222: Based on the camera rotation angle, perform affine transformation on the coordinates of the first spatial point set to obtain a second spatial point set.
具体地，仿射变换公式如下式所示：Specifically, the affine transformation formula is as follows:
Rx、Ry、Rz表示旋转矩阵，具体如上述步骤2023中的描述。按照式(9)，可以对第一空间点集的坐标进行仿射变换。Rx , Ry , and Rz represent rotation matrices, which are specifically described in the above step 2023. According to equation (9), an affine transformation can be performed on the coordinates of the first space point set.
步骤20223，基于第二内参，将第二空间点集转换到在二维像面坐标系下的第二像素点集。Step 20223: Based on the second internal parameter, convert the second spatial point set to a second pixel point set in the two-dimensional image plane coordinate system.
具体地，基于上述公式(3)，可以将相机的三维局部坐标系下的第二空间点集转换到在二维像面坐标系下的第二像素点集。Specifically, based on the above formula (3), the second spatial point set in the three-dimensional local coordinate system of the camera can be converted into a second pixel point set in the two-dimensional image plane coordinate system.
本实现方式通过将第一像素点集转换到三维局部坐标系下进行仿射变换，再从三维局部坐标系下转换回二维坐标系，从而可以准确地对第一像素点集进行视线角度转换，有助于提高确定矫正后图像与待映射图像的坐标映射关系的准确性。This implementation method converts the first pixel point set into a three-dimensional local coordinate system for affine transformation, and then converts it back to a two-dimensional coordinate system, so that the line of sight angle of the first pixel point set can be accurately converted, which helps to improve the accuracy of determining the coordinate mapping relationship between the corrected image and the image to be mapped.
示例性装置Exemplary Devices
图5是本公开一示例性实施例提供的图像转换装置的结构示意图。本实施例可应用在电子设备上，如图5所示，图像转换装置包括：矫正模块501，用于基于用于采集原始图像的原始相机的第一畸变参数和第一内参，对原始图像进行畸变矫正处理，得到矫正后图像；确定模块502，用于基于矫正后图像，确定待映射图像；映射模块503，用于基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像；还原模块504，用于基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像。FIG5 is a schematic diagram of the structure of an image conversion device provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices. As shown in FIG5 , the image conversion device includes: a correction module 501, which is used to perform distortion correction processing on the original image based on the first distortion parameter and the first internal parameter of the original camera used to acquire the original image, so as to obtain a corrected image; a determination module 502, which is used to determine the image to be mapped based on the corrected image; a mapping module 503, which is used to map the image to be mapped to the two-dimensional image plane coordinate system of the target camera based on the first internal parameter of the original camera and the second internal parameter of the target camera, so as to obtain a mapped image; and a restoration module 504, which is used to perform distortion restoration processing on the mapped image based on the second distortion parameter and the second internal parameter of the target camera, so as to obtain a target image.
在本实施例中，矫正模块501可以基于用于采集原始图像的原始相机的第一畸变参数和第一内参，对原始图像进行畸变矫正处理，得到矫正后图像。其中，第一畸变参数可以是预先对上述原始相机进行畸变参数标定得到的。In this embodiment, the correction module 501 can perform distortion correction processing on the original image based on the first distortion parameter and the first internal parameter of the original camera used to capture the original image to obtain a corrected image. The first distortion parameter can be obtained by pre-calibrating the distortion parameter of the original camera.
在本实施例中，确定模块502可以基于矫正后图像，确定待映射图像。In this embodiment, the determination module 502 may determine the image to be mapped based on the rectified image.
其中，待映射图像为用于被映射到另一相机的二维像面坐标下的图像。可选的，可以直接将矫正后图像确定为待映射图像，还可对矫正后图像进行其他方式的处理后(例如相机拍摄角度转换)得到待映射图像。The image to be mapped is an image to be mapped to the two-dimensional image plane coordinates of another camera. Optionally, the rectified image can be directly determined as the image to be mapped, or the rectified image can be processed in other ways (such as camera shooting angle conversion) to obtain the image to be mapped.
在本实施例中，映射模块503可以基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像。In this embodiment, the mapping module 503 may map the image to be mapped to the two-dimensional image plane coordinate system of the target camera based on the first intrinsic parameter of the original camera and the second intrinsic parameter of the target camera to obtain a mapped image.
其中，第一内参和第二内参可以是基于现有的内参标定方法对原始相机和目标相机进行内参标定得到的。目标相机是与原始相机的参数(包括内参、畸变参数等)不同的相机，由于目标相机与原始相机的参数不同，因此，目标相机拍摄的图像与原始相机拍摄的图像的风格不同。例如，原始相机为普通镜头相机，目标相机为鱼眼镜头相机。The first internal parameter and the second internal parameter can be obtained by performing internal parameter calibration on the original camera and the target camera based on an existing internal parameter calibration method. The target camera is a camera with different parameters (including internal parameters, distortion parameters, etc.) from the original camera. Since the parameters of the target camera and the original camera are different, the style of the image taken by the target camera is different from that of the image taken by the original camera. For example, the original camera is a normal lens camera, and the target camera is a fisheye lens camera.
第一内参用于表征原始相机的三维局部坐标系与原始相机的二维像面坐标系的映射关系，第二内参用于表征目标相机的三维局部坐标系与目标相机的二维像面坐标系的映射关系。由于本实施例的目的是将原始图像的风格转换为另一风格，可以模拟在待映射图像对应的相机位置和姿态下，使用目标相机拍摄图像。The first internal parameter is used to characterize the mapping relationship between the 3D local coordinate system of the original camera and the 2D image plane coordinate system of the original camera, and the second internal parameter is used to characterize the mapping relationship between the 3D local coordinate system of the target camera and the 2D image plane coordinate system of the target camera. Since the purpose of this embodiment is to convert the style of the original image into another style, it is possible to simulate shooting an image with the target camera at the camera position and posture corresponding to the image to be mapped.
通常，可以基于第一内参，将待映射图像中的像素点转换到三维局部坐标系下，再基于第二内参，将三维局部坐标系下的像素点转换到二维像面坐标系下，从而得到映射图像。Typically, based on the first intrinsic parameter, the pixels in the image to be mapped are converted to a three-dimensional local coordinate system, and then based on the second intrinsic parameter, the pixels in the three-dimensional local coordinate system are converted to a two-dimensional image plane coordinate system to obtain a mapped image.
在本实施例中，还原模块504可以基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像。具体地，还原模块504可以基于上述式(1)和式(2)以实现对映射图像进行畸变还原处理。In this embodiment, the restoration module 504 can perform distortion restoration processing on the mapping image based on the second distortion parameter and the second internal parameter of the target camera to obtain the target image. Specifically, the restoration module 504 can implement distortion restoration processing on the mapping image based on the above formula (1) and formula (2).
参照图6，图6是本公开另一示例性实施例提供的图像转换装置的结构示意图。6 , which is a schematic diagram of the structure of an image conversion device provided by another exemplary embodiment of the present disclosure.
在一些可选的实现方式中，映射模块503包括：第一确定单元5031，用于基于第一内参，确定原始相机的二维像面坐标系与原始相机的局部三维坐标系的第一映射关系；第二确定单元5032，用于基于第二内参，确定原始相机的局部三维坐标系与目标相机的二维像面坐标系的第二映射关系；映射单元5033，用于基于第一映射关系和第二映射关系，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像。In some optional implementations, the mapping module 503 includes: a first determination unit 5031, used to determine a first mapping relationship between the two-dimensional image plane coordinate system of the original camera and the local three-dimensional coordinate system of the original camera based on a first internal parameter; a second determination unit 5032, used to determine a second mapping relationship between the local three-dimensional coordinate system of the original camera and the two-dimensional image plane coordinate system of the target camera based on a second internal parameter; a mapping unit 5033, used to map the image to be mapped to the two-dimensional image plane coordinate system of the target camera based on the first mapping relationship and the second mapping relationship, to obtain a mapped image.
在一些可选的实现方式中，确定模块502进一步用于：在原始相机的三维局部坐标系下，基于第一内参和预设的相机旋转角度，将矫正后图像转换为待映射图像。In some optional implementations, the determination module 502 is further configured to: in the three-dimensional local coordinate system of the original camera, based on the first intrinsic parameter and a preset camera rotation angle, convert the rectified image into an image to be mapped.
在一些可选的实现方式中，确定模块502包括：提取单元5021，用于从矫正后图像中提取包含预设数量个点的第一像素点集；第一转换单元5022，用于在矫正后图像的二维像面坐标系下，基于第一内参和相机旋转角度，将第一像素点集转换为第二像素点集；第三确定单元5023，用于确定第一像素点集和第二像素点集在矫正后图像的二维像面坐标系下坐标映射关系；第二转换单元5024，用于根据坐标映射关系，将矫正后图像转换为待映射图像。In some optional implementations, the determination module 502 includes: an extraction unit 5021, used to extract a first pixel point set containing a preset number of points from the corrected image; a first conversion unit 5022, used to convert the first pixel point set into a second pixel point set based on a first intrinsic parameter and a camera rotation angle in a two-dimensional image plane coordinate system of the corrected image; a third determination unit 5023, used to determine a coordinate mapping relationship between the first pixel point set and the second pixel point set in the two-dimensional image plane coordinate system of the corrected image; and a second conversion unit 5024, used to convert the corrected image into an image to be mapped according to the coordinate mapping relationship.
在一些可选的实现方式中，第一转换单元5022包括：第一转换子单元50221，用于基于第一内参，将第一像素点集转换为三维局部坐标系下的第一空间点集；第二转换子单元50222，用于基于相机旋转角度，对第一空间点集的坐标进行仿射变换，得到第二空间点集；第三转换子单元50223，用于基于第二内参，将第二空间点集转换到在二维像面坐标系下的第二像素点集。In some optional implementations, the first conversion unit 5022 includes: a first conversion subunit 50221, used to convert the first pixel point set into a first space point set in a three-dimensional local coordinate system based on a first intrinsic parameter; a second conversion subunit 50222, used to perform an affine transformation on the coordinates of the first space point set based on the camera rotation angle to obtain a second space point set; and a third conversion subunit 50223, used to convert the second space point set into a second pixel point set in a two-dimensional image plane coordinate system based on a second intrinsic parameter.
在一些可选的实现方式中，提取单元5021进一步用于：从矫正后图像中提取包含预设数量个不共线的点的第一像素点集。In some optional implementations, the extraction unit 5021 is further used to extract a first pixel point set including a preset number of non-collinear points from the rectified image.
本公开上述实施例提供的图像转换装置，通过对原始图像进行畸变矫正处理，得到矫正后图像，然后基于矫正后图像，确定待映射图像，接着基于原始相机的第一内参和目标相机的第二内参，将待映射图像映射到目标相机的二维像面坐标系下，得到映射图像，最后基于目标相机的第二畸变参数和第二内参，对映射图像进行畸变还原处理，得到目标图像，从而实现了根据相机内参和畸变系数即可得到模拟另一相机拍摄的不同风格的图像，实现方式更加简便高效。由于生成的图像的像素由原始图像的像素转变而来，从而使生成的图像更接近真实场景。The image conversion device provided by the above embodiment of the present disclosure performs distortion correction processing on the original image to obtain a corrected image, and then determines the image to be mapped based on the corrected image, and then maps the image to be mapped to the two-dimensional image plane coordinate system of the target camera based on the first internal parameter of the original camera and the second internal parameter of the target camera to obtain a mapped image, and finally performs distortion restoration processing on the mapped image based on the second distortion parameter and the second internal parameter of the target camera to obtain a target image, thereby achieving different styles of images simulated by another camera shot according to the camera internal parameter and the distortion coefficient, and the implementation method is simpler and more efficient. Since the pixels of the generated image are converted from the pixels of the original image, the generated image is closer to the real scene.
示例性电子设备Exemplary Electronic Devices
下面，参考图7来描述根据本公开实施例的电子设备。该电子设备可以是如图1所示的终端设备101和服务器103中的任一个或两者、或与它们独立的单机设备，该单机设备可以与终端设备101和服务器103进行通信，以从它们接收所采集到的输入信号。Below, an electronic device according to an embodiment of the present disclosure is described with reference to Figure 7. The electronic device may be any one or both of the terminal device 101 and the server 103 as shown in Figure 1, or a stand-alone device independent of them, and the stand-alone device may communicate with the terminal device 101 and the server 103 to receive the collected input signals from them.
图7图示了根据本公开实施例的电子设备的框图。FIG. 7 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure.
如图7所示，电子设备700包括一个或多个处理器701和存储器702。As shown in FIG. 7 , the electronic device 700 includes one or more processors 701 and a memory 702 .
处理器701可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备700中的其他组件以执行期望的功能。The processor 701 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 700 to perform desired functions.
存储器702可以包括一个或多个计算机程序产品，计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器701可以运行程序指令，以实现上文的本公开的各个实施例的图像转换方法以及/或者其他期望的功能。在计算机可读存储介质中还可以存储诸如原始图像、目标图像等各种内容。The memory 702 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 701 may run the program instructions to implement the image conversion method of each embodiment of the present disclosure above and/or other desired functions. Various contents such as original images, target images, etc. may also be stored in the computer-readable storage medium.
在一个示例中，电子设备700还可以包括：输入装置703和输出装置704，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。In one example, the electronic device 700 may further include: an input device 703 and an output device 704, and these components are interconnected via a bus system and/or other forms of connection mechanisms (not shown).
例如，在该电子设备是终端设备101或服务器103时，该输入装置703可以是相机、鼠标、键盘等设备，用于输入图像。在该电子设备是单机设备时，该输入装置703可以是通信网络连接器，用于从终端设备101和服务器103接收所输入的图像。For example, when the electronic device is a terminal device 101 or a server 103, the input device 703 may be a camera, a mouse, a keyboard, etc., for inputting images. When the electronic device is a stand-alone device, the input device 703 may be a communication network connector for receiving input images from the terminal device 101 and the server 103.
该输出装置704可以向外部输出各种信息，包括生成的目标图像。该输出设备704可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。The output device 704 can output various information to the outside, including the generated target image. The output device 704 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
当然，为了简化，图7中仅示出了该电子设备700中与本公开有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备700还可以包括任何其他适当的组件。Of course, for simplicity, FIG7 only shows some of the components related to the present disclosure in the electronic device 700, omitting components such as a bus, an input/output interface, etc. In addition, the electronic device 700 may further include any other appropriate components according to specific application scenarios.
示例性计算机程序产品和计算机可读存储介质Exemplary computer program products and computer-readable storage media
除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本说明书上述“示例性方法”部分中描述的根据本公开各种实施例的图像转换方法中的步骤。In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image conversion method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.
计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
此外，本公开的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，计算机程序指令在被处理器运行时使得处理器执行本说明书上述“示例性方法”部分中描述的根据本公开各种实施例的图像转换方法中的步骤。In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the image conversion method according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.
计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
以上结合具体实施例描述了本公开的基本原理，但是，需要指出的是，在本公开中提及的优点、优势、效果等仅是示例而非限制，不能认为这些优点、优势、效果等是本公开的各个实施例必须具备的。另外，上述公开的具体细节仅是为了示例的作用和便于理解的作用，而非限制，上述细节并不限制本公开为必须采用上述具体的细节来实现。The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
本说明书中各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似的部分相互参见即可。对于系统实施例而言，由于其与方法实施例基本对应，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
本公开中涉及的器件、装置、设备、系统的方框图仅作为例示性的例子并且不意图要求或暗示必须按照方框图示出的方式进行连接、布置、配置。如本领域技术人员将认识到的，可以按任意方式连接、布置、配置这些器件、装置、设备、系统。诸如“包括”、“包含”、“具有”等等的词语是开放性词汇，指“包括但不限于”，且可与其互换使用。这里所使用的词汇“或”和“和”指词汇“和/或”，且可与其互换使用，除非上下文明确指示不是如此。这里所使用的词汇“诸如”指词组“诸如但不限于”，且可与其互换使用。The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and/or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
可能以许多方式来实现本公开的方法和装置。例如，可通过软件、硬件、固件或者软件、硬件、固件的任何组合来实现本公开的方法和装置。用于方法的步骤的上述顺序仅是为了进行说明，本公开的方法的步骤不限于以上具体描述的顺序，除非以其它方式特别说明。此外，在一些实施例中，还可将本公开实施为记录在记录介质中的程序，这些程序包括用于实现根据本公开的方法的机器可读指令。因而，本公开还覆盖存储用于执行根据本公开的方法的程序的记录介质。The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
还需要指出的是，在本公开的装置、设备和方法中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本公开的等效方案。It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and/or recombined. Such decomposition and/or recombination should be regarded as equivalent solutions of the present disclosure.
提供所公开的方面的以上描述以使本领域的任何技术人员能够做出或者使用本公开。对这些方面的各种修改对于本领域技术人员而言是非常显而易见的，并且在此定义的一般原理可以应用于其他方面而不脱离本公开的范围。因此，本公开不意图被限制到在此示出的方面，而是按照与在此公开的原理和新颖的特征一致的最宽范围。The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本公开的实施例限制到在此公开的形式。尽管以上已经讨论了多个示例方面和实施例，但是本领域技术人员将认识到其某些变型、修改、改变、添加和子组合。The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011344168.6ACN112489114B (en) | 2020-11-25 | 2020-11-25 | Image conversion method, device, computer readable storage medium and electronic device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011344168.6ACN112489114B (en) | 2020-11-25 | 2020-11-25 | Image conversion method, device, computer readable storage medium and electronic device |
| Publication Number | Publication Date |
|---|---|
| CN112489114A CN112489114A (en) | 2021-03-12 |
| CN112489114Btrue CN112489114B (en) | 2024-05-10 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011344168.6AActiveCN112489114B (en) | 2020-11-25 | 2020-11-25 | Image conversion method, device, computer readable storage medium and electronic device |
| Country | Link |
|---|---|
| CN (1) | CN112489114B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465716B (en)* | 2020-11-25 | 2025-01-03 | 深圳地平线机器人科技有限公司 | Image conversion method, device, computer readable storage medium and electronic device |
| CN113067984B (en)* | 2021-03-30 | 2023-01-17 | Oppo广东移动通信有限公司 | Binocular shooting correction method, binocular shooting correction device and electronic equipment |
| CN113055601B (en)* | 2021-03-30 | 2023-01-10 | Oppo广东移动通信有限公司 | Binocular photography correction method, binocular photography correction device and electronic equipment |
| CN112884769B (en)* | 2021-04-12 | 2021-09-28 | 深圳中科飞测科技股份有限公司 | Image processing method, image processing apparatus, optical system, and computer-readable storage medium |
| CN113096192B (en)* | 2021-04-25 | 2024-05-07 | 西安四维图新信息技术有限公司 | Image sensor internal parameter calibration method, device, equipment and storage medium |
| CN115439548B (en)* | 2021-08-18 | 2025-08-08 | 北京车和家信息技术有限公司 | Camera calibration method, image stitching method, device, medium, camera and vehicle |
| CN115880355A (en)* | 2021-09-27 | 2023-03-31 | 北京小米移动软件有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN114821497B (en)* | 2022-02-24 | 2024-11-08 | 广州文远知行科技有限公司 | Method, device, equipment and storage medium for determining target position |
| CN115205134A (en)* | 2022-05-13 | 2022-10-18 | 北京旷视科技有限公司 | Distortion mapping data generation method and distortion correction method of camera module |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002158915A (en)* | 2000-09-05 | 2002-05-31 | Inst Of Physical & Chemical Res | How to correct lens distortion in images |
| CN104363986A (en)* | 2014-10-31 | 2015-02-18 | 华为技术有限公司 | Image processing method and device |
| CN107424126A (en)* | 2017-05-26 | 2017-12-01 | 广州视源电子科技股份有限公司 | Image correction method, device, equipment and system, image pickup equipment and display equipment |
| CN109155822A (en)* | 2017-11-28 | 2019-01-04 | 深圳市大疆创新科技有限公司 | Image processing method and device |
| WO2019041651A1 (en)* | 2017-08-30 | 2019-03-07 | 广州视源电子科技股份有限公司 | Image correction method, apparatus and device, and computer readable storage medium |
| CN110517202A (en)* | 2019-08-30 | 2019-11-29 | 的卢技术有限公司 | A kind of vehicle body camera calibration method and its caliberating device |
| CN111028290A (en)* | 2019-11-26 | 2020-04-17 | 北京光年无限科技有限公司 | Graph processing method and device for picture book reading robot |
| CN111047651A (en)* | 2019-12-12 | 2020-04-21 | 中航华东光电有限公司 | Method for correcting distorted image |
| CN111539880A (en)* | 2020-04-15 | 2020-08-14 | 上海摩象网络科技有限公司 | Image processing method and device and handheld camera |
| CN111951193A (en)* | 2020-08-21 | 2020-11-17 | 安谋科技（中国）有限公司 | Image horizontal distortion correction method and horizontal distortion correction device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9196022B2 (en)* | 2014-03-10 | 2015-11-24 | Omnivision Technologies, Inc. | Image transformation and multi-view output systems and methods |
| CN107113376B (en)* | 2015-07-31 | 2019-07-19 | 深圳市大疆创新科技有限公司 | An image processing method, device and camera |
| CN106910210B (en)* | 2017-03-03 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating image information |
| CN109754427A (en)* | 2017-11-01 | 2019-05-14 | 虹软科技股份有限公司 | A method and apparatus for calibration |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002158915A (en)* | 2000-09-05 | 2002-05-31 | Inst Of Physical & Chemical Res | How to correct lens distortion in images |
| CN104363986A (en)* | 2014-10-31 | 2015-02-18 | 华为技术有限公司 | Image processing method and device |
| CN107424126A (en)* | 2017-05-26 | 2017-12-01 | 广州视源电子科技股份有限公司 | Image correction method, device, equipment and system, image pickup equipment and display equipment |
| WO2019041651A1 (en)* | 2017-08-30 | 2019-03-07 | 广州视源电子科技股份有限公司 | Image correction method, apparatus and device, and computer readable storage medium |
| CN109155822A (en)* | 2017-11-28 | 2019-01-04 | 深圳市大疆创新科技有限公司 | Image processing method and device |
| CN110517202A (en)* | 2019-08-30 | 2019-11-29 | 的卢技术有限公司 | A kind of vehicle body camera calibration method and its caliberating device |
| CN111028290A (en)* | 2019-11-26 | 2020-04-17 | 北京光年无限科技有限公司 | Graph processing method and device for picture book reading robot |
| CN111047651A (en)* | 2019-12-12 | 2020-04-21 | 中航华东光电有限公司 | Method for correcting distorted image |
| CN111539880A (en)* | 2020-04-15 | 2020-08-14 | 上海摩象网络科技有限公司 | Image processing method and device and handheld camera |
| CN111951193A (en)* | 2020-08-21 | 2020-11-17 | 安谋科技（中国）有限公司 | Image horizontal distortion correction method and horizontal distortion correction device |
| Title |
|---|
| 线扫描相机标定及畸变矫正方法;朱瑶;王治乐;贺磊;周程灏;;航天返回与遥感(06);全文* |
| Publication number | Publication date |
|---|---|
| CN112489114A (en) | 2021-03-12 |
| Publication | Publication Date | Title |
|---|---|---|
| CN112489114B (en) | Image conversion method, device, computer readable storage medium and electronic device | |
| JP7411114B2 (en) | Spatial geometric information estimation model generation method and device | |
| CN114399597B (en) | Method and device for constructing scene space model and storage medium | |
| CN107113381B (en) | Tolerance video stitching method, device and computer readable medium for spatiotemporal local deformation and seam finding | |
| CN110517214B (en) | Method and apparatus for generating image | |
| CN115294275A (en) | Three-dimensional model reconstruction method, device and computer-readable storage medium | |
| CN111612842B (en) | Method and device for generating pose estimation model | |
| CN112308910B (en) | Data generation method, device and storage medium | |
| CN111784776B (en) | Visual positioning method and device, computer readable medium and electronic equipment | |
| CN111402136B (en) | Panorama generation method and device, computer readable storage medium and electronic equipment | |
| CN111402404B (en) | Panorama complementing method and device, computer readable storage medium and electronic equipment | |
| CN112037279A (en) | Article position identification method and device, storage medium and electronic equipment | |
| WO2022262273A1 (en) | Optical center alignment test method and apparatus, and storage medium and electronic device | |
| CN112991542B (en) | House three-dimensional reconstruction method, device and electronic equipment | |
| CN112714263B (en) | Video generation method, device, equipment and storage medium | |
| CN112270242B (en) | Trajectory display method, device, readable medium and electronic device | |
| CN114612341A (en) | Image distortion correction method and device, computer readable medium and electronic device | |
| CN112465716B (en) | Image conversion method, device, computer readable storage medium and electronic device | |
| CN117994471A (en) | Object modeling method, apparatus, electronic device, storage medium, and program product | |
| CN114241029B (en) | Image three-dimensional reconstruction method and device | |
| CN113570694B (en) | Model point rendering method and device, storage medium, and electronic device | |
| CN114332603A (en) | Appearance processing method, device and electronic device for a white mold | |
| CN118071965B (en) | Augmented reality anchor point placement method and device, electronic equipment and readable storage medium | |
| CN113495278B (en) | Method and apparatus for enhancing point cloud data | |
| CN118485703A (en) | Method, device and apparatus for restoring image depth |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |