CN112967345A - External parameter calibration method, device and system of fisheye camera - Google Patents
External parameter calibration method, device and system of fisheye cameraDownload PDFInfo
- Publication number
- CN112967345A CN112967345ACN202110254664.0ACN202110254664ACN112967345ACN 112967345 ACN112967345 ACN 112967345ACN 202110254664 ACN202110254664 ACN 202110254664ACN 112967345 ACN112967345 ACN 112967345A
- Authority
- CN
- China
- Prior art keywords
- image
- fisheye camera
- focal length
- perception information
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本申请涉及计算机技术中的人工智能、自动驾驶、智能交通、车路协同路侧感知以及计算机视觉，尤其涉及一种鱼眼相机的外参标定方法、装置及系统。The present application relates to artificial intelligence, automatic driving, intelligent transportation, vehicle-road cooperative roadside perception and computer vision in computer technology, and in particular to a method, device and system for calibrating external parameters of a fisheye camera.
背景技术Background technique
随着车辆网技术地发展，车用无线通信技术(Vehicle to X，V2X)路侧感知系统为车路协同的车辆提供了超视距的感知信息，路侧相机为路侧感知系统的最主要的传感器之一，路侧相机通常包括枪机和鱼眼相机。With the development of vehicle network technology, the vehicle wireless communication technology (Vehicle to X, V2X) roadside perception system provides over-the-horizon perception information for the vehicle-road coordination vehicle, and the roadside camera is the most important part of the roadside perception system. One of the sensors, roadside cameras usually include bolt and fisheye cameras.
在现有技术中，感知信息通常由枪机获取到的图像确定，鱼眼相机通常用于监控，从而一般无需对鱼眼相机的外参进行标定，且当鱼眼相机应用于其他场景时，通常采用的外参标定方法为“借助棋盘格”标定法，如制作棋盘格(每个格子的大小可测量)，提取棋盘格的内角点，基于鱼眼相机采集图像，并将图像中的像素点进行腐蚀和膨胀等处理，并基于预设的标定函数对处理后的像素点与棋盘格中的内角点进行计算，从而获取鱼眼相机的外参。In the prior art, the perception information is usually determined by the image obtained by the gun, and the fisheye camera is usually used for monitoring, so it is generally unnecessary to calibrate the external parameters of the fisheye camera, and when the fisheye camera is applied to other scenes, The commonly used external parameter calibration method is the "checkerboard" calibration method, such as making a checkerboard (the size of each grid can be measured), extracting the inner corner points of the checkerboard, collecting images based on a fisheye camera, and converting the pixels in the image Corrosion and expansion are performed on the points, and the processed pixels and the inner corners of the checkerboard are calculated based on the preset calibration function, so as to obtain the external parameters of the fisheye camera.
然而，采用上述对鱼眼相机的外参进行标定，可能存在对于未能检测到所有棋盘格内角点的图像都会被弃之不用而造成的可靠性偏低，成本偏高的问题。However, using the above-mentioned calibration of the external parameters of the fisheye camera may have the problems of low reliability and high cost that images that fail to detect all the inner corners of the checkerboard will be discarded.
发明内容SUMMARY OF THE INVENTION
本申请提供了一种用于提高标定可靠性的鱼眼相机的外参标定方法、装置及系统。The present application provides an external parameter calibration method, device and system for a fisheye camera for improving calibration reliability.
根据本申请的第一方面，提供了一种鱼眼相机的外参标定方法，包括：According to a first aspect of the present application, a method for calibrating external parameters of a fisheye camera is provided, including:
根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像；其中，所述目标焦距为所述鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距；According to the target focal length, the original image collected by the fisheye camera is subjected to de-distortion processing to obtain a de-distorted image; wherein, the target focal length is the area between the fisheye camera and the gun that has a preset image collection overlap area. the corresponding focal length;
根据所述去畸变图像确定所述鱼眼相机的外参。Extrinsic parameters of the fisheye camera are determined according to the undistorted image.
根据本申请的第二方面，提供了一种鱼眼相机的外参标定装置，包括：According to a second aspect of the present application, a device for calibrating external parameters of a fisheye camera is provided, comprising:
第一处理单元，用于根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像；其中，所述目标焦距为所述鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距；The first processing unit is configured to perform de-distortion processing on the original image collected by the fish-eye camera according to the target focal length to obtain a de-distorted image; wherein the target focal length is a predetermined distance between the fish-eye camera and the gun. Set the focal length corresponding to the overlapping area of image acquisition;
第一确定单元，用于根据所述去畸变图像确定所述鱼眼相机的外参。a first determining unit, configured to determine the extrinsic parameters of the fisheye camera according to the dedistorted image.
根据本申请的第三方面，提供了一种电子设备，包括：According to a third aspect of the present application, an electronic device is provided, comprising:
至少一个处理器；以及at least one processor; and
与所述至少一个处理器通信连接的存储器；其中，a memory communicatively coupled to the at least one processor; wherein,
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上第一方面所述的方法。The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the first aspect above.
根据本申请的第四方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行如上第一实施例所述的方法。According to a fourth aspect of the present application, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method as described in the first embodiment above.
根据本申请的第五方面，提供了一种计算机程序产品，所述程序产品包括：计算机程序，所述计算机程序存储在可读存储介质中，电子设备的至少一个处理器可以从所述可读存储介质读取所述计算机程序，所述至少一个处理器执行所述计算机程序使得电子设备执行第一方面所述的方法。According to a fifth aspect of the present application, there is provided a computer program product, the program product comprising: a computer program, the computer program being stored in a readable storage medium, from which at least one processor of an electronic device can read The storage medium reads the computer program, and the at least one processor executes the computer program to cause the electronic device to perform the method of the first aspect.
根据本申请的第六方面，提供了一种路侧设备，包括如上第三方面所述的电子设备。According to a sixth aspect of the present application, a roadside device is provided, including the electronic device as described in the third aspect.
根据本申请的第七方面，提供了一种云控平台，包括如上第三方面所述的电子设备。According to a seventh aspect of the present application, a cloud control platform is provided, including the electronic device described in the third aspect.
根据本申请的第八方面，提供了一种鱼眼相机的外参标定系统，包括：鱼眼相机、枪机、以及如上第二方面所述的装置；其中，According to an eighth aspect of the present application, an external parameter calibration system for a fisheye camera is provided, comprising: a fisheye camera, a bolt, and the device according to the second aspect; wherein,
所述鱼眼相机和所述枪机用于确定目标焦距。The fisheye camera and the gun are used to determine the focal length of the target.
应当理解，本部分所描述的内容并非旨在标识本申请的实施例的关键或重要特征，也不用于限制本申请的范围。本申请的其它特征将通过以下的说明书而变得容易理解。It should be understood that the content described in this section is not intended to identify key or critical features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the present application will become readily understood from the following description.
附图说明Description of drawings
附图用于更好地理解本方案，不构成对本申请的限定。其中：The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present application. in:
图1是根据本申请第一实施例的示意图；1 is a schematic diagram according to a first embodiment of the present application;
图2是根据本申请第二实施例的示意图；2 is a schematic diagram according to a second embodiment of the present application;
图3是鱼眼相机采集到的原始图像的示意图；3 is a schematic diagram of an original image collected by a fisheye camera;
图4是对原始图像进行去畸变处理，得到的去畸变图像的示意图；FIG. 4 is a schematic diagram of a de-distorted image obtained by performing de-distortion processing on an original image;
图5是根据本申请第三实施例的示意图；5 is a schematic diagram according to a third embodiment of the present application;
图6是根据本申请第四实施例的示意图；6 is a schematic diagram according to a fourth embodiment of the present application;
图7是用来实现本申请实施例的电子设备的框图。FIG. 7 is a block diagram of an electronic device used to implement an embodiment of the present application.
具体实施方式Detailed ways
以下结合附图对本申请的示范性实施例做出说明，其中包括本申请实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本申请的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
为了提高车辆行驶的安全性和可靠性，路侧感知系统可以为车辆提供感知信息，感知信息可以包括障碍物(如车辆以及行人等)的相关信息，如障碍物的位置信息等。In order to improve the safety and reliability of vehicle driving, the roadside perception system can provide perception information for the vehicle, and the perception information can include information about obstacles (such as vehicles and pedestrians), such as the location information of obstacles.
其中，路侧相机为路侧感知系统的最主要的传感器之一，路侧相机通常可以包括枪机和鱼眼相机，且枪机可以包括前视枪机和后视枪机。一般情况下，路口(如十字路口，或者，丁字形路口)处可以设置监控杆(即立杆)，监控杆上可以同时设置鱼眼相机、后视枪机和前视枪机。Among them, the roadside camera is one of the most important sensors of the roadside perception system. The roadside camera can usually include a bolt and a fisheye camera, and the bolt can include a front-view bolt and a rear-view bolt. Under normal circumstances, a monitoring pole (ie, a vertical pole) can be set at an intersection (such as an intersection, or a T-shaped intersection), and a fisheye camera, a rear-view camera and a front-view camera can be set on the monitoring pole at the same time.
在相关技术中，鱼眼相机通常用于监控，因此一般无需对鱼眼相机的外参进行标定，且当鱼眼相机应用于其他场景时，通常采用的外参标定方法为“借助棋盘格”标定法，如检测棋盘格中的内角点，进行计算后获取鱼眼相机的外参。In the related art, the fisheye camera is usually used for monitoring, so it is generally unnecessary to calibrate the external parameters of the fisheye camera, and when the fisheye camera is applied to other scenes, the commonly used external parameter calibration method is "with the help of checkerboard" The calibration method, such as detecting the inner corner points in the checkerboard, obtains the external parameters of the fisheye camera after calculation.
然而，受各种因素的影响，鱼眼相机拍摄的图像往往无法采集到棋盘格全部的内角点，这就可能导致鱼眼相机拍摄的大量图像无法使用，只有少数图像可用，从而造成图像淘汰率高，标定效率低，成本偏高，且可靠性偏低等技术问题。However, affected by various factors, the images captured by the fisheye camera are often unable to capture all the inner corners of the checkerboard, which may result in a large number of images captured by the fisheye camera being unusable, and only a few images are available, resulting in an image elimination rate. High, low calibration efficiency, high cost, and low reliability and other technical problems.
为了避免上述技术问题中的至少一种，本申请实施例的发明人经过创造性地劳动，得到了本申请实施例的发明构思：将鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距作为目标焦距，基于目标焦距对鱼眼相机采集的原始图像进行去畸变处理，得到去畸变图像，并基于去畸变图像生成鱼眼相机的外参。In order to avoid at least one of the above-mentioned technical problems, the inventors of the embodiments of the present application have creatively worked to obtain the inventive concept of the embodiments of the present application: a pre-set image acquisition coincidence area between the fisheye camera and the gun is obtained. The corresponding focal length is taken as the target focal length, and the original image collected by the fisheye camera is de-distorted based on the target focal length to obtain a de-distorted image, and the extrinsic parameters of the fish-eye camera are generated based on the de-distorted image.
基于上述发明构思，本申请提供一种鱼眼相机的外参标定方法、装置及路侧设备，应用于计算机技术中的人工智能、自动驾驶、智能交通、车路协同路侧感知以及计算机视觉，以达到节约标定成本，提高标定准确性的技术效果。Based on the above inventive concept, the present application provides an external parameter calibration method, device and roadside equipment for a fisheye camera, which are applied to artificial intelligence, autonomous driving, intelligent transportation, vehicle-road collaborative roadside perception and computer vision in computer technology, so as to provide To achieve the technical effect of saving the cost of calibration and improving the accuracy of calibration.
图1是根据本申请第一实施例的示意图，如图1所示，该方法包括：FIG. 1 is a schematic diagram according to a first embodiment of the present application. As shown in FIG. 1 , the method includes:
S101：根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像。S101: According to the target focal length, perform de-distortion processing on the original image collected by the fisheye camera to obtain a de-distorted image.
其中，目标焦距为鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距。Wherein, the target focal length is the focal length corresponding to the overlapped area between the fisheye camera and the gun when there is a preset image capture area.
示例性地，本实施例的执行主体可以为鱼眼相机的外参标定装置(下文简称外参标定装置)，外参标定装置可以为服务器(包括本地服务器和云端服务器，服务器可以为云控平台、车路协同管理平台、中心子系统、边缘计算平台、云计算平台等)，也可以为路侧设备，也可以为终端设备，也可以为处理器，还可以为芯片，等等，本实施例不做限定。Exemplarily, the execution body of this embodiment may be an external parameter calibration device of a fisheye camera (hereinafter referred to as an external parameter calibration device), and the external parameter calibration device may be a server (including a local server and a cloud server, and the server may be a cloud control platform. , vehicle-road collaborative management platform, central subsystem, edge computing platform, cloud computing platform, etc.), it can also be roadside equipment, terminal equipment, processor, chip, etc., this implementation Examples are not limited.
其中，路侧设备例如有计算功能的路侧感知设备、与路侧感知设备相连接的路侧计算设备，在智能交通车路协同的系统架构中，路侧设备包括路侧感知设备和路侧计算设备，路侧感知设备(例如路侧相机)连接到路侧计算设备(例如路侧计算单元RSCU)，路侧计算设备连接到服务器，服务器可以通过各种方式与自动驾驶或辅助驾驶车辆通信；或者，路侧感知设备自身包括计算功能，则路侧感知设备直接连接到服务器。以上连接可以是有线或是无线。Among them, roadside equipment such as roadside sensing equipment with computing function, roadside computing equipment connected with roadside sensing equipment, in the system architecture of intelligent traffic vehicle-road coordination, roadside equipment includes roadside sensing equipment and roadside sensing equipment. Computing devices, roadside perception devices (such as roadside cameras) are connected to roadside computing devices (such as roadside computing units RSCU), and roadside computing devices are connected to servers, which can communicate with autonomous or assisted driving vehicles in various ways ; Or, if the roadside sensing device itself includes a computing function, the roadside sensing device is directly connected to the server. The above connections can be wired or wireless.
值得说明地是，在本实施例中，引入了根据鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距作为目标焦距，以便基于该目标焦距对鱼眼相机所采集的原始图像进行去畸变处理的特征，通过引入该特征，可以使得在目标焦距的基础上，去畸变处理后的图像(即去畸变图像)与枪机所采集的图像之间具有重合区域，且尤其当重合区域相对较大时，相对而言，可以使得通过本实施例确定出的去畸变图像的范围相对较大，从而当后续基于该去畸变图像确定鱼眼相机的外参时，可以相对确保去畸变图像的全面覆盖性，进而实现提高确定出的鱼眼相机的外参的准确性和可靠性，且相对于采用上述相关技术中的“借助棋盘格”标定法确定鱼眼相机的外参，可以避免受各种因素的影响，鱼眼相机拍摄的图像往往无法采集到棋盘格全部的内角点，从而导致鱼眼相机拍摄的大量图像无法使用的问题，而由于可能只有少数图像可用，图像淘汰率高，则可能导致的标定效率低的问题，且可以避免因图像被弃用而造成的可靠性偏低和拍摄成本偏高的问题。It is worth noting that, in this embodiment, the focal length corresponding to the overlapped area between the fisheye camera and the gun with a preset image acquisition is introduced as the target focal length, so that based on the target focal length, the images captured by the fisheye camera are collected. The original image is de-distorted. By introducing this feature, on the basis of the target focal length, the de-distorted image (that is, the de-distorted image) and the image collected by the camera have an overlapping area, and Especially when the overlapping area is relatively large, relatively speaking, the range of the undistorted image determined by this embodiment can be made relatively large, so that when the external parameters of the fisheye camera are subsequently determined based on the undistorted image, it can be relatively Ensure the comprehensive coverage of the de-distorted image, thereby improving the accuracy and reliability of the determined external parameters of the fisheye camera, and compared with the "checkerboard" calibration method in the above-mentioned related art to determine the external parameters of the fisheye camera. It can avoid the influence of various factors. The images captured by the fisheye camera often cannot capture all the inner corners of the checkerboard, which leads to the problem that a large number of images captured by the fisheye camera cannot be used, and because there may be only a few images available, If the image rejection rate is high, the problem of low calibration efficiency may be caused, and the problems of low reliability and high shooting cost caused by discarded images can be avoided.
S102：根据去畸变图像确定鱼眼相机的外参。S102: Determine the extrinsic parameters of the fisheye camera according to the dedistorted image.
一个示例中，外参标定装置可以采用人工标注的方式，根据去畸变图像确定鱼眼相机的外参。In an example, the external parameter calibration device may use manual annotation to determine the external parameters of the fisheye camera according to the dedistorted image.
例如，外参标定装置可以通过人工的方式基于去畸变图像中的像素点，在世界坐标系中执行采集世界坐标的操作，并基于去畸变图像和采集到的世界坐标进行不同坐标系(即世界坐标系和图像坐标系)的坐标转换计算，从而得到鱼眼相机的外参。For example, the external parameter calibration device can manually perform the operation of collecting world coordinates in the world coordinate system based on the pixels in the undistorted image, and perform different coordinate systems (that is, the world coordinate system) based on the undistorted image and the collected world coordinates. The coordinate conversion calculation of the coordinate system and the image coordinate system) to obtain the external parameters of the fisheye camera.
另一个示例中，外参标定装置可以基于历史图像辅助去畸变图像的方式，确定鱼眼相机的外参。In another example, the extrinsic parameter calibration device may determine the extrinsic parameters of the fisheye camera by assisting in de-distorting images based on historical images.
例如，外参标定装置可以获取由其他相机采集的历史图像，外参标定装置根据其他相机的参数(如外参)和历史图像，确定与历史图像中至少一个像素点的世界坐标，并确定该至少一个像素点在畸变图像中的像素坐标，并基于该像素坐标与确定出的世界坐标进行不同坐标系(即世界坐标系和图像坐标系)的坐标转换计算，从而得到鱼眼相机的外参。For example, the external parameter calibration device can obtain historical images collected by other cameras, and the external parameter calibration device determines the world coordinates of at least one pixel in the historical image according to the parameters of other cameras (such as external parameters) and the historical image, and determines the The pixel coordinates of at least one pixel in the distorted image, and based on the pixel coordinates and the determined world coordinates, the coordinate conversion calculation of different coordinate systems (ie, the world coordinate system and the image coordinate system) is performed, so as to obtain the external parameters of the fisheye camera. .
其中，其他相机的设置位置与鱼眼相机的设置位置可以相同，以便其他相机采集到的历史图像与去畸变图像针对的为相同对象。The setting positions of the other cameras and the fisheye cameras may be the same, so that the historical images collected by the other cameras and the de-distorted images are aimed at the same object.
需要说明地是，上述实施例只是用于示范性地说明，本实施例可以采用的根据去畸变图像确定鱼眼相机的外参的方法，而不能理解为对根据去畸变图像确定鱼眼相机的外参的方法的限定。It should be noted that the above embodiments are only used for exemplary illustration, and the method of determining the external parameters of the fisheye camera according to the dedistorted image can be used in this embodiment, and cannot be understood as determining the method of determining the fisheye camera according to the dedistorted image. Method limitations for external parameters.
基于上述分析可知，本申请实施例提供了一种鱼眼相机的外参标定方法，该方法包括：根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像，其中，目标焦距为鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距，根据去畸变图像确定鱼眼相机的外参，通过根据鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距作为目标焦距，以便基于该目标焦距对鱼眼相机所采集的原始图像进行去畸变处理，以便基于得到的去畸变图像确定鱼眼相机的外参，可以避免受各种因素的影响，鱼眼相机拍摄的图像往往无法采集到棋盘格全部的内角点，从而导致鱼眼相机拍摄的大量图像无法使用的问题，而由于可能只有少数图像可用，图像淘汰率高，则可能导致的标定效率低的问题，而相对而言，通过本实施例确定出的去畸变图像的范围相对较大，与枪机采集的图像之间的盲区减小甚至无盲区，从而当后续基于该去畸变图像确定鱼眼相机的外参时，可以相对确保去畸变图像的全面覆盖性，进而实现提高确定出的鱼眼相机的外参的准确性和可靠性的技术效果。Based on the above analysis, the embodiment of the present application provides a method for calibrating external parameters of a fisheye camera. The method includes: performing a de-distortion process on an original image collected by the fish-eye camera according to the target focal length to obtain a de-distorted image, wherein, The target focal length is the focal length corresponding to the overlapped area between the fisheye camera and the bolt when there is a preset image acquisition, and the external parameters of the fisheye camera are determined according to the undistorted image. The focal length corresponding to the overlapping area of image acquisition is set as the target focal length, so that the original image collected by the fisheye camera can be de-distorted based on the target focal length, so that the extrinsic parameters of the fish-eye camera can be determined based on the obtained de-distorted image. Affected by various factors, the images captured by the fisheye camera are often unable to capture all the inner corners of the checkerboard, resulting in the problem that a large number of images captured by the fisheye camera cannot be used, and since only a few images may be available, the image elimination rate is high. , it may lead to the problem of low calibration efficiency. Relatively speaking, the range of the de-distorted image determined by this embodiment is relatively large, and the blind area between the image collected by the gun is reduced or even no blind area, so that when When the extrinsic parameters of the fisheye camera are subsequently determined based on the undistorted image, the comprehensive coverage of the undistorted image can be relatively ensured, thereby achieving the technical effect of improving the accuracy and reliability of the determined extrinsic parameters of the fisheye camera.
图2是根据本申请第一实施例的示意图，如图2所示，该方法包括：FIG. 2 is a schematic diagram according to the first embodiment of the present application. As shown in FIG. 2 , the method includes:
S201：根据鱼眼相机在每一焦距下的视场角，确定鱼眼相机在每一焦距下的图像采集区域。S201: Determine an image acquisition area of the fisheye camera at each focal length according to the angle of view of the fisheye camera at each focal length.
示例性地，可以基于式1确定焦距与视场角之间的关系，式1：Exemplarily, the relationship between the focal length and the angle of view can be determined based on Equation 1, Equation 1:
θ＝2*atan((L/2)/f)θ=2*atan((L/2)/f)
其中，θ为视场角，L为鱼眼相机的胶片的对角线长度，f为焦距。Among them, θ is the field of view angle, L is the diagonal length of the film of the fisheye camera, and f is the focal length.
本实施例可以理解为：不同的焦距对应的视场角不同，鱼眼相机基于不同的视场角所采集的图像的覆盖范围(即采集区域)不同，针对某焦距，外参标定装置可以基于上述式1确定该焦距对应的视场角，以便确定鱼眼相机在该视场角时，鱼眼相机的图像采集区域。This embodiment can be understood as: different focal lengths correspond to different field of view angles, and the coverage of images (ie, acquisition areas) collected by the fisheye camera based on different field of view angles is different, and for a certain focal length, the external parameter calibration device can be based on The above formula 1 determines the field of view angle corresponding to the focal length, so as to determine the image acquisition area of the fisheye camera when the fisheye camera is at this field of view angle.
在一些实施例中，S201可以包括：根据鱼眼相机在每一焦距下的视场角、以及获取到鱼眼相机所位于的监控杆的高度信息，确定鱼眼相机在每一焦距下的图像采集区域。In some embodiments, S201 may include: determining an image of the fisheye camera at each focal length according to the angle of view of the fisheye camera at each focal length and the acquired height information of the monitoring pole where the fisheye camera is located collection area.
例如，针对某焦距而言，外参标定装置可以确定鱼眼相机在该焦距下的视场角，可以得到基于该视场角得到相应的图像，且外参标定装置可以获取监控杆的高度信息，鱼眼相机安装于监控杆，则可以外参标定装置可以基于视场角得到的图像、以及监控杆的高度信息，确定基于视场角得到图像的左侧至监控杆之间的距离，并确定基于视场角得到图像的右侧至监控杆之间的距离，从而得到该焦距下的图像采集区域。For example, for a certain focal length, the external parameter calibration device can determine the angle of view of the fisheye camera at the focal length, and can obtain a corresponding image based on the angle of view, and the external parameter calibration device can obtain the height information of the monitoring rod , the fisheye camera is installed on the monitoring rod, then the external parameter calibration device can determine the distance from the left side of the image obtained based on the field of view to the monitoring rod based on the image obtained by the field of view angle and the height information of the monitoring rod, and Determine the distance from the right side of the image obtained based on the field of view to the monitoring rod, so as to obtain the image acquisition area under the focal length.
其中，关于基于视场角得到图像的左侧至监控杆之间的距离、以及基于视场角得到图像的右侧至监控杆之间的距离的计算，可以参见相机的成像原理实现，此处不再赘述。Among them, for the calculation of the distance between the left side of the image obtained based on the field of view and the monitoring rod, and the calculation of the distance between the right side of the image obtained based on the field of view and the monitoring rod, you can refer to the realization of the imaging principle of the camera, here No longer.
值得说明地是，在本实施例中，通过结合视场角和监控杆的高度信息，确定鱼眼相机在每一焦距下的图像采集区域，可以提高确定图像采集区域的准确性、可靠性、以及便捷性。It is worth noting that, in this embodiment, by combining the angle of view and the height information of the monitoring rod to determine the image capture area of the fisheye camera at each focal length, the accuracy, reliability, and reliability of determining the image capture area can be improved. and convenience.
S202：根据枪机在每一焦距下视场角，确定枪机在每一焦距下的图像采集区域。S202: Determine the image acquisition area of the gun under each focal length according to the field of view angle of the gun under each focal length.
同理，在该步骤中，可以基于上述式1的方式确定枪机在每一焦距下的图像采集区域，实现原理可以参见S201地描述，此处不再赘述。Similarly, in this step, the image acquisition area of the gun under each focal length can be determined based on the above formula 1, and the implementation principle can be described with reference to S201, which will not be repeated here.
在一些实施例中，S202可以包括：根据枪机在每一焦距下的视场角、以及获取到监控杆的高度信息，确定枪机在每一焦距下的图像采集区域。In some embodiments, S202 may include: determining the image acquisition area of the gun under each focal length according to the field of view of the gun under each focal length and the height information of the monitoring rod obtained.
同理，关于基于每一焦距下的视场角、以及监控杆的高度信息，确定枪机在每一焦距下的图像采集区域，可以参见上述实施例中，基于每一焦距下的视场角、以及监控杆的高度信息，确定鱼眼相机在每一焦距下的图像采集区域的原理，此处不再赘述。且通过结合视场角和监控杆的高度信息，确定枪机在每一焦距下的图像采集区域，可以提高确定图像采集区域的准确性、可靠性、以及便捷性。In the same way, regarding the determination of the image acquisition area of the gun under each focal length based on the field of view angle at each focal length and the height information of the monitoring rod, you can refer to the above-mentioned embodiment, based on the field of view angle at each focal length. , and the height information of the monitoring rod, the principle of determining the image acquisition area of the fisheye camera at each focal length will not be repeated here. And by combining the field of view and the height information of the monitoring rod to determine the image acquisition area of the gun at each focal length, the accuracy, reliability, and convenience of determining the image acquisition area can be improved.
S203：根据鱼眼相机在每一焦距下的图像采集区域、以及枪机在每一焦距下的图像采集区域，确定预设图像采集重合区域。S203: Determine a preset image capture overlap area according to the image capture area of the fisheye camera at each focal length and the image capture area of the gun under each focal length.
示例性地，针对某一焦距，外参标定装置在可以确定鱼眼相机对应的图像采集区域、以及确定枪机对应的图像采集区域，并对两个图像采集区域进行共同区域的确定，确定出的共同区域即为在该焦距下，鱼眼相机的图像与枪机的图像的重合区域。Exemplarily, for a certain focal length, the external parameter calibration device can determine the image acquisition area corresponding to the fisheye camera and the image acquisition area corresponding to the gun, and determine the common area of the two image acquisition areas, and determine the The common area of is the overlapping area of the image of the fisheye camera and the image of the gun at this focal length.
一个示例中，外参标定装置可以预先设置焦距集合，焦距集合中包括多个焦距，外参标定装置可以从焦距集合中依次选取一个焦距，并确定在选取出的焦距的基础上，鱼眼相机对应的图像、以及枪机对应的图像，并确定该两个图像的重合区域，如果两个图像的重合区域的面积大于预设的面积，则将该两个图像的重合区域确定为预设图像采集重合区域，相应地，将选取出的焦距确定为目标焦距。In an example, the external parameter calibration device can preset a focal length set, the focal length set includes a plurality of focal lengths, the external parameter calibration device can sequentially select a focal length from the focal length set, and determine on the basis of the selected focal length, the fisheye camera. The corresponding image and the corresponding image of the gun, and determine the overlapping area of the two images, if the area of the overlapping area of the two images is greater than the preset area, then determine the overlapping area of the two images as the preset image The overlapping area is collected, and accordingly, the selected focal length is determined as the target focal length.
另一个示例中，外参标定装置可以预先设置多个焦距，外参标定装置可以分别计算，在每一个焦距的基础上，鱼眼相机对应的图像、以及枪机对应的图像，并确定该两个图像的重合区域，当所有焦距各自对应的两个图像的重合区域都确定时，从所有两个图像的重合区域中，将重合区域最多的重合区域确定为预设图像采集重合区域，相应地，将重合区域最多的重合区域对应的焦距确定为目标焦距。In another example, the external parameter calibration device may preset multiple focal lengths, and the external parameter calibration device may calculate separately, on the basis of each focal length, the image corresponding to the fisheye camera and the image corresponding to the gun, and determine the two focal lengths. The overlapping areas of the two images, when the overlapping areas of the two images corresponding to all the focal lengths are determined, from the overlapping areas of all the two images, the overlapping area with the most overlapping areas is determined as the preset image acquisition overlapping area, correspondingly , and the focal length corresponding to the overlapping area with the most overlapping area is determined as the target focal length.
需要说明地是，上述示例只是用于示范性地说明，确定预设图像采集重合区域可能实现的方法，而不能理解为对确定预设图像采集重合区域的方法的限定。It should be noted that the above examples are only used to illustrate the possible implementation of the method for determining the overlapped area of preset image acquisition, and should not be understood as a limitation on the method for determining the overlapped area of preset image acquisition.
值得说明地是，在本实施例中，通过每一焦距下的视场角，确定鱼眼相机和枪机各自对应的图像采集区域，并基于各自对应的图像采集区域确定预设图像采集重合区域，可以提高确定预设图像采集重合区域的准确性和可靠性的技术效果。It is worth noting that, in this embodiment, the corresponding image capture areas of the fisheye camera and the gun are determined through the field of view at each focal length, and the preset image capture overlap area is determined based on the respective corresponding image capture areas. , can improve the technical effect of determining the accuracy and reliability of the pre-set image acquisition coincidence area.
S204：根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像。S204: According to the target focal length, perform de-distortion processing on the original image collected by the fisheye camera to obtain a de-distorted image.
其中，目标焦距为鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距。Wherein, the target focal length is the focal length corresponding to the overlapped area between the fisheye camera and the gun when there is a preset image capture area.
示例性地，关于S204地描述，可以参见S101，此处不再赘述。Exemplarily, for the description of S204, reference may be made to S101, and details are not repeated here.
其中，原始图像可以参阅图3。Among them, the original image can refer to Figure 3.
在一些实施例中，S204可以包括：获取鱼眼相机基于目标焦距采集的标定板图像，将标定板图像投影到球面上，得到球面图像，对球面图像进行去畸变处理，得到去畸变图像。In some embodiments, S204 may include: acquiring a calibration plate image collected by the fisheye camera based on the target focal length, projecting the calibration plate image onto a spherical surface to obtain a spherical image, and performing de-distortion processing on the spherical image to obtain a de-distorted image.
示例性地，标定板图像的数量可以为一个，也可以为多个，当标定板图像的数量为多个时，则可以对多个标定板图像进行均值处理，得到用于投影至球面，得到球面图像的标定板图像。而通过基于多个标定板图像得到球面图像，可以减小得到的球面图像的误差，从而使得去畸变处理具有较高的准确性和可靠性的技术效果。Exemplarily, the number of calibration plate images may be one or multiple, and when the number of calibration plate images is multiple, the average value processing may be performed on the multiple calibration plate images to obtain for projecting to a spherical surface, obtaining: Calibration plate image for spherical image. By obtaining a spherical image based on a plurality of calibration plate images, the error of the obtained spherical image can be reduced, so that the de-distortion processing has the technical effect of high accuracy and reliability.
在一些实施例中，对球面图像进行去畸变处理，得到去畸变图像可以包括：确定球面图像与方形图像之间的转换关系，基于转换关系对球面图像的各像素点进行转换，得到各像素点转换至方形图像中的位置信息，从而得到去畸变图像。其中，球面图像对应的球面为方形图像的方形的外切圆或者内切圆。In some embodiments, performing de-distortion processing on the spherical image to obtain the de-distorted image may include: determining a conversion relationship between the spherical image and the square image, and converting each pixel of the spherical image based on the conversion relationship to obtain each pixel Convert to the position information in the square image, resulting in an undistorted image. The spherical surface corresponding to the spherical image is a square circumscribed circle or an inscribed circle of the square image.
其中，去畸变图像可以参阅图4。The de-distorted image can be seen in Figure 4.
也就是说，通过S204的方案，可以基于图3所示的畸变图像，得到如图4所示的去畸变图像。That is to say, through the solution of S204, the distorted image shown in FIG. 4 can be obtained based on the distorted image shown in FIG. 3 .
S205：根据去畸变图像确定鱼眼相机的外参。S205: Determine the external parameters of the fisheye camera according to the dedistorted image.
示例性地，关于S205地描述，可以参见S102，此处不再赘述。Exemplarily, for the description of S205, reference may be made to S102, which will not be repeated here.
在一些实施例中，去畸变图像为二维图像，S205可以包括如下步骤：In some embodiments, the dedistorted image is a two-dimensional image, and S205 may include the following steps:
S2051：获取去畸变图像中的至少一个像素点，在去畸变图像中的二维坐标、以及在世界坐标系下的三维坐标。S2051: Acquire at least one pixel point in the undistorted image, two-dimensional coordinates in the undistorted image, and three-dimensional coordinates in the world coordinate system.
应该理解地是，去畸变图像中包括多个像素点，每一个像素点在去畸变图像中具有二维坐标，二维坐标也可以理解为像素点在图像坐标系中的图像坐标。It should be understood that the undistorted image includes a plurality of pixel points, each pixel point has two-dimensional coordinates in the undistorted image, and the two-dimensional coordinates can also be understood as the image coordinates of the pixel point in the image coordinate system.
世界坐标系也可以理解为地面坐标系(World Geodetic System)，或者称为物理坐标系，即实际的比如地面和其他物体所在的坐标系。The world coordinate system can also be understood as the ground coordinate system (World Geodetic System), or called the physical coordinate system, that is, the actual coordinate system such as the ground and other objects.
在本实施例中，针对去畸变图像中的任一像素点而言，外参标定装置可以确定该任一像素点在图像坐标系下的二维坐标(即该任一像素点在去畸变图像中，以图像坐标系为基础的坐标)，也可以确定该任一像素点在世界坐标系下的三维坐标(即该任一像素点在物理坐标系中，以地面坐标系为基础的坐标，如全球定位系统坐标(Global PositioningSystem，GPS坐标))。In this embodiment, for any pixel point in the undistorted image, the external parameter calibration device can determine the two-dimensional coordinates of the any pixel point in the image coordinate system (that is, the any pixel point in the undistorted image , the coordinates based on the image coordinate system), you can also determine the three-dimensional coordinates of any pixel point in the world coordinate system (that is, the coordinates of any pixel point in the physical coordinate system, based on the ground coordinate system, Such as global positioning system coordinates (Global PositioningSystem, GPS coordinates)).
在一些实施例中，某一像素点在世界坐标系下的三维坐标，可以为该像素点在地面上所对应的地面点的三维坐标。In some embodiments, the three-dimensional coordinates of a certain pixel point in the world coordinate system may be the three-dimensional coordinates of the ground point corresponding to the pixel point on the ground.
示例性地，可以预先通过高精地图或者人工实时动态(real-time kinematic，简称RTK)设备打点的方式，确定任一像素点的GPS坐标，本实施例不做限定。Exemplarily, the GPS coordinates of any pixel point may be determined in advance by means of a high-precision map or an artificial real-time kinematic (RTK for short) device, which is not limited in this embodiment.
在本实施例中，对像素点的数量不做限定，可以为一个，也可以为多个。当像素点的数量为多个时，可以随机的选取多个像素点，且可以选择在去畸变图像中均匀分布的多个像素点。In this embodiment, the number of pixel points is not limited, and may be one or multiple. When the number of pixel points is multiple, multiple pixel points can be randomly selected, and multiple pixel points that are uniformly distributed in the dedistorted image can be selected.
值得说明地是，若选择的像素点的数量为多个，则相较而言，可以提高后续基于二维坐标和三维坐标，确定鱼眼相机的外参时的准确性和可靠性，且尤其当多个像素点为在去畸变图像中均匀分布的多个像素点时，可以充分考虑去畸变图像中各区域的像素点对确定出的鱼眼相机的外参的影响，从而可以进一步提高确定出的鱼眼相机的外参的准确性和可靠性的技术效果。It is worth noting that if the number of selected pixel points is multiple, in comparison, the accuracy and reliability of the subsequent determination of the external parameters of the fisheye camera based on two-dimensional coordinates and three-dimensional coordinates can be improved, and especially When the multiple pixels are evenly distributed in the de-distorted image, the influence of the pixels of each area in the de-distorted image on the determined extrinsic parameters of the fisheye camera can be fully considered, so that the determination can be further improved. The technical effect of the accuracy and reliability of the extrinsic parameters of the fisheye camera.
S2052：根据二维坐标和三维坐标，确定鱼眼相机的外参。S2052: Determine the external parameters of the fisheye camera according to the two-dimensional coordinates and the three-dimensional coordinates.
其中，鱼眼相机的外参包括：鱼眼相机在世界坐标系下的旋转矩阵和平移向量。Among them, the external parameters of the fisheye camera include: the rotation matrix and translation vector of the fisheye camera in the world coordinate system.
示例性地，外参标定装置可以结合相关技术中，图像坐标系与世界坐标系之间的坐标转换原理，对二维坐标和三维坐标进行转换，从而得到鱼眼相机在世界坐标系下的旋转矩阵和平移向量，即得到鱼眼相机的外参。Exemplarily, the external parameter calibration device can convert the two-dimensional coordinates and the three-dimensional coordinates in combination with the coordinate conversion principle between the image coordinate system and the world coordinate system in the related art, so as to obtain the rotation of the fisheye camera in the world coordinate system. The matrix and translation vector, that is, the external parameters of the fisheye camera.
在一些实施例中，可以通过对二维坐标和三维坐标进行PNP(pespective-n-point，简称PNP)解算处理，得到鱼眼相机的外参。In some embodiments, the external parameters of the fisheye camera may be obtained by performing PNP (pespective-n-point, PNP for short) calculation processing on the two-dimensional coordinates and the three-dimensional coordinates.
值得说明地是，在本实施例中，通过分别确定去畸变图像中的至少一个像素点，在去畸变图像中的二维坐标、以及在世界坐标系下的三维坐标，并基于二维坐标和三维坐标，确定鱼眼相机的外参，可以避免相关技术中可能存在对于未能检测到所有棋盘格内角点的图像都会被弃之不用而造成的可靠性偏低，成本偏高的问题，提高了确定鱼眼相机的外参的准确性和可靠性的技术效果，且降低了采集图像的成本。It is worth noting that, in this embodiment, by respectively determining at least one pixel point in the undistorted image, the two-dimensional coordinates in the undistorted image, and the three-dimensional coordinates in the world coordinate system, and based on the two-dimensional coordinates and Three-dimensional coordinates to determine the external parameters of the fisheye camera can avoid the problems of low reliability and high cost that may exist in the related technology for images that fail to detect all the inner corners of the checkerboard. The technical effect of determining the accuracy and reliability of the external parameters of the fisheye camera is obtained, and the cost of collecting images is reduced.
S206：根据鱼眼相机的外参，将鱼眼相机所采集的图像中目标对象的二维感知信息进行处理，得到目标对象的第一感知信息。S206: Process the two-dimensional perception information of the target object in the image collected by the fisheye camera according to the external parameters of the fisheye camera, to obtain the first perception information of the target object.
其中，第一感知信息为三维感知信息。三维感知信息可以包括位置信息等，如目标对象的世界坐标。The first perception information is three-dimensional perception information. The three-dimensional perception information may include position information and the like, such as the world coordinates of the target object.
需要说明的是，S206至S208的执行主体可以与上述实施例中的执行主体相同，也可以与上述实施例中的执行主体不同。例如：It should be noted that the execution subjects of S206 to S208 may be the same as the execution subjects in the foregoing embodiments, or may be different from the execution subjects in the foregoing embodiments. E.g:
上述实施例中的执行主体可以为路侧单元，也可以为与路侧单元连接的服务器，相应地，本实施例的执行主体可以为相同的路侧单元，或者与路侧单元连接的服务器。或者，The execution body in the above embodiment may be a roadside unit or a server connected to the roadside unit. Correspondingly, the execution body of this embodiment may be the same roadside unit or a server connected to the roadside unit. or,
上述实施例中的执行主体可以为路侧单元，本实施例的执行主体可以为与路侧单元连接的服务器。或者，The execution body in the above embodiment may be a roadside unit, and the execution body in this embodiment may be a server connected to the roadside unit. or,
上述实施例中的执行主体可以为与路侧单元连接的服务器，本实施例的执行主体可以为路侧单元。或者，The execution body in the above embodiment may be a server connected to the roadside unit, and the execution body in this embodiment may be the roadside unit. or,
上述实施例中的执行主体可以为与路侧单元，或者，路侧单元连接的服务器，本实施例的执行主体可以为车辆。The executive body in the above embodiment may be a server connected to the roadside unit or the roadside unit, and the executive body in this embodiment may be a vehicle.
例如，若S201至205的执行主体为路侧单元，或者路侧单元连接的服务器，则在路侧单元，或者路侧单元连接的服务器确定出鱼眼相机的外参之后，可以将鱼眼相机所采集的图像、以及鱼眼相机的外参发送接入至路侧单元，或者路侧单元连接的服务器的车辆，而车辆可以基于鱼眼相机的外参执行S206至S208。For example, if the execution subject of S201 to 205 is the roadside unit or the server connected to the roadside unit, after the roadside unit or the server connected to the roadside unit determines the external parameters of the fisheye camera, the fisheye camera can be The collected images and the external parameters of the fisheye camera are sent to the vehicle connected to the roadside unit, or the server connected to the roadside unit, and the vehicle can perform S206 to S208 based on the external parameters of the fisheye camera.
以该步骤的执行主体为车辆，S201至205的执行主体为路侧单元为例，本实施例可以理解为：车辆接收由路侧单元发送的鱼眼相机采集的图像、以及鱼眼相机的外参，鱼眼相机采集的图像中包括目标对象，确定目标对象的二维感知信息，并根据鱼眼相机的外参对目标的二维感知信息进行处理，得到目标对象的第一感知信息，该第一感知信息为三维感知信息。Taking the execution subject of this step as the vehicle, and the execution subject of S201 to 205 as the roadside unit as an example, this embodiment can be understood as: the vehicle receives the image collected by the fisheye camera sent by the roadside unit, and the external image of the fisheye camera. parameters, the image collected by the fisheye camera includes the target object, the two-dimensional perception information of the target object is determined, and the two-dimensional perception information of the target is processed according to the external parameters of the fisheye camera, so as to obtain the first perception information of the target object. The first perception information is three-dimensional perception information.
其中，目标对象可以为车辆基于驾驶需求等从鱼眼相机采集的图像中确定的目标对象，目标对象可以包括其他车辆和行人等，本实施例不做限定。The target object may be a target object determined by the vehicle from an image collected by a fisheye camera based on driving requirements, etc., and the target object may include other vehicles, pedestrians, etc., which is not limited in this embodiment.
S207：根据预设的枪机的外参，将枪机所采集的图像中目标对象的二维感知信息进行处理，得到目标对象的第二感知信息。S207: According to the preset external parameters of the trigger, process the two-dimensional perception information of the target object in the image collected by the trigger to obtain the second perception information of the target object.
其中，第二感知信息为三维感知信息。The second perception information is three-dimensional perception information.
同理，在本实施例中，车辆可以接收由路侧单元发送的枪机的外参，并基于枪机的外参对目标对象的二维感知信息进行处理，得到目标对象的第二感知信息，该第二感知信息为三维感知信息。Similarly, in this embodiment, the vehicle can receive the external parameters of the trigger sent by the roadside unit, and process the two-dimensional perception information of the target object based on the external parameters of the trigger to obtain the second perception information of the target object. , the second perception information is three-dimensional perception information.
S208：对第一感知信息和第二感知信息进行融合处理，得到目标对象的融合后的三维感知信息。S208: Perform fusion processing on the first perception information and the second perception information to obtain fused three-dimensional perception information of the target object.
值得说明地是，在本实施例中，通过分别确定第一感知信息和第二感知信息，并对第一感知信息和第二感知信息进行融合，得到目标对象的融合后的三维感知信息，而第一感知信息是基于鱼眼相机的外参确定的，也就是说，在本实施例中，引入了结合鱼眼相机和枪机，共同确定目标对象的三维感知信息的特征，从而可以避免相关技术中，基于枪机确定目标对象的三维感知信息时，由于盲区等造成的准确性偏低的问题，实现了提高确定目标对象的三维感知信息的准确性和可靠性的技术效果。It is worth noting that, in this embodiment, by respectively determining the first perception information and the second perception information, and fusing the first perception information and the second perception information, the fused three-dimensional perception information of the target object is obtained, and The first perception information is determined based on the external parameters of the fisheye camera, that is to say, in this embodiment, the combination of the fisheye camera and the gun is introduced to jointly determine the characteristics of the three-dimensional perception information of the target object, so as to avoid correlation. In the technology, when the 3D perception information of the target object is determined based on the bolt, the problem of low accuracy due to blind spots, etc., achieves the technical effect of improving the accuracy and reliability of determining the 3D perception information of the target object.
在一些实施例中，第一感知信息和第二感知信息均为目标对象的世界坐标信息，S208包括：对第一感知信息和第二感知信息进行加权处理，得到目标对象的融合后的三维感知信息。In some embodiments, the first perception information and the second perception information are both world coordinate information of the target object, and S208 includes: performing weighting processing on the first perception information and the second perception information to obtain the fused three-dimensional perception of the target object information.
例如，可以为枪机和鱼眼相机分配不同的权重系数，优选地，枪机的权重系数可以大于鱼眼相机的权重系数，以便尽可能减小由于去畸变处理造成的信息丢失等问题，从而提高最终确定出的三维感知信息的准确性和可靠性的技术效果。For example, different weight coefficients can be assigned to the bolt and the fisheye camera. Preferably, the weight coefficient of the bolt can be larger than that of the fisheye camera, so as to minimize the loss of information caused by the de-distortion process, and thus The technical effect of improving the accuracy and reliability of the finally determined three-dimensional perception information.
基于上述分析可知，在一些实施例中，S206至S208的执行主体与S201至205的执行主体可能相同，例如，均为路侧单元，则在S208之后，一个示例中，该方法还可以包括：路侧单元将目标对象的融合后的三维感知信息发送给车辆，其中，目标对象的融合后的三维感知信息用于生成调整或维持车辆的行驶信息的行驶策略。Based on the above analysis, in some embodiments, the execution subjects of S206 to S208 may be the same as the execution subjects of S201 to 205, for example, both are roadside units, then after S208, in an example, the method may further include: The roadside unit sends the fused three-dimensional perception information of the target object to the vehicle, wherein the fused three-dimensional perception information of the target object is used to generate a driving strategy for adjusting or maintaining the driving information of the vehicle.
例如，路侧单元将目标对象的融合后的三维感知信息发送给车辆，车辆根据目标对象的融合后的三维感知信息、以及车辆当前的行驶信息，如定位信息和速度，确定目标对象与车辆之间的距离，并可以基于速度确定是否需要进行减速或绕行，即得到用于调整或维持车辆的行驶信息的行驶策略。For example, the roadside unit sends the fused 3D perception information of the target object to the vehicle, and the vehicle determines the relationship between the target object and the vehicle according to the fused 3D perception information of the target object and the current driving information of the vehicle, such as positioning information and speed. It can determine whether to decelerate or detour based on the speed, that is, to obtain a driving strategy for adjusting or maintaining the driving information of the vehicle.
另一个示例中，路侧单元获取车辆的行驶信息，并根据行驶信息和目标对象的融合后的三维感知信息，生成并输出(如向车辆发送)用于指示调整或维持车辆的行驶信息的行驶策略。In another example, the roadside unit acquires the driving information of the vehicle, and generates and outputs (eg, sends it to the vehicle) according to the fusion of the driving information and the three-dimensional perception information of the target object, which is used to instruct the driving to adjust or maintain the driving information of the vehicle. Strategy.
而在一些实施例中，S206至S208的执行主体与S201至205的执行主体可能不同，例如，S201至205的执行主体为路侧单元，S206至S208的执行主体为车辆，则在S208之后，该方法还可以包括：车辆基于车辆的行驶信息、以及目标对象的融合后的三维感知信息生成调整或维持车辆的行驶信息的行驶策略。In some embodiments, the execution subjects of S206 to S208 may be different from the execution subjects of S201 to 205. For example, the execution subjects of S201 to 205 are roadside units, and the execution subjects of S206 to S208 are vehicles. After S208, The method may further include: the vehicle generates a driving strategy for adjusting or maintaining the driving information of the vehicle based on the driving information of the vehicle and the fused three-dimensional perception information of the target object.
值得说明地是，在本实施例中，通过基于行驶信息、目标对象的融合后的三维感知信息，生成用于调整或维持车辆的行驶信息的行驶策略，可以提高车辆行驶的安全性和可靠性的技术效果。It should be noted that, in this embodiment, by generating a driving strategy for adjusting or maintaining the driving information of the vehicle based on the fused three-dimensional perception information of the driving information and the target object, the safety and reliability of the driving of the vehicle can be improved. technical effect.
图5是根据本申请第三实施例的示意图，如图5所示，鱼眼相机的外参标定装置500，包括：FIG. 5 is a schematic diagram according to a third embodiment of the present application. As shown in FIG. 5 , an external
第一处理单元501，用于根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像；其中，目标焦距为所述鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距。The
第一确定单元502，用于根据去畸变图像确定鱼眼相机的外参。The first determining
图6是根据本申请第四实施例的示意图，如图6所示，鱼眼相机的外参标定装置600，包括：FIG. 6 is a schematic diagram according to a fourth embodiment of the present application. As shown in FIG. 6 , an external
第二确定单元601，根据鱼眼相机在每一焦距下的视场角，确定鱼眼相机在每一焦距下的图像采集区域，并根据枪机在每一焦距下视场角，确定枪机在每一焦距下的图像采集区域。The second determining
第三确定单元602，用于根据鱼眼相机在每一焦距下的图像采集区域、以及枪机在每一焦距下的图像采集区域，确定预设图像采集重合区域。The third determining
结合图6可知，在一些实施例中，第二确定单元601包括：6 , in some embodiments, the second determining
第一确定子单元6011，用于根据鱼眼相机在每一焦距下的视场角、以及获取到鱼眼相机所位于的监控杆的高度信息，确定鱼眼相机在每一焦距下的图像采集区域。The
第二确定子单元6012，用于根据枪机在每一焦距下的视场角、以及获取到监控杆的高度信息，确定枪机在每一焦距下的图像采集区域。The
第一处理单元603，用于根据目标焦距，对鱼眼相机所采集的原始图像进行去畸变处理，得到去畸变图像；其中，目标焦距为所述鱼眼相机与枪机之间的具有预设图像采集重合区域时所对应的焦距。The
结合图6可知，在一些实施例中，第一处理单元603包括：6 , in some embodiments, the
第二获取子单元6031，用于获取鱼眼相机基于目标焦距采集的标定板图像。The
投影子单元6032，用于将标定板图像投影到球面上，得到球面图像。The
处理子单元6033，用于对球面图像进行去畸变处理，得到去畸变图像。The
第一确定单元604，用于根据去畸变图像确定鱼眼相机的外参。The first determining
结合图6可知，在一些实施例中，去畸变图像为二维图像；第一确定单元604包括：6 , in some embodiments, the undistorted image is a two-dimensional image; the first determining
第一获取子单元6041，用于获取去畸变图像中的至少一个像素点，在去畸变图像中的二维坐标、以及在世界坐标系下的三维坐标。The first obtaining
第三确定子单元6042，用于根据二维坐标和三维坐标，确定鱼眼相机的外参。The
在一些实施例中，鱼眼相机的外参包括：鱼眼相机在世界坐标系下的旋转矩阵和平移向量。In some embodiments, the extrinsic parameters of the fisheye camera include: a rotation matrix and a translation vector of the fisheye camera in the world coordinate system.
在一些实施例中，像素点的数量为多个，且多个像素点为去畸变图像中均匀分布的多个像素点。In some embodiments, the number of pixel points is multiple, and the multiple pixel points are multiple pixel points evenly distributed in the dedistorted image.
第二处理单元605，用于根据鱼眼相机的外参，将鱼眼相机所采集的图像中目标对象的二维感知信息进行处理，得到目标对象的第一感知信息，其中，第一感知信息为三维感知信息。The
第三处理单元606，用于根据预设的枪机的外参，将枪机所采集的图像中目标对象的二维感知信息进行处理，得到目标对象的第二感知信息，其中，第二感知信息为三维感知信息。The
融合单元607，用于对第一感知信息和第二感知信息进行融合处理，得到目标对象的融合后的三维感知信息。The
在一些实施例中，第一感知信息和第二感知信息均为目标对象的世界坐标信息；融合单元607用于，对第一感知信息和第二感知信息进行加权处理，得到目标对象的融合后的三维感知信息。In some embodiments, the first perception information and the second perception information are both world coordinate information of the target object; the
获取单元608，用于获取车辆的行驶信息。The obtaining
生成单元609，用于根据行驶信息、以及目标对象的融合后的三维感知信息，生成用于调整或维持车辆的行驶信息的行驶策略。The generating
根据本申请的实施例，本申请还提供了一种电子设备和一种可读存储介质。According to the embodiments of the present application, the present application further provides an electronic device and a readable storage medium.
根据本申请的实施例，本申请还提供了一种计算机程序产品，程序产品包括：计算机程序，计算机程序存储在可读存储介质中，电子设备的至少一个处理器可以从可读存储介质读取计算机程序，至少一个处理器执行计算机程序使得电子设备执行上述任一实施例提供的方案。According to an embodiment of the present application, the present application further provides a computer program product, the program product includes: a computer program, where the computer program is stored in a readable storage medium, and at least one processor of the electronic device can be read from the readable storage medium A computer program, where at least one processor executes the computer program to cause the electronic device to execute the solution provided by any of the foregoing embodiments.
图7示出了可以用来实施本申请的实施例的示例电子设备700的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。FIG. 7 shows a schematic block diagram of an example
如图7所示，电子设备700包括计算单元701，其可以根据存储在只读存储器(ROM)702中的计算机程序或者从存储单元708加载到随机访问存储器(RAM)703中的计算机程序，来执行各种适当的动作和处理。在RAM 703中，还可存储设备700操作所需的各种程序和数据。计算单元701、ROM 702以及RAM 703通过总线704彼此相连。输入/输出(I/O)接口705也连接至总线704。As shown in FIG. 7 , the
设备700中的多个部件连接至I/O接口705，包括：输入单元706，例如键盘、鼠标等；输出单元707，例如各种类型的显示器、扬声器等；存储单元708，例如磁盘、光盘等；以及通信单元709，例如网卡、调制解调器、无线通信收发机等。通信单元709允许设备700通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Various components in the
计算单元701可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元701的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元701执行上文所描述的各个方法和处理，例如鱼眼相机的外参标定方法。例如，在一些实施例中，鱼眼相机的外参标定方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元708。在一些实施例中，计算机程序的部分或者全部可以经由ROM 702和/或通信单元709而被载入和/或安装到设备700上。当计算机程序加载到RAM 7703并由计算单元701执行时，可以执行上文描述的鱼眼相机的外参标定方法的一个或多个步骤。备选地，在其他实施例中，计算单元701可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行鱼眼相机的外参标定方法。
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、负载可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与VPS服务("Virtual Private Server"，或简称"VPS")中，存在的管理难度大，业务扩展性弱的缺陷。服务器也可以为分布式系统的服务器，或者是结合了区块链的服务器。A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the traditional physical host and VPS service ("Virtual Private Server", or "VPS" for short). , there are the defects of difficult management and weak business expansion. The server can also be a server of a distributed system, or a server combined with a blockchain.
根据本申请实施例的另一个方面，本申请实施例还提供了一种路侧设备，包括如上实施例所述的电子设备。According to another aspect of the embodiments of the present application, the embodiments of the present application further provide a roadside device, including the electronic device described in the above embodiments.
根据本申请实施例的另一个方面，本申请实施例还提供了一种云控平台，包括如上实施例所述的电子设备。According to another aspect of the embodiments of the present application, the embodiments of the present application further provide a cloud control platform, including the electronic device described in the above embodiments.
根据本申请实施例的另一个方面，本申请实施例还提供了一种鱼眼相机的外参标定系统，包括：鱼眼相机、枪机、以及如上实施例所述的鱼眼相机的外参标定装置；其中，According to another aspect of the embodiments of the present application, the embodiments of the present application further provide an external parameter calibration system for a fisheye camera, including: a fisheye camera, a bolt, and the external parameter calibration of the fisheye camera described in the above embodiments device; wherein,
鱼眼相机和枪机用于确定目标焦距。A fisheye camera and bolt are used to determine the focal length of the target.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发申请中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present application can be performed in parallel, sequentially or in different orders, and as long as the desired results of the technical solutions disclosed in the present application can be achieved, no limitation is imposed herein.
上述具体实施方式，并不构成对本申请保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本申请的精神和原则之内所作的修改、等同替换和改进等，均应包含在本申请保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.
Claims (26)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110254664.0ACN112967345B (en) | 2021-03-09 | 2021-03-09 | External parameter calibration method, device and system of fish-eye camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110254664.0ACN112967345B (en) | 2021-03-09 | 2021-03-09 | External parameter calibration method, device and system of fish-eye camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112967345Atrue CN112967345A (en) | 2021-06-15 |
| CN112967345B CN112967345B (en) | 2024-04-26 |
Family
ID=76277381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110254664.0AActiveCN112967345B (en) | 2021-03-09 | 2021-03-09 | External parameter calibration method, device and system of fish-eye camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112967345B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113256540A (en)* | 2021-07-14 | 2021-08-13 | 智道网联科技(北京)有限公司 | Image distortion removal method and apparatus, electronic device, and computer-readable storage medium |
| CN113344906A (en)* | 2021-06-29 | 2021-09-03 | 阿波罗智联(北京)科技有限公司 | Vehicle-road cooperative camera evaluation method and device, road side equipment and cloud control platform |
| CN114742726A (en)* | 2022-03-31 | 2022-07-12 | 阿波罗智联(北京)科技有限公司 | Blind area detection method and device, electronic equipment and storage medium |
| CN115376313A (en)* | 2022-07-26 | 2022-11-22 | 四川智慧高速科技有限公司 | A Realization Method of Image Fusion and Distortion Correction Based on Surveillance Camera Group |
| CN115423804A (en)* | 2022-11-02 | 2022-12-02 | 依未科技(北京)有限公司 | Image calibration method and device and image processing method |
| CN116681774A (en)* | 2023-05-23 | 2023-09-01 | 深圳市美力高集团有限公司 | Method and device for determining internal parameters of fish-eye module |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106683139A (en)* | 2017-02-20 | 2017-05-17 | 南京航空航天大学 | Fisheye-camera calibration system based on genetic algorithm and image distortion correction method thereof |
| CN106846415A (en)* | 2017-01-24 | 2017-06-13 | 长沙全度影像科技有限公司 | A kind of multichannel fisheye camera binocular calibration device and method |
| CN107248178A (en)* | 2017-06-08 | 2017-10-13 | 上海赫千电子科技有限公司 | A kind of fisheye camera scaling method based on distortion parameter |
| US20190043219A1 (en)* | 2018-07-02 | 2019-02-07 | Intel Corporation | Dual Model for Fisheye Lens Distortion and an Algorithm for Calibrating Model Parameters |
| US20200267310A1 (en)* | 2020-05-07 | 2020-08-20 | Intel Corporation | Single image ultra-wide fisheye camera calibration via deep learning |
- 2021
- 2021-03-09CNCN202110254664.0Apatent/CN112967345B/enactiveActive
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106846415A (en)* | 2017-01-24 | 2017-06-13 | 长沙全度影像科技有限公司 | A kind of multichannel fisheye camera binocular calibration device and method |
| CN106683139A (en)* | 2017-02-20 | 2017-05-17 | 南京航空航天大学 | Fisheye-camera calibration system based on genetic algorithm and image distortion correction method thereof |
| CN107248178A (en)* | 2017-06-08 | 2017-10-13 | 上海赫千电子科技有限公司 | A kind of fisheye camera scaling method based on distortion parameter |
| US20190043219A1 (en)* | 2018-07-02 | 2019-02-07 | Intel Corporation | Dual Model for Fisheye Lens Distortion and an Algorithm for Calibrating Model Parameters |
| US20200267310A1 (en)* | 2020-05-07 | 2020-08-20 | Intel Corporation | Single image ultra-wide fisheye camera calibration via deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 杨宇;赵成星;张晓玲;: "鱼眼相机的视觉标定及畸变校正", 激光杂志, no. 09, pages 26 - 29* |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113344906A (en)* | 2021-06-29 | 2021-09-03 | 阿波罗智联(北京)科技有限公司 | Vehicle-road cooperative camera evaluation method and device, road side equipment and cloud control platform |
| CN113344906B (en)* | 2021-06-29 | 2024-04-23 | 阿波罗智联(北京)科技有限公司 | Camera evaluation method and device in vehicle-road cooperation, road side equipment and cloud control platform |
| CN113256540A (en)* | 2021-07-14 | 2021-08-13 | 智道网联科技(北京)有限公司 | Image distortion removal method and apparatus, electronic device, and computer-readable storage medium |
| CN114742726A (en)* | 2022-03-31 | 2022-07-12 | 阿波罗智联(北京)科技有限公司 | Blind area detection method and device, electronic equipment and storage medium |
| CN115376313A (en)* | 2022-07-26 | 2022-11-22 | 四川智慧高速科技有限公司 | A Realization Method of Image Fusion and Distortion Correction Based on Surveillance Camera Group |
| CN115376313B (en)* | 2022-07-26 | 2024-07-16 | 四川智慧高速科技有限公司 | Image fusion and distortion correction implementation method based on monitoring camera group |
| CN115423804A (en)* | 2022-11-02 | 2022-12-02 | 依未科技(北京)有限公司 | Image calibration method and device and image processing method |
| CN115423804B (en)* | 2022-11-02 | 2023-03-21 | 依未科技(北京)有限公司 | Image calibration method and device and image processing method |
| CN116681774A (en)* | 2023-05-23 | 2023-09-01 | 深圳市美力高集团有限公司 | Method and device for determining internal parameters of fish-eye module |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112967345B (en) | 2024-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112967345A (en) | External parameter calibration method, device and system of fisheye camera | |
| CN112686877B (en) | Construction and measurement method and system of 3D house damage model based on binocular camera | |
| CN112598750B (en) | Road side camera calibration method and device, electronic equipment and storage medium | |
| CN113989450A (en) | Image processing method, apparatus, electronic device and medium | |
| CN112560769B (en) | Method for detecting obstacle, electronic device, road side device and cloud control platform | |
| CN112967344B (en) | Method, device, storage medium and program product for calibrating camera external parameters | |
| CN112509126A (en) | Method, device, equipment and storage medium for detecting three-dimensional object | |
| CN113610702B (en) | Picture construction method and device, electronic equipment and storage medium | |
| CN114332487A (en) | Image-based accumulated water early warning method, device, equipment, storage medium and product | |
| CN113344906A (en) | Vehicle-road cooperative camera evaluation method and device, road side equipment and cloud control platform | |
| CN116129422A (en) | Monocular 3D target detection method, monocular 3D target detection device, electronic equipment and storage medium | |
| US11893884B2 (en) | Method for acquiring three-dimensional perception information based on external parameters of roadside camera, and roadside device | |
| CN115773759A (en) | Indoor positioning method, device, equipment and storage medium for autonomous mobile robot | |
| CN115346020A (en) | Point cloud processing method, obstacle avoidance method, device, robot and storage medium | |
| JP2018097588A (en) | Three-dimensional space specifying device, method, and program | |
| CN113313764B (en) | Positioning method, positioning device, electronic equipment and storage medium | |
| CN116091709B (en) | Three-dimensional reconstruction method, device, electronic device and storage medium of building | |
| CN114742726B (en) | Blind spot detection method, device, electronic device and storage medium | |
| CN113112551B (en) | Camera parameter determining method and device, road side equipment and cloud control platform | |
| CN113470103B (en) | Method and device for determining camera acting distance in vehicle-road cooperation and road side equipment | |
| CN116883652A (en) | A drivable area segmentation method, device, readable storage medium and robot | |
| CN115908551A (en) | Vehicle distance measuring method and device, electronic equipment and storage medium | |
| WO2023070441A1 (en) | Movable platform positioning method and apparatus | |
| CN115063490A (en) | Vehicle camera external parameter calibration method and device, electronic equipment and storage medium | |
| CN113312979B (en) | Image processing method and device, electronic equipment, road side equipment and cloud control platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20211022 Address after:100176 101, floor 1, building 1, yard 7, Ruihe West 2nd Road, Beijing Economic and Technological Development Zone, Daxing District, Beijing Applicant after:Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd. Address before:2 / F, baidu building, 10 Shangdi 10th Street, Haidian District, Beijing 100085 Applicant before:BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. | |
| GR01 | Patent grant | ||
| GR01 | Patent grant |