技术领域technical field
本发明涉及图像处理技术领域，尤其涉及一种基于鱼眼相机的环视图像合成方法和装置。The present invention relates to the technical field of image processing, in particular to a fisheye camera-based surround view image synthesis method and device.
背景技术Background technique
在汽车行驶尤其在倒车过程中，为了行驶安全，能够清楚地显示车辆周围的实景图像信息至关重要。When the car is running, especially in the process of reversing, for driving safety, it is very important to be able to clearly display the real-world image information around the vehicle.
现有技术中，车辆上设置的相机大多是独立工作，这些相机采集的图像通常不进行拼接就返回给用户，用户体验不好；另外，虽然一些高端车型具备环视功能，主要是在车辆上设置有广角镜头，但是由于广角镜头视角狭窄，并且返回的一般都是平面视图，用户体验较差。In the existing technology, most of the cameras installed on the vehicle work independently, and the images collected by these cameras are usually returned to the user without splicing, and the user experience is not good; There is a wide-angle lens, but because the wide-angle lens has a narrow viewing angle and generally returns a flat view, the user experience is poor.
发明内容Contents of the invention
为解决现有技术中存在的问题，本发明提供一种基于鱼眼相机的环视图像合成方法和装置，以提供给用户车辆四周实景的环视图像，提高了用户体验，同时消除了车辆行驶时的盲区，提升了行驶安全性。In order to solve the problems existing in the prior art, the present invention provides a method and device for synthesizing surround-view images based on a fisheye camera to provide users with surround-view images of the real scene around the vehicle, which improves the user experience and eliminates the need for ambiguity when the vehicle is driving. Blind spots improve driving safety.
第一方面，本发明实施例提供一种基于鱼眼相机的环视图像合成方法，包括：In the first aspect, an embodiment of the present invention provides a fisheye camera-based surround view image synthesis method, including:
获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像；Obtaining multiple fisheye images collected by multiple fisheye cameras installed on the vehicle;
根据预先获取的所述多个鱼眼相机之间的对应关系，对所述多个鱼眼图像进行拼接，获取所述多个鱼眼图像对应的平面环视图像，所述对应关系用于表示所述多个鱼眼相机采集的鱼眼图像之间的位置关系；According to the pre-acquired correspondence between the plurality of fisheye cameras, the plurality of fisheye images are spliced to obtain a plane look-around image corresponding to the plurality of fisheye images, and the correspondence is used to represent the plurality of fisheye images. Describe the positional relationship between the fisheye images collected by multiple fisheye cameras;
对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取所述平面环视图像对应的立体环视图像。performing spatial alignment processing on the same spatial objects in the plurality of fisheye images, and acquiring a stereoscopic surround-view image corresponding to the planar surround-view image.
可选的，所述方法还包括：Optionally, the method also includes:
对所述平面环视图像进行优化，获取优化后的平面环视图像。The plane look-around image is optimized to obtain the optimized plane look-around image.
可选的，所述方法还包括：Optionally, the method also includes:
获取所述多个鱼眼相机采集的多个标定板图像，所述鱼眼图像包括所述标定板图像；Obtaining a plurality of calibration plate images collected by the plurality of fisheye cameras, the fisheye images including the calibration plate images;
根据所述多个标定板图像，获取所述多个鱼眼相机的对应关系。According to the multiple calibration plate images, the corresponding relationship of the multiple fisheye cameras is acquired.
可选的，所述根据所述多个标定板图像，获取所述多个鱼眼相机的对应关系，包括：Optionally, the obtaining the corresponding relationship of the multiple fisheye cameras according to the multiple calibration plate images includes:
根据特征点检测算法，获取所述多个标定板图像的特征点；According to the feature point detection algorithm, the feature points of the multiple calibration plate images are obtained;
根据所述多个标定板图像的特征点，获取所述特征点的位置坐标；Acquiring the position coordinates of the feature points according to the feature points of the plurality of calibration plate images;
根据所述特征点的位置坐标，获取所述多个鱼眼相机的对应关系。According to the position coordinates of the feature points, the corresponding relationship of the plurality of fisheye cameras is obtained.
可选的，所述对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取所述平面环视图像对应的立体环视图像，包括：Optionally, performing spatial alignment processing on the same spatial objects in the plurality of fisheye images, and obtaining a stereoscopic surround-view image corresponding to the planar surround-view image, includes:
对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取第一立体环视图像；Performing spatial alignment processing on the same spatial objects in the plurality of fisheye images to obtain a first stereoscopic surround view image;
对所述第一立体环视图像进行颜色混合处理，获取第二立体环视图像；performing color mixing processing on the first stereoscopic surround view image to obtain a second stereoscopic surround view image;
根据所述第二立体环视图像，以及预先建立的三维模型，获取所述平面环视图像对应的立体环视图像，所述立体环视图像为第二立体环视图像在所述预先建立的三维模型上的投影。Acquire a stereoscopic surround view image corresponding to the planar surround view image according to the second stereoscopic surround view image and a pre-established three-dimensional model, where the stereoscopic surround view image is a projection of the second stereoscopic surround view image on the pre-established three-dimensional model .
可选的，所述方法还包括：Optionally, the method also includes:
显示所述平面环视图像对应的立体环视图像。A stereoscopic surround-view image corresponding to the planar surround-view image is displayed.
可选的，所述鱼眼相机的数量为四个。Optionally, the number of the fisheye cameras is four.
第二方面，本发明实施例提供一种基于鱼眼相机的环视图像合成装置，包括：In a second aspect, an embodiment of the present invention provides a fisheye camera-based surround view image synthesis device, including:
获取模块，用于获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像；An acquisition module, configured to acquire a plurality of fisheye images collected by a plurality of fisheye cameras arranged on the vehicle;
处理模块，用于根据预先获取的所述多个鱼眼相机之间的对应关系，对所述多个鱼眼图像进行拼接，获取所述多个鱼眼图像对应的平面环视图像，所述对应关系用于表示所述多个鱼眼相机采集的鱼眼图像之间的位置关系；A processing module, configured to splice the plurality of fisheye images according to the pre-acquired correspondence between the plurality of fisheye cameras, and obtain a plane look-around image corresponding to the plurality of fisheye images, the correspondence The relationship is used to represent the positional relationship between the fisheye images collected by the multiple fisheye cameras;
对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取所述平面环视图像对应的立体环视图像。performing spatial alignment processing on the same spatial objects in the plurality of fisheye images, and acquiring a stereoscopic surround-view image corresponding to the planar surround-view image.
可选的，所述处理模块，还用于对所述平面环视图像进行优化，获取优化后的平面环视图像。Optionally, the processing module is further configured to optimize the plane surround-view image, and obtain the optimized plane surround-view image.
可选的，所述获取模块，还用于获取所述多个鱼眼相机采集的多个标定板图像，所述鱼眼图像包括所述标定板图像；Optionally, the acquiring module is further configured to acquire multiple calibration plate images collected by the multiple fisheye cameras, the fisheye images including the calibration plate images;
根据所述多个标定板图像，获取所述多个鱼眼相机的对应关系。According to the multiple calibration plate images, the corresponding relationship of the multiple fisheye cameras is acquired.
可选的，所述获取模块，还用于根据特征点检测算法，获取所述多个标定板图像的特征点；Optionally, the acquiring module is further configured to acquire feature points of the plurality of calibration plate images according to a feature point detection algorithm;
根据所述多个标定板图像的特征点，获取所述特征点的位置坐标；Acquiring the position coordinates of the feature points according to the feature points of the plurality of calibration plate images;
根据所述特征点的位置坐标，获取所述多个鱼眼相机的对应关系。According to the position coordinates of the feature points, the corresponding relationship of the plurality of fisheye cameras is acquired.
可选的，所述处理模块，还用于对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取第一立体环视图像；Optionally, the processing module is further configured to perform spatial alignment processing on the same spatial objects in the plurality of fisheye images to obtain a first stereoscopic surround view image;
对所述第一立体环视图像进行颜色混合处理，获取第二立体环视图像；performing color mixing processing on the first stereoscopic surround view image to obtain a second stereoscopic surround view image;
所述获取模块，还用于根据所述第二立体环视图像，以及预先建立的三维模型，获取所述平面环视图像对应的立体环视图像，所述立体环视图像为第二立体环视图像在所述预先建立的三维模型上的投影。The acquiring module is further configured to acquire a stereoscopic surround view image corresponding to the planar surround view image according to the second stereoscopic surround view image and a pre-established three-dimensional model, and the stereoscopic surround view image is the second stereoscopic surround view image in the Projection on pre-built 3D models.
可选的，所述装置还包括：Optionally, the device also includes:
显示模块，用于显示所述平面环视图像对应的立体环视图像。A display module, configured to display a stereoscopic surround-view image corresponding to the planar surround-view image.
可选的，所述鱼眼相机的数量为四个。Optionally, the number of the fisheye cameras is four.
第三方面，本发明实施例提供一种车载设备，包括：In a third aspect, an embodiment of the present invention provides a vehicle-mounted device, including:
处理器；processor;
存储器；以及storage; and
计算机程序；Computer program;
其中，所述计算机程序被存储在所述存储器中，并且被配置为由所述处理器执行，所述计算机程序包括用于执行第一方面所述的方法的指令。Wherein, the computer program is stored in the memory and is configured to be executed by the processor, the computer program including instructions for performing the method described in the first aspect.
第四方面，本发明实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序使得服务器执行第一方面所述的方法。In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program causes a server to execute the method described in the first aspect.
本发明提供的基于鱼眼相机的环视图像合成方法和装置，该方法包括：获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像，根据预先获取的多个鱼眼相机之间的对应关系，对多个鱼眼图像进行拼接，获取多个鱼眼图像对应的平面环视图像，对应关系用于表示多个鱼眼相机采集的鱼眼图像之间的位置关系，对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取平面环视图像对应的立体环视图像。通过该方法，能够获得车辆四周实景的环视图像，提高了用户体验，同时消除了车辆行驶时的盲区，提升了行驶安全性。The fisheye camera-based surround view image synthesis method and device provided by the present invention, the method includes: acquiring a plurality of fisheye images collected by a plurality of fisheye cameras installed on the vehicle, and according to the pre-acquired Correspondence relationship, multiple fisheye images are spliced, and plane surround images corresponding to multiple fisheye images are obtained. The correspondence relationship is used to represent the positional relationship between fisheye images collected by multiple fisheye cameras. For multiple fisheye The same spatial objects in the eye image are spatially aligned to obtain the stereoscopic surround-view image corresponding to the planar surround-view image. By using this method, a surround-view image of the real scene around the vehicle can be obtained, which improves user experience, eliminates blind spots when the vehicle is driving, and improves driving safety.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.
图1为本发明实施例提供的基于鱼眼相机的环视图像合成方法的流程示意图一；Fig. 1 is a schematic flow chart 1 of a fisheye camera-based surround view image synthesis method provided by an embodiment of the present invention;
图2为本发明实施例提供的鱼眼图像的示意图；FIG. 2 is a schematic diagram of a fisheye image provided by an embodiment of the present invention;
图3为本发明实施例提供的俯视平面环视图像的示意图；FIG. 3 is a schematic diagram of a top-view plane surround-view image provided by an embodiment of the present invention;
图4为本发明实施例提供的基于鱼眼相机的环视图像合成方法的流程示意图二；FIG. 4 is a second schematic flow diagram of a fisheye camera-based surround-view image synthesis method provided by an embodiment of the present invention;
图5为本发明实施例提供的标定板的示意图；5 is a schematic diagram of a calibration plate provided by an embodiment of the present invention;
图6为本发明实施例提供的基于鱼眼相机的环视图像合成方法的流程示意图三；FIG. 6 is a schematic flow diagram three of a fisheye camera-based surround-view image synthesis method provided by an embodiment of the present invention;
图7为本发明实施例提供的拼接鱼眼图像的示意图；FIG. 7 is a schematic diagram of a stitched fisheye image provided by an embodiment of the present invention;
图8为本发明实施例提供的立体环视图像的示意图；FIG. 8 is a schematic diagram of a stereoscopic surround view image provided by an embodiment of the present invention;
图9为本发明实施例提供的基于鱼眼相机的环视图像合成装置的结构示意图；9 is a schematic structural diagram of a fisheye camera-based surround-view image synthesis device provided by an embodiment of the present invention;
图10为本发明实施例提供的车载设备的结构示意图。FIG. 10 is a schematic structural diagram of a vehicle-mounted device provided by an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
本发明实施例提供的鱼眼相机的环视图像合成方法可以应用于需要查看车辆四周环境的应用场景中，如车辆行驶或停车过程中。车辆行驶或停车过程，为保证安全，司机需要查看车辆四周是否有障碍物，目前大多数车辆上设置的相机大多是独立工作，这些相机采集的图像通常不进行拼接就返回给用户，给予用户不自然的实景体验，用户体验不好；另外，虽然一些高端车型具备环视功能，主要是在车辆上设置有广角镜头，但是由于广角镜头视角狭窄比较固定，并且返回的一般都是平面视图，并不能提供给用户三维立体环视图像，以供用户手动触摸旋转三维力图环视图像至任一视角，并且返回图像的速度较慢、价格昂贵，导致用户体验差。The surround-view image synthesis method of the fisheye camera provided by the embodiment of the present invention can be applied to application scenarios that need to view the surrounding environment of the vehicle, such as when the vehicle is driving or parking. During the driving or parking process of the vehicle, in order to ensure safety, the driver needs to check whether there are obstacles around the vehicle. At present, most of the cameras installed on the vehicle work independently. The images collected by these cameras are usually returned to the user without stitching, giving the user no inconvenience. Natural real-scene experience, user experience is not good; in addition, although some high-end models have a surround view function, mainly because of the wide-angle lens installed on the vehicle, but because the wide-angle lens has a narrow and fixed viewing angle, and the return is generally a flat view, it cannot be provided to The user's three-dimensional surround view image is provided for the user to manually touch and rotate the three-dimensional view image to any angle of view, and the speed of returning the image is slow and expensive, resulting in poor user experience.
本发明实施例考虑到上述问题，提出一种鱼眼相机的环视图像合成方法，该方法中车载设备获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像，根据预先获取的多个鱼眼相机之间的对应关系，对多个鱼眼图像进行拼接，获取多个鱼眼图像对应的平面环视图像，对应关系用于表示多个鱼眼相机采集的鱼眼图像之间的位置关系，对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取平面环视图像对应的立体环视图像。通过该方法，能够获得车辆四周实景的环视图像，提高了用户体验，同时消除了车辆行驶时的盲区，提升了行驶安全性。Considering the above problems, the embodiment of the present invention proposes a fisheye camera surround view image synthesis method. In this method, the vehicle-mounted device acquires multiple fisheye images collected by multiple fisheye cameras installed on the vehicle, and according to multiple pre-acquired Correspondence between fisheye cameras, splicing multiple fisheye images, and obtaining plane surround images corresponding to multiple fisheye images, the correspondence is used to represent the positional relationship between fisheye images collected by multiple fisheye cameras , perform spatial alignment processing on the same spatial objects in multiple fisheye images, and obtain stereoscopic surround-view images corresponding to planar surround-view images. By using this method, a surround-view image of the real scene around the vehicle can be obtained, which improves user experience, eliminates blind spots when the vehicle is driving, and improves driving safety.
下面以具体的实施例对本发明的技术方案进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例不再赘述。The technical solution of the present invention will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
图1为本发明实施例提供的基于鱼眼相机的环视图像合成方法的流程示意图一。本发明实施例提供了一种鱼眼相机的环视图像合成方法，该方法可以由任意执行鱼眼相机的环视图像合成方法的装置来执行，该装置可以通过软件和/或硬件实现。本实施例中，该装置可以集成在车载设备中。如图1所示，本发明实施例提供的鱼眼相机的环视图像合成方法包括如下步骤：FIG. 1 is a first schematic flowchart of a fisheye camera-based surround-view image synthesis method provided by an embodiment of the present invention. An embodiment of the present invention provides a fisheye camera surround view image synthesis method, which can be executed by any device that executes the fisheye camera surround view image synthesis method, and the device can be implemented by software and/or hardware. In this embodiment, the device can be integrated in the vehicle equipment. As shown in Figure 1, the surround view image synthesis method of the fisheye camera provided by the embodiment of the present invention includes the following steps:
S101、获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像。S101. Acquire multiple fisheye images collected by multiple fisheye cameras installed on the vehicle.
车辆上可以设置有多个鱼眼相机，可以分别设置在车辆的四周，用于采集车辆不同方向的图像。本方案对于鱼眼相机的数量没有特别限制，只要鱼眼相机采集的图像能合成，并完整显示车辆四周的环境即可。Multiple fisheye cameras can be installed on the vehicle, which can be respectively installed around the vehicle to collect images from different directions of the vehicle. This solution has no special limitation on the number of fisheye cameras, as long as the images collected by the fisheye cameras can be synthesized and fully display the environment around the vehicle.
可选的，鱼眼相机的数量为4个，可以分别设置在车辆的车头、车尾、左侧和右侧。Optionally, the number of fisheye cameras is 4, which can be respectively arranged at the front, rear, left and right sides of the vehicle.
S102、根据预先获取的多个鱼眼相机之间的对应关系，对多个鱼眼图像进行拼接，获取多个鱼眼图像对应的平面环视图像。S102. According to the pre-acquired correspondence between the multiple fish-eye cameras, splice the multiple fish-eye images, and acquire a planar view image corresponding to the multiple fish-eye images.
该对应关系用于表示多个鱼眼相机采集的鱼眼图像之间的位置关系，若鱼眼相机的位置没有发生变化，则该对应关系就是固定的。The corresponding relationship is used to represent the positional relationship between the fisheye images collected by multiple fisheye cameras, and if the positions of the fisheye cameras do not change, the corresponding relationship is fixed.
可选的，该对应关系可以为几何转换关系。Optionally, the corresponding relationship may be a geometric conversion relationship.
以车辆上设置有4个鱼眼相机为例，结合图2、图3进行说明，图2为本发明实施例提供的鱼眼图像的示意图，图3为本发明实施例提供的俯视平面环视图像的示意图。Taking four fisheye cameras installed on the vehicle as an example, it will be described in conjunction with Fig. 2 and Fig. 3. Fig. 2 is a schematic diagram of a fisheye image provided by an embodiment of the present invention, and Fig. 3 is a top view plane surround view image provided by an embodiment of the present invention schematic diagram.
由图2可知，车辆上设置的4个鱼眼相机分别设置在车辆的车头、车尾、左侧和右侧，由于鱼眼相机的最大拍摄范围为180°，则鱼眼相机1采集的鱼眼图像和鱼眼相机2采集的鱼眼图像有重叠区域，鱼眼相机2和鱼眼相机3采集的鱼眼图像有重叠区域，鱼眼相机3和鱼眼相机4采集的鱼眼图像有重叠区域，鱼眼相机4和鱼眼相机1采集的鱼眼图像有重叠区域，则可以根据四个鱼眼相机采集的四个鱼眼图像之间的位置关系，对各个鱼眼相机采集的各个鱼眼图像进行拼接，获得这四张鱼眼图像对应的平面环视图像。It can be seen from Figure 2 that the four fisheye cameras installed on the vehicle are respectively installed at the front, rear, left and right sides of the vehicle. Since the maximum shooting range of the fisheye camera is 180°, the fish captured by fisheye camera 1 The eye image and the fisheye image collected by fisheye camera 2 have overlapping areas, the fisheye images collected by fisheye camera 2 and fisheye camera 3 have overlapping areas, and the fisheye images collected by fisheye camera 3 and fisheye camera 4 have overlapping areas Area, the fisheye images collected by fisheye camera 4 and fisheye camera 1 have overlapping areas, then according to the positional relationship between the four fisheye images collected by four fisheye cameras, each fisheye image collected by each fisheye camera The eye images are stitched together to obtain the plane surround view images corresponding to the four fisheye images.
需要说明的是，这里的拼接就是将有重叠区域的任意两张鱼眼图像进行拼接，当重叠区域完全重合，拼接完成，然后进行第二轮拼接，直到四张鱼眼图像拼接在一起，得到一个完整的平面图像。It should be noted that the stitching here is to stitch any two fisheye images with overlapping areas. When the overlapping areas are completely overlapped, the stitching is completed, and then the second round of stitching is performed until the four fisheye images are stitched together to obtain A complete flat image.
进一步，预先建立二维模型，过滤不需要的信息展示给用户的目标，比如：天空中的物体，结合该二维模型，可以得到图3所示的俯视平面环视图像。Further, a 2D model is established in advance, and unnecessary information is filtered to show the target to the user, such as objects in the sky. Combining with the 2D model, the top view plane surround view image shown in FIG. 3 can be obtained.
S103、对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取平面环视图像对应的立体环视图像。S103. Perform spatial alignment processing on the same spatial object in multiple fisheye images, and acquire a stereoscopic surround-view image corresponding to the planar surround-view image.
在立体环视图像中俯视的平面环视图像中地面上的物体基本对齐(拼接完成)，而空间中的物体在对齐时，还存在偏差，故需要对空间存在对齐偏差的集合物体进行空间对齐处理。Objects on the ground are basically aligned (stitching is complete) in the plane surround view image viewed from above in the stereo surround view image, but there are still deviations in the alignment of objects in space. Therefore, it is necessary to perform spatial alignment processing on the collective objects with alignment deviations in space.
在一种可能的实现方式中，可以先对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取第一立体环视图像，然后对第一立体环视图像进行颜色混合处理，获取第二立体环视图像，再根据第二立体环视图像，以及预先建立的三维模型，获取平面环视图像对应的立体环视图像，立体环视图像为第二立体环视图像在预先建立的三维模型上的投影。In a possible implementation, spatial alignment processing can be performed on the same spatial object in multiple fisheye images first to obtain the first stereoscopic surround view image, and then color mixing processing is performed on the first stereoscopic surround view image to obtain the second stereoscopic surround view image. The surround view image, and then according to the second stereo surround view image and the pre-established 3D model, obtain the stereo surround view image corresponding to the planar view image, and the stereo surround view image is the projection of the second stereo surround view image on the pre-established 3D model.
具体地，空间兑取处理可以为根据不同鱼眼图像中相同物体几何形状、外观颜色等，计算得到外观函数，该函数可以是自定义的一个反映不同鱼眼图像的变化率，如：颜色深浅变化率等，以颜色举例说明，若该颜色深浅变化率大于预设相同物体颜色深浅变化率阈值，可以对深色区域进行拉伸，或者对浅色区域进行缩小，以使同一个物体在显示上颜色自然。Specifically, the space redemption process can be based on the calculation of the appearance function based on the geometric shape and appearance color of the same object in different fisheye images. This function can be a custom one that reflects the change rate of different fisheye images, such as: color depth Change rate, etc., using color as an example. If the color depth change rate is greater than the preset color depth change rate threshold of the same object, the dark area can be stretched, or the light area can be reduced, so that the same object can be displayed The color is natural.
空间对齐处理后得到第一立体环视图像，由于各个鱼眼相机的入射光线不同，不同鱼眼图像在亮度上存在差异，不同鱼眼图像中的相同物体颜色上的差异可能比较大，因此需要对不同鱼眼图像进行颜色混合以在视觉上自然，具体地，可以计算同一个物体在不同鱼眼图像上的单位亮度值，求取平均值，然后重新给不同鱼眼图像中的该物体重新着色，从而得到第二立体环视图像。After spatial alignment processing, the first stereoscopic surround view image is obtained. Since the incident light of each fisheye camera is different, different fisheye images have differences in brightness, and the color difference of the same object in different fisheye images may be relatively large. Therefore, it is necessary to Color mixing of different fisheye images to be visually natural, specifically, the unit luminance value of the same object on different fisheye images can be calculated, averaged, and then recolored for the object in different fisheye images , so as to obtain the second stereoscopic surround view image.
预先建立三维模型，该三维模型用于过滤不需要展示给用户的物体，该三维模型上的点和立体图像的点有预设对应关系。A three-dimensional model is established in advance, and the three-dimensional model is used to filter objects that do not need to be displayed to the user, and points on the three-dimensional model have a preset corresponding relationship with points on the stereoscopic image.
可选的，该三维模型可以为碗装网络的三维模型，本方案对于三维模型的形状不做限制。Optionally, the three-dimensional model may be a three-dimensional model of the bowl-packed network, and this solution does not limit the shape of the three-dimensional model.
则可以将得到的第二立体环视图像投影到该三维模型上，得到平面环视图像对应的立体环视图像。Then the obtained second stereoscopic surround view image can be projected onto the three-dimensional model to obtain a stereoscopic surround view image corresponding to the planar surround view image.
本实施例提供的鱼眼相机的环视图像合成方法，该方法中车载设备获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像，根据预先获取的多个鱼眼相机之间的对应关系，对多个鱼眼图像进行拼接，获取多个鱼眼图像对应的平面环视图像，对应关系用于表示多个鱼眼相机采集的鱼眼图像之间的位置关系，对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取平面环视图像对应的立体环视图像。通过该方法，能够获得车辆四周实景的环视图像，提高了用户体验，同时消除了车辆行驶时的盲区，提升了行驶安全性。The fisheye camera surround view image synthesis method provided in this embodiment, in this method, the vehicle-mounted device acquires a plurality of fisheye images collected by a plurality of fisheye cameras installed on the vehicle, and according to the pre-acquired Correspondence relationship, splicing multiple fisheye images, and obtaining the plane look-around image corresponding to multiple fisheye images, the correspondence relationship is used to represent the positional relationship between fisheye images collected by multiple fisheye cameras, for multiple fisheye The same spatial objects in the image are spatially aligned to obtain the stereoscopic surround-view image corresponding to the planar surround-view image. By using this method, a surround-view image of the real scene around the vehicle can be obtained, which improves user experience, eliminates blind spots when the vehicle is driving, and improves driving safety.
在上述图1实施例的基础上，图4为本发明实施例提供的基于鱼眼相机的环视图像合成方法的流程示意图二，如图4所示，该方法还包括以下步骤：On the basis of the above-mentioned embodiment in FIG. 1 , FIG. 4 is a second schematic flow diagram of a fisheye camera-based surround-view image synthesis method provided by an embodiment of the present invention. As shown in FIG. 4 , the method further includes the following steps:
S201、获取多个鱼眼相机采集的多个标定板图像。S201. Acquire multiple calibration plate images collected by multiple fisheye cameras.
标定板可以设置在车辆的四周的地面上，鱼眼相机可以设置在车辆的四周，用于采集标定板图像，鱼眼图像包括标定板图像。The calibration board can be arranged on the ground around the vehicle, and the fisheye camera can be arranged around the vehicle to collect images of the calibration board, and the fisheye image includes the image of the calibration board.
可选的，标定板数量为4个，分别设置在车辆的四周。Optionally, the number of calibration plates is 4, which are arranged around the vehicle respectively.
本方案中标定板的类型不限，图5中的标定布为黑白两个正方形(黑在外圈，白在内圈)，还可以为一个正方形(黑白相间)。The type of the calibration plate in this scheme is not limited. The calibration cloth in Figure 5 is two black and white squares (black on the outer circle and white on the inner circle), or one square (black and white).
S202、根据多个标定板图像，获取多个鱼眼相机的对应关系。S202. According to the images of the calibration boards, the corresponding relationships of the multiple fisheye cameras are obtained.
根据鱼眼相机采集的多个标定板图像，可以确定包括同一个标定板在不同标定板图像的位置，可以确定多个标定板图像的位置关系，从而得到多个鱼眼相机的对应关系。According to the multiple calibration plate images collected by the fisheye camera, the position of the same calibration plate in different calibration plate images can be determined, and the positional relationship of the multiple calibration plate images can be determined, thereby obtaining the corresponding relationship of multiple fisheye cameras.
在一种可能的实现方式中，根据特征点检测算法，获取多个标定板图像的特征点，根据多个标定板图像的特征点，获取特征点的位置坐标，根据特征点的位置坐标，获取多个鱼眼相机的对应关系。In a possible implementation, according to the feature point detection algorithm, the feature points of multiple calibration board images are obtained, the position coordinates of the feature points are obtained according to the feature points of the multiple calibration board images, and the position coordinates of the feature points are obtained according to the position coordinates of the feature points. Correspondence of multiple fisheye cameras.
具体地，结合图5对本步骤进行说明，图5为本发明实施例提供的标定板的示意图，如图5所示，标定板设置在车辆的左前、左后、右前、右后，车辆位于最中心，鱼眼相机分别设置在车辆的车头、车尾、左侧和右侧，每个鱼眼相机均可采集至少两个标定板图像。Specifically, this step will be described in conjunction with FIG. 5. FIG. 5 is a schematic diagram of the calibration plate provided by the embodiment of the present invention. As shown in FIG. In the center, the fisheye cameras are respectively set on the front, rear, left and right sides of the vehicle, and each fisheye camera can collect at least two images of the calibration board.
图5中的特征点可以为黑色正方形和白色正方形的顶点，数量为8个。The feature points in FIG. 5 can be the vertices of the black square and the white square, and the number is 8.
然后初始设定左前的标定板的左上角点A为坐标原点，预先测量得到每个标定板相对于坐标原点的位置坐标。Then initially set the upper left corner point A of the front left calibration board as the coordinate origin, and obtain the position coordinates of each calibration board relative to the coordinate origin by pre-measurement.
进一步，根据确定特征点的位置坐标，对不同鱼眼图像进行配对，以两张标定板图像为例，即在标定板图像1中找到标定板图像2中特征点M的匹配点N，由于两个鱼眼图像之间有重叠区域，参考图2的具体过程，再结合每个特征点M、N的位置坐标，获取标定板图像1和标定板图像2的位置关系，同理，还可按照此方法依次完成与标定板图像3、标定板图像4的特征点的匹配，从而得到四个鱼眼相机的对应关系。Further, according to the position coordinates of the determined feature points, different fisheye images are paired, taking two calibration plate images as an example, that is, the matching point N of the feature point M in the calibration plate image 2 is found in the calibration plate image 1, because the two There are overlapping areas between the two fisheye images. Referring to the specific process in Figure 2, combined with the position coordinates of each feature point M and N, the positional relationship between the calibration plate image 1 and the calibration plate image 2 is obtained. Similarly, it can also be obtained according to This method sequentially completes the matching with the feature points of the calibration board image 3 and the calibration board image 4, so as to obtain the corresponding relationship of the four fisheye cameras.
本实施例提供的鱼眼相机的环视图像合成方法，车载设备获取多个鱼眼相机采集的多个标定板图像，鱼眼图像包括所述标定板图像，根据多个标定板图像，获取多个鱼眼相机的对应关系，由此预先获得建立了设置在车辆上的多个鱼眼相机的对应关系，在实际的车辆行驶过程中，可以根据预先获得的对应关系，以及实时获取的鱼眼图像，获得车辆四周实景的环视图像，提高了用户体验。In the fisheye camera surround view image synthesis method provided in this embodiment, the vehicle-mounted device acquires multiple calibration plate images collected by multiple fisheye cameras, the fisheye image includes the calibration plate image, and according to the multiple calibration plate images, multiple calibration plate images are obtained. The corresponding relationship of fisheye cameras, thus pre-obtaining and establishing the corresponding relationship of multiple fisheye cameras installed on the vehicle, in the actual driving process of the vehicle, it can , to obtain a surround-view image of the real scene around the vehicle, which improves the user experience.
在上述图1、图4实施例的基础上，结合图6对本方案的具体实施过程进行说明，图6为本发明实施例提供的基于鱼眼相机的环视图像合成方法的流程示意图三，如图6所示，该方法具体包括以下步骤：On the basis of the above-mentioned embodiments in Fig. 1 and Fig. 4, the specific implementation process of this solution will be described in conjunction with Fig. 6. Fig. 6 is a schematic flow chart of the method for synthesizing surround-view images based on fisheye cameras provided by the embodiment of the present invention, as shown in Fig. 6, the method specifically includes the following steps:
S301、获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像。S301. Acquire multiple fisheye images collected by multiple fisheye cameras installed on the vehicle.
S302、根据预先获取的多个鱼眼相机之间的对应关系，对多个鱼眼图像进行拼接，获取多个鱼眼图像对应的平面环视图像。S302. According to the pre-acquired correspondence between the multiple fish-eye cameras, splice the multiple fish-eye images, and acquire a planar view image corresponding to the multiple fish-eye images.
S303、对平面环视图像进行优化，获取优化后的平面环视图像。S303. Optimizing the plane surround view image, and obtaining the optimized plane surround view image.
由于步骤S202中预先测量得到每个标定板相对于坐标原点的位置坐标，由于各种原因，测量得到的位置坐标存在一定的误差，因此，得到的鱼眼相机之间的对应关系也会存在一定误差，则得到的平面环视图像中可能会存在不同鱼眼图像中的同一物体不对齐的情况，使得拼接后的图像不是那么完美，需要进行优化。Since the position coordinates of each calibration plate relative to the coordinate origin are pre-measured in step S202, due to various reasons, there is a certain error in the measured position coordinates, so there will also be a certain correspondence between the obtained fisheye cameras. If there is an error, the same object in different fisheye images may not be aligned in the obtained planar view image, which makes the spliced image not so perfect and needs to be optimized.
具体地，结合图7进行具体说明，图7为本发明实施例提供的拼接鱼眼图像的示意图，如图7所示，以两张鱼眼图像拼接为例，鱼眼图像1和鱼眼图像2包括重叠区域，按照以上方法，鱼眼图像1和鱼眼图像2初步拼接后，没有完全重合，误差为区域A，当检测到没有完全重合后，将鱼眼图像1和鱼眼图像2的重叠区域分别朝对方方向进行拉伸，以消除误差区域A。Specifically, it will be described in conjunction with FIG. 7. FIG. 7 is a schematic diagram of a spliced fisheye image provided by an embodiment of the present invention. As shown in FIG. 7, taking two fisheye image splicing as an example, fisheye image 1 and fisheye image 2 Including the overlapping area, according to the above method, after the initial splicing of fisheye image 1 and fisheye image 2, there is no complete overlap, and the error is area A. When it is detected that there is no complete overlap, the fisheye image 1 and fisheye image 2 are combined The overlapping regions are stretched toward each other to eliminate the error region A.
S304、对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取平面环视图像对应的立体环视图像。S304. Perform spatial alignment processing on the same spatial object in multiple fisheye images, and acquire a stereoscopic surround-view image corresponding to the planar surround-view image.
步骤S301-S302和步骤S101-S103的实现过程类似，步骤S304和步骤S103的实现过程类似，在此不再赘述。The implementation process of steps S301-S302 is similar to that of steps S101-S103, and the implementation process of step S304 is similar to that of step S103, which will not be repeated here.
S305、显示平面环视图像对应的立体环视图像。S305. Display a stereoscopic surround-view image corresponding to the planar surround-view image.
图8为本发明实施例提供的立体环视图像的示意图，如图8所示，呈现给用户的是一个车辆四周实景的立体环视图像，中间的黑色区域为车辆，具体地，可以在车载设备的显示屏上将得到的立体环视图像进行显示，并且根据用户需求，可以将该立体环视图像旋转至任一视角，使得用户更加直观且真实的感知车辆四周实景信息。Fig. 8 is a schematic diagram of a stereoscopic surround-view image provided by an embodiment of the present invention. As shown in Fig. 8, what is presented to the user is a stereoscopic surround-view image of a real scene around a vehicle, and the black area in the middle is the vehicle. The obtained stereo surround view image is displayed on the display screen, and the stereo surround view image can be rotated to any angle of view according to the user's needs, so that the user can more intuitively and truly perceive the real scene information around the vehicle.
本实施例提供的鱼眼相机的环视图像合成方法，该方法中车载设备获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像，根据预先获取的多个鱼眼相机之间的对应关系，对多个鱼眼图像进行拼接，获取多个鱼眼图像对应的平面环视图像，对应关系用于表示多个鱼眼相机采集的鱼眼图像之间的位置关系，对平面环视图像进行优化，获取优化后的平面环视图像，对多个鱼眼图像中相同的空间物体进行空间对齐处理，获取平面环视图像对应的立体环视图像。通过该方法，能够获得车辆四周实景的环视图像，提高了用户体验，同时消除了车辆行驶时的盲区，提升了行驶安全性。The fisheye camera surround view image synthesis method provided in this embodiment, in this method, the vehicle-mounted device acquires a plurality of fisheye images collected by a plurality of fisheye cameras installed on the vehicle, and according to the pre-acquired Correspondence relationship, splicing multiple fisheye images to obtain the plane look-around image corresponding to multiple fisheye images, the correspondence relationship is used to represent the positional relationship between the fisheye images collected by multiple fisheye cameras, and the plane look-around image Optimization, obtaining the optimized planar view image, spatially aligning the same spatial objects in multiple fisheye images, and obtaining the stereoscopic view image corresponding to the planar view image. By using this method, a surround-view image of the real scene around the vehicle can be obtained, which improves user experience, eliminates blind spots when the vehicle is driving, and improves driving safety.
图9为本发明实施例提供的基于鱼眼相机的环视图像合成装置的结构示意图，该基于鱼眼相机的环视图像合成装置可以为独立的车载设备，也可以为集成在车载设备中的装置，该装置可以通过软件、硬件或者软硬件结合的方式实现。如图9所示，基于鱼眼相机的环视图像合成装置10包括：获取模块11、处理模块12、显示模块13；9 is a schematic structural diagram of a fisheye camera-based surround-view image synthesis device provided by an embodiment of the present invention. The fisheye-based surround-view image synthesis device can be an independent vehicle-mounted device, or a device integrated in a vehicle-mounted device. The device can be realized by means of software, hardware or a combination of software and hardware. As shown in Figure 9, the fisheye camera-based surround view image synthesis device 10 includes: an acquisition module 11, a processing module 12, and a display module 13;
获取模块11，用于获取设置在车辆上的多个鱼眼相机采集的多个鱼眼图像；An acquisition module 11, configured to acquire a plurality of fisheye images collected by a plurality of fisheye cameras arranged on the vehicle;
处理模块12，用于根据预先获取的所述多个鱼眼相机之间的对应关系，对所述多个鱼眼图像进行拼接，获取所述多个鱼眼图像对应的平面环视图像，所述对应关系用于表示所述多个鱼眼相机采集的鱼眼图像之间的位置关系；The processing module 12 is configured to splice the plurality of fisheye images according to the pre-acquired correspondence between the plurality of fisheye cameras, and obtain a plane look-around image corresponding to the plurality of fisheye images, the The corresponding relationship is used to represent the positional relationship between the fisheye images collected by the plurality of fisheye cameras;
对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取所述平面环视图像对应的立体环视图像。performing spatial alignment processing on the same spatial objects in the plurality of fisheye images, and acquiring a stereoscopic surround-view image corresponding to the planar surround-view image.
可选的，所述处理模块12，还用于对所述平面环视图像进行优化，获取优化后的平面环视图像。Optionally, the processing module 12 is further configured to optimize the plane look-around image, and obtain the optimized plane look-around image.
可选的，所述获取模块11，还用于获取所述多个鱼眼相机采集的多个标定板图像，所述鱼眼图像包括所述标定板图像；Optionally, the acquiring module 11 is further configured to acquire multiple calibration plate images collected by the multiple fisheye cameras, the fisheye images including the calibration plate images;
根据所述多个标定板图像，获取所述多个鱼眼相机的对应关系。According to the multiple calibration plate images, the corresponding relationship of the multiple fisheye cameras is acquired.
可选的，所述获取模块11，还用于根据特征点检测算法，获取所述多个标定板图像的特征点；Optionally, the acquiring module 11 is further configured to acquire feature points of the plurality of calibration plate images according to a feature point detection algorithm;
根据所述多个标定板图像的特征点，获取所述特征点的位置坐标；Acquiring the position coordinates of the feature points according to the feature points of the plurality of calibration plate images;
根据所述特征点的位置坐标，获取所述多个鱼眼相机的对应关系。According to the position coordinates of the feature points, the corresponding relationship of the plurality of fisheye cameras is acquired.
可选的，所述处理模块12，还用于对所述多个鱼眼图像中相同的空间物体进行空间对齐处理，获取第一立体环视图像；Optionally, the processing module 12 is further configured to perform spatial alignment processing on the same spatial objects in the plurality of fisheye images, to acquire a first stereoscopic surround view image;
对所述第一立体环视图像进行颜色混合处理，获取第二立体环视图像；performing color mixing processing on the first stereoscopic surround view image to obtain a second stereoscopic surround view image;
所述获取模块11，还用于根据所述第二立体环视图像，以及预先建立的三维模型，获取所述平面环视图像对应的立体环视图像，所述立体环视图像为第二立体环视图像在所述预先建立的三维模型上的投影。The acquiring module 11 is further configured to acquire a stereoscopic surround view image corresponding to the planar surround view image according to the second stereoscopic surround view image and a pre-established three-dimensional model, and the stereoscopic surround view image is the second stereoscopic surround view image in the Projection on the pre-built 3D model described above.
显示模块12，用于显示所述平面环视图像对应的立体环视图像。The display module 12 is configured to display a stereoscopic surround-view image corresponding to the planar surround-view image.
可选的，所述鱼眼相机的数量为四个。Optionally, the number of the fisheye cameras is four.
本发明实施例提供的鱼眼相机的环视图像合成装置，可以执行上述方法实施例，其实现原理和技术效果类似，在此不再赘述。The surround-view image synthesis device for a fisheye camera provided in the embodiment of the present invention can execute the above-mentioned method embodiment, and its implementation principle and technical effect are similar, and will not be repeated here.
图10为本发明实施例提供的车载设备的硬件结构示意图，如图10所示，本实施例的车载设备80包括：处理器801以及存储器802；其中：FIG. 10 is a schematic diagram of the hardware structure of the vehicle-mounted device provided by the embodiment of the present invention. As shown in FIG. 10 , the vehicle-mounted device 80 of this embodiment includes: a processor 801 and a memory 802; wherein:
存储器802，用于存储计算机执行指令；memory 802, for storing computer-executable instructions;
处理器801，用于执行存储器存储的计算机执行指令，以实现上述实施例中接收设备所执行的各个步骤。具体可以参见前述方法实施例中的相关描述。The processor 801 is configured to execute computer-executable instructions stored in a memory, so as to implement various steps executed by the receiving device in the foregoing embodiments. For details, refer to the related descriptions in the foregoing method embodiments.
可选地，存储器802既可以是独立的，也可以跟处理器801集成在一起。Optionally, the memory 802 can be independent or integrated with the processor 801 .
当存储器802独立设置时，该语音交互设备还包括总线803，用于连接所述存储器802和处理器801。When the memory 802 is set independently, the voice interaction device further includes a bus 803 for connecting the memory 802 and the processor 801 .
本发明实施例还提供一种计算机可读存储介质，其中，计算机可读存储介质存储有计算机程序，所述计算机程序使得服务器执行前述任一实施例提供的基于鱼眼相机的环视图像合成方法。An embodiment of the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables the server to execute the fisheye camera-based surround-view image synthesis method provided in any of the foregoing embodiments.
在本发明所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。例如，以上所描述的设备实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
另外，在本发明各个实施例中的各功能模块可以集成在一个处理单元中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个单元中。上述模块成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may physically exist separately, or two or more modules may be integrated into one unit. The units formed by the above modules can be implemented in the form of hardware, or in the form of hardware plus software functional units.
上述以软件功能模块的形式实现的集成的模块，可以存储在一个计算机可读取存储介质中。上述软件功能模块存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(英文：processor)执行本申请各个实施例所述方法的部分步骤。The above-mentioned integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software functional modules are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or a processor (English: processor) to execute the functions described in various embodiments of the present application. part of the method.
应理解，上述处理器可以是中央处理单元(英文：Central Processing Unit，简称：CPU)，还可以是其他通用处理器、数字信号处理器(英文：Digital Signal Processor，简称：DSP)、专用集成电路(英文：Application Specific Integrated Circuit，简称：ASIC)等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合发明所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。It should be understood that the above-mentioned processor can be a central processing unit (English: Central Processing Unit, referred to as: CPU), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, referred to as: DSP), application-specific integrated circuits (English: Application Specific Integrated Circuit, referred to as: ASIC) and so on. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in conjunction with the invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
存储器可能包含高速RAM存储器，也可能还包括非易失性存储NVM，例如至少一个磁盘存储器，还可以为U盘、移动硬盘、只读存储器、磁盘或光盘等。The storage may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage, and may also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.
总线可以是工业标准体系结构(Industry Standard Architecture，ISA)总线、外部设备互连(Peripheral Component，PCI)总线或扩展工业标准体系结构(ExtendedIndustry Standard Architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，本申请附图中的总线并不限定仅有一根总线或一种类型的总线。The bus may be an Industry Standard Architecture (Industry Standard Architecture, ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, the buses in the drawings of the present application are not limited to only one bus or one type of bus.
上述存储介质可以是由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。存储介质可以是通用或专用计算机能够存取的任何可用介质。The above-mentioned storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable In addition to programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于专用集成电路(Application Specific Integrated Circuits，简称：ASIC)中。当然，处理器和存储介质也可以作为分立组件存在于电子设备或主控设备中。An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in application specific integrated circuits (Application Specific Integrated Circuits, ASIC for short). Of course, the processor and the storage medium can also exist in the electronic device or the main control device as discrete components.
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.
最后应说明的是：以上各实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述各实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811598527.3ACN109754363B (en) | 2018-12-26 | 2018-12-26 | Around-the-eye image synthesis method and device based on fish eye camera |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811598527.3ACN109754363B (en) | 2018-12-26 | 2018-12-26 | Around-the-eye image synthesis method and device based on fish eye camera |
| Publication Number | Publication Date |
|---|---|
| CN109754363A CN109754363A (en) | 2019-05-14 |
| CN109754363Btrue CN109754363B (en) | 2023-08-15 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811598527.3AActiveCN109754363B (en) | 2018-12-26 | 2018-12-26 | Around-the-eye image synthesis method and device based on fish eye camera |
| Country | Link |
|---|---|
| CN (1) | CN109754363B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110466533A (en)* | 2019-07-25 | 2019-11-19 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of control method for vehicle, apparatus and system |
| CN110580724B (en)* | 2019-08-28 | 2022-02-25 | 贝壳技术有限公司 | Method and device for calibrating binocular camera set and storage medium |
| CN110910311B (en)* | 2019-10-30 | 2023-09-26 | 同济大学 | Automatic splicing method of multi-path looking-around camera based on two-dimension code |
| CN113066158B (en)* | 2019-12-16 | 2023-03-10 | 杭州海康威视数字技术股份有限公司 | A vehicle surround view method and device |
| CN111754390A (en)* | 2020-05-11 | 2020-10-09 | 上海欧菲智能车联科技有限公司 | Ring view display method and device, computer equipment and storage medium |
| CN113132708B (en)* | 2021-04-22 | 2022-02-22 | 贝壳找房(北京)科技有限公司 | Method and apparatus for acquiring three-dimensional scene image using fisheye camera, device and medium |
| CN117058749B (en)* | 2023-08-17 | 2024-06-07 | 深圳市华弘智谷科技有限公司 | Multi-camera perspective method and device, intelligent glasses and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104330074A (en)* | 2014-11-03 | 2015-02-04 | 熊友谊 | Intelligent surveying and mapping platform and realizing method thereof |
| CN106023070A (en)* | 2016-06-14 | 2016-10-12 | 北京岚锋创视网络科技有限公司 | Real-time panoramic splicing method and device |
| CN106534670A (en)* | 2016-10-25 | 2017-03-22 | 成都通甲优博科技有限责任公司 | Panoramic video generating method based on fixedly connected fisheye lens camera unit |
| CN106875339A (en)* | 2017-02-22 | 2017-06-20 | 长沙全度影像科技有限公司 | A kind of fish eye images joining method based on strip scaling board |
| CN107256535A (en)* | 2017-06-06 | 2017-10-17 | 斑马信息科技有限公司 | The display methods and device of panoramic looking-around image |
| CN107492125A (en)* | 2017-07-28 | 2017-12-19 | 哈尔滨工业大学深圳研究生院 | The processing method of automobile fish eye lens panoramic view picture |
| WO2018035845A1 (en)* | 2016-08-26 | 2018-03-01 | 深圳市赛亿科技开发有限公司 | Method for eliminating photographic blind spot based on 720 degree panoramic video camera |
| CN108257183A (en)* | 2017-12-20 | 2018-07-06 | 歌尔科技有限公司 | A kind of camera lens axis calibrating method and device |
| CN108629828A (en)* | 2018-04-03 | 2018-10-09 | 中德（珠海）人工智能研究院有限公司 | Scene rendering transition method in the moving process of three-dimensional large scene |
| CN108846796A (en)* | 2018-06-22 | 2018-11-20 | 北京航空航天大学青岛研究院 | Image split-joint method and electronic equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104330074A (en)* | 2014-11-03 | 2015-02-04 | 熊友谊 | Intelligent surveying and mapping platform and realizing method thereof |
| CN106023070A (en)* | 2016-06-14 | 2016-10-12 | 北京岚锋创视网络科技有限公司 | Real-time panoramic splicing method and device |
| WO2018035845A1 (en)* | 2016-08-26 | 2018-03-01 | 深圳市赛亿科技开发有限公司 | Method for eliminating photographic blind spot based on 720 degree panoramic video camera |
| CN106534670A (en)* | 2016-10-25 | 2017-03-22 | 成都通甲优博科技有限责任公司 | Panoramic video generating method based on fixedly connected fisheye lens camera unit |
| CN106875339A (en)* | 2017-02-22 | 2017-06-20 | 长沙全度影像科技有限公司 | A kind of fish eye images joining method based on strip scaling board |
| CN107256535A (en)* | 2017-06-06 | 2017-10-17 | 斑马信息科技有限公司 | The display methods and device of panoramic looking-around image |
| CN107492125A (en)* | 2017-07-28 | 2017-12-19 | 哈尔滨工业大学深圳研究生院 | The processing method of automobile fish eye lens panoramic view picture |
| CN108257183A (en)* | 2017-12-20 | 2018-07-06 | 歌尔科技有限公司 | A kind of camera lens axis calibrating method and device |
| CN108629828A (en)* | 2018-04-03 | 2018-10-09 | 中德（珠海）人工智能研究院有限公司 | Scene rendering transition method in the moving process of three-dimensional large scene |
| CN108846796A (en)* | 2018-06-22 | 2018-11-20 | 北京航空航天大学青岛研究院 | Image split-joint method and electronic equipment |
| Title |
|---|
| 基于车载多视角的鱼眼图像拼接算法;周芳等;《数据通信》;20170531(第5期);第29-34页* |
| Publication number | Publication date |
|---|---|
| CN109754363A (en) | 2019-05-14 |
| Publication | Publication Date | Title |
|---|---|---|
| CN109754363B (en) | Around-the-eye image synthesis method and device based on fish eye camera | |
| US9866752B2 (en) | Systems and methods for producing a combined view from fisheye cameras | |
| CN106846410B (en) | Driving environment imaging method and device based on three dimensions | |
| CN106575432B (en) | Object visualization in bowl imaging system | |
| CN103770706B (en) | Dynamic reversing mirror indicating characteristic | |
| US11330172B2 (en) | Panoramic image generating method and apparatus | |
| CN111770273B (en) | Image shooting method and device, electronic equipment and readable storage medium | |
| JP2017220923A (en) | Image generating apparatus, image generating method, and program | |
| CN108564612A (en) | Model display method, device, storage medium and electronic equipment | |
| CN111815752B (en) | Image processing method and device and electronic equipment | |
| CN109741241B (en) | Fisheye image processing method, device, equipment and storage medium | |
| CN107180406A (en) | Image processing method and equipment | |
| CN114972023A (en) | Image mosaic processing method, device, equipment and computer storage medium | |
| CN112435161A (en) | Panoramic all-around image splicing method and system, electronic equipment and storage medium | |
| CN113496527B (en) | Vehicle surrounding image calibration method, device and system and storage medium | |
| CN113888645A (en) | Driving equipment, computer vision processing method and device and electronic equipment | |
| CN113542463A (en) | Video shooting device and method based on folding screen, storage medium and mobile terminal | |
| CN116051379A (en) | AVM image fusion method and device, vehicle and readable storage medium | |
| CN114638747A (en) | A kind of panoramic reversing image processing method, device and equipment | |
| WO2018042074A1 (en) | A method, apparatus and computer program product for indicating a seam of an image in a corresponding area of a scene | |
| CN114066731A (en) | Method, apparatus, electronic device and storage medium for generating panoramic images | |
| CN117808674A (en) | Vehicle-mounted looking-around system image processing method and device, electronic equipment and vehicle | |
| CN118864682B (en) | Rendering method and device of vehicle model, electronic equipment and storage medium | |
| CN118574018B (en) | Panoramic image processing method and device, related equipment and vehicle | |
| CN115393497A (en) | Virtual imaging method, virtual imaging device, nonvolatile storage medium and computer equipment |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address | Address after:200000 Shanghai City Xuhui District Longyao Road No. 18 10th Floor 1001 Room Patentee after:Zebra Network Technology Co.,Ltd. Country or region after:China Address before:Building D1, 2nd Floor, No. 55 Huaihai West Road, Xuhui District, Shanghai Patentee before:ZEBRED NETWORK TECHNOLOGY Co.,Ltd. Country or region before:China |