技术领域technical field
本申请涉及成像技术领域，尤其涉及一种基于多帧图像的图像处理方法、装置、电子设备。The present application relates to the field of imaging technologies, and in particular, to an image processing method, apparatus, and electronic device based on multi-frame images.
背景技术Background technique
随着智能终端技术的发展，移动终端设备(如智能手机、平板电脑等)的使用越来越普及。绝大多数移动终端设备都内置有摄像头，并且随着移动终端处理能力的增强以及摄像头技术的发展，内置摄像头的性能越来越强大，拍摄图像的质量也越来越高。如今，移动终端设备均操作简单又便于携带，在日常生活中越来越多的用户使用智能手机、平板电脑等移动终端设备拍照。With the development of smart terminal technology, the use of mobile terminal devices (such as smart phones, tablet computers, etc.) is becoming more and more popular. Most mobile terminal devices have built-in cameras, and with the enhancement of processing capabilities of mobile terminals and the development of camera technology, the performance of built-in cameras is becoming more and more powerful, and the quality of captured images is also getting higher and higher. Nowadays, mobile terminal devices are easy to operate and easy to carry. More and more users use mobile terminal devices such as smartphones and tablet computers to take pictures in daily life.
智能移动终端在给人们的日常拍照带来便捷的同时，人们对拍摄的图像质量的要求也越来越高，尤其在夜景这一特殊场景中，图像质量较低。While smart mobile terminals bring convenience to people's daily photography, people have higher and higher requirements for the quality of the captured images, especially in the special scene of night scenes, where the image quality is low.
目前，通常采集多帧原始图像进行高动态合成，但是在采集多帧原始图像过程中会引入噪声，导致最终合成的图像不清晰。因此，在最大限度的保留图像细节的情况下，对图像降噪处理，是一个亟待解决的问题。At present, multiple frames of original images are usually collected for high-dynamic synthesis, but noise will be introduced during the process of collecting multiple frames of original images, resulting in an unclear final synthesized image. Therefore, it is an urgent problem to deal with image noise reduction while preserving the image details to the greatest extent.
发明内容SUMMARY OF THE INVENTION
本申请旨在至少在一定程度上解决相关技术中的技术问题之一。The present application aims to solve one of the technical problems in the related art at least to a certain extent.
为此，本申请的目的在于提出一种基于多帧图像的图像处理方法、装置、电子设备，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，有助于减少原始图像采集帧数，使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。Therefore, the purpose of this application is to propose an image processing method, device and electronic device based on multi-frame images, which can more accurately distinguish the picture noise and effective details of high dynamic range images, and help reduce the original image acquisition frame The total time required for the overall shooting process is shortened, which avoids the situation of blurred pictures caused by too long shooting time, and is conducive to clear shooting of dynamic night scenes.
为达到上述目的，本申请第一方面实施例提出的基于多帧图像的图像处理方法，包括：获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，所述部分帧原始图像为所述多帧原始图像中的至少两帧的原始图像；将所述第一降噪图像转换为第一YUV图像，并将所述第二降噪图像转换为第二YUV图像；根据所述第一YUV图像和所述第二YUV图像，合成得到高动态范围图像。In order to achieve the above purpose, the image processing method based on the multi-frame image proposed by the embodiment of the first aspect of the present application includes: acquiring the multi-frame original image; denoising part of the frame original image based on artificial intelligence to obtain a first noise-reduced image, and Denoising other frames of original images based on artificial intelligence to obtain a second noise-reduced image, where the partial frame of original images is the original image of at least two frames of the multiple frames of original images; converting the first noise-reduced image into A first YUV image, and converting the second noise-reduced image into a second YUV image; and combining the first YUV image and the second YUV image to obtain a high dynamic range image.
本申请第一方面实施例提出的基于多帧图像的图像处理方法，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，有助于减少原始图像采集帧数，使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。The image processing method based on multi-frame images proposed by the embodiment of the first aspect of the present application obtains multi-frame original images; de-noises some frames of original images based on artificial intelligence to obtain a first noise-reduced image, and performs noise reduction for other frames of original images based on Artificial intelligence reduces noise, obtains a second noise reduction image, converts the first noise reduction image into a first YUV image, and converts the second noise reduction image into a second YUV image, according to the first YUV image and the second YUV image, Synthesizing high dynamic range images can more accurately distinguish the picture noise and effective details of high dynamic range images, which helps to reduce the number of original image acquisition frames, shortens the total time required for the overall shooting process, and avoids excessive shooting time. If the picture is blurred for a long time, it is beneficial to shoot dynamic night scenes clearly.
为达到上述目的，本申请第二方面实施例提出的基于多帧图像的图像处理装置，包括：获取模块，用于获取多帧原始图像；降噪模块，用于对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，所述部分帧原始图像为所述多帧原始图像中的至少两帧的原始图像；转换模块，用于将所述第一降噪图像转换为第一YUV图像，并将所述第二降噪图像转换为第二YUV图像；合成模块，用于根据所述第一YUV图像和所述第二YUV图像，合成得到高动态范围图像。In order to achieve the above purpose, the image processing device based on multi-frame images proposed by the embodiment of the second aspect of the present application includes: an acquisition module for acquiring multi-frame original images; Noise reduction, obtaining a first noise reduction image, and denoising other frames of original images based on artificial intelligence to obtain a second noise reduction image, where the partial frame original images are the original images of at least two frames of the multiple frames of original images The conversion module is used to convert the first noise reduction image into the first YUV image, and the second noise reduction image is converted into the second YUV image; the synthesis module is used for according to the first YUV image and The second YUV image is synthesized to obtain a high dynamic range image.
本申请第二方面实施例提出的基于多帧图像的图像处理装置，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，有助于减少原始图像采集帧数，使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。The image processing device based on multi-frame images proposed by the embodiment of the second aspect of the present application obtains multi-frame original images; denoises some frames of original images based on artificial intelligence to obtain a first noise-reduced image, and performs noise reduction for other frames of original images based on Artificial intelligence reduces noise, obtains a second noise reduction image, converts the first noise reduction image into a first YUV image, and converts the second noise reduction image into a second YUV image, according to the first YUV image and the second YUV image, Synthesizing high dynamic range images can more accurately distinguish the picture noise and effective details of high dynamic range images, which helps to reduce the number of original image acquisition frames, shortens the total time required for the overall shooting process, and avoids excessive shooting time. If the picture is blurred for a long time, it is beneficial to shoot dynamic night scenes clearly.
为达到上述目的，本申请第三方面实施例提出的电子设备，包括：图像传感器、存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述图像传感器与所述处理器电连接，所述处理器执行所述程序时，实现本申请第一方面实施例提出的基于多帧图像的图像处理方法。In order to achieve the above purpose, the electronic device proposed in the embodiment of the third aspect of the present application includes: an image sensor, a memory, a processor, and a computer program stored in the memory and running on the processor, the image sensor and the processor The processor is electrically connected, and when the processor executes the program, the multi-frame image-based image processing method proposed by the embodiment of the first aspect of the present application is implemented.
本申请第三方面实施例提出的电子设备，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，有助于减少原始图像采集帧数，使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。The electronic device proposed by the embodiment of the third aspect of the present application obtains a first noise reduction image by acquiring multiple frames of original images; denoising some frames of the original images based on artificial intelligence to obtain a first denoised image, and denoising other frames of the original images based on artificial intelligence to obtain The second noise reduction image is converted into a first YUV image, and the second noise reduction image is converted into a second YUV image, and a high dynamic range image is synthesized according to the first YUV image and the second YUV image , which can more accurately distinguish the picture noise and effective details of high dynamic range images, which helps to reduce the number of original image acquisition frames, shortens the total time required for the overall shooting process, and avoids the situation that the picture is blurred due to too long shooting time. , which is conducive to clear shooting of dynamic night scenes.
为达到上述目的，本申请第四方面实施例提出的图像处理电路，包括：图像信号处理ISP处理器和图形处理器GPU；所述ISP处理器，与图像传感器电连接，用于控制所述图像传感器获取多帧原始图像；所述GPU，与所述ISP处理器电连接，用于对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，所述部分帧原始图像为所述多帧原始图像中的至少两帧的原始图像；所述ISP处理器，还用于将所述第一降噪图像转换为第一YUV图像，并将所述第二降噪图像转换为第二YUV图像；根据所述第一YUV图像和所述第二YUV图像，合成得到高动态范围图像。In order to achieve the above purpose, the image processing circuit proposed in the embodiment of the fourth aspect of the present application includes: an image signal processing ISP processor and a graphics processor GPU; the ISP processor is electrically connected to the image sensor and is used to control the image The sensor acquires multiple frames of original images; the GPU, electrically connected to the ISP processor, is used for denoising some frames of original images based on artificial intelligence to obtain a first noise reduction image, and denoising other frames of original images based on artificial intelligence. noise, to obtain a second noise-reduced image, where the original images of the partial frames are original images of at least two frames of the multiple frames of original images; the ISP processor is further configured to convert the first noise-reduced image into A first YUV image, and converting the second noise-reduced image into a second YUV image; and combining the first YUV image and the second YUV image to obtain a high dynamic range image.
本申请第四方面实施例提出的图像处理电路，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，有助于减少原始图像采集帧数，使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。The image processing circuit proposed by the embodiment of the fourth aspect of the present application obtains multiple frames of original images; performs noise reduction based on artificial intelligence for part of the original images to obtain a first noise reduction image, and performs noise reduction for other frames of original images based on artificial intelligence. Obtain a second noise reduction image, convert the first noise reduction image into a first YUV image, convert the second noise reduction image into a second YUV image, and synthesize the high dynamic range according to the first YUV image and the second YUV image image, which can more accurately distinguish the picture noise and effective details of high dynamic range images, which helps to reduce the number of original image acquisition frames, shortens the total time required for the overall shooting process, and avoids the blurring of the picture caused by too long shooting time. It is conducive to clear shooting of dynamic night scenes.
为达到上述目的，本申请第五方面实施例提出的计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如本申请第一方面实施例提出的基于多帧图像的图像处理方法。In order to achieve the above purpose, the computer-readable storage medium proposed by the embodiment of the fifth aspect of the present application stores a computer program thereon, and when the program is executed by the processor, the multi-frame image-based storage medium as proposed by the embodiment of the first aspect of the present application is implemented. image processing method.
本申请第五方面实施例提出的计算机可读存储介质，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，有助于减少原始图像采集帧数，使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。The computer-readable storage medium proposed by the embodiment of the fifth aspect of the present application obtains multiple frames of original images; de-noises some frames of the original images based on artificial intelligence to obtain a first noise-reduced image, and reduces other frames of original images based on artificial intelligence. noise, obtain a second noise reduction image, convert the first noise reduction image into a first YUV image, and convert the second noise reduction image into a second YUV image, and combine the first YUV image and the second YUV image to obtain a high The dynamic range image can more accurately distinguish the picture noise and effective details of the high dynamic range image, which helps to reduce the number of original image acquisition frames, shortens the total time required for the overall shooting process, and avoids excessive shooting time. The blurred situation is conducive to clear shooting of dynamic night scenes.
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.
附图说明Description of drawings
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:
图1为本申请实施例所提供的第一种基于多帧图像的图像处理方法的流程示意图；1 is a schematic flowchart of a first multi-frame image-based image processing method provided by an embodiment of the present application;
图2为本申请中的一种应用流程示意图；Fig. 2 is a kind of application flow schematic diagram in the application;
图3为本申请实施例所提供的第二种基于多帧图像的图像处理方法的流程示意图；3 is a schematic flowchart of a second multi-frame image-based image processing method provided by an embodiment of the present application;
图4为本申请实施例所提供的第三种基于多帧图像的图像处理方法的流程示意图；4 is a schematic flowchart of a third multi-frame image-based image processing method provided by an embodiment of the present application;
图5为本申请实施例提供的第四种基于多帧图像的图像处理方法的流程示意图；5 is a schematic flowchart of a fourth multi-frame image-based image processing method provided by an embodiment of the present application;
图6为本申请实施例提供的第一种基于多帧图像的图像处理装置的结构示意图；FIG. 6 is a schematic structural diagram of a first image processing apparatus based on a multi-frame image provided by an embodiment of the present application;
图7为本申请实施例提供的第二种基于多帧图像的图像处理装置的结构示意图；7 is a schematic structural diagram of a second multi-frame image-based image processing apparatus provided by an embodiment of the present application;
图8为本申请实施例提供的一种电子设备的结构示意图；FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
图9为本申请实施例提供的一种电子设备的原理示意图；FIG. 9 is a schematic diagram of the principle of an electronic device provided by an embodiment of the present application;
图10为本申请实施例提供的一种图像处理电路的原理示意图。FIG. 10 is a schematic diagram of the principle of an image processing circuit provided by an embodiment of the present application.
具体实施方式Detailed ways
下面详细描述本申请的实施例，实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能理解为对本申请的限制。相反，本申请的实施例包括落入所附加权利要求书的精神和内涵范围内的所有变化、修改和等同物。Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, but should not be construed as a limitation on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents falling within the spirit and scope of the appended claims.
针对相关技术中，高动态高动态范围图像时，拍摄的帧数较多，采帧时间长，可能会由于抖动导致拍摄的图像存在拖影，或者会在拍摄的过程中引入噪声，导致图像画面模糊的问题，本申请提出了一种基于多帧图像的图像处理方法，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像；将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像；根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。In the related art, when a high dynamic and high dynamic range image is used, a large number of frames are taken, and the frame acquisition time is long, which may cause smear in the captured image due to jitter, or introduce noise during the shooting process, resulting in the image picture. To solve the problem of ambiguity, this application proposes an image processing method based on multiple frames of images. By obtaining multiple frames of original images, denoising some frames of original images based on artificial intelligence to obtain a first noise-reduced image, and processing other frames of original images. Based on artificial intelligence noise reduction, a second noise reduction image is obtained, and some of the original frames are original images of at least two frames of the original images; the first noise reduction image is converted into a first YUV image, and the second noise reduction image is converted into a second noise reduction image. The image is converted into a second YUV image; and a high dynamic range image is synthesized according to the first YUV image and the second YUV image.
下面参考附图描述本申请实施例的基于多帧图像的图像处理方法和装置。The following describes the image processing method and apparatus based on multi-frame images according to the embodiments of the present application with reference to the accompanying drawings.
图1为本申请实施例所提供的第一种基于多帧图像的图像处理方法的流程示意图。FIG. 1 is a schematic flowchart of a first image processing method based on a multi-frame image provided by an embodiment of the present application.
本申请实施例的基于多帧图像的图像处理方法，应用于电子设备，该电子设备可以为手机、平板电脑、个人数字助理、穿戴式设备等具有各种操作系统、成像设备的硬件设备。The image processing method based on multi-frame images in the embodiments of the present application is applied to electronic devices, which may be mobile phones, tablet computers, personal digital assistants, wearable devices and other hardware devices with various operating systems and imaging devices.
如图1所示，该基于多帧图像的图像处理方法包括以下步骤：As shown in Figure 1, the image processing method based on multi-frame images includes the following steps:
步骤101，获取多帧原始图像。Step 101, acquiring multiple frames of original images.
其中，原始图像可以例如通过电子设备的图像传感器采集得到的未做任何处理的RAW格式图像，对此不作限制。Wherein, the original image may be, for example, an unprocessed RAW format image acquired by an image sensor of an electronic device, which is not limited.
其中，RAW格式图像就是图像传感器将捕捉到的光源信号转化为数字信号的原始图像。RAW格式图像记录了数码相机传感器的原始信息，同时记录了由相机拍摄所产生的一些元数据，如感光度的设置、快门速度、光圈值、白平衡等。Among them, the RAW format image is the original image that the image sensor converts the captured light source signal into a digital signal. The RAW format image records the original information of the digital camera sensor, and records some metadata generated by the camera, such as the sensitivity setting, shutter speed, aperture value, white balance and so on.
可以通过获取当前拍摄场景的预览图像，以确定当前拍摄场景是否属于夜景场景。由于不同场景下环境亮度值不同，预览图像内容也不相同，可以根据当前拍摄场景预览图像的画面内容以及各区域的环境亮度值，确定当前拍摄场景属于夜景场景后，启动夜景拍摄模式，在不同曝光量下采集多帧原始图像。Whether the current shooting scene belongs to a night scene scene can be determined by acquiring a preview image of the current shooting scene. Since the ambient brightness value is different in different scenes, the preview image content is also different. According to the screen content of the preview image of the current shooting scene and the ambient brightness value of each area, it can be determined that the current shooting scene belongs to the night scene scene, and then start the night scene shooting mode. Capture multiple frames of raw images under exposure.
例如，预览图像的画面内容包括夜晚天空或者夜景灯源等，或者预览图像的各区域中环境亮度值符合夜景环境下图像的亮度分布特性，即可确定当前拍摄场景属于夜景场景。For example, if the screen content of the preview image includes the night sky or night scene light sources, or the ambient brightness value in each area of the preview image conforms to the brightness distribution characteristics of the image in the night scene environment, it can be determined that the current shooting scene belongs to the night scene scene.
由于在夜景拍摄时，拍摄场景中光线强度等环境因素的限制，电子设备在拍摄图像时，若采集单帧原始图像无法较好同时顾及到夜景中的灯光等高亮区域，以及夜景中的低亮区域。Due to the limitation of environmental factors such as light intensity in the shooting scene during night scene shooting, when electronic equipment captures an image, if a single frame of original image is captured, it cannot take into account the bright areas such as lights in the night scene and the low light in the night scene. bright area.
因此，电子设备可以通过拍摄多帧原始图像，用于图像合成，另外还可以用于选取画面清晰的图像进行合成成像。Therefore, the electronic device can be used for image synthesis by shooting multiple frames of original images, and can also be used to select a clear image for synthesis imaging.
为了同时顾及到夜景中的灯光等高亮区域，以及夜景中的低亮区域，可以控制电子设备的图像传感器在不同曝光量下，拍摄得到的多帧原始图像。例如：采用低曝光量拍摄以对高亮区清晰成像，采用高曝光量拍摄以对低亮区清晰成像。In order to take into account the high-brightness areas such as lights in the night scene and the low-brightness areas in the night scene at the same time, the image sensor of the electronic device can be controlled to capture multiple frames of original images under different exposures. For example: shoot with low exposure to clearly image high-brightness areas, and shoot with high exposure to image clearly low-brightness areas.
步骤102，对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像。Step 102 , denoise the original images of some frames based on artificial intelligence to obtain a first denoised image, and denoise the original images of other frames based on artificial intelligence to obtain a second denoised image, and the original images of some frames are among the original images of the multiple frames. of at least two frames of the original image.
本申请实施例中，部分帧原始图像为至少两帧相同曝光量的第一图像，其它帧原始图像为曝光量低于第一图像的至少一帧第二图像。In the embodiment of the present application, some frames of original images are at least two frames of first images with the same exposure, and other frames of original images are at least one frame of second images with a lower exposure than the first image.
本申请实施例中，通过分别对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，是考虑到部分帧原始图像和其它帧原始图像的噪声特性并不完全相同，因此使得降噪更具有针对性，能够有效提升降噪效果。In the embodiment of the present application, the first noise-reduced image is obtained by denoising the original images of some frames based on artificial intelligence, and the second de-noising image is obtained by denoising the original images of other frames based on artificial intelligence. The noise characteristics of the original image and the original images of other frames are not the same, so the noise reduction is more targeted and the noise reduction effect can be effectively improved.
可以理解的是，由于电子设备中的图像传感器在拍摄的过程中会受到不同程度的来自周边电路和本身像素间的光电磁干扰，因此拍摄得到的原始图像不可避免的存在噪声，并且，干扰程度的不同，拍摄得到的图像的清晰度也不相同。因此采集的多帧原始图像也必然存在噪声，可以进一步对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像。It can be understood that since the image sensor in the electronic device will be subject to different degrees of optical and electromagnetic interference from the peripheral circuit and its own pixels during the shooting process, the original image obtained by shooting inevitably has noise, and the degree of interference The sharpness of the captured images is also different. Therefore, there must be noise in the collected multiple frames of original images. It is possible to further denoise some frames of original images based on artificial intelligence to obtain a first denoised image, and to denoise other frames of original images based on artificial intelligence to obtain a second denoised image. .
例如，在夜景拍摄场景中，通常使用较大的光圈和较长的曝光时间拍摄得到图像，此时如果选择较高的感光度来减少了曝光时间，拍摄得到的图像必然会产生噪声。For example, in a night scene shooting scene, an image is usually captured with a larger aperture and a longer exposure time. At this time, if a higher sensitivity is selected to reduce the exposure time, the captured image will inevitably produce noise.
本申请实施例中，可以首先对部分帧原始图像进行多帧融合降噪，得到初始降噪图像。In this embodiment of the present application, multi-frame fusion noise reduction may first be performed on part of the original images of the frames to obtain an initial noise reduction image.
例如，对部分帧原始图像进行图像对齐处理，合成为一张多帧融合图像(可以被称为初始降噪图像)，相当于经过了时域降噪，初步地提升了画面的信噪比。For example, performing image alignment processing on some frames of original images and synthesizing them into a multi-frame fused image (which can be called an initial noise reduction image) is equivalent to temporal noise reduction, which preliminarily improves the signal-to-noise ratio of the picture.
而后，采用第一神经网络模型，对初始降噪图像进行噪声特性识别，并采用第二神经网络模型，对其它帧原始图像中的各帧原始图像进行噪声特性识别，能够同时对初始降噪图像和各帧原始图像中的高光区域和暗光区域降噪，进而可以得到较佳的降噪效果的降噪图像。Then, the first neural network model is used to identify the noise characteristics of the initial noise reduction image, and the second neural network model is used to identify the noise characteristics of each frame of the original image in other frames of the original image, and the initial noise reduction image can be simultaneously identified. And the high-light area and dark-light area in the original image of each frame are denoised, and then a denoised image with better denoising effect can be obtained.
需要说明的是，对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，其中的第一降噪图像和第二降噪图像为未经加工处理的RAW图像。It should be noted that the first noise-reduced image is obtained based on artificial intelligence noise reduction for some frames of original images, and the second noise-reduced image is obtained based on artificial intelligence noise reduction for other frames of original images, wherein the first noise-reduced image and The second noise-reduced image is an unprocessed RAW image.
本申请实施例中，对部分帧原始图像基于人工智能降噪时，可以采用第一神经网络模型，对初始降噪图像进行噪声特性识别，其中，第一神经网络模型，已学习得到初始降噪图像的感光度与噪声特性之间的映射关系。In the embodiment of the present application, when denoising a part of the original image of the frame based on artificial intelligence, a first neural network model may be used to identify the noise characteristics of the initial denoised image, wherein the first neural network model has been learned to obtain the initial denoising The mapping relationship between the sensitivity of the image and the noise characteristics.
本申请实施例中，对其它帧原始图像基于人工智能降噪时，可以分别针对其它帧原始图像中的各帧原始图像，采用第二神经网络模型，对其它帧原始图像中的各帧原始图像进行噪声特性识别，第二神经网络模型，已学习得到各帧原始图像的感光度与噪声特性之间的映射关系。In the embodiment of the present application, when the other frames of original images are denoised based on artificial intelligence, the second neural network model may be used for each frame of the original images in the other frames of the original images. The noise characteristics are identified, and the second neural network model has learned the mapping relationship between the sensitivity of each frame of the original image and the noise characteristics.
作为一种可能的实现方式，由于第一神经网络模型，已学习得到初始降噪图像的感光度与噪声特性之间的映射关系。因此，可以将初始降噪图像输入第一神经网络模型中，以采用第一神经网络模型对初始降噪图像进行噪声特性识别，从而识别出初始降噪图像的噪声特性，根据识别出的噪声特性，对初始降噪图像降噪，得到第一降噪图像，从而达到了降噪的目的，提高了图像的信噪比。As a possible implementation manner, due to the first neural network model, the mapping relationship between the sensitivity of the initial noise reduction image and the noise characteristic has been learned. Therefore, the initial noise reduction image can be input into the first neural network model, so as to use the first neural network model to identify the noise characteristics of the initial noise reduction image, so as to identify the noise characteristics of the initial noise reduction image, according to the identified noise characteristics , denoise the initial noise reduction image to obtain the first noise reduction image, thereby achieving the purpose of noise reduction and improving the signal-to-noise ratio of the image.
针对采用第二神经网络模型，对其它帧原始图像中的各帧原始图像进行噪声特性识别的描述可以以此类推。For the use of the second neural network model, the description of performing noise feature identification on each frame of original images in other frames of original images can be deduced by analogy.
其中，感光度，又称为ISO值，是指衡量底片对于光的灵敏程度的指标。对于感光度较低的底片，需要曝光更长的时间以达到跟感光度较高的底片相同的成像。数码相机的感光度是一种类似于胶卷感光度的一种指标，数码相机的ISO可以通过调整感光器件的灵敏度或者合并感光点来调整，也就是说，可以通过提升感光器件的光线敏感度或者合并几个相邻的感光点来达到提升ISO的目的。Among them, the sensitivity, also known as the ISO value, refers to an index that measures the sensitivity of the film to light. For lower-sensitivity negatives, longer exposure times are required to achieve the same imaging as higher-sensitivity negatives. The sensitivity of a digital camera is an indicator similar to the sensitivity of a film. The ISO of a digital camera can be adjusted by adjusting the sensitivity of the photosensitive device or combining the photosensitive points, that is, by increasing the light sensitivity of the photosensitive device or by merging the photosensitive points. Combine several adjacent photosensitive points to achieve the purpose of increasing ISO.
需要说明的是，无论是数码或是底片摄影，ISO值越低，采集的图像质量越高，图像细节表现越细腻，ISO值越高，光线感应性能越强，也就越能接收更多的光线，从而产生更多的热量，因此，使用相对较高的感光度通常会引入较多的噪声，从而导致图像质量降低。It should be noted that, whether it is digital or film photography, the lower the ISO value, the higher the quality of the captured image and the finer the image details. light, which generates more heat, so using a relatively high ISO usually introduces more noise, which results in lower image quality.
本申请实施例中，噪声特性，可以是由于图像传感器引起的随机噪声的统计特性。这里说的噪声主要包括热噪声和散粒噪声，其中，热噪声符合高斯分布，散粒噪声符合泊松分布，本申请实施例中的统计特性可以指噪声的方差值，当然也可以是其他可能情况的值，在此不做限定。In this embodiment of the present application, the noise characteristic may be a statistical characteristic of random noise caused by an image sensor. The noise mentioned here mainly includes thermal noise and shot noise. Among them, thermal noise conforms to Gaussian distribution, and shot noise conforms to Poisson distribution. The statistical characteristic in this embodiment of the present application may refer to the variance value of noise, and of course it may also be other The possible values are not limited here.
步骤103，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像。Step 103: Convert the first noise-reduced image into a first YUV image, and convert the second noise-reduced image into a second YUV image.
可选地，电子设备的显示器能够处理的图像格式为YUV格式。Optionally, the image format that can be processed by the display of the electronic device is the YUV format.
其中，图像的亮度信号被称作Y，色度信号是由两个互相独立的信号组成，视颜色系统和格式不同，两种色度信号经常被称作U和V。在这种情况下，得到RAW格式的高动态范围图像之后，可以通过图像信号处理器(Image Signal Processing，ISP)对高动态范围图像进行格式转换，将RAW格式的高动态范围图像转换为YUV格式图像。由于显示器的显示界面尺寸有限，为了达到更好的预览效果，可以将转换得到的YUV格式图像压缩至预览尺寸以进行预览显示。本申请实施例中，上述得到的第一降噪图像的数量为一张，而得到的第二降噪图像与其它原始图像中所包含的原始图像的数量相对应。Among them, the luminance signal of the image is called Y, and the chrominance signal is composed of two independent signals. Depending on the color system and format, the two chrominance signals are often called U and V. In this case, after the high dynamic range image in RAW format is obtained, the high dynamic range image can be format-converted through Image Signal Processing (ISP), and the high dynamic range image in RAW format can be converted into YUV format image. Due to the limited size of the display interface of the monitor, in order to achieve a better preview effect, the converted YUV format image can be compressed to the preview size for preview display. In the embodiment of the present application, the number of obtained first noise reduction images is one, and the obtained second noise reduction images correspond to the number of original images included in other original images.
因此，本申请中，可以将第一降噪图像转换为第一YUV图像，并分别将每一张第二降噪图像均转换为第二YUV图像，得到多张第二YUV图像，使得在合成高动态范围图像时，进行合成的各种输入帧图像均已经过精准的降噪，在合成的时候可以有效保证各个亮度的合成不会有太大的噪声不连续现象，也就能更好的保护各个亮度图像细节。Therefore, in the present application, the first noise reduction image can be converted into a first YUV image, and each second noise reduction image can be converted into a second YUV image respectively, so as to obtain multiple second YUV images, so that in the synthesis In the case of high dynamic range images, the various input frame images for synthesis have been accurately denoised, which can effectively ensure that the synthesis of each brightness will not have too much noise discontinuity during synthesis, and it will be better. Protects image details of individual brightness.
本申请实施例中，将第一降噪图像转换为第一YUV图像，包括：根据部分帧原始图像对第一降噪图像进行细节增强处理；将所处理得到的第一降噪图像转换为第一YUV图像，能够保留原始多帧的EV0raw图，使用多帧的EV0raw图，对第一降噪图像进行细节增强处理，实现在融合的时候把人工智能降噪之前可能损失的图像细节再次叠加回来，有效地保障了图像的细节完整性。In the embodiment of the present application, converting the first noise-reduced image into the first YUV image includes: performing detail enhancement processing on the first noise-reduced image according to the original image of some frames; converting the processed first noise-reduced image into the first noise-reduced image. A YUV image can retain the original multi-frame EV0raw image, and use the multi-frame EV0raw image to perform detail enhancement processing on the first noise-reduced image, so as to superimpose the image details that may be lost before the artificial intelligence noise reduction during fusion. , which effectively guarantees the detail integrity of the image.
步骤104，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。Step 104 , synthesize a high dynamic range image according to the first YUV image and the second YUV image.
本申请实施例中，可以对第一YUV图像和第二YUV图像进行高动态合成，合成得到高动态范围图像。In the embodiment of the present application, the first YUV image and the second YUV image may be combined with high dynamic range to obtain a high dynamic range image.
其中，高动态范围图像(High-Dynamic Range，简称HDR)，相比普通的图像，可以提供更多的动态范围和图像细节。Among them, a high dynamic range image (High-Dynamic Range, HDR for short) can provide more dynamic range and image details than ordinary images.
本申请实施例中，可以确定第一YUV图像和第二YUV图像中，对应于不同曝光时间点的，最佳细节的低动态范围图像LDR(Low-Dynamic Range)图像，而后，根据最佳细节的低动态范围图像LDR(Low-Dynamic Range)图像合成高动态范围图像，能够更好的反映真实环境中的视觉效果。In this embodiment of the present application, it may be determined that, in the first YUV image and the second YUV image, corresponding to different exposure time points, the LDR (Low-Dynamic Range) image with the best detail is a Low-Dynamic Range (LDR) image with the best detail, and then, according to the best detail The low dynamic range image LDR (Low-Dynamic Range) image is synthesized into a high dynamic range image, which can better reflect the visual effect in the real environment.
需要说明的是，由于第一YUV图像和各帧第二YUV图像是在不同曝光情况下拍摄并降噪处理得到的，因此，第一YUV图像和各帧第二YUV图像中包含有不同亮度的画面信息。对于同一景物，第一YUV图像和各帧第二YUV图像中可能是过曝的，可能是欠曝的，还可能是恰当曝光的。将第一YUV图像和各帧第二YUV图像合成为高动态范围图像后，能够尽量使得合成的高动态范围图像中各景物恰当曝光，与实际场景也更加相近。It should be noted that, since the first YUV image and the second YUV images of each frame are obtained by shooting under different exposure conditions and noise reduction processing, the first YUV image and the second YUV image of each frame contain different brightness. screen information. For the same scene, the first YUV image and each frame of the second YUV image may be overexposed, may be underexposed, or may be properly exposed. After synthesizing the first YUV image and the second YUV images of each frame into a high dynamic range image, each scene in the synthesized high dynamic range image can be properly exposed as much as possible, and is closer to the actual scene.
本实施例中，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，相较于未进行人工智能的降噪处理，本申请能够在一定程度上有助于减少原始图像采集帧数，对于每一帧原始图像来说有助于增大采集时的感光度以减小拍摄时长，从而使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。另外，本申请中通过分别对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，是考虑到部分帧原始图像和其它帧原始图像的噪声特性并不完全相同，因此使得降噪更具有针对性，能够有效提升降噪效果。In this embodiment, by acquiring multiple frames of original images; denoising some frames of original images based on artificial intelligence to obtain a first denoised image, and denoising other frames of original images based on artificial intelligence to obtain a second denoised image, The first noise reduction image is converted into a first YUV image, and the second noise reduction image is converted into a second YUV image, and a high dynamic range image is synthesized according to the first YUV image and the second YUV image, which can be more accurately distinguished Compared with the noise reduction processing without artificial intelligence for the picture noise and effective details of high dynamic range images, this application can help reduce the number of original image acquisition frames to a certain extent, which is helpful for each frame of original image. In order to increase the sensitivity during acquisition to reduce the shooting time, the total time required for the overall shooting process is shortened, which avoids the situation that the shooting time is too long and causes the picture to be blurred, which is conducive to clear shooting of dynamic night scenes. In addition, in this application, the first noise-reduced image is obtained by denoising part of the original images of frames based on artificial intelligence, and the second de-noised image is obtained by denoising the original images of other frames based on artificial intelligence. The noise characteristics of the image and the original image of other frames are not exactly the same, so the noise reduction is more targeted and the noise reduction effect can be effectively improved.
为了获得较佳的人工智能的降噪效果，可以选用神经网络模型降噪，并采用各感光度的样本图像对该神经网络模型进行训练，以提高神经网络模型识别噪声特性的能力。In order to obtain a better noise reduction effect of artificial intelligence, a neural network model can be selected for noise reduction, and the neural network model is trained with sample images of each sensitivity, so as to improve the ability of the neural network model to identify noise characteristics.
参见图2，图2为本申请中的一种应用流程示意图。Referring to FIG. 2, FIG. 2 is a schematic diagram of an application flow in this application.
本申请实施例中的，神经网络模型包括：第一神经网络模型和第二神经网络模型，可以针对其中的一种神经网络模型，对其进行的具体的训练过程参见图3，针对另一种神经网络模型的训练过程类似，可以以此类推。如图3所示，图3为本申请实施例所提供的第二种基于多帧图像的图像处理方法的流程示意图，具体包括以下步骤：In the embodiment of the present application, the neural network model includes: a first neural network model and a second neural network model. For one of the neural network models, refer to FIG. 3 for the specific training process. The training process of a neural network model is similar, and so on. As shown in FIG. 3, FIG. 3 is a schematic flowchart of a second multi-frame image-based image processing method provided by an embodiment of the present application, which specifically includes the following steps:
步骤301，获取各感光度的样本图像。In step 301, sample images of each sensitivity are acquired.
其中，样本图像中已经标注了图像的噪声特性。Among them, the noise characteristics of the image have been marked in the sample image.
本申请实施例中，样本图像可以是在不同的环境亮度下，设置不同的感光度拍摄得到的图像。In this embodiment of the present application, the sample image may be an image obtained by setting different sensitivities under different ambient brightness.
也就是说，环境亮度应为多种，在每一种环境亮度下，分别在不同感光度情况下拍摄多帧图像，作为样本图像。That is to say, there should be a variety of ambient brightnesses, and under each ambient brightness, multiple frames of images are taken under different sensitivity conditions as sample images.
为了获得更佳准确的噪声特性识别结果，本申请实施例中还可以对环境亮度和ISO进行细分，增加样本图像的帧数，以使初始降噪图像输入第一神经网络模型后，该第一神经网络能准确的识别出初始降噪图像的统计特性。In order to obtain a better and more accurate noise characteristic recognition result, in this embodiment of the present application, the ambient brightness and ISO may also be subdivided, and the number of frames of the sample image may be increased, so that after the initial noise reduction image is input into the first neural network model, the A neural network can accurately identify the statistical properties of the initial denoised image.
步骤302，采用各感光度的样本图像对第一神经网络模型进行训练。Step 302, using the sample images of each sensitivity to train the first neural network model.
本申请实施例中，获取到不同环境光亮度下拍摄得到的各感光度的样本图像后，采用样本图像对第一神经网络模型进行训练。将样本图像中标注的统计特性作为模型训练的特性，将经过统计特性标注的样本图像输入第一神经网络模型，以对第一神经网络模型进行训练，进而识别出图像的统计特性。In the embodiment of the present application, after acquiring the sample images of each sensitivity photographed under different ambient light brightness, the first neural network model is trained by using the sample images. The statistical characteristics marked in the sample images are used as the characteristics of model training, and the sample images marked with the statistical characteristics are input into the first neural network model to train the first neural network model, thereby identifying the statistical characteristics of the images.
当然，神经网络模型仅仅是实现基于人工智能的降噪的一种可能的实现方式，在实际执行过程中，可以通过其他任意可能的方式来实现基于人工智能的降噪，比如，还可以采用传统的编程技术(比如模拟法和工程学方法)实现，又比如，还可以遗传学算法和人工神经网络的方法来实现。Of course, the neural network model is only a possible way to realize artificial intelligence-based noise reduction. In the actual execution process, artificial intelligence-based noise reduction can be achieved by any other possible means. For example, traditional It can be realized by programming techniques (such as simulation method and engineering method), for example, it can also be realized by the method of genetic algorithm and artificial neural network.
需要说明的是，在样本图像中标注统计特性对第一神经网络模型进行训练，是因为已标注的样本图像能够清楚的表示出图像的噪声位置和噪声类型，从而将标注的统计特性作为模型训练的特性，将初始降噪图像输入第一神经网络模型后，能够识别出图像中的统计特性。It should be noted that the first neural network model is trained by labeling statistical features in the sample images because the labeled sample images can clearly represent the noise location and noise type of the image, so that the labeled statistical features are used as model training. After inputting the initial denoised image into the first neural network model, the statistical characteristics in the image can be identified.
步骤303，直至第一神经网络模型识别出的噪声特性与相应样本图像中标注的噪声特性匹配时，第一神经网络模型训练完成。Step 303 , until the noise characteristic identified by the first neural network model matches the noise characteristic marked in the corresponding sample image, the training of the first neural network model is completed.
本申请实施例中，采用各感光度的样本图像对神经网络模型进行训练，直至第一神经网络模型识别出的噪声特性与相应样本图像中标注的统计特性匹配，In the embodiment of the present application, the neural network model is trained by using the sample images of each sensitivity, until the noise characteristics identified by the first neural network model match the statistical characteristics marked in the corresponding sample images,
本申请实施例中，通过获取各感光度的样本图像，采用各感光度的样本图像对第一神经网络模型进行训练，直至第一神经网络模型识别出的统计特性与相应样本图像中标注的统计特性匹配时，第一神经网络模型训练完成。由于，采用各感光度下经过标注统计特性的样本图像对第一神经网络模型进行训练，能够实现将初始降噪图像输入第一神经网络模型后，准确的识别出图像的统计特性，以实现对图像降噪处理，从而提高图像的拍摄质量。In the embodiment of the present application, by acquiring sample images of each sensitivity, the first neural network model is trained by using the sample images of each sensitivity, until the statistical characteristics identified by the first neural network model and the statistics marked in the corresponding sample images When the features are matched, the training of the first neural network model is completed. Because the first neural network model is trained by using the sample images marked with statistical characteristics under each sensitivity, it is possible to accurately identify the statistical characteristics of the image after inputting the initial noise reduction image into the first neural network model, so as to realize the accurate identification of the image. Image noise reduction processing, thereby improving the quality of the image.
在图1实施例的基础上，作为一种可能的实现方式，在步骤101中采集多帧原始图像时，可以根据预览图像的成像质量，确定基准曝光量的图像帧数n，以采集符合基准曝光量的n帧原始图像，并采集低于基准曝光量的至少一帧原始图像。下面结合图4对上述过程进行详细介绍，如图4所示，图4为本申请实施例所提供的第三种基于多帧图像的图像处理方法的流程示意图，步骤101还可以包括：On the basis of the embodiment in FIG. 1 , as a possible implementation manner, when collecting multiple frames of original images in step 101 , the number n of image frames with reference exposure can be determined according to the imaging quality of the preview image, so as to collect images that meet the reference n frames of original images of exposure, and at least one frame of original images that is lower than the reference exposure is collected. The above process will be described in detail below with reference to FIG. 4 . As shown in FIG. 4 , FIG. 4 is a schematic flowchart of a third multi-frame image-based image processing method provided by an embodiment of the present application. Step 101 may further include:
步骤401，根据预览图像的成像质量，确定基准曝光量的图像帧数n。Step 401 , according to the imaging quality of the preview image, determine the image frame number n of the reference exposure amount.
其中的预览图像是预先获取得到的，例如，可以是开启摄像头拍摄得到的预览图像，或者，也可以是从存储器中读取的，对此不作限制。The preview image is obtained in advance, for example, it may be a preview image obtained by turning on the camera, or it may be read from a memory, which is not limited.
其中，n为大于或等于2的自然数。Among them, n is a natural number greater than or equal to 2.
需要说明的是，采集的图像帧数较多时，整个拍摄时长会过长，在拍摄过程中可能会引入较多的噪声，因此本申请实施例中，图像帧数n的取值范围可以为3或4，以降低拍摄时长，获得较高质量的图像。It should be noted that when the number of image frames collected is large, the entire shooting time will be too long, and more noise may be introduced during the shooting process. Therefore, in the embodiment of the present application, the value range of the number of image frames n can be 3 or 4 to reduce shooting time and obtain higher quality images.
本申请实施例中，预览图像的成像质量可以例如采用信噪比和/或成像速度进行衡量，并且成像质量一般是与采集图像帧数为正向关系，即，成像质量越好，则可以采集越多帧的图像。In this embodiment of the present application, the imaging quality of the preview image can be measured by, for example, the signal-to-noise ratio and/or the imaging speed, and the imaging quality is generally positively related to the number of captured image frames, that is, the better the imaging quality, the better the imaging quality can be. more frames of images.
本申请实施例在具体执行的过程中，若基于脚架模式拍摄预览图像，则考虑到画面较稳定，则可采集较多帧数的预览图像进行后续合成，而基于手持模式拍摄预览图像，则由于不可避免的人手的抖动所造成的画面抖动，本申请实施例中为了避免高动态范围图像模糊，可以采集较少帧的预览图像进行后续的合成。In the specific implementation process of the embodiment of the present application, if the preview image is taken based on the tripod mode, considering that the picture is relatively stable, a preview image with a larger number of frames can be collected for subsequent synthesis, and the preview image is taken based on the handheld mode. Due to the unavoidable picture shaking caused by the shaking of the human hand, in the embodiment of the present application, in order to avoid blurring of the high dynamic range image, preview images with fewer frames may be collected for subsequent synthesis.
可以理解的是，采集的原始图像帧数越多，包含有不同的画面信息越多，在高动态合成时得到的高动态范围图像中包含有更多的画面信息，与实际场景也更加相近，因此成像质量与采集图像帧数为正向关系，进而可以根据预览图像的成像质量，确定基准曝光量的图像帧数n。It can be understood that the more original image frames collected, the more different picture information is contained, and the high dynamic range image obtained during high dynamic synthesis contains more picture information, which is closer to the actual scene. Therefore, there is a positive relationship between the imaging quality and the number of captured image frames, and then the number of image frames n of the reference exposure can be determined according to the imaging quality of the preview image.
步骤402，采集符合基准曝光量的n帧原始图像。Step 402 , collecting n frames of original images conforming to the reference exposure amount.
本申请实施例中，根据预览图像的成像质量，确定基准曝光量的图像帧数n后，进一步采集符合基准曝光量的n帧原始图像。In the embodiment of the present application, after determining the number of image frames n of the reference exposure amount according to the imaging quality of the preview image, n frames of original images conforming to the reference exposure amount are further collected.
在一种可能的场景下，可以基于拍摄场景的光照度确定的基准曝光量和设定的基准感光度，确定各帧待采集原始图像的基准曝光时长，以获得不同动态范围的图像，使得合成后的图像具有更高的动态范围，提高图像的整体亮度和质量。In a possible scenario, the reference exposure duration of each frame of the original image to be collected can be determined based on the reference exposure amount determined by the illumination of the shooting scene and the set reference sensitivity, so as to obtain images with different dynamic ranges, so that the synthesized images The image has a higher dynamic range, improving the overall brightness and quality of the image.
下面结合图5对上述过程进行详细介绍，图5为本申请实施例提供的第四种基于多帧图像的图像处理方法的流程示意图，如图5所示，步骤402还可以包括如下子步骤：The above process will be described in detail below with reference to FIG. 5 . FIG. 5 is a schematic flowchart of a fourth multi-frame image-based image processing method provided by an embodiment of the present application. As shown in FIG. 5 , step 402 may further include the following sub-steps:
子步骤4021，根据拍摄场景的光照度，确定基准曝光量。Sub-step 4021: Determine a reference exposure amount according to the illuminance of the shooting scene.
其中，曝光量，是指电子设备中的感光器件在曝光时长内接受到光的多少，曝光量与光圈、曝光时长和感光度有关。其中，光圈也就是通光口径，决定单位时间内光线通过的数量；曝光时长，是指光线通过镜头的时间；感光度，又称为ISO值，是衡量底片对于光的灵敏程度的指标，用于表示感光元件的感光速度，ISO数值越高就说明该感光元器件的感光能力越强。Among them, the exposure amount refers to the amount of light received by the photosensitive device in the electronic equipment during the exposure time, and the exposure amount is related to the aperture, exposure time and sensitivity. Among them, the aperture is the aperture of light, which determines the amount of light passing through the unit time; the exposure time refers to the time for the light to pass through the lens; the sensitivity, also known as the ISO value, is an indicator to measure the sensitivity of the film to light. In expressing the photosensitive speed of the photosensitive element, the higher the ISO value, the stronger the photosensitive ability of the photosensitive element.
其中，曝光量与曝光时长、感光度光圈相关，例如，可以是曝光时长和感光度乘积，相关技术中的基准曝光量，定义为曝光补偿等级为零，即EV0。The exposure amount is related to the exposure duration and the sensitivity aperture. For example, it can be the product of the exposure duration and the sensitivity. The reference exposure amount in the related art is defined as the exposure compensation level of zero, ie EV0.
具体地，通过图像传感器获取当前拍摄场景的预览图像，进一步的通过感光器件测量得到预览图像各区域的环境光亮度，进而根据预览图像的亮度信息，确定基准曝光量。其中，在光圈固定的情况下，基准曝光量具体可以包括基准曝光时长和基准感光度。Specifically, a preview image of the current shooting scene is acquired by an image sensor, and the ambient light brightness of each area of the preview image is further measured by a photosensitive device, and then the reference exposure amount is determined according to the brightness information of the preview image. Wherein, when the aperture is fixed, the reference exposure amount may specifically include a reference exposure duration and a reference sensitivity.
本申请实施例中，基准曝光量，是指通过对预览图像进行测光获取的当前拍摄场景的亮度信息后，确定的与当前环境的亮度信息相适应的曝光量，基准曝光量的取值可以是基准感光度与基准曝光时长之间的乘积。In the embodiment of the present application, the reference exposure refers to the exposure that is adapted to the brightness information of the current environment determined after the brightness information of the current shooting scene obtained by performing light metering on the preview image, and the value of the reference exposure can be is the product of the base sensitivity and the base exposure duration.
子步骤4022，根据预览图像的画面抖动程度，或者根据采集预览图像的图像传感器的抖动程度，设定基准感光度。In sub-step 4022, the reference sensitivity is set according to the degree of screen shake of the preview image, or according to the degree of shake of the image sensor that collects the preview image.
本申请实施例中，基准感光度，可以是根据预览图像的画面抖动程度，设定与当前的抖动程度相适应的感光度；也可以是根据采集预览图像的图像传感器当前的抖动程度，设定与当前的抖动程度相适应的感光度，在此不做限定。其中，基准感光度的取值范围可以为100ISO至200ISO。In this embodiment of the present application, the reference sensitivity may be set according to the degree of screen shake of the preview image to suit the current degree of shake; or may be set according to the current degree of shake of the image sensor that collects the preview image. The sensitivity suitable for the current degree of shaking is not limited here. Wherein, the value range of the reference sensitivity may be 100ISO to 200ISO.
可以理解的是，采集图像的感光度会影响到整体的拍摄时长，拍摄时长过长，可能会导致手持拍摄时图像传感器的抖动程度加剧，从而影响图像质量。因此，可以根据预览图像的画面抖动程度，或者根据采集预览的图像传感器的抖动程度，确定采集预览图像对应的基准感光度，以使得拍摄时长控制在合适的范围内。It is understandable that the sensitivity of the captured image will affect the overall shooting time. If the shooting time is too long, the shake of the image sensor may be aggravated during handheld shooting, thereby affecting the image quality. Therefore, the reference sensitivity corresponding to the captured preview image can be determined according to the degree of screen shake of the preview image, or according to the degree of shake of the image sensor that captures the preview, so that the shooting duration is controlled within an appropriate range.
本申请实施例中，为了确定抖动程度，可以根据电子设备中设置的位移传感器，采集位移信息，进而，根据采集到的电子设备的位移信息，确定预览图像的画面抖动程度或者采集预览图像的图像传感器的抖动程度。In this embodiment of the present application, in order to determine the degree of shaking, displacement information may be collected according to a displacement sensor set in the electronic device, and further, according to the collected displacement information of the electronic device, the degree of screen shaking of the preview image may be determined or the image of the preview image may be collected. The jitter of the sensor.
作为一种示例，可以通过获取电子设备当前的陀螺仪(Gyro-sensor)信息，确定电子设备当前的抖动程度，即采集预览图像的图像传感器的抖动程度。As an example, the current jitter degree of the electronic device, that is, the jitter degree of the image sensor that collects the preview image, can be determined by acquiring the current gyroscope (Gyro-sensor) information of the electronic device.
其中，陀螺仪又叫角速度传感器，可以测量物理量偏转、倾斜时的转动角速度。在电子设备中，陀螺仪可以很好的测量转动、偏转的动作，从而可以精确分析判断出使用者的实际动作。电子设备的陀螺仪信息(gyro信息)可以包括手机在三维空间中三个维度方向上的运动信息，三维空间的三个维度可以分别表示为X轴、Y轴、Z轴三个方向，其中，X轴、Y轴、Z轴为两两垂直关系。Among them, the gyroscope is also called the angular velocity sensor, which can measure the rotational angular velocity when the physical quantity is deflected and tilted. In electronic equipment, the gyroscope can measure the movement of rotation and deflection very well, so that the actual movement of the user can be accurately analyzed and judged. The gyroscope information (gyro information) of the electronic device may include the movement information of the mobile phone in the three-dimensional directions in the three-dimensional space, and the three dimensions of the three-dimensional space may be expressed as the three directions of the X axis, the Y axis, and the Z axis, wherein, The X-axis, Y-axis, and Z-axis are perpendicular to each other.
需要说明的是，可以根据电子设备当前的gyro信息，确定采集预览图像的图像传感器的抖动程度。电子设备在三个方向上的gyro运动的绝对值越大，则采集预览图像的图像传感器的抖动程度越大。It should be noted that, according to the current gyro information of the electronic device, the degree of shaking of the image sensor that collects the preview image can be determined. The greater the absolute value of the gyro motion of the electronic device in the three directions, the greater the degree of jitter of the image sensor that captures the preview image.
具体的，可以预设在三个方向上gyro运动的绝对值阈值，并根据获取到的当前在三个方向上的gyro运动的绝对值之和，与预设的阈值的关系，确定采集预览图像的图像传感器的当前的抖动程度。Specifically, the absolute value thresholds of gyro motion in three directions can be preset, and the acquired preview image is determined according to the relationship between the obtained sum of absolute values of gyro motion in three directions and the preset threshold value. The current shake level of the image sensor.
举例来说，假设预设的阈值为第一阈值A、第二阈值B、第三阈值C，且A<B<C，当前获取到的在三个方向上gyro运动的绝对值之和为S。若S<A，则确定采集预览图像的图像传感器的抖动程度为“无抖动”；若A<S<B，则可以确定采集预览图像的图像传感器的抖动程度为“轻微抖动”；若B<S<C，则可以确定采集预览图像的图像传感器的抖动程度为“小抖动”；若S>C，则可以确定采集预览图像的图像传感器的抖动程度为“大抖动”。For example, assuming that the preset thresholds are the first threshold A, the second threshold B, and the third threshold C, and A<B<C, the sum of the absolute values of the currently acquired gyro motion in the three directions is S . If S<A, the degree of shaking of the image sensor that collects the preview image is determined to be "no shake"; if A<S<B, the degree of shaking of the image sensor that collects the preview image can be determined to be "slight shake"; if B< If S<C, it can be determined that the degree of jitter of the image sensor that captures the preview image is "small jitter"; if S>C, it can be determined that the degree of jitter of the image sensor that captures the preview image is "large jitter".
需要说明的是，上述举例仅为示例性的，不能视为对本申请的限制。实际使用时，可以根据实际需要预设阈值的数量和各阈值的具体数值，以及根据gyro信息与各阈值的关系，预设gyro信息与采集预览图像的图像传感器抖动程度的映射关系。It should be noted that the above examples are only exemplary, and should not be regarded as limitations of the present application. In actual use, the number of thresholds and the specific value of each threshold can be preset according to actual needs, and the mapping relationship between gyro information and the image sensor jitter degree of the image sensor that collects the preview image can be preset according to the relationship between gyro information and each threshold.
具体的，若采集预览图像的图像传感器的抖动程度较小，则可以将每帧待采集图像对应的基准感光度可以适当压缩为较小的值，以有效抑制每帧图像的噪声、提高拍摄图像的质量；若采集预览图像的图像传感器的抖动程度较大，则可以将每帧待采集图像对应的基准感光度可以适当提高为较大的值，以缩短拍摄时长。Specifically, if the degree of shaking of the image sensor that collects the preview image is small, the reference sensitivity corresponding to each frame of the image to be collected can be appropriately compressed to a small value, so as to effectively suppress the noise of each frame of image and improve the captured image. If the image sensor that collects the preview image has a large degree of shaking, the reference sensitivity corresponding to each frame of the image to be collected can be appropriately increased to a larger value to shorten the shooting time.
举例来说，若确定采集预览图像的图像传感器的抖动程度为“无抖动”，则可以将基准感光度确定为较小的值，以尽量获得更高质量的图像，比如确定基准感光度为100；若确定采集预览图像的图像传感器的抖动程度为“轻微抖动”，则可以将基准感光度确定为较大的值，以降低拍摄时长，比如确定基准感光度为120；若确定采集预览图像的图像传感器的抖动程度为“小抖动”，则可以进一步增大基准感光度，以降低拍摄时长，比如确定基准感光度为180；若确定采集预览图像的图像传感器的抖动程度为“大抖动”，则可以确定当前的抖动程度过大，此时可以进一步增大基准感光度，以降低拍摄时长，比如确定基准感光度为200。For example, if it is determined that the degree of shaking of the image sensor that captures the preview image is "no shake", the reference sensitivity can be determined as a smaller value to obtain a higher quality image, for example, the reference sensitivity can be determined to be 100 ; If it is determined that the degree of shaking of the image sensor that captures the preview image is "slight shaking", the reference sensitivity can be determined as a larger value to reduce the shooting time, for example, the reference sensitivity is determined to be 120; If the shake degree of the image sensor is "small shake", the reference sensitivity can be further increased to reduce the shooting time. For example, the reference sensitivity is determined to be 180; if the shake degree of the image sensor that captures the preview image is determined to be "large shake", Then it can be determined that the current degree of shaking is too large, and at this time, the reference sensitivity can be further increased to reduce the shooting time, for example, the reference sensitivity is determined to be 200.
需要说明的是，上述举例仅为示例性的，不能视为对本申请的限制。实际使用时，当采集预览图像的图像传感器的抖动程度变化时，既可以改变基准感光度，以获得最优的方案。其中，采集预览图像的图像传感器的抖动程度与每帧待采集图像对应的基准感光度的映射关系，可以根据实际需要预设。It should be noted that, the above examples are only exemplary and should not be regarded as limitations of the present application. In actual use, when the degree of shaking of the image sensor that collects the preview image changes, the reference sensitivity can be changed to obtain the optimal solution. The mapping relationship between the degree of shaking of the image sensor that collects the preview image and the reference sensitivity corresponding to each frame of the image to be collected may be preset according to actual needs.
本申请实施例中，预览图像的画面抖动程度与采集预览图像的图像传感器的抖动程度呈正相关关系，根据预览图像的画面抖动程度，设定基准感光度的实现过程参见上述过程，在此不再赘述。In the embodiment of the present application, the degree of screen shaking of the preview image is positively correlated with the degree of shaking of the image sensor that collects the preview image. Repeat.
子步骤4023，根据基准曝光量和设定的基准感光度，确定基准曝光时长。Sub-step 4023: Determine the reference exposure duration according to the reference exposure amount and the set reference sensitivity.
本申请实施例中，基准曝光量，包括基准曝光时长和基准感光度，因此，在根据拍摄场景的光照度，确定基准曝光量，以及根据预览图像的画面抖动程度或者采集预览图像的图像传感器的抖动程度确定基准感光度后，即可根据基准曝光量及基准感光度，确定基准曝光时长。In this embodiment of the present application, the reference exposure includes the reference exposure duration and the reference sensitivity. Therefore, the reference exposure is determined according to the illuminance of the shooting scene, and the screen shake degree of the preview image or the shake of the image sensor that collects the preview image. After the reference sensitivity is determined, the reference exposure duration can be determined according to the reference exposure amount and the reference sensitivity.
子步骤4024，根据基准曝光时长和基准感光度，采集n帧原始图像。Sub-step 4024, collect n frames of original images according to the reference exposure duration and the reference sensitivity.
本申请实施例中，在确定各帧待采集原始图像的基准曝光时长和基准感光度后，根据各帧待采集原始图像的曝光时长和基准感光度控制图像传感器进行图像采集，在此不做具体赘述。In the embodiment of the present application, after determining the reference exposure duration and reference sensitivity of the original images to be collected for each frame, the image sensor is controlled to perform image collection according to the exposure duration and reference sensitivity of the original images to be collected for each frame, which is not specifically described here. Repeat.
步骤403，采集低于基准曝光量的至少一帧原始图像。Step 403: Collect at least one frame of original image that is lower than the reference exposure amount.
本申请实施例中，在采集低于基准曝光量的至少一帧原始图像时，可以根据设定的曝光补偿等级，对基准曝光时长进行补偿，得到短于基准曝光时长的补偿曝光时长；根据补偿曝光时长和基准感光度，采集至少一帧原始图像。In the embodiment of the present application, when collecting at least one frame of original image that is lower than the reference exposure amount, the reference exposure duration can be compensated according to the set exposure compensation level to obtain a compensated exposure duration shorter than the reference exposure duration; Exposure duration and benchmark sensitivity, and at least one frame of original image is collected.
可以理解为，通过曝光补偿等级，对至少一帧原始图像分别采取不同的曝光补偿策略，使得待采集图像对应于不同的曝光量，以获得具有不同动态范围的图像。It can be understood that, through the exposure compensation level, different exposure compensation strategies are respectively adopted for at least one frame of the original image, so that the images to be collected correspond to different exposure amounts, so as to obtain images with different dynamic ranges.
需要说明的是，在曝光量最初的定义中，曝光量并不是指一个准确的数值，而是指“能够给出相同的曝光量的所有相机光圈与曝光时长的组合”。感光度、光圈和曝光时长确定了相机的曝光量，不同的参数组合可以产生相等的曝光量。曝光补偿等级是对曝光量进行调整的参数，使得某些图像欠曝光，某些图像过曝光，还可以使得某些图像恰当曝光。本申请实施例中，至少一帧第二图像对应的曝光补偿等级取值范围为EV-5至EV-1。It should be noted that in the original definition of exposure, exposure does not refer to an exact value, but refers to "all combinations of camera apertures and exposure durations that can give the same exposure". Sensitivity, aperture and exposure duration determine the exposure of the camera, and different combinations of parameters can produce equal exposures. Exposure compensation level is a parameter that adjusts the exposure, so that some images are underexposed, some images are overexposed, and some images are properly exposed. In this embodiment of the present application, the exposure compensation level corresponding to at least one frame of the second image ranges from EV-5 to EV-1.
作为一种示例，采集低于基准曝光量的至少一帧原始图像，具体为两帧原始图像，该至少一帧原始图像可以被称为至少一帧第二图像，具体为两帧第二图像，两帧第二图像对应不同的曝光补偿等级，且两帧第二图像的曝光补偿等级小于EV0。As an example, collecting at least one frame of original images, specifically two frames of original images, which is lower than the reference exposure amount, the at least one frame of original images may be referred to as at least one frame of second images, specifically, two frames of second images, The two frames of the second images correspond to different exposure compensation levels, and the exposure compensation levels of the two frames of the second images are less than EV0.
具体地，根据设定的曝光补偿等级，对基准曝光时长进行补偿，得到短于基准曝光时长的补偿曝光时长；根据补偿曝光时长和基准感光度，采集两帧第二图像。Specifically, according to the set exposure compensation level, the reference exposure duration is compensated to obtain a compensation exposure duration shorter than the reference exposure duration; two frames of second images are collected according to the compensation exposure duration and the reference sensitivity.
本申请实施例中，通过根据预览图像的成像质量，确定基准曝光量的图像帧数n，采集符合基准曝光量的n帧原始图像，同时采集低于基准曝光量的至少一帧原始图像。由此，通过采集基准曝光量的n帧原始图像，同时采集低于基准曝光量的至少一帧原始图像，进而提高了图像的成像质量，得到清晰度较高的成像效果。In the embodiment of the present application, the number of image frames n of the reference exposure amount is determined according to the imaging quality of the preview image, n frames of original images that meet the reference exposure amount are collected, and at least one frame of original image that is lower than the reference exposure amount is collected at the same time. Therefore, by collecting n frames of original images of the reference exposure amount and simultaneously collecting at least one frame of original images lower than the reference exposure amount, the imaging quality of the image is improved, and the imaging effect with higher definition is obtained.
图6为本申请实施例提供的第一种基于多帧图像的图像处理装置的结构示意图。FIG. 6 is a schematic structural diagram of a first image processing apparatus based on a multi-frame image provided by an embodiment of the present application.
如图6所示，该基于多帧图像的图像处理装置600包括：获取模块610、降噪模块620、转换模块630以及合成模块640。As shown in FIG. 6 , the image processing apparatus 600 based on multi-frame images includes: an acquisition module 610 , a noise reduction module 620 , a conversion module 630 and a synthesis module 640 .
获取模块610，用于获取多帧原始图像；an acquisition module 610, configured to acquire multiple frames of original images;
降噪模块620，用于对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像；The noise reduction module 620 is configured to perform noise reduction based on artificial intelligence for part of the original image of the frame to obtain a first noise-reduced image, and based on artificial intelligence for the original image of other frames to obtain a second noise-reduced image, and the original image of part of the frame is multi- Original images of at least two frames of the original images;
转换模块630，用于将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像；a conversion module 630, configured to convert the first noise-reduced image into a first YUV image, and convert the second noise-reduced image into a second YUV image;
合成模块640，用于根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。The synthesis module 640 is configured to obtain a high dynamic range image by synthesis according to the first YUV image and the second YUV image.
可选地，一些实施例中，降噪模块620，具体用于：Optionally, in some embodiments, the noise reduction module 620 is specifically configured to:
对部分帧原始图像进行多帧融合降噪，得到初始降噪图像；Perform multi-frame fusion noise reduction on some frames of the original image to obtain the initial noise reduction image;
采用第一神经网络模型，对初始降噪图像进行噪声特性识别，并采用第二神经网络模型，对其它帧原始图像中的各帧原始图像进行噪声特性识别；其中，第一神经网络模型，已学习得到初始降噪图像的感光度与噪声特性之间的映射关系，第二神经网络模型，已学习得到各帧原始图像的感光度与噪声特性之间的映射关系；The first neural network model is used to identify the noise characteristics of the initial noise reduction image, and the second neural network model is used to identify the noise characteristics of each frame of the original image in other frames of the original image; wherein, the first neural network model has been Learning to obtain the mapping relationship between the sensitivity and noise characteristics of the initial noise reduction image, the second neural network model has learned the mapping relationship between the sensitivity and noise characteristics of each frame of the original image;
根据第一神经网络模型识别出的噪声特性，对初始降噪图像降噪，以得到第一降噪图像，并根据第二神经网络模型识别出的噪声特性，分别对各帧原始图像降噪，以得到多帧第二降噪图像。According to the noise characteristics identified by the first neural network model, the initial noise reduction image is denoised to obtain a first noise reduction image, and according to the noise characteristics identified by the second neural network model, the original images of each frame are denoised respectively, to obtain multiple frames of second noise-reduced images.
可选地，一些实施例中，参见图7，图7为本申请实施例提供的第二种基于多帧图像的图像处理装置的结构示意图，还包括：Optionally, in some embodiments, referring to FIG. 7 , FIG. 7 is a schematic structural diagram of a second multi-frame image-based image processing apparatus according to an embodiment of the present application, further comprising:
训练模块650，用于采用各感光度的样本图像对神经网络模型进行训练，直至神经网络模型识别出的噪声特性与相应样本图像中标注的噪声特性匹配时，神经网络模型训练完成，神经网络模型包括：第一神经网络模型和第二神经网络模型。The training module 650 is used for training the neural network model by using the sample images of each sensitivity, until the noise characteristic identified by the neural network model matches the noise characteristic marked in the corresponding sample image, the training of the neural network model is completed, and the neural network model Including: a first neural network model and a second neural network model.
可选地，一些实施例中，部分帧原始图像为至少两帧相同曝光量的第一图像，其它帧原始图像为曝光量低于第一图像的至少一帧第二图像；Optionally, in some embodiments, the original images of some frames are at least two frames of first images with the same exposure, and the original images of other frames are at least one frame of second images with a lower exposure than the first image;
转换模块630，具体用于：The conversion module 630 is specifically used for:
根据部分帧原始图像对第一降噪图像进行细节增强处理；Perform detail enhancement processing on the first noise-reduced image according to the original image of some frames;
将所处理得到的第一降噪图像转换为第一YUV图像。The processed first noise-reduced image is converted into a first YUV image.
可选地，一些实施例中，获取模块610，具体用于：Optionally, in some embodiments, the obtaining module 610 is specifically configured to:
获取预览图像；get a preview image;
根据预览图像的成像质量，确定基准曝光量的图像帧数n；其中，n为大于或等于2的自然数；According to the imaging quality of the preview image, determine the image frame number n of the reference exposure amount; wherein, n is a natural number greater than or equal to 2;
采集符合基准曝光量的n帧原始图像；Collect n frames of original images that meet the reference exposure;
采集低于基准曝光量的至少一帧原始图像。Acquire at least one frame of raw image below the reference exposure.
可选地，一些实施例中，获取模块610，具体用于：Optionally, in some embodiments, the obtaining module 610 is specifically configured to:
根据拍摄场景的光照度，确定基准曝光量；Determine the reference exposure according to the illumination of the shooting scene;
根据基准曝光量和设定的基准感光度，确定基准曝光时长；Determine the reference exposure duration according to the reference exposure amount and the set reference sensitivity;
根据基准曝光时长和基准感光度，采集n帧原始图像。According to the reference exposure time and reference sensitivity, n frames of original images are collected.
可选地，一些实施例中，至少一帧第二图像具体为两帧第二图像；Optionally, in some embodiments, the at least one frame of the second image is specifically two frames of the second image;
两帧第二图像对应不同的曝光补偿等级，且两帧第二图像的曝光补偿等级小于EV0。The two frames of the second images correspond to different exposure compensation levels, and the exposure compensation levels of the two frames of the second images are less than EV0.
可选地，一些实施例中，至少一帧第二图像对应的曝光补偿等级取值范围为EV-5至EV-1。Optionally, in some embodiments, the exposure compensation level corresponding to at least one frame of the second image ranges from EV-5 to EV-1.
需要说明的是，前述对基于多帧图像的图像处理方法实施例的解释说明也适用于该实施例的基于多帧图像的图像处理装置600，此处不再赘述。It should be noted that, the foregoing explanations on the embodiment of the image processing method based on multi-frame images are also applicable to the image processing apparatus 600 based on multi-frame images in this embodiment, and are not repeated here.
本实施例中，通过获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像，根据第一YUV图像和第二YUV图像，合成得到高动态范围图像，能够更加精确地区分出高动态范围图像的画面噪声和有效细节，相较于未进行人工智能的降噪处理，本申请能够在一定程度上有助于减少原始图像采集帧数，对于每一帧原始图像来说有助于增大采集时的感光度以减小拍摄时长，从而使得整体拍摄过程需要的总时长得到缩短，避免了拍摄时长过长导致画面模糊的情况，有利于清晰拍摄动态夜景。另外，本申请中通过分别对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，是考虑到部分帧原始图像和其它帧原始图像的噪声特性并不完全相同，因此使得降噪更具有针对性，能够有效提升降噪效果。In this embodiment, by acquiring multiple frames of original images; denoising some frames of original images based on artificial intelligence to obtain a first denoised image, and denoising other frames of original images based on artificial intelligence to obtain a second denoised image, The first noise reduction image is converted into a first YUV image, and the second noise reduction image is converted into a second YUV image, and a high dynamic range image is synthesized according to the first YUV image and the second YUV image, which can be more accurately distinguished Compared with the noise reduction processing without artificial intelligence for the picture noise and effective details of high dynamic range images, this application can help reduce the number of original image acquisition frames to a certain extent, which is helpful for each frame of original image. In order to increase the sensitivity during acquisition to reduce the shooting time, the total time required for the overall shooting process is shortened, which avoids the situation that the shooting time is too long and causes the picture to be blurred, which is conducive to clear shooting of dynamic night scenes. In addition, in this application, the first noise-reduced image is obtained by denoising part of the original images of frames based on artificial intelligence, and the second de-noised image is obtained by denoising the original images of other frames based on artificial intelligence. The noise characteristics of the image and the original image of other frames are not exactly the same, so the noise reduction is more targeted and the noise reduction effect can be effectively improved.
为了实现上述实施例，本申请还提出一种电子设备200，参见图8，图8为本申请实施例提供的一种电子设备的结构示意图，包括：图像传感器210、处理器220、存储器230及存储在存储器230上并可在处理器220上运行的计算机程序，图像传感器210与处理器220电连接，处理器220执行程序时，实现如上述实施例中的基于多帧图像的图像处理方法。In order to realize the above embodiments, the present application further proposes an electronic device 200. Referring to FIG. 8, FIG. 8 is a schematic structural diagram of an electronic device provided by an embodiment of the present application, including: an image sensor 210, a processor 220, a memory 230 and A computer program stored in the memory 230 and running on the processor 220, the image sensor 210 is electrically connected to the processor 220, and when the processor 220 executes the program, the image processing method based on multi-frame images in the above-mentioned embodiments is implemented.
作为一种可能的情况，处理器220可以包括：图像信号处理ISP处理器。As a possible situation, the processor 220 may include: an image signal processing ISP processor.
其中，ISP处理器，用于控制图像传感器获取多帧原始图像。Among them, the ISP processor is used to control the image sensor to obtain multiple frames of original images.
作为另一种可能的情况，处理器220还可以包括：与ISP处理器连接的图形处理器(Graphics Processing Unit，简称GPU)。As another possible situation, the processor 220 may further include: a graphics processor (Graphics Processing Unit, GPU for short) connected to the ISP processor.
其中，GPU，用于对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像。Among them, the GPU is used to denoise some original images of frames based on artificial intelligence to obtain a first denoised image, and denoise other frames of original images based on artificial intelligence to obtain a second denoised image, and some of the original images of frames are multiple frames Original images of at least two frames in the original image.
GPU，还用于对高动态范围图像进行编码处理。GPU, which is also used to encode high dynamic range images.
ISP处理器，还用于将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像；根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。The ISP processor is also used to convert the first noise reduction image into a first YUV image, and convert the second noise reduction image into a second YUV image; according to the first YUV image and the second YUV image, a high dynamic range is obtained by synthesis image.
作为一种示例，请参阅图9，在图8电子设备的基础上，图9中为本申请实施例提供的一种电子设备的原理示例图。电子设备200的存储器230包括非易失性存储器80、内存储器82和处理器220。存储器230中存储有计算机可读指令。计算机可读指令被存储器执行时，使得处理器230执行上述任一实施方式的基于多帧图像的图像处理方法。As an example, please refer to FIG. 9 . On the basis of the electronic device of FIG. 8 , FIG. 9 is a schematic diagram of an electronic device provided by an embodiment of the present application. Memory 230 of electronic device 200 includes non-volatile memory 80 , internal memory 82 and processor 220 . Computer readable instructions are stored in memory 230 . When the computer-readable instructions are executed by the memory, the processor 230 is made to execute the image processing method based on the multi-frame image of any one of the above embodiments.
如图9所示，该电子设备200包括通过系统总线81连接的处理器220、非易失性存储器80、内存储器82、显示屏83和输入装置84。其中，电子设备200的非易失性存储器80存储有操作系统和计算机可读指令。该计算机可读指令可被处理器220执行，以实现本申请实施方式的基于多帧图像的图像处理方法。该处理器220用于提供计算和控制能力，支撑整个电子设备200的运行。电子设备200的内存储器82为非易失性存储器80中的计算机可读指令的运行提供环境。电子设备200的显示屏83可以是液晶显示屏或者电子墨水显示屏等，输入装置84可以是显示屏83上覆盖的触摸层，也可以是电子设备200外壳上设置的按键、轨迹球或触控板，也可以是外接的键盘、触控板或鼠标等。该电子设备200可以是手机、平板电脑、笔记本电脑、个人数字助理或穿戴式设备(例如智能手环、智能手表、智能头盔、智能眼镜)等。本领域技术人员可以理解，图9中示出的结构，仅仅是与本申请方案相关的部分结构的示意图，并不构成对本申请方案所应用于其上的电子设备200的限定，具体的电子设备200可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。As shown in FIG. 9 , the electronic device 200 includes a processor 220 , a non-volatile memory 80 , an internal memory 82 , a display screen 83 and an input device 84 connected through a system bus 81 . The non-volatile memory 80 of the electronic device 200 stores an operating system and computer-readable instructions. The computer-readable instructions can be executed by the processor 220 to implement the image processing method based on the multi-frame images of the embodiments of the present application. The processor 220 is used to provide computing and control capabilities to support the operation of the entire electronic device 200 . Internal memory 82 of electronic device 200 provides an environment for the execution of computer-readable instructions in non-volatile memory 80 . The display screen 83 of the electronic device 200 may be a liquid crystal display screen or an electronic ink display screen, etc., and the input device 84 may be a touch layer covered on the display screen 83 , or a button, a trackball, or a touch panel provided on the housing of the electronic device 200 . It can also be an external keyboard, trackpad or mouse, etc. The electronic device 200 may be a mobile phone, a tablet computer, a notebook computer, a personal digital assistant, or a wearable device (eg, a smart bracelet, a smart watch, a smart helmet, and smart glasses). Those skilled in the art can understand that the structure shown in FIG. 9 is only a schematic diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device 200 to which the solution of the present application is applied. The specific electronic device 200 may include more or fewer components than shown, or combine certain components, or have a different arrangement of components.
为了实现上述实施例，本申请还提出一种图像处理电路，请参阅图10，图10为本申请实施例提供的一种图像处理电路的原理示意图，如图10所示，图像处理电路90包括图像信号处理ISP处理器91(ISP处理器91作为处理器220)和图形处理器GPU。In order to implement the above-mentioned embodiment, the present application also proposes an image processing circuit. Please refer to FIG. 10 . FIG. 10 is a schematic diagram of the principle of an image processing circuit provided by an embodiment of the present application. As shown in FIG. 10 , the image processing circuit 90 includes The image signal processes the ISP processor 91 (the ISP processor 91 serves as the processor 220) and the graphics processor GPU.
ISP处理器，与图像传感器电连接，用于控制图像传感器获取多帧原始图像；The ISP processor is electrically connected with the image sensor and is used to control the image sensor to obtain multiple frames of original images;
GPU，与ISP处理器电连接，用于对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像。The GPU is electrically connected to the ISP processor, and is used for denoising part of the original images of frames based on artificial intelligence to obtain a first denoised image, and denoising other frames of original images based on artificial intelligence to obtain a second denoised image, part of the frame The original image is an original image of at least two frames of the multiple frames of original images.
ISP处理器，还用于将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像；根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。The ISP processor is also used to convert the first noise reduction image into a first YUV image, and convert the second noise reduction image into a second YUV image; according to the first YUV image and the second YUV image, a high dynamic range is obtained by synthesis image.
摄像头93捕捉的图像数据首先由ISP处理器91处理，ISP处理器91对图像数据进行分析以捕捉可用于确定摄像头93的一个或多个控制参数的图像统计信息。摄像模组310可包括一个或多个透镜932和图像传感器934。图像传感器934可包括色彩滤镜阵列(如Bayer滤镜)，图像传感器934可获取每个成像像素捕捉的光强度和波长信息，并提供可由ISP处理器91处理的一组原始图像数据。传感器94(如陀螺仪)可基于传感器94接口类型把采集的图像处理的参数(如防抖参数)提供给ISP处理器91。传感器94接口可以为SMIA(StandardMobile Imaging Architecture，标准移动成像架构)接口、其它串行或并行照相机接口或上述接口的组合。Image data captured by camera 93 is first processed by ISP processor 91 , which analyzes the image data to capture image statistics that can be used to determine one or more control parameters of camera 93 . Camera module 310 may include one or more lenses 932 and image sensor 934 . Image sensor 934 , which may include an array of color filters (eg, Bayer filters), may obtain light intensity and wavelength information captured by each imaging pixel and provide a set of raw image data that may be processed by ISP processor 91 . The sensor 94 (eg, a gyroscope) may provide the acquired image processing parameters (eg, anti-shake parameters) to the ISP processor 91 based on the sensor 94 interface type. The sensor 94 interface may be an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.
此外，图像传感器934也可将原始图像数据发送给传感器94，传感器94可基于传感器94接口类型把原始图像数据提供给ISP处理器91，或者传感器94将原始图像数据存储到图像存储器95中。In addition, image sensor 934 may also send raw image data to sensor 94, which may provide raw image data to ISP processor 91 based on sensor 94 interface type, or sensor 94 may store raw image data in image memory 95.
ISP处理器91按多种格式逐个像素地处理原始图像数据。例如，每个图像像素可具有8、10、12或14比特的位深度，ISP处理器91可对原始图像数据进行一个或多个图像处理操作、收集关于图像数据的统计信息。其中，图像处理操作可按相同或不同的位深度精度进行。The ISP processor 91 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 91 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.
ISP处理器91还可从图像存储器95接收图像数据。例如，传感器94接口将原始图像数据发送给图像存储器95，图像存储器95中的原始图像数据再提供给ISP处理器91以供处理。图像存储器95可为存储器330、存储器330的一部分、存储设备、或电子设备内的独立的专用存储器，并可包括DMA(Direct Memory Access，直接存取存储器)特征。ISP processor 91 may also receive image data from image memory 95 . For example, the sensor 94 interface sends the raw image data to the image memory 95, and the raw image data in the image memory 95 is provided to the ISP processor 91 for processing. The image memory 95 may be the memory 330, a part of the memory 330, a storage device, or an independent dedicated memory within the electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.
当接收到来自图像传感器934接口或来自传感器94接口或来自图像存储器95的原始图像数据时，ISP处理器91可进行一个或多个图像处理操作，如时域滤波。处理后的图像数据可发送给图像存储器95，以便在被显示之前进行另外的处理。ISP处理器91从图像存储器95接收处理数据，并对处理数据进行原始域中以及RGB和YCbCr颜色空间中的图像数据处理。ISP处理器91处理后的图像数据可输出给显示器97(显示器97可包括显示屏83)，以供用户观看和/或由图形引擎或GPU进一步处理。When receiving raw image data from the image sensor 934 interface or from the sensor 94 interface or from the image memory 95, the ISP processor 91 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 95 for additional processing before being displayed. The ISP processor 91 receives the processed data from the image memory 95 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The image data processed by ISP processor 91 may be output to display 97 (which may include display screen 83) for viewing by a user and/or for further processing by a graphics engine or GPU.
此外，ISP处理器91的输出还可发送给图像存储器95，且显示器97可从图像存储器95读取图像数据。In addition, the output of the ISP processor 91 can also be sent to the image memory 95, and the display 97 can read the image data from the image memory 95.
在一个实施例中，图像存储器95可被配置为实现一个或多个帧缓冲器。此外，ISP处理器91的输出可发送给编码器/解码器96，以便编码/解码图像数据。编码的图像数据可被保存，并在显示于显示器97设备上之前解压缩。编码器/解码器96可由CPU或GPU或协处理器实现。In one embodiment, image memory 95 may be configured to implement one or more frame buffers. In addition, the output of ISP processor 91 may be sent to encoder/decoder 96 for encoding/decoding image data. The encoded image data can be saved and decompressed prior to display on the display 97 device. The encoder/decoder 96 may be implemented by a CPU or GPU or a co-processor.
ISP处理器91确定的统计数据可发送给控制逻辑器92单元。例如，统计数据可包括自动曝光、自动白平衡、自动聚焦、闪烁检测、黑电平补偿、透镜932阴影校正等图像传感器934统计信息。控制逻辑器92可包括执行一个或多个例程(如固件)的处理元件和/或微控制器，一个或多个例程可根据接收的统计数据，确定摄像头93的控制参数及ISP处理器91的控制参数。例如，摄像头93的控制参数可包括传感器94控制参数(例如增益、曝光控制的积分时间、防抖参数等)、照相机闪光控制参数、透镜932控制参数(例如聚焦或变焦用焦距)、或这些参数的组合。ISP控制参数可包括用于自动白平衡和颜色调整(例如，在RGB处理期间)的增益水平和色彩校正矩阵，以及透镜932阴影校正参数。Statistics determined by the ISP processor 91 may be sent to the control logic 92 unit. For example, the statistics may include image sensor 934 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 932 shading correction, and the like. Control logic 92 may include a processing element and/or a microcontroller that executes one or more routines (eg, firmware) that may determine control parameters for camera 93 and an ISP processor based on received statistics 91 control parameters. For example, camera 93 control parameters may include sensor 94 control parameters (eg, gain, integration time for exposure control, stabilization parameters, etc.), camera flash control parameters, lens 932 control parameters (eg, focal length for focusing or zooming), or these parameters The combination. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 932 shading correction parameters.
以下为运用图9中图像处理技术实现基于多帧图像的图像处理方法的步骤：ISP处理器控制图像传感器获取多帧原始图像；GPU对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像，ISP处理器，还用于将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像；根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。The following are the steps of using the image processing technology in Fig. 9 to realize the image processing method based on multiple frames of images: the ISP processor controls the image sensor to obtain multiple frames of original images; image, and denoise other frames of original images based on artificial intelligence to obtain a second noise-reduced image, the original images of some frames are original images of at least two frames of the original images of multiple frames, the ISP processor is also used for denoising the first de-noised image. The noisy image is converted into a first YUV image, and the second noise-reduced image is converted into a second YUV image; and a high dynamic range image is synthesized according to the first YUV image and the second YUV image.
为了实现上述实施例，本申请实施例还提供了一种存储介质，当存储介质中的指令由处理器执行时，使得处理器执行以下步骤：获取多帧原始图像；对部分帧原始图像基于人工智能降噪，得到第一降噪图像，并对其它帧原始图像基于人工智能降噪，得到第二降噪图像，部分帧原始图像为多帧原始图像中的至少两帧的原始图像；将第一降噪图像转换为第一YUV图像，并将第二降噪图像转换为第二YUV图像；根据第一YUV图像和第二YUV图像，合成得到高动态范围图像。In order to implement the above embodiments, the embodiments of the present application also provide a storage medium, when the instructions in the storage medium are executed by the processor, the processor is caused to perform the following steps: acquiring multiple frames of original images; Intelligent noise reduction, obtaining a first noise reduction image, and denoising other frames of original images based on artificial intelligence to obtain a second noise reduction image, and some frame original images are the original images of at least two frames of multiple frames of original images; A noise-reduced image is converted into a first YUV image, and a second noise-reduced image is converted into a second YUV image; and a high dynamic range image is synthesized according to the first YUV image and the second YUV image.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)等。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium , when the program is executed, it may include the flow of the above-mentioned method embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or the like.
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910280172.1ACN110072052B (en) | 2019-04-09 | 2019-04-09 | Image processing method and device based on multi-frame image and electronic equipment |
| PCT/CN2020/081471WO2020207262A1 (en) | 2019-04-09 | 2020-03-26 | Image processing method and apparatus based on multiple frames of images, and electronic device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910280172.1ACN110072052B (en) | 2019-04-09 | 2019-04-09 | Image processing method and device based on multi-frame image and electronic equipment |
| Publication Number | Publication Date |
|---|---|
| CN110072052Atrue CN110072052A (en) | 2019-07-30 |
| CN110072052B CN110072052B (en) | 2021-08-27 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910280172.1AExpired - Fee RelatedCN110072052B (en) | 2019-04-09 | 2019-04-09 | Image processing method and device based on multi-frame image and electronic equipment |
| Country | Link |
|---|---|
| CN (1) | CN110072052B (en) |
| WO (1) | WO2020207262A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020207262A1 (en)* | 2019-04-09 | 2020-10-15 | Oppo广东移动通信有限公司 | Image processing method and apparatus based on multiple frames of images, and electronic device |
| CN111885312A (en)* | 2020-07-27 | 2020-11-03 | 展讯通信（上海）有限公司 | HDR image imaging method, system, electronic device and storage medium |
| CN112288646A (en)* | 2020-01-07 | 2021-01-29 | 影石创新科技股份有限公司 | Stacked noise reduction method, device, electronic device and storage medium |
| CN112529775A (en)* | 2019-09-18 | 2021-03-19 | 华为技术有限公司 | Image processing method and device |
| WO2021082580A1 (en)* | 2019-10-31 | 2021-05-06 | 北京迈格威科技有限公司 | Night scene high dynamic range image generation method, device, and electronic apparatus |
| CN112950503A (en)* | 2021-02-26 | 2021-06-11 | 北京小米松果电子有限公司 | Training sample generation method and device and truth value image generation method and device |
| CN113744120A (en)* | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and image processing method |
| CN113744119A (en)* | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Multimedia processing chip and electronic equipment |
| CN114187192A (en)* | 2021-11-22 | 2022-03-15 | 南通智能感知研究院 | Image processing method based on multi-exposure fusion |
| CN114511112A (en)* | 2022-01-24 | 2022-05-17 | 北京通建泰利特智能系统工程技术有限公司 | Intelligent operation and maintenance method and system based on Internet of things and readable storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112669241B (en)* | 2021-01-29 | 2023-11-14 | 成都国科微电子有限公司 | Image processing method, device, equipment and medium |
| CN112950501B (en)* | 2021-02-26 | 2023-10-13 | 平安科技（深圳）有限公司 | Noise field-based image noise reduction method, device, equipment and storage medium |
| CN113674232B (en)* | 2021-08-12 | 2024-09-03 | Oppo广东移动通信有限公司 | Image noise estimation method, device, electronic equipment and storage medium |
| CN115460343B (en)* | 2022-07-31 | 2023-06-13 | 荣耀终端有限公司 | Image processing method, device and storage medium |
| CN115665562A (en)* | 2022-10-24 | 2023-01-31 | 维沃移动通信有限公司 | Image processing method, circuit, device and medium |
| CN116017172A (en)* | 2022-12-26 | 2023-04-25 | 展讯通信（上海）有限公司 | Raw domain image noise reduction method and device, camera and terminal |
| CN116245962B (en)* | 2023-03-16 | 2023-12-22 | 新疆量子通信技术有限公司 | Data extraction system and method for wireless transmission to blockchain server |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103051841A (en)* | 2013-01-05 | 2013-04-17 | 北京小米科技有限责任公司 | Method and device for controlling exposure time |
| US20130335596A1 (en)* | 2012-06-15 | 2013-12-19 | Microsoft Corporation | Combining multiple images in bracketed photography |
| CN107566739A (en)* | 2017-10-18 | 2018-01-09 | 维沃移动通信有限公司 | A kind of photographic method and mobile terminal |
| CN107635098A (en)* | 2017-10-30 | 2018-01-26 | 广东欧珀移动通信有限公司 | High dynamic range image noise removal method, device and equipment |
| CN107864341A (en)* | 2017-12-29 | 2018-03-30 | Tcl移动通信科技（宁波）有限公司 | One kind drop frame per second photographic method, mobile terminal and storage medium |
| CN108012080A (en)* | 2017-12-04 | 2018-05-08 | 广东欧珀移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN108280811A (en)* | 2018-01-23 | 2018-07-13 | 哈尔滨工业大学深圳研究生院 | A kind of image de-noising method and system based on neural network |
| CN108322646A (en)* | 2018-01-31 | 2018-07-24 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN108737750A (en)* | 2018-06-07 | 2018-11-02 | 北京旷视科技有限公司 | Image processing method, device and electronic equipment |
| CN108900782A (en)* | 2018-08-22 | 2018-11-27 | Oppo广东移动通信有限公司 | Exposal control method, device and electronic equipment |
| CN108924420A (en)* | 2018-07-10 | 2018-11-30 | Oppo广东移动通信有限公司 | Image shooting method, image shooting device, image shooting medium, electronic equipment and model training method |
| CN108989700A (en)* | 2018-08-13 | 2018-12-11 | Oppo广东移动通信有限公司 | Image formation control method, device, electronic equipment and computer readable storage medium |
| CN109005366A (en)* | 2018-08-22 | 2018-12-14 | Oppo广东移动通信有限公司 | Camera module night scene image pickup processing method, device, electronic equipment and storage medium |
| CN109040603A (en)* | 2018-10-15 | 2018-12-18 | Oppo广东移动通信有限公司 | High dynamic range image acquisition method, device and mobile terminal |
| CN109089046A (en)* | 2018-09-25 | 2018-12-25 | Oppo广东移动通信有限公司 | Image noise reduction method and device, computer readable storage medium and electronic equipment |
| CN109194882A (en)* | 2018-08-22 | 2019-01-11 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN109218628A (en)* | 2018-09-20 | 2019-01-15 | Oppo广东移动通信有限公司 | Image processing method, device, electronic device and storage medium |
| CN109360163A (en)* | 2018-09-26 | 2019-02-19 | 深圳积木易搭科技技术有限公司 | A kind of fusion method and emerging system of high dynamic range images |
| CN109361853A (en)* | 2018-10-22 | 2019-02-19 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9633426B2 (en)* | 2014-05-30 | 2017-04-25 | General Electric Company | Remote visual inspection image capture system and method |
| US8346008B2 (en)* | 2009-06-09 | 2013-01-01 | Aptina Imaging Corporation | Systems and methods for noise reduction in high dynamic range imaging |
| CN105827971B (en)* | 2016-03-31 | 2019-01-11 | 维沃移动通信有限公司 | A kind of image processing method and mobile terminal |
| CN109218613B (en)* | 2018-09-18 | 2020-08-14 | Oppo广东移动通信有限公司 | High dynamic range image synthesis method and device, terminal equipment and storage medium |
| CN110072052B (en)* | 2019-04-09 | 2021-08-27 | Oppo广东移动通信有限公司 | Image processing method and device based on multi-frame image and electronic equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130335596A1 (en)* | 2012-06-15 | 2013-12-19 | Microsoft Corporation | Combining multiple images in bracketed photography |
| CN103051841A (en)* | 2013-01-05 | 2013-04-17 | 北京小米科技有限责任公司 | Method and device for controlling exposure time |
| CN107566739A (en)* | 2017-10-18 | 2018-01-09 | 维沃移动通信有限公司 | A kind of photographic method and mobile terminal |
| CN107635098A (en)* | 2017-10-30 | 2018-01-26 | 广东欧珀移动通信有限公司 | High dynamic range image noise removal method, device and equipment |
| CN108012080A (en)* | 2017-12-04 | 2018-05-08 | 广东欧珀移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN107864341A (en)* | 2017-12-29 | 2018-03-30 | Tcl移动通信科技（宁波）有限公司 | One kind drop frame per second photographic method, mobile terminal and storage medium |
| CN108280811A (en)* | 2018-01-23 | 2018-07-13 | 哈尔滨工业大学深圳研究生院 | A kind of image de-noising method and system based on neural network |
| CN108322646A (en)* | 2018-01-31 | 2018-07-24 | 广东欧珀移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN108737750A (en)* | 2018-06-07 | 2018-11-02 | 北京旷视科技有限公司 | Image processing method, device and electronic equipment |
| CN108924420A (en)* | 2018-07-10 | 2018-11-30 | Oppo广东移动通信有限公司 | Image shooting method, image shooting device, image shooting medium, electronic equipment and model training method |
| CN108989700A (en)* | 2018-08-13 | 2018-12-11 | Oppo广东移动通信有限公司 | Image formation control method, device, electronic equipment and computer readable storage medium |
| CN108900782A (en)* | 2018-08-22 | 2018-11-27 | Oppo广东移动通信有限公司 | Exposal control method, device and electronic equipment |
| CN109005366A (en)* | 2018-08-22 | 2018-12-14 | Oppo广东移动通信有限公司 | Camera module night scene image pickup processing method, device, electronic equipment and storage medium |
| CN109194882A (en)* | 2018-08-22 | 2019-01-11 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN109218628A (en)* | 2018-09-20 | 2019-01-15 | Oppo广东移动通信有限公司 | Image processing method, device, electronic device and storage medium |
| CN109089046A (en)* | 2018-09-25 | 2018-12-25 | Oppo广东移动通信有限公司 | Image noise reduction method and device, computer readable storage medium and electronic equipment |
| CN109360163A (en)* | 2018-09-26 | 2019-02-19 | 深圳积木易搭科技技术有限公司 | A kind of fusion method and emerging system of high dynamic range images |
| CN109040603A (en)* | 2018-10-15 | 2018-12-18 | Oppo广东移动通信有限公司 | High dynamic range image acquisition method, device and mobile terminal |
| CN109361853A (en)* | 2018-10-22 | 2019-02-19 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020207262A1 (en)* | 2019-04-09 | 2020-10-15 | Oppo广东移动通信有限公司 | Image processing method and apparatus based on multiple frames of images, and electronic device |
| CN112529775A (en)* | 2019-09-18 | 2021-03-19 | 华为技术有限公司 | Image processing method and device |
| WO2021082580A1 (en)* | 2019-10-31 | 2021-05-06 | 北京迈格威科技有限公司 | Night scene high dynamic range image generation method, device, and electronic apparatus |
| CN112288646A (en)* | 2020-01-07 | 2021-01-29 | 影石创新科技股份有限公司 | Stacked noise reduction method, device, electronic device and storage medium |
| CN113744119A (en)* | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Multimedia processing chip and electronic equipment |
| CN113744120B (en)* | 2020-05-29 | 2025-01-28 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and image processing method |
| CN113744119B (en)* | 2020-05-29 | 2025-01-21 | Oppo广东移动通信有限公司 | Multimedia processing chips and electronic devices |
| CN113744120A (en)* | 2020-05-29 | 2021-12-03 | Oppo广东移动通信有限公司 | Multimedia processing chip, electronic device and image processing method |
| CN111885312B (en)* | 2020-07-27 | 2021-07-09 | 展讯通信（上海）有限公司 | HDR image imaging method, system, electronic device and storage medium |
| CN111885312A (en)* | 2020-07-27 | 2020-11-03 | 展讯通信（上海）有限公司 | HDR image imaging method, system, electronic device and storage medium |
| CN112950503A (en)* | 2021-02-26 | 2021-06-11 | 北京小米松果电子有限公司 | Training sample generation method and device and truth value image generation method and device |
| CN114187192A (en)* | 2021-11-22 | 2022-03-15 | 南通智能感知研究院 | Image processing method based on multi-exposure fusion |
| CN114511112A (en)* | 2022-01-24 | 2022-05-17 | 北京通建泰利特智能系统工程技术有限公司 | Intelligent operation and maintenance method and system based on Internet of things and readable storage medium |
| Publication number | Publication date |
|---|---|
| WO2020207262A1 (en) | 2020-10-15 |
| CN110072052B (en) | 2021-08-27 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110072051B (en) | Image processing method and device based on multi-frame images | |
| CN110072052B (en) | Image processing method and device based on multi-frame image and electronic equipment | |
| CN110062160B (en) | Image processing method and device | |
| CN110290289B (en) | Image noise reduction method, device, electronic device and storage medium | |
| CN110166708B (en) | Night scene image processing method and device, electronic equipment and storage medium | |
| CN110248106B (en) | Image noise reduction method, device, electronic device and storage medium | |
| CN109040609B (en) | Exposure control method, exposure control device, electronic equipment and computer-readable storage medium | |
| CN109068067B (en) | Exposure control method and device and electronic equipment | |
| CN108900782B (en) | Exposure control method, device, and electronic device | |
| CN109788207B (en) | Image synthesis method and device, electronic equipment and readable storage medium | |
| CN110191291B (en) | Image processing method and device based on multi-frame images | |
| CN109194882B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN110166707B (en) | Image processing method, device, electronic device, and storage medium | |
| CN110062159A (en) | Image processing method and device based on multi-frame image and electronic equipment | |
| JP2021530911A (en) | Night view photography methods, devices, electronic devices and storage media | |
| CN109672819B (en) | Image processing method, apparatus, electronic device, and computer-readable storage medium | |
| CN110264420B (en) | Image processing method and device based on multi-frame images | |
| CN109005364A (en) | Image formation control method, device, electronic equipment and computer readable storage medium | |
| WO2020034737A1 (en) | Imaging control method, apparatus, electronic device, and computer-readable storage medium | |
| CN110166709B (en) | Night scene image processing method and device, electronic equipment and storage medium | |
| CN109151333B (en) | Exposure control method, exposure control device and electronic equipment | |
| CN110166706B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| CN110213462B (en) | Image processing method, image processing device, electronic apparatus, image processing circuit, and storage medium | |
| CN110166711B (en) | Image processing method, device, electronic device, and storage medium | |
| CN110276730B (en) | Image processing method and device and electronic equipment |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20210827 | |
| CF01 | Termination of patent right due to non-payment of annual fee |