技术领域Technical field
本申请涉及图像处理技术领域，具体涉及一种运动轨迹图像生成方法、装置、存储介质及电子设备。The present application relates to the field of image processing technology, and specifically to a motion trajectory image generation method, device, storage medium and electronic equipment.
背景技术Background technique
运动轨迹图像是一种将多张图像中的运动主体合成到一张图像上而获得的一张体现主体运动轨迹的轨迹图像，但现有技术中，针对快速运动主体的生成运动轨迹图像存在较大局限性，具体如下：The motion trajectory image is a trajectory image that embodies the subject's motion trajectory obtained by synthesizing the moving subjects in multiple images into one image. However, in the existing technology, there are problems in generating motion trajectory images for fast-moving subjects. Big limitations, as follows:
(1)主体快速运动会引起运动模糊问题，导致合成后的轨迹照片中存在明显的主体运动失真，主体轨迹模糊等问题，影响轨迹照片质量；(1) The rapid movement of the subject will cause motion blur problems, resulting in obvious subject motion distortion and subject trajectory blur in the synthesized trajectory photos, which will affect the quality of the trajectory photos;
(2)采集的图像通常对于大范围移动主体，受限于合成方式限制的较小视场，难以捕捉完整运动过程，生成的运动轨迹图像中运动轨迹完整性较差。(2) The collected images are usually for large-scale moving subjects, and are limited by the small field of view limited by the synthesis method. It is difficult to capture the complete motion process, and the motion trajectory integrity of the generated motion trajectory images is poor.
因此，目前的运动轨迹图像生成方式中，存在多数运动场景下生成的运动轨迹图像的图像质量和轨迹完整性较差的问题，用户体验较差。Therefore, in the current motion trajectory image generation method, there is a problem that the image quality and trajectory integrity of the motion trajectory images generated in most motion scenes are poor, resulting in poor user experience.
发明内容Contents of the invention
本申请实施例提供一种运动轨迹图像生成方案，可以有效提升多数运动场景下生成的运动轨迹图像的图像质量和轨迹完整性，提升用户体验。Embodiments of the present application provide a motion trajectory image generation solution, which can effectively improve the image quality and trajectory integrity of motion trajectory images generated in most sports scenarios, and improve user experience.
本申请实施例提供以下技术方案：The embodiments of this application provide the following technical solutions:
根据本申请的一个实施例，一种运动轨迹图像生成方法，其包括：获取多张候选合成图像；基于每张所述候选合成图像中的非主体背景，将所述多张候选合成图像对齐至统一坐标系，得到每张所述候选合成图像的主体前景和非主体背景在所述统一坐标系下的对齐位置信息；根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型；将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，所述多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。According to an embodiment of the present application, a method for generating motion trajectory images includes: acquiring multiple candidate composite images; based on the non-subject background in each of the candidate composite images, aligning the multiple candidate composite images to Unify the coordinate system to obtain the alignment position information of the subject foreground and non-subject background of each candidate composite image under the unified coordinate system; according to the motion parameters, foreground background information and the corresponding motion parameters of each candidate composite image. Align the position information to determine the area types of different image areas under the unified coordinate system; align the area images in each of the image areas, and perform noise reduction according to the noise reduction method corresponding to the area type of each of the image areas, to obtain A plurality of background noise-reduced images and a plurality of foreground noise-reduced images are used to synthesize the subject's motion trajectory image.
在本申请的一些实施例中，所述根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型，包括：根据每张所述候选合成图像对应的所述运动参数、前景背景信息及所述对齐位置信息，得到对齐至各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型；根据各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，得到各图像区域的区域类型。In some embodiments of the present application, determining the area types of different image areas under the unified coordinate system based on the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image includes: According to the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, obtain the motion and static types, front and rear background types and overlapping and non-overlapping types corresponding to the regional images aligned to each of the image areas; The area type of each image area is obtained according to the combination of the motion and stillness types, the front and rear background types and the overlapping and non-overlapping types corresponding to the area images in each of the image areas.
在本申请的一些实施例中，所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，包括：将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像；根据所述融合降噪区域图像及所述单帧降噪区域图像，得到所述多张背景降噪图像和多张前景降噪图像。In some embodiments of the present application, the area images aligned to each of the image areas are denoised according to the noise reduction method corresponding to the area type of each of the image areas, and multiple background noise reduction images are obtained. Multiple foreground denoising images include: denoising regional images in overlapping image areas through multi-frame fusion denoising methods to obtain fused denoising area images; denoising regional images in non-overlapping image areas, Noise reduction is performed through a single frame fusion noise reduction method to obtain a single frame noise reduction area image; according to the fusion noise reduction area image and the single frame noise reduction area image, the multiple background noise reduction images and multiple foreground images are obtained Noise reduction images.
在本申请的一些实施例中，所述获取多张候选合成图像，包括：获取多张待处理图像；获取各所述待处理图像对应的对焦状态、主体位置及曝光状态中至少一种，得到筛选参数；根据各所述待处理图像对应的筛选参数，从所述多张待处理图像中筛选出符合预定条件的待处理图像，得到所述多张候选合成图像，其中，所述预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种。In some embodiments of the present application, obtaining multiple candidate composite images includes: obtaining multiple images to be processed; obtaining at least one of the focus state, subject position and exposure state corresponding to each of the images to be processed, to obtain Screening parameters; according to the screening parameters corresponding to each of the images to be processed, select the images to be processed that meet predetermined conditions from the multiple images to be processed, and obtain the multiple candidate composite images, wherein the predetermined conditions include Not in the process of focusing, not in the process of exposure convergence, or meeting the subject sharpness requirements.
在本申请的一些实施例中，所述多张待处理图像为根据预定曝光参数采集的；所述方法还包括：获取各所述待处理图像对应的运动参数及允许位移限值；根据所述运动参数及允许位移限值，确定允许曝光时长限值；根据场景亮度获取小于所述允许曝光时长限值的预定优选曝光时长；基于所述预定优选曝光时长更新所述预定曝光参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined exposure parameters; the method further includes: obtaining the motion parameters and allowable displacement limits corresponding to each of the images to be processed; according to the Motion parameters and allowed displacement limits are used to determine the allowed exposure duration limit; a predetermined preferred exposure duration smaller than the allowed exposure duration limit is obtained based on scene brightness; and the predetermined exposure parameters are updated based on the predetermined preferred exposure duration.
在本申请的一些实施例中，所述多张待处理图像为根据预定对焦参数采集的；所述方法还包括：根据各所述待处理图像对应的主体位置计算对焦区域；根据所述对焦区域更新所述预定对焦参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined focus parameters; the method further includes: calculating a focus area according to the subject position corresponding to each of the images to be processed; Update the predetermined focus parameters.
在本申请的一些实施例中，在所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像之后，所述方法还包括：根据所述对齐位置信息，在所述统一坐标系下将所述多张背景降噪图像拼接为的整体背景图像；以及，将所述多张前景降噪图像与所述整体背景图像融合，得到融合轨迹图像；根据所述融合轨迹图像，得到主体运动轨迹图像。In some embodiments of the present application, on the regional images that will be aligned to each of the image regions, noise reduction is performed according to the noise reduction method corresponding to the region type of each of the image regions to obtain multiple background noise reduction images. and multiple foreground noise-reduced images, the method further includes: splicing the multiple background noise-reduced images into an overall background image under the unified coordinate system according to the alignment position information; and, Multiple foreground noise reduction images are fused with the overall background image to obtain a fused trajectory image; based on the fused trajectory image, a subject motion trajectory image is obtained.
根据本申请的一个实施例，一种运动轨迹图像生成装置，所述装置包括：图像获取单元，用于获取多张候选合成图像；图像对齐单元，用于基于每张所述候选合成图像中的非主体背景，将所述多张候选合成图像对齐至统一坐标系，得到每张所述候选合成图像的主体前景和非主体背景在所述统一坐标系下的对齐位置信息；类型划分单元，用于根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型；图像降噪单元，用于将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，所述多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。According to an embodiment of the present application, a device for generating motion trajectory images, the device includes: an image acquisition unit, used to acquire a plurality of candidate synthetic images; an image alignment unit, used to base on each of the candidate synthetic images Non-subject background, align the plurality of candidate composite images to a unified coordinate system, and obtain the alignment position information of the subject foreground and non-subject background of each candidate composite image in the unified coordinate system; the type division unit is used Determine the area types of different image areas under the unified coordinate system based on the motion parameters, foreground background information and alignment position information corresponding to each candidate synthetic image; an image noise reduction unit is used to align the areas to each The regional images in the image areas are denoised according to the noise reduction methods corresponding to the area types of each of the image areas, and multiple background denoised images and multiple foreground denoised images are obtained. The multiple background denoised images and Multiple foreground noise-reduced images are used to synthesize the subject's motion trajectory image.
在本申请的一些实施例中，所述类型划分单元，用于：根据每张所述候选合成图像对应的所述运动参数、前景背景信息及所述对齐位置信息，得到对齐至各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型；根据各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，得到各图像区域的区域类型。In some embodiments of the present application, the type classification unit is used to: obtain the alignment to each of the images based on the motion parameters, foreground background information and alignment position information corresponding to each of the candidate composite images. The motion and stillness types, the front and rear background types and the overlapping and non-overlapping types corresponding to the regional images in the region; according to the combination of the motion and stillness types, the foreground and background types and the overlapping and non-overlapping types corresponding to the regional images in each of the image regions, the region type of each image region is obtained .
在本申请的一些实施例中，所述图像降噪单元，用于：将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像；根据所述融合降噪区域图像及所述单帧降噪区域图像，得到所述多张背景降噪图像和多张前景降噪图像。In some embodiments of the present application, the image denoising unit is used to: denoise regional images in overlapping types of image regions through multi-frame fusion denoising methods to obtain fusion denoising regional images; The regional image in the overlapping type image area is denoised through a single frame fusion noise reduction method to obtain a single frame denoising area image; according to the fused denoising area image and the single frame denoising area image, the above Multiple background noise-reduced images and multiple foreground noise-reduced images.
在本申请的一些实施例中，所述图像获取单元，用于：获取多张待处理图像；获取各所述待处理图像对应的对焦状态、主体位置及曝光状态中至少一种，得到筛选参数；根据各所述待处理图像对应的筛选参数，从所述多张待处理图像中筛选出符合预定条件的待处理图像，得到所述多张候选合成图像，其中，所述预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种。In some embodiments of the present application, the image acquisition unit is used to: acquire multiple images to be processed; acquire at least one of the focus state, subject position and exposure state corresponding to each of the images to be processed, and obtain screening parameters. ; According to the screening parameters corresponding to each of the images to be processed, select the images to be processed that meet predetermined conditions from the multiple images to be processed, and obtain the multiple candidate composite images, wherein the predetermined conditions include non- At least one of the following is during the focusing process, not in the exposure convergence process, or meets the subject sharpness requirements.
在本申请的一些实施例中，所述多张待处理图像为根据预定曝光参数采集的；所述装置还包括曝光调节单元，用于：获取各所述待处理图像对应的运动参数及允许位移限值；根据所述运动参数及允许位移限值，确定允许曝光时长限值；根据场景亮度获取小于所述允许曝光时长限值的预定优选曝光时长；基于所述预定优选曝光时长更新所述预定曝光参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined exposure parameters; the device also includes an exposure adjustment unit for: obtaining the motion parameters and allowed displacements corresponding to each of the images to be processed. limit; determine the allowed exposure duration limit based on the motion parameter and the allowed displacement limit; obtain a predetermined preferred exposure duration that is less than the allowed exposure duration limit based on scene brightness; update the predetermined exposure time based on the predetermined preferred exposure duration exposure parameters.
在本申请的一些实施例中，所述多张待处理图像为根据预定对焦参数采集的；所述装置还包括对焦调节单元，用于：根据各所述待处理图像对应的主体位置计算对焦区域；根据所述对焦区域更新所述预定对焦参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined focus parameters; the device further includes a focus adjustment unit for: calculating the focus area according to the subject position corresponding to each of the images to be processed. ;Update the predetermined focus parameter according to the focus area.
在本申请的一些实施例中，在所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像之后，所述装置还包括融合单元，用于：根据所述对齐位置信息，在所述统一坐标系下将所述多张背景降噪图像拼接为的整体背景图像；以及，将所述多张前景降噪图像与所述整体背景图像融合，得到融合轨迹图像；根据所述融合轨迹图像，得到主体运动轨迹图像。In some embodiments of the present application, on the regional images that will be aligned to each of the image regions, noise reduction is performed according to the noise reduction method corresponding to the region type of each of the image regions to obtain multiple background noise reduction images. and multiple foreground noise-reduced images, the device further includes a fusion unit, configured to: splice the multiple background noise-reduced images into an overall background image under the unified coordinate system according to the alignment position information; And, fuse the plurality of foreground noise reduction images with the overall background image to obtain a fused trajectory image; and obtain a subject motion trajectory image based on the fused trajectory image.
根据本申请的另一实施例，一种存储介质，其上存储有计算机程序，当所述计算机程序被计算机的处理器执行时，使计算机执行本申请实施例所述的方法。According to another embodiment of the present application, a storage medium has a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is caused to execute the method described in the embodiment of the present application.
根据本申请的另一实施例，一种电子设备可以包括：存储器，存储有计算机程序；处理器，读取存储器存储的计算机程序，以执行本申请实施例所述的方法。According to another embodiment of the present application, an electronic device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiment of the present application.
根据本申请的另一实施例，一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行本申请实施例所述的各种可选实现方式中提供的方法。According to another embodiment of the present application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations described in the embodiments of this application.
本申请实施例中，获取多张候选合成图像；基于每张所述候选合成图像中的非主体背景，将所述多张候选合成图像对齐至统一坐标系，得到每张所述候选合成图像的主体前景和非主体背景在所述统一坐标系下的对齐位置信息；根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型；将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，所述多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。In the embodiment of the present application, multiple candidate composite images are obtained; based on the non-subject background in each candidate composite image, the multiple candidate composite images are aligned to a unified coordinate system to obtain the coordinates of each candidate composite image. Alignment position information of the subject foreground and non-subject background in the unified coordinate system; determine different images in the unified coordinate system based on the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image The area type of the area; align the area images in each of the image areas, perform noise reduction according to the noise reduction method corresponding to the area type of each of the image areas, and obtain multiple background noise reduction images and multiple foreground noise reduction images. , the multiple background noise-reduced images and the multiple foreground noise-reduced images are used to synthesize the subject motion trajectory image.
以这种方式，将多张候选合成图像通过背景分离后，根据非主体背景将多张候选合成图像独立对齐至统一坐标系，使得原始图像的采集可以不受视场的限制，图像采集过程可以跟随主体移动提升视场范围，提高了大范围运动场景下运动轨迹图像的轨迹完整性；进一步的，通过将统一坐标系下划分图像区域的区域类型，根据将对齐至各图像区域中的区域图像，根据各图像区域的区域类型所对应降噪方式进行降噪，可以有效避免主体快速运动过程中曝光参数等导致运动模糊，进一步确保合成的运动轨迹图像的图像质量。进而，本申请整体上可以有效提升多数运动场景下生成的运动轨迹图像的图像质量和轨迹完整性，提升用户体验。In this way, after separating multiple candidate composite images through the background, the multiple candidate composite images are independently aligned to a unified coordinate system according to the non-subject background, so that the collection of original images can not be restricted by the field of view, and the image collection process can Following the movement of the subject increases the field of view and improves the trajectory integrity of motion trajectory images in large-scale motion scenes; further, by dividing the area types of image areas under a unified coordinate system, the area images aligned to each image area are , perform noise reduction according to the noise reduction method corresponding to the area type of each image area, which can effectively avoid motion blur caused by exposure parameters during the rapid movement of the subject, and further ensure the image quality of the synthesized motion trajectory image. Furthermore, this application as a whole can effectively improve the image quality and trajectory integrity of motion trajectory images generated in most sports scenarios, and improve user experience.
附图说明Description of the drawings
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.
图1示出了根据本申请的一个实施例的运动轨迹图像生成方法的流程图。Figure 1 shows a flow chart of a motion trajectory image generation method according to an embodiment of the present application.
图2示出了一种场景下通过应用本申请的实施例进行运动轨迹图像生成的装置框图。Figure 2 shows a block diagram of a device for generating motion trajectory images by applying embodiments of the present application in a scenario.
图3示出了一种场景下通过应用本申请的实施例进行运动轨迹图像生成的流程图。Figure 3 shows a flow chart of motion trajectory image generation by applying embodiments of the present application in one scenario.
图4示出了根据本申请的一个实施例的运动轨迹图像生成装置的框图。Figure 4 shows a block diagram of a motion trajectory image generating device according to an embodiment of the present application.
图5示出了根据本申请的一个实施例的电子设备的框图。Figure 5 shows a block diagram of an electronic device according to one embodiment of the present application.
具体实施方式Detailed ways
以下结合附图及实施例，对本公开进行进一步详细说明。应当理解，此处所提供的实施例仅仅用以解释本公开，并不用于限定本公开。另外，以下所提供的实施例是用于实施本公开的部分实施例，而非提供实施本公开的全部实施例，在不冲突的情况下，本公开实施例记载的技术方案可以任意组合的方式实施。The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided here are only used to explain the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are for implementing some of the embodiments of the present disclosure, rather than providing all the embodiments for implementing the present disclosure. The technical solutions recorded in the embodiments of the present disclosure can be combined in any way unless there is any conflict. implementation.
需要说明的是，在本公开实施例中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的方法或者装置不仅包括所明确记载的要素，而且还包括没有明确列出的其他要素，或者是还包括为实施方法或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的方法或者装置中还存在另外的相关要素(例如方法中的步骤或者装置中的单元，例如的单元可以是部分电路、部分处理器、部分程序或软件等等)。It should be noted that in the embodiments of the present disclosure, the terms "comprising", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements not only includes the explicitly stated elements, but also other elements not expressly listed, or elements inherent to the implementation of the method or apparatus. Without further limitations, an element defined by the statement "comprises a..." does not exclude the presence of other related elements (such as steps in the method or units in the device) in the method or device including the element. , for example, the unit may be part of a circuit, part of a processor, part of a program or software, etc.).
例如，本公开实施例提供的运动轨迹图像生成方法包含了一系列的步骤，但是本公开实施例提供的运动轨迹图像生成方法不限于所记载的步骤，同样地，本公开实施例提供的运动轨迹图像生成装置包括了一系列单元，但是本公开实施例提供的装置不限于包括所明确记载的单元，还可以包括为获取相关信息、或基于信息进行处理时所需要设置的单元。For example, the motion trajectory image generation method provided by the embodiment of the present disclosure includes a series of steps, but the motion trajectory image generation method provided by the embodiment of the disclosure is not limited to the recorded steps. Similarly, the motion trajectory image generation method provided by the embodiment of the disclosure The image generation device includes a series of units, but the device provided by the embodiments of the present disclosure is not limited to include the explicitly described units, and may also include units that are required to obtain relevant information or perform processing based on the information.
除非另有定义，本文所使用的所有的技术和科学术语与属于本公开的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述具体的实施例的目的，不是旨在限制本公开。Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the disclosure.
图1示意性示出了根据本申请的一个实施例的运动轨迹图像生成方法的流程图。该运动轨迹图像生成方法的执行主体可以是任意具有处理能力的设备或服务器，设备例如电视、电脑、手机、智能手表以及家电设备等，服务器例如云端服务器、物理服务器等。本申请的一种具体实施例中，作为执行主体的设备具体为手机。Figure 1 schematically shows a flow chart of a motion trajectory image generation method according to an embodiment of the present application. The execution subject of the motion trajectory image generation method can be any device or server with processing capabilities. Devices such as TVs, computers, mobile phones, smart watches and home appliances, etc. Servers such as cloud servers, physical servers, etc. In a specific embodiment of the present application, the device as the execution subject is specifically a mobile phone.
如图1所示，该运动轨迹图像生成方法可以包括步骤S110至步骤S140。As shown in Figure 1, the motion trajectory image generating method may include steps S110 to S140.
步骤S110，获取多张候选合成图像；步骤S120，基于每张所述候选合成图像中的非主体背景，将所述多张候选合成图像对齐至统一坐标系，得到每张所述候选合成图像的主体前景和非主体背景在所述统一坐标系下的对齐位置信息；步骤S130，根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型；步骤S140，将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，所述多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。Step S110, obtain multiple candidate composite images; Step S120, based on the non-subject background in each candidate composite image, align the multiple candidate composite images to a unified coordinate system to obtain the coordinates of each candidate composite image. Alignment position information of the subject foreground and non-subject background in the unified coordinate system; step S130, determine the unified coordinate system based on the motion parameters, foreground background information and the alignment position information corresponding to each candidate composite image. Region types of different image regions; step S140, align the regional images in each of the image regions, perform noise reduction according to the noise reduction method corresponding to the region type of each of the image regions, and obtain multiple background noise reduction images and Multiple foreground noise-reduced images, the multiple background noise-reduced images and the multiple foreground noise-reduced images are used to synthesize the subject motion trajectory image.
多张候选合成图像为用于合成主体运动轨迹图像的图像。一些场景下，多张候选合成图像可以是设备实时针对运动主体(例如运动的人物)拍摄的图像序列中的图像，从而，基于本申请实施例可以实时在线进行运动轨迹图像生成；还有一些场景下，多张候选合成图像可以是预先存储的图像序列中图像，从而，基于本申请实施例可以实现离线进行运动轨迹图像生成。The plurality of candidate composite images are images used to synthesize the subject's motion trajectory image. In some scenarios, multiple candidate composite images may be images in an image sequence captured by the device in real time for moving subjects (such as moving people). Therefore, based on the embodiments of the present application, motion trajectory images can be generated online in real time; there are also some scenarios. In this case, multiple candidate composite images may be images in a pre-stored image sequence, so that offline motion trajectory image generation can be achieved based on the embodiments of the present application.
从每张候选合成图像中可以分离得到主体前景(即候选合成图像中主体所在的区域)和非主体背景(即候选合成图像中主体前景之外的区域)。From each candidate composite image, the subject foreground (i.e., the area where the subject is located in the candidate composite image) and the non-subject background (i.e., the area outside the subject foreground in the candidate composite image) can be separated.
基于每张候选合成图像中的非主体背景，可以将多张候选合成图像映射对齐至统一坐标系下，得到每张候选合成图像中的主体前景和非主体背景在统一坐标系下的对齐位置信息，对齐位置信息可以包括主体前景和非主体背景中像素点在该统一坐标系下的坐标位置。Based on the non-subject background in each candidate composite image, multiple candidate composite images can be mapped and aligned to a unified coordinate system to obtain the alignment position information of the subject foreground and non-subject background in each candidate composite image under the unified coordinate system. , the alignment position information may include the coordinate positions of pixels in the subject foreground and non-subject background under the unified coordinate system.
将多张候选合成图像映射对齐至统一坐标系下，从而，将运动主体大范围移动的图像映射至统一坐标系下得到大范围的运动范围图像，根据非主体背景将多张候选合成图像独立对齐至统一坐标系，使得原始图像的采集可以不受视场的限制，图像采集过程可以跟随主体移动提升视场范围。Map and align multiple candidate composite images to a unified coordinate system, thereby mapping images of large-scale movements of moving subjects to a unified coordinate system to obtain a wide range of motion images, and independently align multiple candidate composite images based on non-subject backgrounds To a unified coordinate system, the collection of original images can be not limited by the field of view, and the image collection process can follow the movement of the subject to increase the field of view.
统一坐标系下大范围的运动范围图像可以根据预定划分方式划分为不同图像区域，根据每张候选合成图像对应的运动参数、前景背景信息及对齐位置信息，可以得到对齐至各图像区域的部分区域图像的动静类型、前后景类型及重叠非重叠类型，从而确定统一坐标系下不同图像区域的区域类型。A large motion range image under a unified coordinate system can be divided into different image areas according to a predetermined division method. According to the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, partial areas aligned to each image area can be obtained The dynamic and static types, front and rear background types, and overlapping and non-overlapping types of the image are used to determine the area types of different image areas under a unified coordinate system.
其中，运动参数具体可以是候选合成图像中各像素点的运动状态，可以通过运动检测得到；前景背景信息具体可以是描述候选合成图像中各像素点属于前景还是背景的信息；对齐位置信息具体可以是候选合成图像中各像素点在统一坐标系中的坐标位置。Among them, the motion parameter can be the motion state of each pixel in the candidate composite image, which can be obtained through motion detection; the foreground background information can be information describing whether each pixel in the candidate composite image belongs to the foreground or the background; the alignment position information can be is the coordinate position of each pixel in the candidate composite image in the unified coordinate system.
将对齐至各图像区域中的区域图像，根据各图像区域的区域类型所对应预设的降噪方式进行降噪，得到各图像区域中的降噪区域图像，将归属于各候选合成图像中的主体前景和非主体背景的降噪区域图像可以重新拼接得到降噪后的主体前景(即前景降噪图像)以及降噪后非主体背景(即背景降噪图像)，从而得到多张背景降噪图像和多张前景降噪图像。The regional images aligned to each image area will be denoised according to the preset noise reduction method corresponding to the area type of each image area, and the denoised area images in each image area will be obtained, which will belong to each candidate synthetic image. The noise-reduced area images of the subject foreground and non-subject background can be re-spliced to obtain the denoised subject foreground (i.e., the foreground noise-reduced image) and the denoised non-subject background (i.e., the background noise-reduced image), thereby obtaining multiple background noise-reduced images. image and multiple foreground noise-reduced images.
多张前景降噪图像为降噪后清晰的运动主体的运动图像，根据多张背景降噪图像和多张前景降噪图像即可合成运动主体的主体运动轨迹图像，主体运动轨迹图像可以清晰呈现大范围的运动主体的主体运动轨迹。Multiple foreground noise-reduced images are moving images of moving subjects that are clear after noise reduction. Based on multiple background noise-reduced images and multiple foreground noise-reduced images, the subject motion trajectory image of the moving subject can be synthesized, and the subject motion trajectory image can be clearly presented. The subject movement trajectories of a large range of moving subjects.
以这种方式，基于步骤S110至步骤S140，将多张候选合成图像通过背景分离后，根据非主体背景将多张候选合成图像独立对齐至统一坐标系，使得原始图像的采集可以不受视场的限制，图像采集过程可以跟随主体移动提升视场范围，提高了大范围运动场景下运动轨迹图像的轨迹完整性；进一步的，通过将统一坐标系下划分图像区域的区域类型，根据将对齐至各图像区域中的区域图像，根据各图像区域的区域类型所对应降噪方式进行降噪，可以有效避免主体快速运动过程中曝光参数等导致运动模糊，进一步确保合成的运动轨迹图像的图像质量。进而，本申请整体上可以有效提升多数运动场景下生成的运动轨迹图像的图像质量和轨迹完整性，提升用户体验。In this way, based on steps S110 to S140, after the multiple candidate composite images are separated by the background, the multiple candidate composite images are independently aligned to a unified coordinate system according to the non-subject background, so that the original image can be collected regardless of the field of view. With the limitation of The regional images in each image area are denoised according to the noise reduction method corresponding to the area type of each image area, which can effectively avoid motion blur caused by exposure parameters during the rapid movement of the subject, and further ensure the image quality of the synthesized motion trajectory image. Furthermore, this application as a whole can effectively improve the image quality and trajectory integrity of motion trajectory images generated in most sports scenarios, and improve user experience.
下面描述图1实施例下进行运动轨迹图像生成时，所进行的各步骤下进一步可选的具体实施例。The following describes further optional specific embodiments of each step performed when generating a motion trajectory image in the embodiment of FIG. 1 .
一种实施例中，所述获取多张候选合成图像，包括：获取多张待处理图像；获取各所述待处理图像对应的对焦状态、主体位置及曝光状态中至少一种，得到筛选参数；根据各所述待处理图像对应的筛选参数，从所述多张待处理图像中筛选出符合预定条件的待处理图像，得到所述多张候选合成图像，其中，所述预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种。In one embodiment, obtaining multiple candidate composite images includes: obtaining multiple images to be processed; obtaining at least one of the focus state, subject position and exposure state corresponding to each of the images to be processed to obtain screening parameters; According to the screening parameters corresponding to each of the images to be processed, the images to be processed that meet predetermined conditions are selected from the multiple images to be processed, and the multiple candidate composite images are obtained, wherein the predetermined conditions include not being in focus. At least one of the following: in process, not in the process of exposure convergence, and in compliance with subject definition requirements.
多张待处理图像可以是实时针对运动主体(例如运动的人物)拍摄的图像序列，此时，可以通过对每张待处理图像进行自动追焦，得到每张所述待处理图像对应的对焦状态(用于反映待处理图像是否为对焦过程中的图像)和主体位置(即运动主体所在位置)；同时可以检测拍摄每张待处理图像的曝光状态(用于反映待处理图像是否为曝光收敛过程中的图像)。The multiple images to be processed can be a sequence of images taken in real time against moving subjects (such as moving people). At this time, the focus state corresponding to each of the images to be processed can be obtained by automatically focusing on each image to be processed. (used to reflect whether the image to be processed is an image in the process of focusing) and the position of the subject (that is, the position of the moving subject); at the same time, the exposure status of each image to be processed can be detected (used to reflect whether the image to be processed is in the process of exposure convergence) image in ).
多张待处理图像也可以是预先存储的图像序列，存储多张待处理图像时可以将待处理图像相应的运动参数、曝光状态、对焦状态及主体位置等信息打包并逐帧存储。Multiple images to be processed can also be pre-stored image sequences. When storing multiple images to be processed, information such as motion parameters, exposure status, focus status, and subject position corresponding to the images to be processed can be packaged and stored frame by frame.
获取各待处理图像对应的对焦状态、主体位置及曝光状态中至少一种，得到各待处理图像对应的筛选参数；根据各待处理图像对应的筛选参数，可以从多张待处理图像中筛选出符合预定条件的待处理图像，得到多张候选合成图像，其中，预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种。Obtain at least one of the focus state, subject position and exposure state corresponding to each image to be processed, and obtain the filtering parameters corresponding to each image to be processed; according to the filtering parameters corresponding to each image to be processed, multiple images to be processed can be filtered out Multiple candidate composite images are obtained from images to be processed that meet predetermined conditions, where the predetermined conditions include at least one of not being in the focusing process, not being in the exposure convergence process, and meeting subject sharpness requirements.
其中，根据对焦状态可以判断待处理图像是否处于对焦过程中，从而筛选出非处于对焦过程中的待处理图像。根据曝光状态可以判断待处理图像是否处于曝光收敛过程中，从而筛选出非处于曝光收敛过程中的待处理图像。根据待处理图像中主体位置在中心还是边缘以及主体位置的占比等可以分析待处理图像中主体清晰度，从而筛选出符合主体清晰度要求的待处理图像。Among them, according to the focus state, it can be determined whether the image to be processed is in the process of focusing, thereby filtering out the images to be processed that are not in the process of focusing. According to the exposure status, it can be determined whether the image to be processed is in the process of exposure convergence, thereby filtering out the images to be processed that are not in the process of exposure convergence. The subject clarity in the image to be processed can be analyzed based on whether the subject position in the image to be processed is in the center or at the edge and the proportion of the subject position, thereby screening out the images to be processed that meet the subject clarity requirements.
进而，多张候选合成图像为从多张待处理图像中筛选出的满足非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种的待处理图像，筛选出的候选合成图像满足预定条件也即候选合成图像的图像质量较高，基于筛选出的候选合成图像来合成主体运动轨迹图像可以进一步提升运动轨迹图像的图像质量。Furthermore, the multiple candidate composite images are images to be processed that are selected from multiple images to be processed and satisfy at least one of not being in the focusing process, not being in the exposure convergence process, and meeting the subject sharpness requirements, and the selected candidates are The synthesized image satisfies the predetermined conditions, that is, the image quality of the candidate synthesized image is higher. Synthesizing the subject's motion trajectory image based on the selected candidate synthesized images can further improve the image quality of the motion trajectory image.
可以理解，其它实施例中，可以获取多张待处理图像，并直接将多张待处理图像作为多张候选合成图像，而不进行筛选。It can be understood that in other embodiments, multiple images to be processed can be obtained and directly used as multiple candidate composite images without filtering.
进一步的，一种实施例中，所述多张待处理图像为根据预定曝光参数采集的；所述方法还包括：获取各所述待处理图像对应的运动参数及允许位移限值；根据所述运动参数及允许位移限值，确定允许曝光时长限值；根据场景亮度获取小于所述允许曝光时长限值的预定优选曝光时长；基于所述预定优选曝光时长更新所述预定曝光参数。Further, in one embodiment, the plurality of images to be processed are collected according to predetermined exposure parameters; the method further includes: obtaining the motion parameters and allowable displacement limits corresponding to each of the images to be processed; according to the Motion parameters and allowed displacement limits are used to determine the allowed exposure duration limit; a predetermined preferred exposure duration smaller than the allowed exposure duration limit is obtained based on scene brightness; and the predetermined exposure parameters are updated based on the predetermined preferred exposure duration.
多张待处理图像为根据预定曝光参数实采集的，例如，采用设备实时拍摄得到多张待处理图像的序列，基于运动参数及允许位移限值选择合适的曝光参数更新预定曝光参数，可以使得后续采集的待处理图像降低运动失真及模糊的发生情况，保证图像曝光效果的稳定，整体上进一步提升运动轨迹图像的图像质量。Multiple images to be processed are actually collected according to predetermined exposure parameters. For example, a sequence of multiple images to be processed is obtained by using equipment to shoot in real time. Appropriate exposure parameters are selected based on the motion parameters and allowable displacement limits to update the predetermined exposure parameters, which can make subsequent The collected images to be processed reduce the occurrence of motion distortion and blur, ensure the stability of image exposure, and further improve the image quality of motion trajectory images as a whole.
其中，获取各待处理图像对应的运动参数及允许位移限值，具体可以是通过运动检测算法，根据相邻图像帧之间的光流信息变化逐帧估计图像动静区及动区各点的运动速度，并完成运动速度与图像帧的匹配，得到各待处理图像中像素点对应的运动速度(即运动参数)。同时，可以预先设定的图像画面无运动失真及模糊的最大允许位移作为允许位移限值。Among them, the motion parameters and allowable displacement limits corresponding to each image to be processed are obtained. Specifically, a motion detection algorithm can be used to estimate the motion of each point in the image's dynamic and static areas and moving areas frame by frame based on changes in optical flow information between adjacent image frames. speed, and completes the matching of the movement speed with the image frame to obtain the movement speed (i.e., motion parameters) corresponding to the pixels in each image to be processed. At the same time, the maximum allowable displacement of the image without motion distortion and blur can be preset as the allowable displacement limit.
根据运动参数及允许位移限值，可以计算出计算最大允许曝光时长作为允许曝光时长限值。根据场景亮度(即采集待处理图像的环境亮度)可以获取小于允许曝光时长限值的多个预定曝光时长中，匹配该场景亮度的一个作为预定优选曝光时长。Based on the motion parameters and allowable displacement limits, the maximum allowable exposure time can be calculated as the allowable exposure time limit. According to the brightness of the scene (that is, the brightness of the environment in which the image to be processed is collected), among multiple predetermined exposure durations that are less than the allowed exposure duration limit, the one matching the scene brightness can be obtained as the predetermined preferred exposure duration.
基于预定优选曝光时长更新预定曝光参数用于进行拍摄待处理图像，则保证合成运动轨迹图像时拍摄待处理图像过程中，不同运动状态下待处理图像无运动失真及模糊和曝光效果的一致性，进一步保证拼接融合后运动轨迹图像画面效果的清晰性与一致性。Update the predetermined exposure parameters based on the predetermined preferred exposure duration for shooting the image to be processed, thereby ensuring that during the process of capturing the image to be processed when synthesizing the motion trajectory image, the image to be processed will have no motion distortion and the consistency of blur and exposure effects under different motion states. This further ensures the clarity and consistency of the motion trajectory image effects after splicing and fusion.
进一步的，一些方式中，基于预定优选曝光时长更新预定曝光参数的同时，可以同步调整拍摄待处理图像的亮度补偿增益，进一步保证拼接融合后运动轨迹图像画面效果的清晰性与一致性。其中，亮度补偿增益可以是针对运动速度预定设定的对应的增益。Furthermore, in some methods, while updating the predetermined exposure parameters based on the predetermined preferred exposure duration, the brightness compensation gain of the captured image to be processed can be simultaneously adjusted to further ensure the clarity and consistency of the motion trajectory image effects after splicing and fusion. The brightness compensation gain may be a corresponding gain predetermined for the movement speed.
进一步的，一种实施例中，所述多张待处理图像为根据预定对焦参数采集的；所述方法还包括：根据各所述待处理图像对应的主体位置计算对焦区域；根据所述对焦区域更新所述预定对焦参数。Further, in one embodiment, the plurality of images to be processed are collected according to predetermined focus parameters; the method further includes: calculating the focus area according to the subject position corresponding to each of the images to be processed; Update the predetermined focus parameters.
根据各待处理图像对应的主体位置可以计算得到图像中的对焦区域，根据所述对焦区域更新所述预定对焦参数，可以进一步保证拍摄过程中焦点锁定在运动主体上，运动主体不出现失焦模糊，进一步提升运动轨迹图像的图像质量，其中，运动主体可以是通过用户主动指定或基于目标识别算法自动选择的。The focus area in the image can be calculated according to the subject position corresponding to each image to be processed, and the predetermined focus parameters are updated according to the focus area, which can further ensure that the focus is locked on the moving subject during the shooting process, and the moving subject does not suffer from out-of-focus blur. , to further improve the image quality of motion trajectory images, in which the moving subject can be actively specified by the user or automatically selected based on a target recognition algorithm.
进一步的，一种实施例中，所述根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型，包括：根据每张所述候选合成图像对应的所述运动参数、前景背景信息及所述对齐位置信息，得到对齐至各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型；根据各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，得到各图像区域的区域类型。Further, in one embodiment, determining the area types of different image areas under the unified coordinate system based on the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image includes: According to the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, obtain the motion and static types, front and rear background types and overlapping and non-overlapping types corresponding to the regional images aligned to each of the image areas; The area type of each image area is obtained according to the combination of the motion and stillness types, the front and rear background types and the overlapping and non-overlapping types corresponding to the area images in each of the image areas.
统一坐标系下大范围的运动范围图像可以根据预定划分方式划分为不同图像区域，根据每张候选合成图像对应的运动参数、前景背景信息及对齐位置信息，可以得到对齐至各图像区域的部分区域图像的动静类型、前后景类型及重叠非重叠类型。动静类型可以包括运动区和静态区，前后景类型可以包括前景及背景，重叠非重叠类型可以包括重叠区及非重叠区。A large motion range image under a unified coordinate system can be divided into different image areas according to a predetermined division method. According to the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, partial areas aligned to each image area can be obtained The dynamic and static types of the image, the front and rear background types, and the overlapping and non-overlapping types. The dynamic and static types can include moving areas and static areas, the front and rear background types can include foreground and background, and the overlapping and non-overlapping types can include overlapping areas and non-overlapping areas.
从而根据各图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，可以得到各图像区域的区域类型，区域类型可以包括前景动区、前景静重叠区、前景静非重叠区、背景动重叠区、背景动非重叠区、背景静重叠区、背景静非重叠区，其中，前景动区即前景和运动区的组合对应区域类型，前景静重叠区即前景、静态区及重叠区的组合对应区域类型，依此类推。Therefore, according to the combination of the dynamic and static types, the front and rear foreground types and the overlapping and non-overlapping types corresponding to the regional images in each image area, the area type of each image area can be obtained. The area type can include a foreground moving area, a foreground static overlapping area, and a foreground static non-overlapping area. area, background dynamic overlapping area, background moving non-overlapping area, background static overlapping area, and background static non-overlapping area. Among them, the foreground dynamic area is the combination of the foreground and the moving area corresponding to the area type, and the foreground static overlapping area is the foreground, static area, and Combinations of overlapping areas correspond to area types, and so on.
进一步的，一种实施例中，所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，可以包括：Further, in one embodiment, the area images aligned to each of the image areas are denoised according to the noise reduction method corresponding to the area type of each of the image areas, and multiple background noise reduction images are obtained. Multiple foreground noise-reduced images, which can include:
将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像；根据所述融合降噪区域图像及所述单帧降噪区域图像，得到所述多张背景降噪图像和多张前景降噪图像。The regional images in overlapping image areas are denoised through multi-frame fusion denoising to obtain fused denoising area images; the regional images in non-overlapping image areas are denoised through single-frame fusion denoising. Noise is obtained to obtain a single frame noise reduction area image; according to the fused noise reduction area image and the single frame noise reduction area image, the multiple background noise reduction images and multiple foreground noise reduction images are obtained.
重叠类型的图像区域即区域类型对应的组合中包括重叠区的图像区域，非重叠类型的图像区域即区域类型对应的组合中包括非重叠区的图像区域。An image area of the overlapping type is an image area that includes an overlapping area in the combination corresponding to the area type, and an image area of the non-overlapping type is an image area that includes a non-overlapping area in the combination corresponding to the area type.
将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像，可以极为有效避免主体快速运动过程中曝光参数等导致运动模糊，进一步确保合成的运动轨迹图像的图像质量。其中，多帧融合降噪方式和单帧融合降噪方式可以选用现有的帧融合降噪方式及单帧融合降噪方式。The regional images in overlapping image areas are denoised through multi-frame fusion denoising to obtain fused denoising area images; the regional images in non-overlapping image areas are denoised through single-frame fusion denoising. Noise is obtained to obtain a single-frame noise-reduced area image, which can be extremely effective in avoiding motion blur caused by exposure parameters during the rapid movement of the subject, and further ensuring the image quality of the synthesized motion trajectory image. Among them, the multi-frame fusion noise reduction method and the single-frame fusion noise reduction method can use the existing frame fusion noise reduction method and single-frame fusion noise reduction method.
根据所述融合降噪区域图像及所述单帧降噪区域图像，将归属于各候选合成图像中的主体前景和非主体背景的降噪区域图像可以重新拼接得到降噪后的主体前景(即前景降噪图像)以及降噪后非主体背景(即背景降噪图像)，从而得到多张背景降噪图像和多张前景降噪图像。According to the fused noise reduction area image and the single frame noise reduction area image, the noise reduction area images belonging to the subject foreground and non-subject background in each candidate composite image can be re-spliced to obtain the denoised subject foreground (i.e. The foreground noise-reduced image) and the denoised non-subject background (i.e. the background noise-reduced image) are obtained, thereby obtaining multiple background noise-reduced images and multiple foreground noise-reduced images.
进一步的，一种实施例中，在所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像之后，所述方法还包括：Further, in one embodiment, in the area images that are to be aligned to each of the image areas, noise reduction is performed according to the noise reduction method corresponding to the area type of each of the image areas, and multiple background noise reduction images are obtained. and multiple foreground noise reduction images, the method also includes:
根据所述对齐位置信息，在所述统一坐标系下将所述多张背景降噪图像拼接为的整体背景图像；以及，将所述多张前景降噪图像与所述整体背景图像融合，得到融合轨迹图像；根据所述融合轨迹图像，得到主体运动轨迹图像。According to the alignment position information, the plurality of background noise reduction images are spliced into an overall background image under the unified coordinate system; and, the plurality of foreground noise reduction images and the overall background image are fused to obtain Fusion of trajectory images; based on the fusion of trajectory images, an image of the subject's motion trajectory is obtained.
根据对齐位置信息，在统一坐标系下可以将多张背景降噪图像通过裁剪拼接为运动主体整体运动范围下的整体背景图像。然后，将多张前景降噪图像与整体背景图像融合，使得多张前景降噪图像中运动主体合成到整体背景图像上，称为一张完整的融合轨迹图像。根据融合轨迹图像，即可得到表现运动主体的运动轨迹的主体运动轨迹图像。According to the alignment position information, multiple background noise reduction images can be cropped and spliced into an overall background image within the overall motion range of the moving subject in a unified coordinate system. Then, multiple foreground denoised images are fused with the overall background image, so that the moving subjects in the multiple foreground denoised images are synthesized onto the overall background image, which is called a complete fused trajectory image. According to the fused trajectory image, the subject movement trajectory image representing the movement trajectory of the moving subject can be obtained.
其中，根据所述融合轨迹图像，得到主体运动轨迹图像，具体可以是：将融合轨迹图像，作为得到的主体运动轨迹图像；或，将所述融合轨迹图像利用图像平滑算法进行平滑处理使前景与背景过渡自然，得到平滑图像，然后通过对平滑图像进行裁剪生成选定区域的运动轨迹照片，作为最终的主体运动轨迹图像。Wherein, according to the fused trajectory image, the subject movement trajectory image is obtained. Specifically, the fusion trajectory image can be used as the obtained subject movement trajectory image; or, the fused trajectory image is smoothed using an image smoothing algorithm to make the foreground and The background transition is natural and a smooth image is obtained, and then the motion trajectory photos of the selected area are generated by cropping the smooth image as the final subject motion trajectory image.
为便于更好的实施本申请实施例提供的运动轨迹图像生成方法，以下结合一种场景下进行运动轨迹图像生成的流程进一步描述前述实施例，参阅图2及图3，图2示出了一种场景下通过应用本申请的实施例进行运动轨迹图像生成的装置框图，图3示出了一种场景下通过应用本申请的实施例进行运动轨迹图像生成的流程图。In order to facilitate better implementation of the motion trajectory image generation method provided by the embodiments of the present application, the foregoing embodiments are further described below in conjunction with the process of generating motion trajectory images in a scenario. Refer to Figures 2 and 3. Figure 2 shows a A block diagram of a device for generating motion trajectory images by applying embodiments of the present application in this scenario. Figure 3 shows a flow chart of generating motion trajectory images by applying embodiments of the present application in one scenario.
如图2所示，该场景下，运动轨迹图像生成的装置可以包括图像采集模块210、智能追焦模块220、运动检测模块230、动态曝光模块240、图像筛选模块250及图像处理模块260。As shown in Figure 2, in this scenario, the device for generating motion trajectory images may include an image acquisition module 210, an intelligent focus tracking module 220, a motion detection module 230, a dynamic exposure module 240, an image screening module 250, and an image processing module 260.
参阅图3，该场景下，运动轨迹图像生成的流程可以包括步骤S310至步骤S360。Referring to FIG. 3 , in this scenario, the process of generating motion trajectory images may include steps S310 to S360.
步骤S310，采集图像。具体地，获取多张待处理图像。Step S310, collect images. Specifically, multiple images to be processed are obtained.
其中，通过图像采集模块210可以实时采集图像，从而得到多张待处理图像的序列。Among them, images can be collected in real time through the image acquisition module 210, thereby obtaining a sequence of multiple images to be processed.
步骤S320，获取图像参数。具体地，获取各待处理图像对应的对焦状态、主体位置及曝光状态。Step S320: Obtain image parameters. Specifically, the focus state, subject position and exposure state corresponding to each image to be processed are obtained.
其中，通过智能追焦模块220可以对每张待处理图像进行自动追焦，得到每张所述待处理图像对应的对焦状态(用于反映待处理图像是否为对焦过程中的图像)和主体位置(即运动主体所在位置)。Among them, the intelligent focus tracking module 220 can automatically focus on each image to be processed, and obtain the focus status (used to reflect whether the image to be processed is an image in the process of focusing) and the subject position corresponding to each image to be processed. (That is, the location of the moving subject).
其中，通过动态曝光模块240可以检测拍摄每张待处理图像的曝光状态(用于反映待处理图像是否为曝光收敛过程中的图像)。The dynamic exposure module 240 can detect the exposure status of each image to be processed (used to reflect whether the image to be processed is an image in the process of exposure convergence).
步骤S330，图像筛选。具体地，根据各所述待处理图像对应的筛选参数，从所述多张待处理图像中筛选出符合预定条件的待处理图像，得到所述多张候选合成图像，其中，所述预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求。Step S330, image screening. Specifically, according to the screening parameters corresponding to each of the images to be processed, the images to be processed that meet predetermined conditions are screened out from the multiple images to be processed, and the multiple candidate composite images are obtained, wherein the predetermined conditions include Not in the process of focusing, not in the process of exposure convergence, and meeting the subject sharpness requirements.
其中，通过图像筛选模块250可以进行图像筛选，得到多张候选合成图像。Among them, the image screening module 250 can perform image screening to obtain multiple candidate composite images.
步骤S340，曝光调节。具体地，获取各待处理图像对应的运动参数及允许位移限值；根据运动参数及允许位移限值，确定允许曝光时长限值；根据场景亮度获取小于允许曝光时长限值的预定优选曝光时长；基于预定优选曝光时长更新预定曝光参数。Step S340, exposure adjustment. Specifically, obtain the motion parameters and allowable displacement limits corresponding to each image to be processed; determine the allowable exposure duration limit based on the motion parameters and the allowable displacement limit; obtain a predetermined preferred exposure duration that is less than the allowed exposure duration limit based on the scene brightness; The predetermined exposure parameters are updated based on the predetermined preferred exposure duration.
其中，通过动态曝光模块240可以基于预定优选曝光时长对图像采集模块210中的预定曝光参数进行更新。The dynamic exposure module 240 can update the predetermined exposure parameters in the image acquisition module 210 based on the predetermined preferred exposure duration.
步骤S350，对焦调节。具体地，根据各待处理图像对应的主体位置计算对焦区域；根据对焦区域更新预定对焦参数。Step S350, focus adjustment. Specifically, the focus area is calculated according to the subject position corresponding to each image to be processed; and the predetermined focus parameters are updated according to the focus area.
其中，通过智能追焦模块220可以根据对焦区域对图像采集模块210中的预定对焦参数进行更新。Among them, the predetermined focus parameters in the image acquisition module 210 can be updated according to the focus area through the intelligent focus tracking module 220 .
步骤S360，图像处理。步骤S360具体可以包括步骤S361至步骤S364。其中，图像处理模块260中可以包括：图像对齐单元用于执行步骤S361，类型划分单元用于执行步骤S362，图像降噪单元用于执行步骤S363，图像融合单元用于执行步骤S364。Step S360, image processing. Step S360 may specifically include steps S361 to S364. The image processing module 260 may include: an image alignment unit for performing step S361, a type classification unit for performing step S362, an image noise reduction unit for performing step S363, and an image fusion unit for performing step S364.
步骤S361，对齐处理。具体地，基于每张候选合成图像中的非主体背景，将多张候选合成图像对齐至统一坐标系，得到每张候选合成图像的主体前景和非主体背景在统一坐标系下的对齐位置信息。Step S361, alignment processing. Specifically, based on the non-subject background in each candidate composite image, multiple candidate composite images are aligned to a unified coordinate system, and the alignment position information of the subject foreground and non-subject background in each candidate composite image under the unified coordinate system is obtained.
步骤S362，区域类型划分。具体地，根据每张候选合成图像对应的运动参数、前景背景信息及对齐位置信息，确定统一坐标系下不同图像区域的区域类型。Step S362: Region type classification. Specifically, based on the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, the area types of different image areas under the unified coordinate system are determined.
其中，具体可以是根据每张候选合成图像对应的运动参数、前景背景信息及对齐位置信息，得到对齐至各图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型；根据各图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，得到各图像区域的区域类型。区域类型可以包括前景动区、前景静重叠区、前景静非重叠区、背景动重叠区、背景动非重叠区、背景静重叠区、背景静非重叠区。Specifically, based on the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, the motion and static types, front and rear background types and overlapping and non-overlapping types corresponding to the regional images aligned to each image area are obtained; according to each image The combination of dynamic and static types, front and rear background types and overlapping and non-overlapping types corresponding to the area images in the area is used to obtain the area type of each image area. Area types may include foreground dynamic areas, foreground static overlapping areas, foreground static non-overlapping areas, background dynamic overlapping areas, background dynamic non-overlapping areas, background static overlapping areas, and background static non-overlapping areas.
步骤S363，按类型进行降噪。具体地，将对齐至各图像区域中的区域图像，根据各图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。Step S363: perform noise reduction by type. Specifically, the regional images aligned to each image area are denoised according to the noise reduction method corresponding to the area type of each image area, and multiple background denoising images and multiple foreground denoising images are obtained. Multiple background denoising images are obtained. The image and multiple foreground noise-reduced images are used to synthesize the subject's motion trajectory image.
其中，具体可以将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像；根据融合降噪区域图像及单帧降噪区域图像，得到多张背景降噪图像和多张前景降噪图像。Specifically, the regional images in overlapping image areas can be denoised through multi-frame fusion denoising to obtain fused denoising area images; the area images in non-overlapping image areas can be reduced through single-frame fusion. Perform noise reduction using the noise reduction method to obtain a single frame noise reduction area image; according to the fusion of the noise reduction area image and the single frame noise reduction area image, multiple background noise reduction images and multiple foreground noise reduction images are obtained.
步骤S364，图像融合。具体地，根据对齐位置信息，在统一坐标系下将多张背景降噪图像拼接为的整体背景图像；以及，将多张前景降噪图像与整体背景图像融合，得到融合轨迹图像；根据融合轨迹图像，得到主体运动轨迹图像。Step S364, image fusion. Specifically, according to the alignment position information, multiple background noise reduction images are spliced into an overall background image under a unified coordinate system; and, multiple foreground noise reduction images are fused with the overall background image to obtain a fusion trajectory image; according to the fusion trajectory image to obtain the subject's motion trajectory image.
以这种方式，该场景下，通过应用本申请的实施例，至少具有有益效果：多张候选合成图像通过背景分离后，根据非主体背景将多张候选合成图像独立对齐至统一坐标系，使得原始图像的采集可以不受视场的限制，图像采集过程可以跟随主体移动提升视场范围，提高了大范围运动场景下运动轨迹图像的轨迹完整性；进一步的，通过将统一坐标系下划分图像区域的区域类型，根据将对齐至各图像区域中的区域图像，根据各图像区域的区域类型所对应降噪方式进行降噪，可以有效避免主体快速运动过程中曝光参数等导致运动模糊，进一步确保合成的运动轨迹图像的图像质量。整体上可以有效提升多数运动场景下生成的运动轨迹图像的图像质量和轨迹完整性，提升用户体验。In this way, in this scenario, by applying the embodiments of the present application, at least there are beneficial effects: after multiple candidate composite images are separated by background, the multiple candidate composite images are independently aligned to a unified coordinate system according to the non-subject background, so that The collection of original images is not limited by the field of view. The image collection process can follow the movement of the subject to increase the field of view, improving the trajectory integrity of motion trajectory images in large-scale motion scenes; further, by dividing the image under a unified coordinate system According to the area type of the area, the area image will be aligned to each image area, and the noise reduction method is performed according to the noise reduction method corresponding to the area type of each image area. This can effectively avoid motion blur caused by exposure parameters during the rapid movement of the subject, further ensuring Image quality of synthesized motion trajectory images. Overall, it can effectively improve the image quality and trajectory integrity of motion trajectory images generated in most sports scenarios, and improve user experience.
为便于更好的实施本申请实施例提供的运动轨迹图像生成方法，本申请实施例还提供一种基于上述运动轨迹图像生成方法的运动轨迹图像生成装置。其中名词的含义与上述运动轨迹图像生成方法中相同，具体实现细节可以参考方法实施例中的说明。图4示出了根据本申请的一个实施例的运动轨迹图像生成装置的框图。In order to facilitate better implementation of the motion trajectory image generation method provided by the embodiment of the present application, the embodiment of the present application also provides a motion trajectory image generation device based on the above motion trajectory image generation method. The meanings of the nouns are the same as in the above motion trajectory image generation method. For specific implementation details, please refer to the description in the method embodiment. Figure 4 shows a block diagram of a motion trajectory image generating device according to an embodiment of the present application.
如图4所示，运动轨迹图像生成装置400中可以包括：图像获取单元410可以用于获取多张候选合成图像；图像对齐单元420可以用于基于每张所述候选合成图像中的非主体背景，将所述多张候选合成图像对齐至统一坐标系，得到每张所述候选合成图像的主体前景和非主体背景在所述统一坐标系下的对齐位置信息；类型划分单元430可以用于根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型；图像降噪单元440可以用于将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，所述多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。As shown in Figure 4, the motion trajectory image generation device 400 may include: an image acquisition unit 410 may be used to acquire multiple candidate composite images; an image alignment unit 420 may be used to base the non-subject background in each candidate composite image , align the plurality of candidate composite images to a unified coordinate system, and obtain the alignment position information of the subject foreground and non-subject background of each candidate composite image under the unified coordinate system; the type classification unit 430 can be used to divide the candidate composite images according to The motion parameters, foreground background information and alignment position information corresponding to each candidate synthetic image determine the area types of different image areas under the unified coordinate system; the image denoising unit 440 can be used to align to each of the The regional images in the image area are denoised according to the noise reduction method corresponding to the area type of each image area, and multiple background noise reduction images and multiple foreground noise reduction images are obtained. The multiple background noise reduction images and multiple background noise reduction images are obtained. A foreground noise-reduced image is used to synthesize the subject's motion trajectory image.
在本申请的一些实施例中，所述类型划分单元，用于：根据每张所述候选合成图像对应的所述运动参数、前景背景信息及所述对齐位置信息，得到对齐至各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型；根据各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，得到各图像区域的区域类型。In some embodiments of the present application, the type classification unit is used to: obtain the alignment to each of the images based on the motion parameters, foreground background information and alignment position information corresponding to each of the candidate composite images. The motion and stillness types, the front and rear background types and the overlapping and non-overlapping types corresponding to the regional images in the region; according to the combination of the motion and stillness types, the foreground and background types and the overlapping and non-overlapping types corresponding to the regional images in each of the image regions, the region type of each image region is obtained .
在本申请的一些实施例中，所述图像降噪单元，用于：将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像；根据所述融合降噪区域图像及所述单帧降噪区域图像，得到所述多张背景降噪图像和多张前景降噪图像。In some embodiments of the present application, the image denoising unit is used to: denoise regional images in overlapping types of image regions through multi-frame fusion denoising methods to obtain fusion denoising regional images; The regional image in the overlapping type image area is denoised through a single frame fusion noise reduction method to obtain a single frame denoising area image; according to the fused denoising area image and the single frame denoising area image, the above Multiple background noise-reduced images and multiple foreground noise-reduced images.
在本申请的一些实施例中，所述图像获取单元，用于：获取多张待处理图像；获取各所述待处理图像对应的对焦状态、主体位置及曝光状态中至少一种，得到筛选参数；根据各所述待处理图像对应的筛选参数，从所述多张待处理图像中筛选出符合预定条件的待处理图像，得到所述多张候选合成图像，其中，所述预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种。In some embodiments of the present application, the image acquisition unit is used to: acquire multiple images to be processed; acquire at least one of the focus state, subject position and exposure state corresponding to each of the images to be processed, and obtain screening parameters. ; According to the screening parameters corresponding to each of the images to be processed, select the images to be processed that meet predetermined conditions from the multiple images to be processed, and obtain the multiple candidate composite images, wherein the predetermined conditions include non- At least one of the following is during the focusing process, not in the exposure convergence process, or meets the subject sharpness requirements.
在本申请的一些实施例中，所述多张待处理图像为根据预定曝光参数采集的；所述装置还包括曝光调节单元，用于：获取各所述待处理图像对应的运动参数及允许位移限值；根据所述运动参数及允许位移限值，确定允许曝光时长限值；根据场景亮度获取小于所述允许曝光时长限值的预定优选曝光时长；基于所述预定优选曝光时长更新所述预定曝光参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined exposure parameters; the device also includes an exposure adjustment unit for: obtaining the motion parameters and allowed displacements corresponding to each of the images to be processed. limit; determine the allowed exposure duration limit based on the motion parameter and the allowed displacement limit; obtain a predetermined preferred exposure duration that is less than the allowed exposure duration limit based on scene brightness; update the predetermined exposure time based on the predetermined preferred exposure duration exposure parameters.
在本申请的一些实施例中，所述多张待处理图像为根据预定对焦参数采集的；所述装置还包括对焦调节单元，用于：根据各所述待处理图像对应的主体位置计算对焦区域；根据所述对焦区域更新所述预定对焦参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined focus parameters; the device further includes a focus adjustment unit for: calculating the focus area according to the subject position corresponding to each of the images to be processed. ;Update the predetermined focus parameter according to the focus area.
在本申请的一些实施例中，在所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像之后，所述装置还包括融合单元，用于：根据所述对齐位置信息，在所述统一坐标系下将所述多张背景降噪图像拼接为的整体背景图像；以及，将所述多张前景降噪图像与所述整体背景图像融合，得到融合轨迹图像；根据所述融合轨迹图像，得到主体运动轨迹图像。In some embodiments of the present application, on the regional images that will be aligned to each of the image regions, noise reduction is performed according to the noise reduction method corresponding to the region type of each of the image regions to obtain multiple background noise reduction images. and multiple foreground noise-reduced images, the device further includes a fusion unit, configured to: splice the multiple background noise-reduced images into an overall background image under the unified coordinate system according to the alignment position information; And, fuse the plurality of foreground noise reduction images with the overall background image to obtain a fused trajectory image; and obtain a subject motion trajectory image based on the fused trajectory image.
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本申请的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。It should be noted that although several modules or units of equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into being embodied by multiple modules or units.
此外，本申请实施例还提供一种电子设备，如图5所示，图5示出了根据本申请的一个实施例的电子设备的框图，具体来讲：In addition, an embodiment of the present application also provides an electronic device, as shown in Figure 5. Figure 5 shows a block diagram of an electronic device according to an embodiment of the present application. Specifically:
该电子设备可以包括一个或者一个以上处理核心的处理器501、一个或一个以上计算机可读存储介质的存储器502、电源503和输入单元504等部件。本领域技术人员可以理解，图5中示出的电子设备结构并不构成对电子设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。The electronic device may include components such as a processor 501 of one or more processing cores, a memory 502 of one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art can understand that the structure of the electronic device shown in FIG. 5 does not constitute a limitation of the electronic device, and may include more or fewer components than shown, or combine certain components, or arrange different components.
其中：in:
处理器501是该电子设备的控制中心，利用各种接口和线路连接整个计算机设备的各个部分，通过运行或执行存储在存储器502内的软件程序和/或模块，以及调用存储在存储器502内的数据，执行计算机设备的各种功能和处理数据，从而对电子设备进行整体监控。可选的，处理器501可包括一个或多个处理核心；优选的，处理器501可集成应用处理器和调制解调处理器，其中，应用处理器主要处理操作系统、用户页面和应用程序等，调制解调处理器主要处理无线通讯。可以理解的是，上述调制解调处理器也可以不集成到处理器501中。The processor 501 is the control center of the electronic device, using various interfaces and lines to connect various parts of the entire computer device, by running or executing software programs and/or modules stored in the memory 502, and calling programs stored in the memory 502. Data, perform various functions of computer equipment and process data to provide overall monitoring of electronic equipment. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, where the application processor mainly processes operating systems, user pages, application programs, etc. , the modem processor mainly handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 501.
存储器502可用于存储软件程序以及模块，处理器501通过运行存储在存储器502的软件程序以及模块，从而执行各种功能应用以及数据处理。存储器502可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据计算机设备的使用所创建的数据等。此外，存储器502可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。相应地，存储器502还可以包括存储器控制器，以提供处理器501对存储器502的访问。The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502 . The memory 502 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may store data based on Data created by the use of computer equipment, etc. In addition, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502 .
电子设备还包括给各个部件供电的电源503，优选的，电源503可以通过电源管理系统与处理器501逻辑相连，从而通过电源管理系统实现管理充电、放电、以及功耗管理等功能。电源503还可以包括一个或一个以上的直流或交流电源、再充电系统、电源故障检测电路、电源转换器或者逆变器、电源状态指示器等任意组件。The electronic device also includes a power supply 503 that supplies power to various components. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so that functions such as charging, discharging, and power consumption management can be implemented through the power management system. The power supply 503 may also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
该电子设备还可包括输入单元504，该输入单元504可用于接收输入的数字或字符信息，以及产生与用户设置以及功能控制有关的键盘、鼠标、操作杆、光学或者轨迹球信号输入。The electronic device may also include an input unit 504 that may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
尽管未示出，电子设备还可以包括显示单元等，在此不再赘述。具体在本实施例中，电子设备中的处理器501会按照如下的指令，将一个或一个以上的计算机程序的进程对应的可执行文件加载到存储器502中，并由处理器501来运行存储在存储器502中的计算机程序，从而实现本申请前述实施例中各种功能，如处理器501可以执行下述步骤：Although not shown, the electronic device may also include a display unit and the like, which will not be described again here. Specifically, in this embodiment, the processor 501 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 502 according to the following instructions, and the processor 501 will run the executable files stored in the computer program. The computer program in the memory 502 can realize various functions in the aforementioned embodiments of the present application. For example, the processor 501 can perform the following steps:
获取多张候选合成图像；基于每张所述候选合成图像中的非主体背景，将所述多张候选合成图像对齐至统一坐标系，得到每张所述候选合成图像的主体前景和非主体背景在所述统一坐标系下的对齐位置信息；根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型；将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，所述多张背景降噪图像和多张前景降噪图像用于合成主体运动轨迹图像。Obtain multiple candidate composite images; based on the non-subject background in each candidate composite image, align the multiple candidate composite images to a unified coordinate system to obtain the subject foreground and non-subject background of each candidate composite image Alignment position information under the unified coordinate system; determine the area types of different image areas under the unified coordinate system according to the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image; Align to the regional images in each of the image areas, perform noise reduction according to the noise reduction method corresponding to the area type of each of the image areas, and obtain multiple background noise reduction images and multiple foreground noise reduction images. The multiple background noise reduction images The noise-reduced image and multiple foreground noise-reduced images are used to synthesize the subject's motion trajectory image.
在本申请的一些实施例中，所述根据每张所述候选合成图像对应的运动参数、前景背景信息及所述对齐位置信息，确定所述统一坐标系下不同图像区域的区域类型，包括：根据每张所述候选合成图像对应的所述运动参数、前景背景信息及所述对齐位置信息，得到对齐至各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型；根据各所述图像区域中区域图像对应的动静类型、前后景类型及重叠非重叠类型的组合，得到各图像区域的区域类型。In some embodiments of the present application, determining the area types of different image areas under the unified coordinate system based on the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image includes: According to the motion parameters, foreground background information and alignment position information corresponding to each candidate composite image, obtain the motion and static types, front and rear background types and overlapping and non-overlapping types corresponding to the regional images aligned to each of the image areas; The area type of each image area is obtained according to the combination of the motion and stillness types, the front and rear background types and the overlapping and non-overlapping types corresponding to the area images in each of the image areas.
在本申请的一些实施例中，所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像，包括：将重叠类型的图像区域中的区域图像，通过多帧融合降噪方式进行降噪，得到融合降噪区域图像；将非重叠类型的图像区域中的区域图像，通过单帧融合降噪方式进行降噪，得到单帧降噪区域图像；根据所述融合降噪区域图像及所述单帧降噪区域图像，得到所述多张背景降噪图像和多张前景降噪图像。In some embodiments of the present application, the area images aligned to each of the image areas are denoised according to the noise reduction method corresponding to the area type of each of the image areas, and multiple background noise reduction images are obtained. Multiple foreground denoising images include: denoising regional images in overlapping image areas through multi-frame fusion denoising methods to obtain fused denoising area images; denoising regional images in non-overlapping image areas, Noise reduction is performed through a single frame fusion noise reduction method to obtain a single frame noise reduction area image; according to the fusion noise reduction area image and the single frame noise reduction area image, the multiple background noise reduction images and multiple foreground images are obtained Noise reduction images.
在本申请的一些实施例中，所述获取多张候选合成图像，包括：获取多张待处理图像；获取各所述待处理图像对应的对焦状态、主体位置及曝光状态中至少一种，得到筛选参数；根据各所述待处理图像对应的筛选参数，从所述多张待处理图像中筛选出符合预定条件的待处理图像，得到所述多张候选合成图像，其中，所述预定条件包括非处于对焦过程中、非处于曝光收敛过程中及符合主体清晰度要求中至少一种。In some embodiments of the present application, obtaining multiple candidate composite images includes: obtaining multiple images to be processed; obtaining at least one of the focus state, subject position and exposure state corresponding to each of the images to be processed, to obtain Screening parameters; according to the screening parameters corresponding to each of the images to be processed, select the images to be processed that meet predetermined conditions from the multiple images to be processed, and obtain the multiple candidate composite images, wherein the predetermined conditions include Not in the process of focusing, not in the process of exposure convergence, or meeting the subject sharpness requirements.
在本申请的一些实施例中，所述多张待处理图像为根据预定曝光参数采集的；还包括：获取各所述待处理图像对应的运动参数及允许位移限值；根据所述运动参数及允许位移限值，确定允许曝光时长限值；根据场景亮度获取小于所述允许曝光时长限值的预定优选曝光时长；基于所述预定优选曝光时长更新所述预定曝光参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined exposure parameters; it also includes: obtaining motion parameters and allowable displacement limits corresponding to each of the images to be processed; according to the motion parameters and The allowed displacement limit determines the allowed exposure duration limit; obtains a predetermined preferred exposure duration that is less than the allowed exposure duration limit based on scene brightness; and updates the predetermined exposure parameters based on the predetermined preferred exposure duration.
在本申请的一些实施例中，所述多张待处理图像为根据预定对焦参数采集的；还包括：根据各所述待处理图像对应的主体位置计算对焦区域；根据所述对焦区域更新所述预定对焦参数。In some embodiments of the present application, the plurality of images to be processed are collected according to predetermined focus parameters; it also includes: calculating a focus area according to the subject position corresponding to each of the images to be processed; and updating the focus area according to the focus area. Predetermined focus parameters.
在本申请的一些实施例中，在所述将对齐至各所述图像区域中的区域图像，根据各所述图像区域的区域类型所对应降噪方式进行降噪，得到多张背景降噪图像和多张前景降噪图像之后，还包括：根据所述对齐位置信息，在所述统一坐标系下将所述多张背景降噪图像拼接为的整体背景图像；以及，将所述多张前景降噪图像与所述整体背景图像融合，得到融合轨迹图像；根据所述融合轨迹图像，得到主体运动轨迹图像。In some embodiments of the present application, on the regional images that will be aligned to each of the image regions, noise reduction is performed according to the noise reduction method corresponding to the region type of each of the image regions to obtain multiple background noise reduction images. and multiple foreground noise reduction images, the method further includes: splicing the multiple background noise reduction images into an overall background image under the unified coordinate system according to the alignment position information; and, splicing the multiple foreground noise reduction images into an overall background image. The noise reduction image is fused with the overall background image to obtain a fused trajectory image; based on the fused trajectory image, a subject motion trajectory image is obtained.
本领域普通技术人员可以理解，上述实施例的各种方法中的全部或部分步骤可以通过计算机程序来完成，或通过计算机程序控制相关的硬件来完成，该计算机程序可以存储于一计算机可读存储介质中，并由处理器进行加载和执行。Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a computer program, or by controlling relevant hardware by a computer program. The computer program can be stored in a computer-readable storage. media and loaded and executed by the processor.
为此，本申请实施例还提供一种存储介质，其中存储有计算机程序，该计算机程序能够被处理器进行加载，以执行本申请实施例所提供的任一种方法中的步骤。To this end, embodiments of the present application also provide a storage medium in which a computer program is stored, and the computer program can be loaded by the processor to execute steps in any method provided by the embodiments of the present application.
其中，该存储介质可以是计算机可读存储介质，该存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)、磁盘或光盘等。The storage medium may be a computer-readable storage medium, and the storage medium may include: read only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.
由于该存储介质中所存储的计算机程序，可以执行本申请实施例所提供的任一种方法中的步骤，因此，可以实现本申请实施例所提供的方法所能实现的有益效果，详见前面的实施例，在此不再赘述。Since the computer program stored in the storage medium can execute steps in any of the methods provided by the embodiments of the present application, it is possible to achieve the beneficial effects that can be achieved by the methods provided by the embodiments of the present application. For details, see above The embodiments will not be described again here.
本领域技术人员在考虑说明书及实践这里公开的实施方式后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary technical means in the technical field that are not disclosed in this application. .
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的实施例，而可以在不脱离其范围的情况下进行各种修改和改变。It should be understood that the present application is not limited to the embodiments which have been described above and shown in the drawings, but various modifications and changes may be made without departing from the scope thereof.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311558456.5ACN117576155A (en) | 2023-11-21 | 2023-11-21 | Motion trajectory image generation method, device, storage medium and electronic equipment |
| PCT/CN2024/126582WO2025107973A1 (en) | 2023-11-21 | 2024-10-22 | Motion trajectory image generation method and apparatus, storage medium, and electronic device |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311558456.5ACN117576155A (en) | 2023-11-21 | 2023-11-21 | Motion trajectory image generation method, device, storage medium and electronic equipment |
| Publication Number | Publication Date |
|---|---|
| CN117576155Atrue CN117576155A (en) | 2024-02-20 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311558456.5APendingCN117576155A (en) | 2023-11-21 | 2023-11-21 | Motion trajectory image generation method, device, storage medium and electronic equipment |
| Country | Link |
|---|---|
| CN (1) | CN117576155A (en) |
| WO (1) | WO2025107973A1 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025107973A1 (en)* | 2023-11-21 | 2025-05-30 | 惠州Tcl移动通信有限公司 | Motion trajectory image generation method and apparatus, storage medium, and electronic device |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4926499B2 (en)* | 2006-02-28 | 2012-05-09 | オリンパス株式会社 | Noise reduction processing apparatus, method, program, and camera apparatus |
| CN111400550A (en)* | 2019-12-30 | 2020-07-10 | 深圳市商汤科技有限公司 | A target motion trajectory construction method, device and computer storage medium |
| CN116134829B (en)* | 2020-12-31 | 2025-10-03 | 华为技术有限公司 | Image processing method and device |
| CN113592887B (en)* | 2021-06-25 | 2022-09-02 | 荣耀终端有限公司 | Video shooting method, electronic device and computer-readable storage medium |
| CN114429484A (en)* | 2022-01-21 | 2022-05-03 | 惠州Tcl移动通信有限公司 | Image processing method and device, intelligent equipment and storage medium |
| CN117576155A (en)* | 2023-11-21 | 2024-02-20 | 惠州Tcl移动通信有限公司 | Motion trajectory image generation method, device, storage medium and electronic equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025107973A1 (en)* | 2023-11-21 | 2025-05-30 | 惠州Tcl移动通信有限公司 | Motion trajectory image generation method and apparatus, storage medium, and electronic device |
| Publication number | Publication date |
|---|---|
| WO2025107973A1 (en) | 2025-05-30 |
| Publication | Publication Date | Title |
|---|---|---|
| CN109348089B (en) | Night scene image processing method, device, electronic device and storage medium | |
| CN108335279B (en) | Image fusion and HDR imaging | |
| US8988529B2 (en) | Target tracking apparatus, image tracking apparatus, methods of controlling operation of same, and digital camera | |
| US9307134B2 (en) | Automatic setting of zoom, aperture and shutter speed based on scene depth map | |
| US10237473B2 (en) | Depth map calculation in a stereo camera system | |
| CN111028189A (en) | Image processing method, device, storage medium and electronic device | |
| CN113379615B (en) | Image processing method and device, storage medium and electronic device | |
| CN109618102B (en) | Focusing processing method, device, electronic device and storage medium | |
| CN111586308B (en) | Image processing method and device and electronic equipment | |
| JP7117872B2 (en) | IMAGE PROCESSING DEVICE, IMAGING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM | |
| US20130222621A1 (en) | Information processing apparatus, terminal apparatus, image capturing apparatus, information processing method, and information provision method for an image capturing apparatus | |
| WO2025107973A1 (en) | Motion trajectory image generation method and apparatus, storage medium, and electronic device | |
| JP6320053B2 (en) | Image processing apparatus, image processing method, and computer program | |
| JP2009111716A (en) | Imaging apparatus, program, and template generation method | |
| CN111263079B (en) | Image processing method, device, storage medium and electronic device | |
| WO2023109389A1 (en) | Image fusion method and apparatus, and computer device and computer-readable storage medium | |
| CN114063864A (en) | Image display method, apparatus, electronic device, and computer-readable storage medium | |
| WO2025082108A1 (en) | Image processing method and apparatus, storage medium, and electronic device | |
| CN110933304B (en) | Method and device for determining to-be-blurred region, storage medium and terminal equipment | |
| WO2025044439A1 (en) | Image processing method and apparatus, and electronic device and storage medium | |
| US10880457B2 (en) | Image processing apparatus, image capturing apparatus, image processing method, and storage medium | |
| US20130342735A1 (en) | Image processing method and image processing apparatus for performing defocus operation according to image alignment related information | |
| US8629925B2 (en) | Image processing apparatus, image processing method, and computer program | |
| CN113807124B (en) | Image processing method, device, storage medium and electronic equipment | |
| JP2019191724A (en) | Image processing apparatus, image processing method, and program |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |