WO2015085922A1 - Method and apparatus for frame rate up-conversion - Google Patents

Abstract

Disclosed in the embodiments of the present invention are a method and apparatus for frame rate up-conversion, which relate to the field of image processing and are aimed at reducing blocking artifacts in interpolation frames to improve the watching experience of a user watching a video. The method provided in the embodiment of the present invention comprises: determining the position of a current block in a first image, a motion vector of the current block relative to a matching block in a second image as well as a similar block corresponding to the current block; obtaining the position of a block to be interpolated, corresponding to the current block, in a frame to be interpolated according to the position of the current block and the motion vector of the current block relative to the matching block; obtaining a weight value of the pixel value of the similar block relative to the pixel value of the current block on the basis of a weighting algorithm; multiplying the pixel value of the similar block by the weight value to obtain the pixel value of the block to be interpolated; and inserting the pixel value of the block to be interpolated to the position of the block to be interpolated.

Description

一种帧率上采样的方法和装置Method and device for frame rate upsampling

本申请要求于2013年12月10日提交中国专利局、申请号为201310669856.3、发明名称为“一种帧率上采样的方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。The present application claims priority to Chinese Patent Application No. 201310669856.3, entitled "A Method and Apparatus for Frame Rate Upsampling", filed on Dec. 10, 2013, the entire contents of In this application.

技术领域Technical field

本发明涉及图像处理领域，尤其涉及一种帧率上采样的方法和装置。The present invention relates to the field of image processing, and in particular, to a method and apparatus for frame rate upsampling.

背景技术Background technique

视频帧率(Frame rate)是指每秒显示的帧数(Frames per Second，FPS)。视频帧率越大，运动画面的平顺度就越好，这样，用户的观赏体验就会越好。一般地，可以通过在待播放视频中相邻的两帧图像之间插入一个或者多个插值帧来提高视频帧率，可以将该方法称为帧率上采样(Frame Rate Up Conversion，FRUC)。The frame rate refers to the number of frames per second (FPS). The larger the video frame rate, the better the smoothness of the moving picture, so that the user's viewing experience will be better. Generally, the video frame rate can be increased by inserting one or more interpolated frames between two adjacent frames of the video to be played. This method can be referred to as Frame Rate Up Conversion (FRUC).

运动补偿帧率上采样(Motion Compensated Frame Rate Up Conversion，MC-FRUC)是一种帧率上采样的方法，主要包括运动估计(Motion Estimation，ME)过程和运动补偿插值(Motion Compensated Interpolation，MCI)过程两大部分。Motion Compensated Frame Rate Up Conversion (MC-FRUC) is a method of frame rate upsampling, which mainly includes Motion Estimation (ME) process and Motion Compensated Interpolation (MCI). The two parts of the process.

运动估计过程具体可以为：将视频流中的每一帧图像分成许多互不重叠的宏块，并认为宏块内所有像素的位移量都相同；将当前帧的前一帧(第n-1帧图像)中的一个宏块作为当前块，在当前帧(第n帧图像)的某一特定搜索范围内根据一定的匹配准则找出与当前块最匹配的匹配块；计算当前块相对匹配块的运动向量。运动补偿插值具体可以为：根据运动估计的结果和一定的插值方法生成插值帧。移动块插值方法是一种常用的插值方法，具体的：将当前块的像素值插入与当前块相距L的位置，其中，L是指当前块相对匹配块的运动向量(mv)的一半，如图1所示。The motion estimation process may specifically be: dividing each frame image in the video stream into a plurality of non-overlapping macroblocks, and considering that all pixels in the macroblock have the same amount of displacement; and the previous frame of the current frame (n-1) One macroblock in the frame image is used as the current block, and the matching block that best matches the current block is found according to a certain matching criterion in a certain search range of the current frame (the nth frame image); the current block relative matching block is calculated. Motion vector. The motion compensation interpolation may specifically be: generating an interpolation frame according to the result of the motion estimation and a certain interpolation method. The moving block interpolation method is a commonly used interpolation method. Specifically, the pixel value of the current block is inserted into the position L from the current block, where L is half of the motion vector (mv) of the current block relative to the matching block, such as Figure 1 shows.

由于相邻宏块(当前块)的运动向量一般不同，因此，移动块插值方法容易导致插值帧中块效应严重的问题，从而影响用户的观赏体验。Since the motion vectors of adjacent macroblocks (current blocks) are generally different, the moving block interpolation method easily causes a problem of serious blockiness in the interpolation frame, thereby affecting the viewing experience of the user.

发明内容Summary of the invention

本发明实施例提供了一种帧率上采样的方法和装置，用以减少插值帧中的块效应，从而提高用户观赏视频时的观赏体验。Embodiments of the present invention provide a method and apparatus for frame rate upsampling, which are used to reduce blockiness in an interpolated frame, thereby improving the viewing experience when a user views a video.

第一方面，提供一种帧率上采样的方法，包括：In a first aspect, a method for frame rate upsampling is provided, including:

确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；Determining a position of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;

根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；Obtaining a position of the to-be-interpolated block corresponding to the current block in the frame to be interpolated according to a position of the current block and a motion vector of the current block relative to the matching block;

基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；Obtaining, according to a weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;

将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；Multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated;

将所述待插值块的像素值插入所述待插值块的位置。Inserting the pixel value of the block to be interpolated into the position of the block to be interpolated.

结合第一方面，在第一种可能的实现方式中，在所述获取所述相似块的像素值相对所述当前块的像素值的权重值之前，所述方法还包括：获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；With reference to the first aspect, in a first possible implementation, before the obtaining a weight value of a pixel value of the similar block relative to a pixel value of the current block, the method further includes: acquiring the similar block The absolute value of the difference between the pixel value and the pixel value of the current block;

所述基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。The obtaining, by the weighting algorithm, the weight value of the pixel value of the similar block relative to the pixel value of the current block, comprising: weighting an absolute value of the difference value based on the weighting algorithm, to obtain the similar block A weight value of a pixel value relative to a pixel value of the current block.

结合第一方面，在第二种可能的实现方式中，在所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置之前，所述方法还包括：With reference to the first aspect, in a second possible implementation, the current block in the frame to be interpolated is obtained according to the location of the current block and the motion vector of the current block relative to the matching block. Before the location of the corresponding block to be interpolated, the method further includes:

按照相同的扩展倍率分别扩展所述当前块和所述相似块；Expanding the current block and the similar block respectively according to the same expansion ratio;

所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置，包括：Obtaining, according to the location of the current block, the location of the current block and the motion vector of the matching block, the location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated, including:

根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；Obtaining, according to the expanded position of the current block and the motion vector of the current block relative to the matching block, a bit of the to-be-interpolated block corresponding to the extended current block in the frame to be interpolatedSet

所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：And multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated, including:

将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。Multiplying the expanded pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated.

结合第一方面，在第三种可能的实现方式中，在所述将所述待插值块的像素值插入所述待插值块的位置之后，所述方法还包括：With reference to the first aspect, in a third possible implementation, after the pixel value of the to-be-interpolated block is inserted into the position of the block to be interpolated, the method further includes:

对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；And performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。Determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.

结合第一方面或者第一方面的第一种可能的实现方式至第三种可能的实现方式任一种，在第四种可能的实现方式中，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。With reference to the first aspect, or the first possible implementation manner of the first aspect, to any one of the third possible implementation manners, in a fourth possible implementation manner, the similarity block includes: the matching block; or m macroblocks in the first region of the first image, m≥1, and m is an integer.

结合第一方面的第四种可能的实现方式，在第五种可能的实现方式中，所述相似块为所述匹配块；所述方法还包括：With reference to the fourth possible implementation of the first aspect, in a fifth possible implementation, the similar block is the matching block; the method further includes:

根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；Obtaining, according to the pixel value of the current block and the pixel value of the matching block, a pixel value of the modified block; the modified block is a virtual macro block, and is used to correct a pixel value of the block to be interpolated;

所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：And multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated, including:

根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。Obtaining a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the correction block, and the weight value.

结合第一方面的第五种可能的实现方式，在第六种可能的实现方式中，所述根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值，包括：With reference to the fifth possible implementation manner of the first aspect, in a sixth possible implementation, the acquiring, according to the pixel value of the current block and the pixel value of the matching block, acquiring a pixel value of the correction block, including :

将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。Using the average value of the pixel value of the current block and the pixel value of the matching block as a correction blockPixel values.

结合第一方面的第四种可能的实现方式，在第七种可能的实现方式中，所述相似块为所述第一图像的第一区域中的m个宏块；With reference to the fourth possible implementation of the first aspect, in a seventh possible implementation, the similar block is the m macroblocks in the first area of the first image;

所述获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：The obtaining the weight value of the pixel value of the similar block relative to the pixel value of the current block includes:

分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；Obtaining n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block, respectively; m≥n≥1, where n is an integer;

所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：And multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated, including:

分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。Multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain pixel values of the to-be-interpolated block .

第二方面，提供一种帧率上采样的装置，包括：In a second aspect, a device for upsampling a frame rate is provided, including:

确定单元，用于确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；a determining unit, configured to determine a location of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;

位置获取单元，用于根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；a location acquiring unit, configured to acquire, according to a location of the current block and a motion vector of the current block with respect to the matching block, a location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated;

权重值获取单元，用于基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；a weight value obtaining unit, configured to acquire, according to a weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;

像素值获取单元，用于将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；a pixel value obtaining unit, configured to multiply a pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated;

插值单元，用于将所述待插值块的像素值插入所述待插值块的位置。An interpolation unit, configured to insert a pixel value of the block to be interpolated into a position of the block to be interpolated.

结合第二方面，在第一种可能的实现方式中，所述权重值获取单元还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；With reference to the second aspect, in a first possible implementation, the weight value obtaining unit is further configured to acquire an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;

所述权重值获取单元具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。The weight value obtaining unit is configured to perform a weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.

结合第二方面，在第二种可能的实现方式中，所述装置还包括：With reference to the second aspect, in a second possible implementation manner, the device further includes:

扩展单元，用于按照相同的扩展倍率分别扩展所述当前块和所述相似块；An expansion unit, configured to separately expand the current block and the similar block according to the same expansion ratio;

所述位置获取单元具体用于，根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；The location obtaining unit is configured to: obtain, according to the extended location of the current block, and the motion vector of the current block relative to the matching block, a to-be-interpolated frame corresponding to the extended current block. The position of the interpolation block;

所述像素值获取单元具体用于，将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。The pixel value obtaining unit is configured to multiply the pixel value of the extended similar block by the weight value to obtain a pixel value of the block to be interpolated.

结合第二方面，在第三种可能的实现方式中，所述像素值获取单元还用于：With reference to the second aspect, in a third possible implementation, the pixel value obtaining unit is further configured to:

对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；And performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。Determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.

结合第二方面或者第二方面的第一种可能的实现方式至第三种可能的实现方式任一种，在第四种可能的实现方式中，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。With reference to the second aspect, or the first possible implementation manner of the second aspect, to the third possible implementation manner, in a fourth possible implementation, the similarity block includes: the matching block; or m macroblocks in the first region of the first image, m≥1, and m is an integer.

结合第二方面的第四种可能的实现方式，在第五种可能的实现方式中，所述相似块为所述匹配块；With reference to the fourth possible implementation of the second aspect, in a fifth possible implementation, the similar block is the matching block;

所述像素值获取单元还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；The pixel value obtaining unit is further configured to: acquire a pixel value of the correction block according to the pixel value of the current block and the pixel value of the matching block; and the correction block is a virtual macro block, and the block to be interpolated is modified. Pixel value;

所述像素值获取单元具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。The pixel value obtaining unit is configured to obtain a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the modified block, and the weight value.

结合第二方面的第五种可能的实现方式，在第六种可能的实现方式中，所述像素值获取单元具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。With reference to the fifth possible implementation of the second aspect, in a sixth possible implementation, the pixel value acquiring unit is specifically configured to: the pixel value of the current block and the pixel value of the matching block The mean value is used as the pixel value of the correction block.

结合第二方面的第四种可能的实现方式，在第七种可能的实现方式中，所述相似块为所述第一图像的第一区域中的m个宏块；With reference to the fourth possible implementation of the second aspect, in a seventh possible implementation, the similar block is m macroblocks in a first region of the first image;

所述权重值获取单元具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；The weight value obtaining unit is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m≥n≥1, where n is an integer ;

分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。Multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain pixel values of the to-be-interpolated block .

本发明实施例提供的帧率上采样的方法和装置，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The method and apparatus for frame rate upsizing provided by the embodiment of the present invention, by determining a current block and a similar block corresponding to the current block, acquiring a weight value of a pixel value of a similar block corresponding to the current block relative to a pixel value of the current block, and The obtained weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

附图说明DRAWINGS

为了更清楚地说明本发明实施例或现有技术的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are merely Some embodiments of the present invention may also be used to obtain other drawings based on these drawings without departing from the art.

图1为现有技术中的一种移动块插值方法的示意图；1 is a schematic diagram of a moving block interpolation method in the prior art;

图2为实施例一提供的一种帧率上采样的方法的流程示意图；2 is a schematic flowchart of a method for sampling a frame rate up according toEmbodiment 1;

图3为实施例1提供的一种帧率上采样的方法的流程示意图；3 is a schematic flowchart of a method for sampling a frame rate up byEmbodiment 1;

图4为实施例1提供的一种表示权重值与绝对误差的变化关系的示意图；4 is a schematic diagram showing a relationship between a weight value and an absolute error according toEmbodiment 1;

图5为实施例1提供的一种移动块插值方法的示意图；FIG. 5 is a schematic diagram of a moving block interpolation method according toEmbodiment 1; FIG.

图6为实施例1提供的另一种帧率上采样的方法的流程示意图；FIG. 6 is a schematic flowchart diagram of another method for sampling frame rate up byEmbodiment 1;

图7为实施例2提供的一种帧率上采样的方法的流程示意图；FIG. 7 is a schematic flowchart diagram of a method for frame rate upsizing provided inEmbodiment 2;

图8为实施例2提供的一种表示扩展后的当前块与当前块的位置关系的示意图；8 is a schematic diagram showing a positional relationship between an extended current block and a current block according toEmbodiment 2;

图9为实施例2提供的一种移动块插值方法的示意图；9 is a schematic diagram of a mobile block interpolation method according toEmbodiment 2;

图10为实施例2提供的另一种帧率上采样的方法的流程示意图；10 is a schematic flowchart diagram of another method for sampling frame rate up byEmbodiment 2;

图11为实施例3提供的一种帧率上采样的方法的流程示意图；FIG. 11 is a schematic flowchart diagram of a method for frame rate upsizing provided by Embodiment 3;

图12为实施例3提供的一种表示当前块与其对应的相似块的位置关系的示意图；12 is a schematic diagram showing a positional relationship between a current block and a corresponding similar block according to Embodiment 3;

图13为实施例3提供的一种移动块插值方法的示意图；FIG. 13 is a schematic diagram of a moving block interpolation method according to Embodiment 3; FIG.

图14为实施例二提供的一种帧率上采样的装置的结构示意图；14 is a schematic structural diagram of an apparatus for frame rate upsampling provided inEmbodiment 2;

图15为实施例二提供的另一种帧率上采样的装置的结构示意图；FIG. 15 is a schematic structural diagram of another apparatus for sampling frame rate up according toEmbodiment 2; FIG.

图16为实施例三提供的一种帧率上采样的装置的结构示意图。16 is a schematic structural diagram of an apparatus for frame rate upsampling provided in Embodiment 3.

具体实施方式detailed description

下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

为了清楚理解本发明实施例，下面对本发明实施例中的一些要素进行解释：In order to clearly understand the embodiments of the present invention, some elements of the embodiments of the present invention are explained below:

(1)当前帧、当前帧的前一帧(1) current frame, the previous frame of the current frame

当前帧是指第n帧图像；前帧的前一帧是指第n-1帧图像。其中，n≥2，n为整数。The current frame refers to the nth frame image; the previous frame of the previous frame refers to the n-1th frame image. Where n≥2, n is an integer.

(2)插值帧、待插值帧(2) Interpolation frame, frame to be interpolated

插值帧是指已经完成插值的图像；待插值帧是指还未完成插值的图像。插值帧和待插值帧均可以称为虚拟帧(即并非视频流本身中的图像，而是通过帧率上采样方法得到的图像)。An interpolated frame refers to an image that has been interpolated; an interpolated frame refers to an image that has not been interpolated. Both the interpolated frame and the frame to be interpolated may be referred to as a virtual frame (ie, not an image in the video stream itself, but an image obtained by a frame rate upsampling method).

(3)第一图像、第二图像(3) First image, second image

第一图像和第二图像可以为一视频流中的任意两帧图像，本发明实施例中均以其为相邻的两帧图像为例进行说明。其中，第一图像可以为第二图像的前一帧图像，也可以为第二图像的后一帧图像。例如，第一图像为当前帧图像，第二图像为当前帧的前一帧图像；又如，第一图像为当前帧的前一帧图像，第二图像为当前帧图像。The first image and the second image may be any two frames in a video stream. In the embodiment of the present invention, the two adjacent frames are taken as an example for description. The first image may be the previous frame image of the second image, or may be the next frame image of the second image. For example, the first image is the current frame image, and the second image is the previous frame image of the current frame; for example, the first image is the previous frame image of the current frame, and the second image is the current frame image.

在视频流的相邻帧之间插入一帧图像的场景中，第一图像和第二图像可以为该视频流本身的图像；在视频流的相邻帧之间插入多帧图像的场景中，第一图像和第二图像可以为该视频流本身的图像，也可以为该视频流插值完成后的图像(即插值帧)。In a scene in which one frame of image is inserted between adjacent frames of a video stream, the first image and the second image may be images of the video stream itself; in a scene in which a plurality of frames of images are inserted between adjacent frames of the video stream, The first image and the second image may be images of the video stream itself, or may be images after the video stream is interpolated (ie, interpolated frames).

(4)宏块(4) Macroblock

将一帧图像等分成若干部分，每一部分称为一个宏块(Macro Block)。一般地，每个宏块为一正方形，若该图像的宽(width)和/或高(height)不能整除该宏块的边长，则将图像宽高扩展成宏块的整数倍。One frame of image is equally divided into sections, each section being called a Macro Block. Generally, each macroblock is a square. If the width and/or height of the image cannot divide the side length of the macroblock, the image width is expanded to an integral multiple of the macroblock.

(5)当前块(5) Current block

将一帧图像中、当前时刻正在处理的的宏块称为当前块。本发明实施例中，当前块所在的图像为第一图像。一般地，按照宏块在图像中所处位置从上到下、从左到右的顺序依次处理该图像中的宏块。A macroblock that is being processed at a current time in one frame of an image is referred to as a current block. In the embodiment of the present invention, the image in which the current block is located is the first image. Generally, macroblocks in the image are processed sequentially in order from top to bottom and left to right in the position of the macroblock.

(6)匹配块(6) Matching block

匹配块是相对当前块而言的，可以描述为与当前块对应的匹配块。本发明实施例中，匹配块所在的图像为第二图像。匹配块可以为一完整宏块，也可以为由多个部分宏块组合而成的一个块；一般地，匹配块为一正方形，其边长与宏块的边长相等。The matching block is relative to the current block and can be described as a matching block corresponding to the current block. In the embodiment of the present invention, the image in which the matching block is located is the second image. The matching block may be a complete macroblock or a block composed of a plurality of partial macroblocks; generally, the matching block is a square whose side length is equal to the side length of the macroblock.

(7)待插值块(7) to be inserted into the block

待插值块是相对当前块而言的，可以描述为与当前块对应的待插值块。待插值块所在的图像为待插值帧，The block to be interpolated is relative to the current block and can be described as the block to be interpolated corresponding to the current block. The image to be interpolated is the frame to be interpolated.

(8)相似块(8) Similar blocks

相似块是相对当前块而言的，可以描述为与当前块对应的相似块。具体是指，获取与当前块对应的待插值块的像素时所使用的、除当前块之外的块。The similarity block is relative to the current block and can be described as a similar block corresponding to the current block. Specifically, it refers to a block other than the current block used when acquiring pixels of the block to be interpolated corresponding to the current block.

(9)修正块(9) Correction block

修正块为一虚拟宏块，用于修正待插值块的像素值。The correction block is a virtual macroblock for correcting the pixel value of the block to be interpolated.

(10)块的像素值(10) the pixel value of the block

块的像素值是指该块中的所有点的像素值之和。例如，匹配块的像素值是指该匹配块中所有点的像素值之和；又如，当前块的像素值是指当前块中的所有点的像素值之和。The pixel value of a block refers to the sum of the pixel values of all points in the block. For example, the pixel value of the matching block refers to the sum of the pixel values of all the points in the matching block; for example, the pixel value of the current block refers to the sum of the pixel values of all the points in the current block.

另外，本发明实施例中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。本发明实施例中字符“/”，一般表示前后关联对象是一种“或”的关系。In addition, the term “and/or” in the embodiment of the present invention is merely an association relationship describing an associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, and A exists at the same time. And B, there are three cases of B alone. In the embodiment of the present invention, the character "/" generally indicates that the contextual object is an "or" relationship.

实施例一Embodiment 1

如图2所示，为本发明实施例提供的一种帧率上采样的方法，包括：As shown in FIG. 2, a method for sampling a frame rate according to an embodiment of the present invention includes:

201：确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块。201: Determine a location of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block.

示例性的，本发明实施例是对运动补偿帧率上采样的方法的改进，主要是对运动补偿帧率上采样的方法中的运动补偿插值过程的改进。Exemplarily, the embodiment of the present invention is an improvement of the method for up-sampling the motion compensation frame rate, and is mainly an improvement of the motion compensation interpolation process in the method of motion compensation frame rate upsampling.

当前块的位置可以用坐标表示，具体可以用当前块的中心点、任一顶点或者该当前块中的其他点的坐标表示。需要说明的是，在具体实现时，需要合理选择使用当前块中的某一个点的坐标表示该当前块的位置。The position of the current block can be represented by coordinates, and can be expressed by the coordinates of the center point of the current block, any vertex, or other points in the current block. It should be noted that, in a specific implementation, it is necessary to reasonably select the coordinates of a certain point in the current block to represent the position of the current block.

第二图像中与当前块对应的匹配块，以及当前块相对该匹配块的运动向量可以在运动估计过程中获得。本发明实施例对实现运动估计的方法不进行限定，可以为现有技术中的任一种方法。例如，可以使用全搜索法(Full Search)、钻石搜索法等获取第二图像中与当前块对应的匹配块。A matching block corresponding to the current block in the second image, and a motion vector of the current block relative to the matching block may be obtained in the motion estimation process. The method for implementing motion estimation is not limited in the embodiment of the present invention, and may be any method in the prior art. For example, a matching block corresponding to the current block in the second image may be acquired using a full search method (Full Search), a diamond search method, or the like.

需要说明的是，具体实现时，在运动估计过程之后，为了提高运动估计过程获得的运动向量的精确度，可以对运动向量进行优化处理，具体可以包括：运动向量判定优化过程、运动向量平滑过程等。因此，步骤201中的“当前块相对第二图像中的一匹配块的运动向量”还可以为：对运动估计过程获得的运动向量进行优化处理之后，得到的运动向量。本发明实施例对具体的优化处理方法不进行限定。以下实施例中所提到的运动向量，均以运动估计过程中得到的运动向量为例进行说明。It should be noted that, in the specific implementation, after the motion estimation process, in order to improve the accuracy of the motion vector obtained by the motion estimation process, the motion vector may be optimized, which may include: a motion vector decision optimization process, a motion vector smoothing process. Wait. Therefore, the “motion vector of a current block relative to a matching block in the second image” instep 201 may also be a motion vector obtained after performing optimization processing on the motion vector obtained by the motion estimation process. The specific optimization processing method is not limited in the embodiment of the present invention. The motion vectors mentioned in the following embodiments are all described by taking the motion vector obtained during the motion estimation process as an example.

另外需要说明的是，由于运动估计的过程可以描述为：以第一图像中的当前块为参考，在第二图像中搜索与当前块对应的匹配块。因此，还可以将第一图像称为参考帧，第二图像称为搜索帧。例如，若第一图像为当前帧(第n帧图像)、第二图像为当前帧的前一帧(第n-1帧图像)，则第n帧为参考帧、第n-1帧为搜索帧；又如，若第一图像为当前帧的前一帧(第n-1帧图像)，第二图像为当前帧(第n帧图像)，则第n-1帧为参考帧，第n帧为搜索帧。It should be further noted that the process of motion estimation may be described as: searching for a matching block corresponding to the current block in the second image with reference to the current block in the first image. Therefore, the first image can also be referred to as a reference frame, and the second image is referred to as a search frame. For example, if the first image is the current frame (the nth frame image) and the second image is the previous frame of the current frame (the n-1th frame image), the nth frame is the reference frame and the n-1th frame is the search. For example, if the first image is the previous frame of the current frame (the n-1th frame image) and the second image is the current frame (the nth frame image), the n-1th frame is the reference frame, the nth The frame is a search frame.

可选的，所述相似块包括但不限于以下两种：Optionally, the similar blocks include but are not limited to the following two types:

(1)所述匹配块；(2)所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。(1) the matching block; (2) m macroblocks in the first region of the first image, m≥1, and m is an integer.

示例性的，第一区域可以由当前块的邻域和/或时域对应的宏块的邻域构成，本发明实施例对m的取值不进行限定。Illustratively, the first area may be formed by the neighborhood of the current block and/or the neighborhood of the macroblock corresponding to the time domain. The value of m is not limited in the embodiment of the present invention.

202：根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置。202: Obtain a location of the to-be-interpolated block corresponding to the current block in the to-be-interpolated frame according to the location of the current block and the motion vector of the current block relative to the matching block.

示例性的，步骤202具体可以为：将待插值帧中与当前块相距L的位置作为待插值帧中的与该当前块对应的待插值块的位置，其中，L是指当前块相对所述匹配块的运动向量的一半。一般地，假设当前块的坐标为(x1，y1)，当前块相对所述匹配块的运动向量为(a，b)，则当第一图像为第二图像的前一帧图像时，待插值帧中的与该当前块对应的待插值块的位置的坐标为：

当第一图像为第二图像的后一帧图像时，待插值帧中的与该当前块对应的待插值块的位置的坐标为：

Exemplarily, the step 202 may be: determining, by using the position of the frame to be interpolated from the current block as the position of the block to be interpolated corresponding to the current block in the frame to be interpolated, where L means that the current block is relative to the current block. Matches half of the motion vector of the block. Generally, assuming that the coordinates of the current block are (x1, y1), and the motion vector of the current block relative to the matching block is (a, b), when the first image is the previous frame image of the second image, the value to be interpolated The coordinates of the position of the block to be interpolated corresponding to the current block in the frame are:

When the first image is the next frame image of the second image, the coordinates of the position of the block to be interpolated corresponding to the current block in the frame to be interpolated are:

203：基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值。203: Acquire a weight value of a pixel value of the similar block relative to a pixel value of the current block based on a weighting algorithm.

示例性的，本发明实施例对确定相似块的像素值相对当前块的像素值的权重值时，所使用的参考指标不进行限定。例如，该参考指标可以为相似块的像素值相对当前块的像素值的差值的绝对值(Sum of Absolute Difference，SAD)，相似块中的某一点的像素值相对该点在当前块中对应的点的像素值的绝对误差(Absolute Difference，AD)等，其中，相似块中的某一点相对该点在当前块中对应的点可以具体为：相似块中的左上顶点相对当前块中的左上顶点，或者，相似块中的中心点相对当前块中的中心点等。Exemplarily, when determining the weight value of the pixel value of the similar block relative to the pixel value of the current block, the reference index used in the embodiment of the present invention is not limited. For example, the reference indicator may be an absolute value (SAD of Absolute Difference, SAD) of a pixel value of a similar block relative to a pixel value of the current block, and a pixel value of a certain point in the similar block corresponds to the point in the current block. Absolute Difference (AD) of the pixel value of the point, etc., wherein a point in the similar block relative to the point in the current block may be specifically: the upper left vertex in the similar block is relative to the upper left in the current block The vertex, or the center point in the similar block, is relative to the center point in the current block.

本发明实施例对具体的加权算法不进行限定，可以为现有技术中的满足“参考指标(例如SAD或者SD)越大权重值越小”的规律的任一种加权算法。例如，可以为：高斯加权算法、线性加权算法、三角函数加权算法、自定义窗函数加权算法等。The embodiment of the present invention does not limit the specific weighting algorithm, and may be any weighting algorithm in the prior art that satisfies the rule that the reference indicator (such as SAD or SD) has a smaller weight value. For example, it may be: a Gaussian weighting algorithm, a linear weighting algorithm, a trigonometric function weighting algorithm, a custom window function weighting algorithm, and the like.

在本发明的一个实施例中，在步骤203之前，该方法还可以包括：获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值。该情况下，步骤203具体可以包括：基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。In an embodiment of the present invention, beforestep 203, the method may further include: acquiring an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block. In this case, thestep 203 may include: performing a weighting operation on the absolute value of the difference based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.

进一步地，为了减少插值帧中的空洞(空洞是指没有插入像素值的部分)，从而提高用户的观赏体验，发明提供下述实施例，具体的：Further, in order to reduce the hole in the interpolation frame (the hole refers to the portion where the pixel value is not inserted), thereby improving the viewing experience of the user, the invention provides the following embodiments, specifically:

在本发明的一个实施例中，在步骤202之前，该方法还包括：按照相同的扩展倍率分别扩展所述当前块和所述相似块。In an embodiment of the invention, beforestep 202, the method further comprises: expanding the current block and the similar block respectively according to the same expansion ratio.

该情况下，步骤202具体包括：根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置。步骤204具体包括：将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。In this case, thestep 202 includes: obtaining the to-be-interpolated frame corresponding to the extended current block according to the expanded position of the current block and the motion vector of the current block relative to the matching block. The location of the interpolation block. Step 204 specifically includes: multiplying the expanded pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated.

示例性的，本发明实施例扩展倍率的取值不进行限定，可以根据经验和具体场景进行设置。例如，经反复实验得出：在智真小幅度运动场景中，扩展倍率为2时，去除空洞的效果较好，且运动物体边界处扭曲和重影较小。For example, the value of the expansion ratio of the embodiment of the present invention is not limited, and may be set according to experience and a specific scenario. For example, after repeated experiments, it is concluded that in the small-magnification motion scene, when the expansion ratio is 2, the effect of removing the hole is better, and the distortion and ghosting at the boundary of the moving object are smaller.

另外，需要说明的是，该情况下，步骤203还可以为：获取扩展后的相似块的像素相对扩展后的当前块像素值的权重值，将该权重值作为相似块的像素相对当前块像素值的权重值。In addition, it should be noted that, in this case, step 203 may be: obtaining a weight value of the pixel of the current block that is expanded relative to the expanded similar block, and using the weight value as a pixel of the similar block relative to the current block pixel. The weight value of the value.

204：将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。204: Multiply the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated.

在本发明的一个实施例中，所述相似块为所述第一图像的第一区域中的m个宏块；步骤203具体包括：分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数。步骤204具体包括：分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。In an embodiment of the present invention, the similarity block is m macroblocks in the first region of the first image; and thestep 203 specifically includes: acquiring pixel values of n macroblocks in the m macroblocks, respectively. n weight values relative to the pixel values of the current block; m≥n≥1, the n being an integer. Thestep 204 includes: multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain the The pixel value of the interpolation block.

205：将所述待插值块的像素值插入所述待插值块的位置。205: Insert a pixel value of the block to be interpolated into a position of the block to be interpolated.

示例性的，将第一图像中的每个宏块作为当前块，执行步骤201-205之后，运动补偿插值过程完成，待插值帧成为了插值帧。该插值帧中的任一点的像素值为覆盖在该点的所有像素的和。Exemplarily, each macroblock in the first image is taken as the current block. After performing steps 201-205, the motion compensation interpolation process is completed, and the frame to be interpolated becomes an interpolation frame. The pixel value of any point in the interpolated frame is the sum of all pixels covering the point.

进一步地，在图像(包括第一图像和第二图像)中包含两个/多个运动物体，或者运动物体旋转、缩放、光照变化的场景中，运动估计过程确定的运动向量往往不精确。另外，当图像中运动物体出现遮挡时，由于被遮挡的块只存在于当前帧或者当前帧的前一帧中，所以利用两帧进行运动估计过程搜索不到遮挡部分的匹配块，该情况下，获得的匹配块可能是错误的。为了减少不精确的运动向量对帧率上采样的结果产生的影响，本发明提供下述实施例，具体的：Further, in a scene in which two or more moving objects are included in an image (including the first image and the second image), or a moving object is rotated, scaled, or changed in illumination, the motion vector determined by the motion estimation process is often inaccurate. In addition, when the moving object in the image is occluded, since the occluded block exists only in the current frame or the previous frame of the current frame, the motion estimation process using two frames cannot search for the matching block of the occlusion portion, in which case The obtained matching block may be wrong. In order to reduce the impact of inaccurate motion vectors on the results of frame rate upsampling, the present invention provides the following embodiments, specifically:

在本发明的一个实施例中，所述相似块为所述匹配块；该方法还可以包括：根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值。该情况下，步骤204具体包括：根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。In an embodiment of the present invention, the similarity block is the matching block; the method may further include: acquiring a pixel value of the modified block according to the pixel value of the current block and the pixel value of the matching block; The block is a virtual macroblock for correcting pixel values of the block to be interpolated. In this case, thestep 204 specifically includes: obtaining, according to the pixel value of the matching block, the pixel value of the correction block, and the weight value, a pixel value of the block to be interpolated.

示例性的，本发明实施例对根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值的具体方法不进行限定。例如，可以具体为：将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。Illustratively, the specific method for obtaining the pixel value of the correction block according to the pixel value of the current block and the pixel value of the matching block is not limited. For example, it may be specifically that the average value of the pixel value of the current block and the pixel value of the matching block is used as the pixel value of the correction block.

可选的，在步骤205之后，该方法还可以包括：Optionally, afterstep 205, the method may further include:

A)对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；A) performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

B)根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。B) determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.

示例性的，重叠像素是指被重复覆盖的像素。由于步骤201中的当前块可以为第一图像中的任一宏块，因此，在执行步骤205之后，待插值帧中可能存在重叠像素。Illustratively, overlapping pixels refer to pixels that are repeatedly covered. Since the current block instep 201 can be any macroblock in the first image, there may be overlapping pixels in the frame to be interpolated after performingstep 205.

关于步骤A)和步骤B)的应用可以包括但不限于以下两种方式：方式1、在将第一图像中的任一宏块作为当前块执行步骤201-204之后，执行步骤A)和步骤B)；方式2、在将第一图像中的每个宏块作为当前块执行步骤201-204之后，执行步骤A)和步骤B)。由于在待插值帧生成插值帧的过程中，一重复像素可能被覆盖多次，若按照方式1，需要多次执行步骤A)和步骤B)，因此，具体实现时，一般使用方式2，下述具体实施例中以方式2为例进行说明。The application regarding step A) and step B) may include, but is not limited to, the following two modes:mode 1, after performing steps 201-204 as any current block in the first image, performing step A) and step B);Mode 2, after performing steps 201-204 for each macroblock in the first image as the current block, performing steps A) and B). Since a repeated pixel may be covered multiple times during the process of generating an interpolation frame in the frame to be interpolated, according tomode 1,Steps A) and B) need to be performed multiple times. Therefore, in the specific implementation, themode 2 is generally used, and themode 2 in the following specific embodiments is taken as an example for description.

进一步地，本实施例中将方式2得到的待插值帧称为插值帧，由于插值帧中可能存在空洞，具体实现时，该方法还可以包括：对插值帧进行填补空洞的处理，其中，填补空洞的处理方法可以使用现有技术中的方法实现，此处不再描述。Further, in the embodiment, the frame to be interpolated obtained by themode 2 is referred to as an interpolated frame. Since there may be a hole in the interpolated frame, the method may further include: performing a filling hole processing on the interpolated frame, where The method of processing the holes can be implemented using methods in the prior art and will not be described here.

本发明实施例提供的帧率上采样的方法，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The frame rate upsampling method provided by the embodiment of the present invention obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block by determining the current block and the similar block corresponding to the current block, and acquires the weight value of the pixel value of the current block. The weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

下面通过具体的实施例对实施例一中描述的帧率上采样的方法进行示例性说明。The method of frame rate upsampling described in the first embodiment will be exemplarily described below through a specific embodiment.

实施例1Example 1

本实施例中，相似块为匹配块，且不对当前块和相似块进行扩展；另外，第一图像和第二图像为一视频流中相邻的两帧图像。本实施例中将第一图像分为R个宏块，每个宏块包含S个像素，S＝16×16。In this embodiment, the similar block is a matching block, and the current block and the similar block are not expanded; in addition, the first image and the second image are two adjacent frames in a video stream. In this embodiment, the first image is divided into R macroblocks, and each macroblock includes S pixels, S=16×16.

如图3所示，为本发明实施例提供的一种帧率上采样的方法，包括：As shown in FIG. 3, a method for frame rate upsampling according to an embodiment of the present invention includes:

301：输入第一图像和第二图像。301: Input a first image and a second image.

302：在第二图像中确定与第一图像中的每个宏块对应的匹配块，以及每个宏块相对其对应的匹配块的运动向量。302: Determine, in the second image, a matching block corresponding to each macroblock in the first image, and a motion vector of each macroblock relative to its corresponding matching block.

示例性的，该步骤302可以称为运动估计过程，本实施例对具体的实现方式不进行限定。Exemplarily, thestep 302 may be referred to as a motion estimation process, and the specific implementation manner is not limited in this embodiment.

303：将第一图像中的第i个宏块作为当前块；确定当前块的位置、与当前块对应的匹配块，当前块相对匹配块的运动向量。303: The i-th macro block in the first image is used as a current block; determining a position of the current block, a matching block corresponding to the current block, and a current block relatively matching the motion vector of the block.

其中，1≤i≤R，i为整数，i的初始值为1。Where 1 ≤ i ≤ R, i is an integer, and the initial value of i is 1.

示例性的，“当前块相对匹配块的运动向量”中的“匹配块”是指：与当前块对应的匹配块，以下步骤中的“匹配块”均是指：与当前块对应的匹配块。Exemplarily, the “matching block” in the “moving vector of the current block relative to the matching block” refers to: the matching block corresponding to the current block, and the “matching block” in the following steps refers to: the matching block corresponding to the current block. .

由于本实施例中不对当前块和相似块(匹配块)进行扩展，因此，可以使用当前块的中心点的坐标表示当前块的位置，使用匹配块的中心点的坐标表示匹配块的位置。Since the current block and the similar block (matching block) are not expanded in this embodiment, the coordinates of the center point of the current block can be used to represent the position of the current block, and the coordinates of the center point of the matching block are used to indicate the position of the matching block.

假设当前块的中心点的坐标为(x1，y1)，匹配块的中心点的坐标为(x2，y2)，则当第一图像为第n-1帧图像、第二图像为第n帧图像时，当前块相对匹配块的运动向量为：(x2-x1，y2-y1)；当第一图像为第n帧图像、第二图像为第n-1帧图像时，当前块相对匹配块的运动向量为：(x1-x2，y1-y2)。Assuming that the coordinates of the center point of the current block are (x1, y1) and the coordinates of the center point of the matching block are (x2, y2), then the first image is the n-1th frame image and the second image is the nth frame image. The motion vector of the current block relative to the matching block is: (x2-x1, y2-y1); when the first image is the nth frame image and the second image is the n-1th frame image, the current block is relatively matched to the block. The motion vector is: (x1-x2, y1-y2).

304：根据当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与当前块对应的待插值块的位置。304: Obtain a position of the to-be-interpolated block corresponding to the current block in the frame to be interpolated according to the position of the current block and the motion vector of the current block relative to the matching block.

示例性的，按照实施例一中的描述，针对步骤303中的示例，步骤304可以具体为：Exemplarily, according to the description in the first embodiment, for the example in step 303, step 304 may be specifically:

当第一图像为第n-1帧图像、第二图像为第n帧图像时，插值帧中的与当前块对应的待插值块的位置可以表示为:

即

When the first image is the n-1th frame image and the second image is the nth frame image, the position of the interpolated block corresponding to the current block in the interpolated frame may be expressed as:

which is

当第一图像为第n帧图像、第二图像为第n-1帧图像时，插值帧中的与当前块对应的待插值块的位置可以表示为:

即

When the first image is the nth frame image and the second image is the n-1th frame image, the position of the interpolated block corresponding to the current block in the interpolated frame may be expressed as:

which is

由此可知，无论第一图像是第二图像的前一帧图像还是后一帧图像，只要确定当前块的坐标和匹配块的坐标，即可确定与当前块对应的待插值块的坐标。It can be seen that whether the first image is the previous frame image or the subsequent frame image of the second imageFor example, as long as the coordinates of the current block and the coordinates of the matching block are determined, the coordinates of the block to be interpolated corresponding to the current block can be determined.

305：获取匹配块的像素值相对当前块的像素值的差值的绝对值。305: Obtain an absolute value of a difference between a pixel value of the matching block and a pixel value of the current block.

示例性的，匹配块中的第s个像素的像素值相对当前块中的第s个像素的像素值的绝对误差AD(s)可以通过如下公式计算：Exemplarily, the absolute error AD(s) of the pixel value of the sth pixel in the matching block relative to the pixel value of the sth pixel in the current block can be calculated by the following formula:

AD(s)＝|MB_ref(s)-MB_search(s)|；AD(s)=|MB_ref (s)-MB_search (s)|;

其中，MB_ref(s)是指第一图像(即参考帧)的当前块(MB_ref)中的第s个像素的像素值，MB_search(s)是指第二图像(即搜索帧)中与当前块对应的匹配块(MB_search)中的第s个像素的像素值。另外，按照步骤302中的示例，s∈{1，2，…，S}。Where MB_ref (s) refers to the pixel value of the sth pixel in the current block (MB_ref ) of the first image (ie, the reference frame), and MB_search (s) refers to the second image (ie, the search frame) The pixel value of the sth pixel in the matching block (MB_search ) corresponding to the current block. In addition, according to the example instep 302, s ∈ {1, 2, ..., S}.

按照步骤303中的示例，匹配块的像素值相对当前块的像素值的差值的绝对值SAD可以通过如下公式计算：According to the example in step 303, the absolute value SAD of the difference between the pixel value of the matching block and the pixel value of the current block can be calculated by the following formula:

306：基于加权算法对该差值的绝对值进行加权运算，得到匹配块的像素值相对当前块的像素值的权重值。306: Perform a weighting operation on the absolute value of the difference based on the weighting algorithm to obtain a weight value of the pixel value of the matching block relative to the pixel value of the current block.

示例性的，下面以加权算法为高斯加权算法对步骤306进行说明：Exemplarily, step 306 is described below as a Gaussian weighting algorithm by a weighting algorithm:

匹配块中的第s个像素的像素值相对当前块中的第s个像素的像素值的权重值w(s)可以根据以下公式获得：The weight value w(s) of the pixel value of the sth pixel in the matching block relative to the pixel value of the sth pixel in the current block can be obtained according to the following formula:

其中，μ是指期望值可以固定取0；σ是指方差，一般可取固定值5或者10，当然不限于此。Wherein, μ means that the expected value can be fixedly taken as 0; σ means the variance, and generally can take a fixed value of 5 or 10, of course, not limited to this.

匹配块的像素值相对当前块的像素值的权重值可以根据以下公式获得：The weight value of the pixel value of the matching block relative to the pixel value of the current block can be obtained according to the following formula:

如图4所示，为σ取不同值时，w(s)与AD(s)之间的关系。由该图4可以看出，w(s)随着AD(s)的增加而减小，当AD(s)增大到一定程度时，w(s)趋于0。也就是说，当匹配块中的第s个像素的像素值与当前块中的第s个像素的像素值的差超过一定数值时，当前块中的第s个像素的像素值不会插入插值帧中。例如，取σ＝10，由图4可知，当AD(s)超过30时，w(s)几乎降为0，也就是说，当匹配块中的第s个像素的像素值与当前块中的第s个像素的像素值的差超过30时，当前块中的第s个像素的像素值不会插入插值帧中。由此可知，当匹配块的像素值与当前块的像素值的差超过一定数值时，当前块中的像素值不会插入插值帧中。As shown in FIG. 4, the relationship between w(s) and AD(s) when σ takes different values. As can be seen from FIG. 4, w(s) decreases as AD(s) increases, and when AD(s) increases to a certain extent, w(s) tends to zero. That is, when the difference between the pixel value of the sth pixel in the matching block and the pixel value of the sth pixel in the current block exceeds a certain value, the pixel value of the sth pixel in the current block is not inserted into the interpolation. In the frame. For example, taking σ=10, as can be seen from FIG. 4, when AD(s) exceeds 30, w(s) is almost reduced to 0, that is, when the pixel value of the sth pixel in the matching block is in the current block. When the difference of the pixel values of the sth pixel exceeds 30, the pixel value of the sth pixel in the current block is not inserted into the interpolation frame. It can be seen that when the difference between the pixel value of the matching block and the pixel value of the current block exceeds a certain value, the pixel value in the current block is not inserted into the interpolation frame.

307：将匹配块的像素值乘以权重值，得到待插值块的像素值。307: Multiply the pixel value of the matching block by the weight value to obtain the pixel value of the block to be interpolated.

示例性的，待插值块中的第s个像素的像素值MB_interpolate(s)可以根据以下公式获得：Exemplarily, the pixel value MB_interpolate (s) of the_sth pixel in the block to be interpolated can be obtained according to the following formula:

MB_interpolate(s)＝MB_search(s)w(s)；MB_interpolate (s)=MB_search (s)w(s);

待插值块的像素值

可以根据以下公式获得：Pixel value of the block to be interpolated

It can be obtained according to the following formula:

308：将待插值块的像素值插入待插值块的位置。308: Insert the pixel value of the block to be interpolated into the position of the block to be interpolated.

示例性的，本实施例中结合当前块的像素值、匹配块的像素值得到待插值块的像素值，并将待插值块的像素值插入待插值块的位置，如图5所示，其中，图5中第一图像为第n-1帧图像，第二图像为第n帧图像。Exemplarily, in this embodiment, the pixel value of the block to be interpolated is obtained by combining the pixel value of the current block and the pixel value of the matching block, and the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, as shown in FIG. 5 , wherein In FIG. 5, the first image is the n-1th frame image, and the second image is the nth frame image.

309：判断i是否大于等于R。309: Determine whether i is greater than or equal to R.

若否，则执行步骤310；若是，则执行步骤311。If no, step 310 is performed; if yes, step 311 is performed.

示例性的，步骤309的判断结果为“是”，说明：已经在待插值帧中插入了第一图像中的每个宏块对应的待插入块。Exemplarily, the determination result ofstep 309 is “Yes”, indicating that the to-be-inserted block corresponding to each macroblock in the first image has been inserted in the frame to be interpolated.

310：i自加1。310: i is incremented by 1.

步骤310之后，返回步骤303。Afterstep 310, the process returns to step 303.

311：分别对待插值帧中每个重叠像素的像素值进行加权累加处理，得到每个重叠像素的中间像素值。311: Perform weighted accumulation processing on pixel values of each overlapping pixel in the interpolation frame, respectively, to obtain an intermediate pixel value of each overlapping pixel.

假设像素t为一重叠像素，则像素t的中间像素值ws(t)可以根据以下公式获得：Assuming that the pixel t is an overlapping pixel, the intermediate pixel value ws(t) of the pixel t can be obtained according to the following formula:

其中，J是指该像素t被重复覆盖的次数，即覆盖该像素t的待插值块的个数；MB_{int erpolate}(s)_j是指覆盖该像素t的第j个待插值块(MB_{int erpolate})，w(t)_j是指覆盖像素t的第j个待插值块(MB_{int erpolate})相对与第j个待插值块对应的当前块中相应像素的像素值的权重值。Where J is the number of times the pixel t is repeatedly covered, that is, the number of blocks to be interpolated that cover the pixel t; MB_{int erpolate} (s)_j refers to the jth block to be interpolated (MB_int ) covering the pixel t_Erpolate ), w(t)_j refers to the weight value of the pixel value of the corresponding pixel in the current block corresponding to the j-th interpolated block corresponding to the j-th interpolated block (MB_{int erpolate} ) of the pixel t.

312：根据步骤306中使用的加权算法和每个重叠像素的中间像素值，分别确定每个重叠像素的目标像素值，得到插值帧。312: Determine a target pixel value of each overlapping pixel according to the weighting algorithm used instep 306 and the intermediate pixel value of each overlapping pixel, to obtain an interpolated frame.

示例性的，按照步骤311中的示例，步骤312具体可以实现为：Exemplarily, according to the example in step 311, step 312 may be specifically implemented as:

像素t的权重和sw(t)可以通过以下公式获得：

The weight of the pixel t and sw(t) can be obtained by the following formula:

像素t的目标像素值

可以通过以下公式获得：

Target pixel value of pixel t

It can be obtained by the following formula:

313：输出该插值帧。313: Output the interpolation frame.

可选的，如图6所示，步骤307可以用以下步骤307A-307B替换：Optionally, as shown in FIG. 6, step 307 can be replaced by the followingsteps 307A-307B:

307A：将当前块的像素值与匹配块的像素值的均值作为修正块的像素值。307A: using the mean value of the pixel value of the current block and the pixel value of the matching block as a correction blockPixel values.

示例性的，分别将当前块中的每个像素的像素值与匹配块中相应像素的像素值求均值，得到修正块中相应像素的像素值。例如，将当前块中的第3个像素的像素值与匹配块中的第3个像素的像素值求均值，得到修正块中第3个像素的像素值。Exemplarily, the pixel values of each pixel in the current block and the pixel values of the corresponding pixels in the matching block are respectively averaged to obtain pixel values of corresponding pixels in the modified block. For example, the pixel value of the third pixel in the current block and the pixel value of the third pixel in the matching block are averaged to obtain the pixel value of the third pixel in the correction block.

307B：根据匹配块的像素值、修正块的像素值和步骤306中获得的权重值，得到待插值块的像素值。307B: Obtain a pixel value of the block to be interpolated according to the pixel value of the matching block, the pixel value of the correction block, and the weight value obtained instep 306.

示例性的，步骤307B可以包括但不限于通过以下方式实现：Exemplarily,step 307B may include, but is not limited to, being implemented in the following manner:

待插值块中的第s个像素的像素值MB_{int erpolate}(s)可以根据以下公式获得：The pixel value MB_{int erpolate} (s) of the_sth pixel in the block to be interpolated can be obtained according to the following formula:

MB_{int erpolate}(s)＝MB_search(s)w(s)+MB_correct(1-w(s))；其中，MB_correct是指修正块。MB_{int erpolate} (s)=MB_search (s)w(s)+MB_correct (1-w(s)); where MB_correct refers to the correction block.

待插值块的像素值

可以根据以下公式获得：Pixel value of the block to be interpolated

It can be obtained according to the following formula:

需要说明的是，用步骤307A-307B替换步骤307的方案，可以减少不精确的运动向量对帧率上采样的结果产生的影响。It should be noted that, by replacing the scheme ofstep 307 withsteps 307A-307B, the influence of the inaccurate motion vector on the result of the frame rate upsampling can be reduced.

本发明实施例提供的帧率上采样的方法，通过动态获取与当前块对应的匹配块的像素值相对当前块的像素值的权重值，并将动态获取的权重值用于获得待插值块的像素值的过程中，从而实现帧率上采样。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The frame rate up-sampling method provided by the embodiment of the present invention dynamically obtains the weight value of the pixel value of the matching block corresponding to the current block relative to the pixel value of the current block, and uses the dynamically obtained weight value to obtain the block to be interpolated. In the process of pixel values, frame rate upsampling is achieved. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

实施例2Example 2

本实施例中，相似块为匹配块，且对当前块和相似块进行扩展；另外，第一图像和第二图像为一视频流中相邻的两帧图像。本实施例中将第一图像分为R个宏块，每个宏块包含16×16个像素。In this embodiment, the similar block is a matching block, and the current block and the similar block are expanded; in addition, the first image and the second image are two adjacent frames in a video stream. In this embodiment, the first image is divided into R macroblocks, each of which contains 16×16 pixels.

如图7所示，为本发明实施例提供的一种帧率上采样的方法，包括：As shown in FIG. 7, a method for sampling a frame rate according to an embodiment of the present invention includes:

701：输入第一图像和第二图像。701: Input a first image and a second image.

702：在第二图像中确定与第一图像中的每个宏块对应的匹配块，以及每个宏块相对其对应的匹配块的运动向量。702: Determine, in the second image, a matching block corresponding to each macroblock in the first image, and a motion vector of each macroblock relative to its corresponding matching block.

703：将第一图像中的第i个宏块作为当前块；确定当前块的位置、与当前块对应的匹配块，当前块相对匹配块的运动向量。其中，1≤i≤R，i为整数，i的初始值为1。703: The i-th macroblock in the first image is used as a current block; the location of the current block, the matching block corresponding to the current block, and the motion vector of the current block relative to the matching block are determined. Where 1 ≤ i ≤ R, i is an integer, and the initial value of i is 1.

示例性的，步骤701-703的相关解释和具体示例均可以参考上述实施例1的步骤301-303，下面仅对与实施例1的不同之处进行说明：For example, the related explanations and specific examples of steps 701-703 can refer to steps 301-303 of theabove embodiment 1, and only the differences from theembodiment 1 are explained below:

由于本实施例中对当前块和相似块(匹配块)进行扩展，这里的“扩展”一般是指以原点为中心扩展当前块和匹配块的像素，扩展前后一个块(包括当前块块或者匹配块)的中心点的坐标不改变，因此，一般不使用块(包括当前块块或者匹配块)的中心点的坐标表示该块的位置，本实施例中使用当前块的左顶点坐标表示当前块的位置，使用匹配块的左顶点坐标表示匹配块的位置。Since the current block and the similar block (matching block) are extended in this embodiment, the "extension" herein generally refers to expanding the pixels of the current block and the matching block centering on the origin, and expanding one block before and after (including the current block or matching). The coordinates of the center point of the block are not changed. Therefore, the coordinates of the center point of the block (including the current block or the matching block) are generally not used to indicate the position of the block. In this embodiment, the coordinates of the left vertex of the current block are used to represent the current block. The position, using the coordinates of the left vertex of the matching block to indicate the position of the matching block.

704：按照相同的扩展倍率分别扩展当前块和匹配块。704: Expand the current block and the matching block respectively according to the same expansion ratio.

示例性的，本实施例中以扩展倍率为2进行说明，具体是指，将宏块的边长扩展2倍。按照扩展倍率为2进行扩展后，每个宏块中包含S个像素，S＝32×32。Exemplarily, in the present embodiment, the expansion ratio is 2, and specifically, the side length of the macro block is expanded by 2 times. After spreading according to the expansion ratio of 2, each macroblock contains S pixels, S = 32 × 32.

705：根据扩展后的当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与扩展后的当前块对应的待插值块的位置。705. Acquire, according to the position of the extended current block and the motion vector of the current block relative matching block, a position of the to-be-interpolated block corresponding to the expanded current block in the frame to be interpolated.

示例性的，如图8所示，假设当前块的左顶点为A，当前块的位置具体可以为A点的坐标，扩展后的当前块的左顶点为B，则扩展后的当前块的位置具体为B点的坐标。Exemplarily, as shown in FIG. 8 , it is assumed that the left vertex of the current block is A, the position of the current block may specifically be the coordinates of point A, and the left vertex of the expanded current block is B, and the position of the expanded current block is Specifically, the coordinates of point B.

706：获取匹配块的像素值相对当前块的像素值的差值的绝对值。706: Obtain an absolute value of a difference between a pixel value of the matching block and a pixel value of the current block.

707：基于加权算法对该差值的绝对值进行加权云端，得到匹配块的像素相对当前块的像素值的权重值。707: Weighting the absolute value of the difference based on a weighting algorithm to obtain a matching blockThe weight value of the pixel relative to the pixel value of the current block.

可选的，步骤706可以用以下步骤替换：获取扩展后的匹配块的像素值相对扩展后的当前块的像素值的差值的绝对值；该情况下，步骤707可以用以下步骤替换：根据加权算法和该差值的绝对值，获取扩展后的匹配块的像素相对扩展后的当前块的像素值的权重值。Optionally,step 706 may be replaced by: obtaining an absolute value of a difference between a pixel value of the extended matching block and a pixel value of the expanded current block; in this case, step 707 may be replaced by the following steps: The weighting algorithm and the absolute value of the difference value obtain the weight value of the pixel value of the current block of the expanded matching block relative to the current block.

本实施例中的步骤706-707与实施例1中的步骤305-306相同。可选的方式中的具体示例可以参考实施例1中的步骤305-306，不同之处在于，S的取值不同。Steps 706-707 in this embodiment are the same as steps 305-306 inEmbodiment 1. For a specific example in the alternative manner, reference may be made to steps 305-306 inEmbodiment 1, except that the values of S are different.

708：将扩展后的匹配块的像素值乘以该权重值，得到待插值块的像素值。708: Multiply the pixel value of the extended matching block by the weight value to obtain a pixel value of the block to be interpolated.

709：将待插值块的像素值插入待插值块的位置。709: Insert the pixel value of the block to be interpolated into the position of the block to be interpolated.

示例性的，本实施例中结合扩展后的当前块的像素值、扩展后的匹配块的像素值得到待插值块的像素值，并将待插值块的像素值插入待插值块的位置，如图9所示，其中，图9中第一图像为第n-1帧图像，第二图像为第n帧图像。Exemplarily, in this embodiment, the pixel value of the expanded current block is combined with the pixel value of the expanded matching block, and the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, such as As shown in FIG. 9, the first image in FIG. 9 is the n-1th frame image, and the second image is the nth frame image.

710：判断i是否大于等于R。710: Determine whether i is greater than or equal to R.

若否，则执行步骤711；若是，则执行步骤712。If no, step 711 is performed; if yes, step 712 is performed.

示例性的，步骤710的判断结果为“是”，说明：已经在待插值帧中插入了第一图像中的每个宏块对应的待插入块。Exemplarily, the determination result ofstep 710 is “Yes”, indicating that the to-be-inserted block corresponding to each macroblock in the first image has been inserted in the frame to be interpolated.

711：i自加1。711: i is incremented by 1.

步骤711之后，返回步骤703。Afterstep 711, the process returns to step 703.

712：分别对待插值帧中每个重叠像素的像素值进行加权累加处理，得到每个重叠像素的中间像素值。712: Perform weighted accumulation processing on pixel values of each overlapping pixel in the interpolation frame, respectively, to obtain an intermediate pixel value of each overlapping pixel.

713：根据步骤707中使用的加权算法和每个重叠像素的中间像素值，分别确定每个重叠像素的目标像素值，得到插值帧。713: Determine a target pixel value of each overlapping pixel according to the weighting algorithm used instep 707 and the intermediate pixel value of each overlapping pixel, to obtain an interpolation frame.

714：输出该插值帧。714: Output the interpolation frame.

示例性的，步骤710-714的具体示例可以参考实施例1中的步骤309-313，此处不再赘述。For example, specific examples of steps 710-714 may refer to steps 309-313 inEmbodiment 1, and details are not described herein again.

可选的，如图10所示，步骤708可以用以下步骤708A-708B替换：Alternatively, as shown in FIG. 10,step 708 can be replaced with the followingsteps 708A-708B:

708A：将扩展后的当前块的像素值与扩展后的匹配块的像素值的均值作为修正块的像素值。708A: The average value of the pixel value of the expanded current block and the pixel value of the expanded matching block is used as the pixel value of the correction block.

708B：根据扩展后的匹配块的像素值、修正块的像素值和步骤707中获得的权重值，得到待插值块的像素值。708B: Obtain a pixel value of the block to be interpolated according to the pixel value of the expanded matching block, the pixel value of the modified block, and the weight value obtained instep 707.

需要说明的是，用步骤708A-708B替换步骤708的方案，可以减少不精确的运动向量对帧率上采样的结果产生的影响。It should be noted that replacing the scheme ofstep 708 withsteps 708A-708B can reduce the impact of inaccurate motion vectors on the result of frame rate upsampling.

示例性的，本实施例中的步骤708-714、708A、708B中的相关解释和具体示例可以参考实施例1中相关，其中，具体实现时，需要将实施例1中的“匹配块”的替换为“扩展后的匹配块”，将实施例1中的“当前块”替换为“扩展后的当前块”；将实施例1中S的取值由16×16替换为32×32。For example, related explanations and specific examples in steps 708-714, 708A, and 708B in this embodiment may be referred to inEmbodiment 1, wherein, in specific implementation, the "matching block" inEmbodiment 1 is required. Replaced with "expanded matching block", the "current block" inEmbodiment 1 is replaced with "expanded current block"; the value of S inEmbodiment 1 is replaced by 16x16 to 32x32.

2)S的取值不同。2) The value of S is different.

另外，实施例1中获得的每个待插值块中包含16×16个像素，实施例2中获得的每个待插值块中包含32×32个像素。Further, each of the blocks to be interpolated obtained inEmbodiment 1 contains 16 × 16 pixels, and each block to be interpolated obtained inEmbodiment 2 contains 32 × 32 pixels.

本发明实施例提供的帧率上采样的方法，通过动态获取与当前块对应的匹配块的像素值相对当前块的像素值的权重值，并对当前块和匹配块进行扩展，结合动态获取的权重值、扩展后的当前块的像素的像素值和扩展后的匹配块的像素的像素值获取待插值块的像素值，从而实现帧率上采样。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The frame rate upsampling method provided by the embodiment of the present invention dynamically obtains the weight value of the pixel value of the matching block corresponding to the current block relative to the pixel value of the current block, and expands the current block and the matching block, and combines the dynamic acquisition. The weight value, the pixel value of the pixel of the expanded current block, and the pixel value of the pixel of the expanded matching block acquire the pixel value of the block to be interpolated, thereby achieving frame rate upsampling. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

实施例3Example 3

本实施例中，相似块为第一图像的第一区域中的m个宏块，且不对相似块进行扩展。另外，第一图像和第二图像为一视频流中相邻的两帧图像。本实施例中将第一图像分为R个宏块，每个宏块包含S个像素，S＝16×16。In this embodiment, the similar block is m macroblocks in the first region of the first image, and is notExtend similar blocks. In addition, the first image and the second image are two adjacent frames in a video stream. In this embodiment, the first image is divided into R macroblocks, and each macroblock includes S pixels, S=16×16.

如图11所示，为本发明实施例提供的另一种帧率上采样的方法，包括：As shown in FIG. 11, another method for sampling a frame rate according to an embodiment of the present invention includes:

1101：输入第一图像和第二图像。1101: Input a first image and a second image.

1102：在第二图像中确定与第一图像中的每个宏块对应的匹配块，以及每个宏块相对其对应的匹配块的运动向量。1102: Determine, in the second image, a matching block corresponding to each macroblock in the first image, and a motion vector of each macroblock relative to its corresponding matching block.

示例性的，该步骤1102可以称为运动估计过程，本实施例对具体的实现方式不进行限定。For example, thestep 1102 may be referred to as a motion estimation process, and the specific implementation manner is not limited in this embodiment.

1103：将第一图像中的第i个宏块作为当前块；确定当前块的位置、当前块相对第二图像中一匹配块的运动向量、当前块对应的相似块，其中，相似块为第一图像的第一区域中的m个宏块。1103: The i-th macroblock in the first image is used as a current block; determining a location of the current block, a motion vector of a current block relative to a matching block in the second image, and a similar block corresponding to the current block, where the similar block is m macroblocks in the first region of an image.

其中，1≤i≤R，i为整数，i的初始值为1；m≥1，m为整数。Where 1 ≤ i ≤ R, i is an integer, the initial value of i is 1; m ≥ 1, and m is an integer.

示例性的，第一区域可以由当前块的邻域和/或时域对应的宏块的邻域构成。如图12所示，为当前块对应的相似块的示例，当然，具体实现时不限于此。Exemplarily, the first region may be composed of a neighborhood of a current block and/or a neighborhood of a macroblock corresponding to the time domain. As shown in FIG. 12, it is an example of a similar block corresponding to the current block. Of course, the specific implementation is not limited thereto.

1104：根据当前块的位置和当前块相对匹配块的运动向量，获取待插值帧中的与当前块对应的待插值块的位置。1104: Obtain a position of the to-be-interpolated block corresponding to the current block in the frame to be interpolated according to the position of the current block and the motion vector of the current block relative to the matching block.

1105：获取第j个相似块的像素值相对当前块的像素值的差值的绝对值。1105: Acquire an absolute value of a difference between a pixel value of the jth similar block and a pixel value of the current block.

1106：基于加权算法，对该差值的绝对值进行加权运算，得到第j个相似块的像素值相对当前块的像素值的权重值。1106: Perform a weighting operation on the absolute value of the difference based on the weighting algorithm to obtain a weight value of the pixel value of the jth similar block relative to the pixel value of the current block.

其中，j的初始值为1，1≤j≤m，j为整数。Wherein, the initial value of j is 1, 1 ≤ j ≤ m, and j is an integer.

示例性的，步骤1105-1106的具体示例可以参考上述实施例1中的步骤305-306。其中，具体实现时，将步骤305-306中的“匹配块的像素值”替换为“第j个相似块的像值素”。For example, specific examples of steps 1105-1106 may refer to steps 305-306 in theabove embodiment 1. In the specific implementation, the “pixel value of the matching block” in steps 305-306 is replaced with “the image value element of the jth similar block”.

1107：将第j个相似块的像素值乘以该权重值，得到待插值块的第j像素值。1107: Multiply the pixel value of the jth similar block by the weight value to obtain a jth pixel value of the block to be interpolated.

需要说明的是，为了清楚说明本实施例的方法，引入“插值块的第j像素值”的概念。“插值块的第j像素值”具体是指，第j次获得与第i个当前块对应的待插值块的像素值。It should be noted that, in order to clearly explain the method of the present embodiment, the concept of "jth pixel value of an interpolated block" is introduced. The “jth pixel value of the interpolation block” specifically means that the pixel value of the block to be interpolated corresponding to the i th current block is obtained in the jth time.

另外需要说明，在实施例1和实施例2中，针对第i个当前块，只获取一次与该第i个当前块对应的待插值块的像素值；本实施例中，针对第i个当前块，获取m次与该第i个当前块对应的待插值块的像素值。It should be noted that, in the first embodiment and the second embodiment, the pixel value of the block to be interpolated corresponding to the i th current block is acquired only once for the i th current block; in this embodiment, for the i th current Block, obtaining pixel values of the block to be interpolated corresponding to the i-th current block m times.

1108：判断j是否大于或者等于m。1108: Determine whether j is greater than or equal to m.

若否，则执行步骤1109；若是，则执行步骤1111。If no, go to step 1109; if yes, go tostep 1111.

示例性的，步骤1107的判断结果为“是”，说明：已经获取了m次与第i个当前块对应的待插值块的像素值。Exemplarily, the determination result ofstep 1107 is “Yes”, indicating that the pixel values of the block to be interpolated corresponding to the i-th current block have been acquired m times.

1109：j自加1。1109: j is added by 1.

执行步骤1109之后，返回步骤1105。After step 1109 is performed, the process returns to step 1105.

1110：对待插值块的m个像素值进行求和运算，得到待插值块的像素值。1110: Perform a summation operation on the m pixel values of the interpolation block to obtain a pixel value of the block to be interpolated.

1111：将待插值块的像素值插入待插值块的位置。1111: Insert the pixel value of the block to be interpolated into the position of the block to be interpolated.

示例性的，本实施例中结合当前块的像素值、当前块周围的相似块的像素值得到待插值块的像素值，并将待插值块的像素值插入待插值块的位置，如图13所示，其中，图13中第一图像为第n-1帧图像，第二图像为第n帧图像。Exemplarily, in this embodiment, the pixel value of the block to be interpolated is obtained by combining the pixel value of the current block and the pixel value of the similar block around the current block, and the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, as shown in FIG. As shown therein, the first image in FIG. 13 is the n-1th frame image, and the second image is the nth frame image.

1112：判断i是否大于等于R。1112: Determine whether i is greater than or equal to R.

若否，则执行步骤1113；若是，则执行步骤1114。If no, go tostep 1113; if yes, go tostep 1114.

示例性的，步骤1112的判断结果为“是”，说明：已经在待插值帧中插入了第一图像中的每个宏块对应的待插入块。Exemplarily, the determination result ofstep 1112 is “Yes”, indicating that the to-be-inserted block corresponding to each macroblock in the first image has been inserted in the frame to be interpolated.

1113：i自加1。1113: i is incremented by 1.

步骤1113之后，返回步骤1103。Afterstep 1113, the process returns to step 1103.

1114：分别对待插值帧中每个重叠像素的像素值进行加权累加处理，得到每个重叠像素的中间像素值。1114: Perform weighted accumulation processing on pixel values of each overlapping pixel in the interpolation frame, respectively, to obtain an intermediate pixel value of each overlapping pixel.

1115：根据步骤1106中使用的加权算法和每个重叠像素的中间像素值，分别确定每个重叠像素的目标像素值，得到插值帧。1115: Determine a target pixel value of each overlapping pixel according to the weighting algorithm used instep 1106 and the intermediate pixel value of each overlapping pixel, to obtain an interpolated frame.

1116：输出该插值帧。1116: Output the interpolation frame.

示例性的，步骤1112-1116的具体示例可以参考实施例1中的步骤309-313，此处不再赘述。For example, the specific examples of the steps 1112-1116 may refer to steps 309-313 inEmbodiment 1, and details are not described herein again.

本发明实施例提供的帧率上采样的方法，通过动态获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将动态获取的权重值用于获得待插值块的像素值的过程中，从而实现帧率上采样。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The frame rate upsampling method provided by the embodiment of the present invention dynamically obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block, and uses the dynamically obtained weight value to obtain the block to be interpolated. In the process of pixel values, frame rate upsampling is achieved. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

实施例二Embodiment 2

如图14所示，为本发明实施例提供的一种帧率上采样的装置14，用以执行图1所示的帧率上采样的方法，该装置14包括：As shown in FIG. 14, a frame rate upsampling apparatus 14 for performing the frame rate upsampling method shown in FIG. 1 is provided. The apparatus 14 includes:

确定单元141，用于确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；a determining unit 141, configured to determine a location of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;

位置获取单元142，用于根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；a location obtaining unit 142, configured to acquire, according to a location of the current block and a motion vector of the current block with respect to the matching block, a location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated;

权重值获取单元143，用于基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；The weightvalue obtaining unit 143 is configured to acquire, according to the weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;

像素值获取单元144，用于将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；a pixelvalue obtaining unit 144, configured to multiply a pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated;

插值单元145，用于将所述待插值块的像素值插入所述待插值块的位置。Theinterpolation unit 145 is configured to insert a pixel value of the block to be interpolated into a position of the block to be interpolated.

可选的，所述权重值获取单元143还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；Optionally, the weightvalue obtaining unit 143 is further configured to acquire an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;

所述权重值获取单元143具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。The weightvalue obtaining unit 143 is specifically configured to perform weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.

可选的，如图15所示，所述装置14还包括：Optionally, as shown in FIG. 15, the device 14 further includes:

扩展单元146，用于按照相同的扩展倍率分别扩展所述当前块和所述相似块；The expansion unit 146 is configured to separately expand the current block and the similar block according to the same expansion ratio;

所述位置获取单元142具体用于，根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；The location obtaining unit 142 is specifically configured to: obtain, according to the expanded location of the current block, and a motion vector of the current block relative to the matching block, a corresponding to the extended current block in the to-be-interpolated frame. The position of the block to be interpolated;

所述像素值获取单元144具体用于，将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。The pixelvalue obtaining unit 144 is specifically configured to multiply the pixel value of the extended similar block by the weight value to obtain a pixel value of the block to be interpolated.

可选的，所述像素值获取单元144还用于：Optionally, the pixelvalue obtaining unit 144 is further configured to:

对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；And performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。Determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.

可选的，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。Optionally, the similar block includes: the matching block; or m macroblocks in the first region of the first image, m≥1, where m is an integer.

可选的，所述相似块为所述匹配块；Optionally, the similar block is the matching block;

所述像素值获取单元144还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；The pixelvalue obtaining unit 144 is further configured to: acquire, according to the pixel value of the current block and the pixel value of the matching block, a pixel value of the modified block; the modified block is a virtual macro block, and is used to correct the to-be-interpolated value. The pixel value of the block;

所述像素值获取单元144具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。The pixelvalue obtaining unit 144 is specifically configured to: according to the pixel value of the matching block,The pixel value of the correction block and the weight value obtain a pixel value of the block to be interpolated.

可选的，所述像素值获取单元144具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。Optionally, the pixelvalue obtaining unit 144 is specifically configured to use, as a pixel value of the modified block, a mean value of a pixel value of the current block and a pixel value of the matching block.

可选的，所述相似块为所述第一图像的第一区域中的m个宏块；Optionally, the similar block is m macroblocks in the first region of the first image;

所述权重值获取单元143具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；The weightvalue obtaining unit 143 is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m≥n≥1, where n is Integer

分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。Multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain pixel values of the to-be-interpolated block .

本发明实施例提供的帧率上采样的装置，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The frame rate upsizing apparatus provided by the embodiment of the present invention obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block by determining the current block and the similar block corresponding to the current block, and acquires the weight value of the pixel value of the current block. The weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

实施例三Embodiment 3

如图16所示，为本发明实施例提供的一种帧率上采样的装置14，用以执行图1所示的帧率上采样的方法，该装置14包括：存储器161和处理器162，其中，As shown in FIG. 16, a frame rate upsampling apparatus 14 for performing the frame rate upsampling method shown in FIG. 1 is provided. The apparatus 14 includes a memory 161 and a processor 162. among them,

存储器161用于存储一组代码，该代码用于控制处理器162执行以下动作：The memory 161 is used to store a set of codes for controlling the processor 162 to perform the following actions:

确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；Determining a position of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;

根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；And according to a position of the current block and a motion direction of the current block relative to the matching blockAnd obtaining a position of the block to be interpolated corresponding to the current block in the frame to be interpolated;

基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；Obtaining, according to a weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;

将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；Multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated;

将所述待插值块的像素值插入所述待插值块的位置。Inserting the pixel value of the block to be interpolated into the position of the block to be interpolated.

可选的，处理器162还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；Optionally, the processor 162 is further configured to: obtain an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;

处理器162具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。The processor 162 is specifically configured to perform weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.

可选的，处理器162还用于，按照相同的扩展倍率分别扩展所述当前块和所述相似块；Optionally, the processor 162 is further configured to separately expand the current block and the similar block according to the same expansion ratio;

处理器162具体用于：The processor 162 is specifically configured to:

根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；Obtaining, according to the expanded position of the current block and the motion vector of the current block relative to the matching block, a position of the to-be-interpolated block corresponding to the extended current block in the frame to be interpolated;

将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。Multiplying the expanded pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated.

可选的，处理器162还用于：Optionally, the processor 162 is further configured to:

对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；And performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。Determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.

可选的，所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。Optionally, the similar block includes: the matching block; or m macroblocks in the first region of the first image, m≥1, where m is an integer.

可选的，所述相似块为所述匹配块；Optionally, the similar block is the matching block;

处理器162还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；The processor 162 is further configured to: obtain, according to the pixel value of the current block and the pixel value of the matching block, a pixel value of the modified block; the modified block is a virtual macro block, and is used to correct a pixel value of the block to be interpolated ;

处理器162具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。The processor 162 is specifically configured to obtain a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the modified block, and the weight value.

可选的，处理器162具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。Optionally, the processor 162 is specifically configured to use, as a pixel value of the modified block, a mean value of a pixel value of the current block and a pixel value of the matching block.

可选的，所述相似块为所述第一图像的第一区域中的m个宏块；Optionally, the similar block is m macroblocks in the first region of the first image;

处理器162具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；The processor 162 is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m≥n≥1, where n is an integer;

分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。Multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain pixel values of the to-be-interpolated block .

本发明实施例提供的帧率上采样的装置，通过确定当前块和与当前块对应的相似块获取与当前块对应的相似块的像素值相对当前块的像素值的权重值，并将获取的权重值乘以相似块的像素值，得到待插值块的像素值，利用得到的待插值块的像素值实现帧率上采样。由于当前块和与当前块对应的相似块在不断变化，因此针对不同的当前块可以获得不同的权重值，即可以动态获取权重值。相比现有技术的移动块插值方法中，直接将当前块的像素值作为待插值块的像素，本方案中在使用当前块的像素值以及动态获取的权重值获得待插值块的像素值，可以降低块效应，从而增强用户的观赏体验。The frame rate upsizing apparatus provided by the embodiment of the present invention obtains the weight value of the pixel value of the similar block corresponding to the current block relative to the pixel value of the current block by determining the current block and the similar block corresponding to the current block, and acquires the weight value of the pixel value of the current block. The weight value is multiplied by the pixel value of the similar block to obtain the pixel value of the block to be interpolated, and the frame rate upsampling is implemented by using the obtained pixel value of the block to be interpolated. Since the current block and the similar block corresponding to the current block are constantly changing, different weight values can be obtained for different current blocks, that is, the weight value can be dynamically obtained. Compared with the prior art mobile block interpolation method, the pixel value of the current block is directly used as the pixel of the block to be interpolated. In this solution, the pixel value of the block to be interpolated is obtained by using the pixel value of the current block and the dynamically obtained weight value. Block effects can be reduced to enhance the user's viewing experience.

所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device described aboveThe embodiment is merely illustrative. For example, the division of the unit is only a logical function division, and the actual implementation may have another division manner. For example, multiple units or components may be combined or integrated into another system. Or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理包括，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。The above-described integrated unit implemented in the form of a software functional unit can be stored in a computer readable storage medium. The software functional units described above are stored in a storage medium and include instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform portions of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the claims.

Claims (16)

1. 一种帧率上采样的方法，其特征在于，包括：A method for frame rate upsampling, comprising:

   确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；Determining a position of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;

   根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；Obtaining a position of the to-be-interpolated block corresponding to the current block in the frame to be interpolated according to a position of the current block and a motion vector of the current block relative to the matching block;

   基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；Obtaining, according to a weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;

   将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；Multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated;

   将所述待插值块的像素值插入所述待插值块的位置。Inserting the pixel value of the block to be interpolated into the position of the block to be interpolated.
2. 根据权利要求1所述的方法，其特征在于，The method of claim 1 wherein

   在所述获取所述相似块的像素值相对所述当前块的像素值的权重值之前，所述方法还包括：获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；Before the obtaining the weight value of the pixel value of the similar block relative to the pixel value of the current block, the method further includes: acquiring a difference between a pixel value of the similar block and a pixel value of the current block Absolute value

   所述基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。The obtaining, by the weighting algorithm, the weight value of the pixel value of the similar block relative to the pixel value of the current block, comprising: weighting an absolute value of the difference value based on the weighting algorithm, to obtain the similar block A weight value of a pixel value relative to a pixel value of the current block.
3. 根据权利要求1所述的方法，其特征在于，在所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置之前，所述方法还包括：The method according to claim 1, wherein the obtaining, according to the location of the current block and the motion vector of the current block relative to the matching block, the corresponding to the current block in the frame to be interpolated Before the location of the block to be interpolated, the method further includes:

   按照相同的扩展倍率分别扩展所述当前块和所述相似块；Expanding the current block and the similar block respectively according to the same expansion ratio;

   所述根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置，包括：Obtaining, according to the location of the current block, the location of the current block and the motion vector of the matching block, the location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated, including:

   根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；Obtaining, to be interpolated, the block to be interpolated corresponding to the extended current block in the frame to be interpolated according to the extended position of the current block and the motion vector of the current block relative to the matching blocks position;

   所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：And multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated, including:

   将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。Multiplying the expanded pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated.
4. 根据权利要求1所述的方法，其特征在于，在所述将所述待插值块的像素值插入所述待插值块的位置之后，所述方法还包括：The method according to claim 1, wherein after the pixel value of the block to be interpolated is inserted into the position of the block to be interpolated, the method further comprises:

   对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；And performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

   根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。Determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.
5. 根据权利要求1-4任一项所述的方法，其特征在于，A method according to any one of claims 1 to 4, characterized in that

   所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。The similarity block includes: the matching block; or m macroblocks in the first region of the first image, m≥1, and the m is an integer.
6. 根据权利要求5所述的方法，其特征在于，所述相似块为所述匹配块；所述方法还包括：The method according to claim 5, wherein the similar block is the matching block; the method further comprises:

   根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；Obtaining, according to the pixel value of the current block and the pixel value of the matching block, a pixel value of the modified block; the modified block is a virtual macro block, and is used to correct a pixel value of the block to be interpolated;

   所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：And multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated, including:

   根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。Obtaining a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the correction block, and the weight value.
7. 根据权利要求6所述的方法，其特征在于，所述根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值，包括：The method according to claim 6, wherein the obtaining the pixel value of the correction block according to the pixel value of the current block and the pixel value of the matching block comprises:

   将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。The mean value of the pixel value of the current block and the pixel value of the matching block is used as the pixel value of the correction block.
8. 根据权利要求5所述的方法，其特征在于，所述相似块为所述第一图像的第一区域中的m个宏块；The method of claim 5 wherein said similar block is m macroblocks in a first region of said first image;

   所述获取所述相似块的像素值相对所述当前块的像素值的权重值，包括：The obtaining the weight value of the pixel value of the similar block relative to the pixel value of the current block includes:

   分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；Obtaining n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block, respectively; m≥n≥1, where n is an integer;

   所述将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值，包括：And multiplying the pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated, including:

   分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。Multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain pixel values of the to-be-interpolated block .
9. 一种帧率上采样的装置，其特征在于，包括：A device for upsampling a frame rate, comprising:

   确定单元，用于确定第一图像中的当前块的位置、所述当前块相对第二图像中的一匹配块的运动向量以及与所述当前块对应的相似块；a determining unit, configured to determine a location of a current block in the first image, a motion vector of the current block relative to a matching block in the second image, and a similar block corresponding to the current block;

   位置获取单元，用于根据所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与所述当前块对应的待插值块的位置；a location acquiring unit, configured to acquire, according to a location of the current block and a motion vector of the current block with respect to the matching block, a location of the to-be-interpolated block corresponding to the current block in the frame to be interpolated;

   权重值获取单元，用于基于加权算法获取所述相似块的像素值相对所述当前块的像素值的权重值；a weight value obtaining unit, configured to acquire, according to a weighting algorithm, a weight value of a pixel value of the similar block relative to a pixel value of the current block;

   像素值获取单元，用于将所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值；a pixel value obtaining unit, configured to multiply a pixel value of the similar block by the weight value to obtain a pixel value of the block to be interpolated;

   插值单元，用于将所述待插值块的像素值插入所述待插值块的位置。An interpolation unit, configured to insert a pixel value of the block to be interpolated into a position of the block to be interpolated.
10. 根据权利要求9所述的装置，其特征在于，The device of claim 9 wherein:

    所述权重值获取单元还用于，获取所述相似块的像素值相对所述当前块的像素值的差值的绝对值；The weight value obtaining unit is further configured to acquire an absolute value of a difference between a pixel value of the similar block and a pixel value of the current block;

    所述权重值获取单元具体用于，基于所述加权算法对所述差值的绝对值进行加权运算，得到所述相似块的像素值相对所述当前块的像素值的权重值。The weight value obtaining unit is configured to perform a weighting operation on the absolute value of the difference value based on the weighting algorithm to obtain a weight value of a pixel value of the similar block relative to a pixel value of the current block.
11. 根据权利要求9所述的装置，其特征在于，所述装置还包括：The device according to claim 9, wherein the device further comprises:

    扩展单元，用于按照相同的扩展倍率分别扩展所述当前块和所述相似块；An expansion unit, configured to separately expand the current block and the similar block according to the same expansion ratio;

    所述位置获取单元具体用于，根据扩展后的所述当前块的位置和所述当前块相对所述匹配块的运动向量，获取待插值帧中的与扩展后的所述当前块对应的待插值块的位置；The location obtaining unit is configured to: obtain, according to the extended location of the current block, and the motion vector of the current block relative to the matching block, a to-be-interpolated frame corresponding to the extended current block. The position of the interpolation block;

    所述像素值获取单元具体用于，将扩展后的所述相似块的像素值乘以所述权重值，得到所述待插值块的像素值。The pixel value obtaining unit is configured to multiply the pixel value of the extended similar block by the weight value to obtain a pixel value of the block to be interpolated.
12. 根据权利要求9所述的装置，其特征在于，所述像素值获取单元还用于：The device according to claim 9, wherein the pixel value obtaining unit is further configured to:

    对所述待插值帧中重叠像素的像素值进行加权累加处理，得到所述重叠像素的中间像素值；And performing weighted accumulation processing on pixel values of the overlapping pixels in the frame to be interpolated to obtain intermediate pixel values of the overlapping pixels;

    根据所述加权算法和所述重叠像素的中间像素值，确定所述重叠像素的目标像素值。Determining a target pixel value of the overlapping pixel according to the weighting algorithm and an intermediate pixel value of the overlapping pixel.
13. 根据权利要求9-12任一项所述的装置，其特征在于，A device according to any one of claims 9-12, wherein

    所述相似块包括：所述匹配块；或者，所述第一图像的第一区域中的m个宏块，m≥1，所述m为整数。The similarity block includes: the matching block; or m macroblocks in the first region of the first image, m≥1, and the m is an integer.
14. 根据权利要求13所述的装置，其特征在于，所述相似块为所述匹配块；The apparatus according to claim 13, wherein said similar block is said matching block;

    所述像素值获取单元还用于，根据所述当前块的像素值和所述匹配块的像素值，获取修正块的像素值；修正块为一虚拟宏块，用于修正所述待插值块的像素值；The pixel value obtaining unit is further configured to: acquire a pixel value of the correction block according to the pixel value of the current block and the pixel value of the matching block; and the correction block is a virtual macro block, and the block to be interpolated is modified. Pixel value;

    所述像素值获取单元具体用于，根据所述匹配块的像素值、所述修正块的像素值和所述权重值，得到所述待插值块的像素值。The pixel value obtaining unit is configured to obtain a pixel value of the block to be interpolated according to a pixel value of the matching block, a pixel value of the modified block, and the weight value.
15. 根据权利要求14所述的装置，其特征在于，The device of claim 14 wherein:

    所述像素值获取单元具体用于，将所述当前块的像素值与所述匹配块的像素值的均值作为修正块的像素值。The pixel value obtaining unit is specifically configured to use, as a pixel value of the correction block, a mean value of a pixel value of the current block and a pixel value of the matching block.
16. 根据权利要求13所述的装置，其特征在于，所述相似块为所述第一图像的第一区域中的m个宏块；The apparatus according to claim 13, wherein said similar block is m macroblocks in a first region of said first image;

    所述权重值获取单元具体用于，分别获取m个宏块中的n个宏块的像素值相对所述当前块的像素值的n个权重值；m≥n≥1，所述n为整数；The weight value obtaining unit is specifically configured to respectively acquire n weight values of pixel values of n macroblocks in the m macroblocks relative to pixel values of the current block; m≥n≥1, where n is an integer ;

    分别将所述n个相似块的像素值乘以n个权重值中对应的权重值，得到n个计算结果；对所述n个计算结果进行求和运算，得到所述待插值块的像素值。Multiplying the pixel values of the n similar blocks by the corresponding weight values of the n weight values to obtain n calculation results; performing a summation operation on the n calculation results to obtain pixel values of the to-be-interpolated block .

Images