本申请是原案申请号为201180029771.8的发明专利申请(国际申请号：PCT/KR2011/002763，申请日：2011年4月18日，发明名称：视频编码/解码设备和方法)的分案申请。This application is a divisional application of the original patent application number 201180029771.8 (international application number: PCT/KR2011/002763, filing date: April 18, 2011, invention name: video encoding/decoding device and method).
技术领域technical field
本发明涉及视频编码/解码设备和方法。更具体地说，本发明涉及一种用于通过将扩展宏块划分为各种大小和形状的子块并根据这些子块的大小和形状利用不同的预测单元和变换单元进行视频编码来提高视频压缩效率的设备和方法。此外，本发明涉及一种用于通过根据从这种视频编码设备接收到的比特流进行视频解码来提高视频重构效率的设备和方法。The present invention relates to video encoding/decoding devices and methods. More specifically, the present invention relates to a method for improving video quality by dividing an extended macroblock into sub-blocks of various sizes and shapes and performing video coding using different prediction units and transform units according to the sizes and shapes of the sub-blocks. Apparatus and methods for compression efficiency. Furthermore, the present invention relates to a device and a method for improving video reconstruction efficiency by performing video decoding from a bitstream received from such a video encoding device.
背景技术Background technique
本部分的说明仅提供与本发明有关的背景信息，并不构成现有技术。The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
当前的视频数据压缩技术包括H.261、H.263、H.264、MPEG-2和MPEG-4。根据现有的视频压缩技术，在将各个图像分割为由亮度分量的矩形16×16像素区域和色度分量的矩形8×8像素区域组成的固定大小宏块之后，对各个图像进行编码。各个宏块的所有亮度分量和色度分量是空间或时间预测的，并且结果预测残留在最终压缩之前经受变换、量化和熵编码等。Current video data compression technologies include H.261, H.263, H.264, MPEG-2, and MPEG-4. According to existing video compression techniques, each image is encoded after being divided into fixed-size macroblocks consisting of a rectangular 16×16 pixel area for luma components and a rectangular 8×8 pixel area for chrominance components. All luma and chroma components of each macroblock are spatially or temporally predicted, and the resulting prediction residue is subjected to transformation, quantization, entropy coding, etc. before final compression.
根据H.264/AVC压缩标准的编码设备将各个宏块再次划分为较小大小16x16,8x8和4x4的块，以进入帧内预测编码，其中，按照四个预测模式中的一个来处理16x16像素块，并且按照四个预测模式中的一个来处理8x8像素块和4x4像素块。关于帧间预测编码，各个宏块可以首先划分为像素大小16x16,16x8,8x16,8x8,8x4,4x8和4x4的块。按照8x8或4x4像素块为单位进行变换，并且变换系数的量化采用标量量化。Coding devices according to the H.264/AVC compression standard subdivide individual macroblocks into smaller blocks of size 16x16, 8x8 and 4x4 for intra-frame predictive coding, where 16x16 pixels are processed according to one of four prediction modes blocks, and 8x8 pixel blocks and 4x4 pixel blocks are processed according to one of four prediction modes. Regarding inter-frame predictive coding, each macroblock can be first divided into blocks of pixel sizes 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4. Transformation is performed in units of 8x8 or 4x4 pixel blocks, and scalar quantization is used for quantization of transform coefficients.
然而，16x16宏块被认为是针对QCIF和CIF类型的小图像的合适大小，而不是适于诸如4Kx2K视频的高分辨率视频的大小。此外，由于通常的视频压缩技术在进行视频编码时采用固定大小的宏块(尽管H.264/AVC规定对宏块进行再次划分和编码，但是宏块的大小仍然是固定的)，仍然要实现对高分辨率视频的有效编码。However, a 16x16 macroblock is considered a suitable size for small pictures of the QCIF and CIF types, but not a size suitable for high-resolution video such as 4Kx2K video. In addition, since the usual video compression technology uses fixed-size macroblocks during video encoding (although H.264/AVC stipulates that macroblocks are divided and encoded again, the size of macroblocks is still fixed), it is still necessary to implement Efficient encoding of high-resolution video.
此外，视频的编码和解码之间的统一操作使得难以从高分辨率视频编码的已经劣化的压缩效率实现高效率的视频解码。Furthermore, the unified operation between encoding and decoding of video makes it difficult to achieve high-efficiency video decoding from the already degraded compression efficiency of high-resolution video encoding.
发明内容Contents of the invention
技术问题technical problem
因此，为了解决上述问题，本发明尝试通过以下处理来提高视频压缩效率和视频重构效率：将宏块扩展为各种大小，将扩展宏块划分为各种大小和形状的子块，以及利用根据这些子块的大小的不同的预测单元和变换单元来进行视频编码/解码。Therefore, in order to solve the above-mentioned problems, the present invention attempts to improve video compression efficiency and video reconstruction efficiency by expanding the macroblock into various sizes, dividing the expanded macroblock into subblocks of various sizes and shapes, and utilizing Video encoding/decoding is performed according to different prediction units and transformation units of the sizes of these sub-blocks.
采用的技术方案Technical solution adopted
本发明的实施方式提供了一种视频编码/解码设备，该视频编码/解码设备包括：视频编码器，其用于通过预测宏块划分成的子块来生成预测子块，通过从所述子块中减去所述预测子块来生成残留子块，通过利用根据所述子块的大小选择的变换对所述残留子块进行变换和量化来生成量化变换系数，以及当使用上下文自适应二进制算术编码(CABAC)时利用根据所述宏块和所述子块的不同的上下文表来生成编码视频数据，以对所述量化变换系数进行编码；以及视频解码器，其用于通过利用CABAC并且利用根据宏块和子块的不同的上下文表对编码数据进行解码来重构分割类型信息、运动信息、变换类型信息和量化变换信息，通过利用有所述变换类型信息识别的变换来对所述量化变换系数进行逆量化和逆变换，通过利用所述运动信息预测根据所述分割类型信息识别的子块来生成预测子块，通过将所述预测子块与所述残留子块相加来重构所述子块，并且基于所述子块重构所述宏块。An embodiment of the present invention provides a video encoding/decoding device, the video encoding/decoding device includes: a video encoder for generating a predicted sub-block by predicting sub-blocks into which a macroblock is divided, by Subtracting the predicted sub-block from the block to generate a residual sub-block, generating quantized transform coefficients by transforming and quantizing the residual sub-block with a transform selected according to the size of the sub-block, and when using context-adaptive binary Arithmetic coding (CABAC) for generating coded video data using different context tables for said macroblock and said subblock to code said quantized transform coefficients; and a video decoder for coding said quantized transform coefficients by using CABAC and Reconstructing partition type information, motion information, transform type information, and quantized transform information by decoding encoded data using different context tables according to macroblocks and subblocks, said quantization is performed by utilizing a transform identified by said transform type information Inverse quantization and inverse transformation of the transform coefficients, generating a predictive subblock by predicting a subblock identified from the partition type information using the motion information, reconstructing by adding the predictive subblock to the residual subblock the sub-blocks, and reconstructing the macroblocks based on the sub-blocks.
本发明的另一实施方式提供了一种视频编码设备，该视频编码设备包括：预测器，其用于通过预测宏块划分成的子块来生成预测子块；减法器，其用于通过从所述子块中减去所述预测子块来生成残留子块；变换器和量化器，其用于通过利用根据所述子块的大小选择的变换对所述残留子块进行变换和量化来生成量化变换系数；以及编码器，其用于当使用上下文自适应二进制算术编码(CABAC)时利用根据所述宏块和所述子块的不同的上下文表来生成编码视频数据，以对所述量化变换系数进行编码。Another embodiment of the present invention provides a video encoding device, which includes: a predictor for generating a predicted sub-block by predicting sub-blocks into which a macroblock is divided; subtracting the predicted sub-block from the sub-block to generate a residual sub-block; a transformer and a quantizer for transforming and quantizing the residual sub-block by using a transform selected according to the size of the sub-block to generate generating quantized transform coefficients; and an encoder for generating encoded video data using different context tables from the macroblock and the subblock when using Context Adaptive Binary Arithmetic Coding (CABAC) for the The quantized transform coefficients are encoded.
这里，所述编码器可以根据相邻块的情况计算并改变要编码的当前块的二进制信号的出现概率。Here, the encoder may calculate and change the occurrence probability of the binary signal of the current block to be encoded according to conditions of adjacent blocks.
所述编码器可以与要编码的当前块的二进制信号的出现概率对应地设置多个上下文表，并且根据相邻块和当前块的情况利用所述多个上下文表当中的不同的上下文表生成所述编码视频数据。The encoder may set a plurality of context tables corresponding to the occurrence probability of the binary signal of the current block to be encoded, and use a different context table among the plurality of context tables to generate the coded video data.
所述编码器可以通过基于图片类型、句法的类型和对所述句法进行编码时与所述相邻块对应的值中的至少一个选择上下文表来生成所述编码视频数据。The encoder may generate the encoded video data by selecting a context table based on at least one of a picture type, a type of syntax, and a value corresponding to the neighboring block when the syntax is encoded.
所述编码器可以根据要编码的当前块的上侧块和左侧块是否都是跳过模式或者所述上侧块和所述左侧块中的任一个是否是跳过模式来针对skipX_flag使用不同的上下文表。The encoder may use skipX_flag according to whether both the upper block and the left block of the current block to be encoded are in skip mode or whether either of the upper block and the left block is in skip mode Different context tables.
所述编码器可以根据要编码的当前块的上侧块和左侧块是否是帧间模式还是帧内模式，或者所述上侧块和所述左侧块中的任一个是否是帧间模式来针对interX_flag使用不同的上下文表。The encoder may be based on whether the upper block and the left block of the current block to be encoded are inter mode or intra mode, or whether either of the upper block and the left block is an inter mode to use a different context table for interX_flag.
如果所述上侧块或所述左侧块大于16x16像素块，则编码器可以在帧内模式和帧间模式按照混合模式存在时以及仅存在帧间模式时针对interX_flag使用不同的上下文表。If the upper side block or the left side block is larger than a 16x16 pixel block, the encoder may use different context tables for interX_flag when intra and inter modes exist in mixed mode and when only inter mode exists.
所述编码器可以根据要编码的当前块的上侧块和左侧块是否都是直接模式或者所述上侧块和所述左侧块中的任一个是否是直接模式来针对directX_flag使用不同的上下文表。The encoder may use different values for directX_flag according to whether the upper block and the left block of the current block to be encoded are both in direct mode or whether either of the upper block and the left block is in direct mode context table.
所述编码器可以根据要编码的当前块的上侧块和左侧块是否都按照扩展宏块为单位进行编码或者所述上侧块和所述左侧块中的任一个是否按照扩展宏块为单位进行编码来针对extended_mb_flag使用不同的上下文表。The encoder may encode according to whether both the upper block and the left block of the current block to be encoded are encoded in units of extended macroblocks or whether any of the upper block and the left block is encoded in units of extended macroblocks Encoded for units to use a different context table for extended_mb_flag.
所述编码器可以根据要编码的当前块的上侧块和左侧块是否都被用作基准块或者所述上侧块和所述左侧块中的任一个是否被用作基准块来针对B_pdir使用不同的上下文表。The encoder may target according to whether both the upper block and the left block of the current block to be encoded are used as a reference block or whether any one of the upper block and the left block is used as a reference block. B_pdir uses a different context table.
所述编码器可以根据要编码的所述当前块的所述上侧块或所述左侧块是否假设L0预测、L1预测或双向预测来针对B_pdir使用不同的上下文表。The encoder may use different context tables for B_pdir depending on whether the upper block or the left block of the current block to be encoded assumes L0 prediction, L1 prediction or bidirectional prediction.
针对MVD/ref_idx,a CBPX_flag和变换类型中的至少一个的不同的上下文表可以根据要编码的当前块的上侧块和下侧块来进行使用。Different context tables for at least one of MVD/ref_idx, a CBPX_flag, and transform type may be used according to upper and lower blocks of the current block to be encoded.
针对所述变换类型的所述上下文表可以根据要编码的所述当前块的可用变换类型而改变。The context table for the transform type may change according to available transform types of the current block to be encoded.
本发明的另一实施方式提供了一种视频解码设备，该视频解码设备包括：解码器，其用于通过利用CABAC和利用根据宏块和子块的不同的上下文表对编码数据进行解码来重构分割类型信息、运动信息、变换类型信息和量化变换系数；逆量化器和逆变换器，其用于通过利用由所述变换类型信息识别的变换对所述量化变换系数进行逆量化和逆变换来重构残留子块；预测器，其用于通过利用所述运动信息预测根据所述分割类型信息识别的子块来生成预测块；以及加法器，其用于通过将所述预测子块与所述残留子块相加来重构所述子块，其中，所述宏块基于所述子块来进行重构。Another embodiment of the present invention provides a video decoding apparatus including: a decoder for reconstructing coded data by using CABAC and using different context tables according to macroblocks and subblocks partition type information, motion information, transform type information, and quantized transform coefficients; an inverse quantizer and an inverse transformer for dequantizing and inverse transforming the quantized transform coefficients by using a transform identified by the transform type information a reconstructed residual sub-block; a predictor for generating a prediction block by predicting a sub-block identified according to the partition type information using the motion information; and an adder for generating a prediction block by combining the prediction sub-block with the The sub-blocks are reconstructed by adding the residual sub-blocks, wherein the macroblocks are reconstructed based on the sub-blocks.
这里，所述解码器可以基于根据要解码的当前块的上侧块和左侧块而不同地设置了二进制信号的出现概率的上下文表来重构所述编码数据。Here, the decoder may reconstruct the encoded data based on a context table in which occurrence probabilities of binary signals are set differently according to upper and left blocks of the current block to be decoded.
本发明的另一实施方式提供了一种视频编码/解码方法，该视频编码/解码方法包括以下步骤：通过预测宏块被划分成的子块来生成预测子块，通过从所述子块中减去所述预测子块来生成残留子块，通过利用根据所述子块的大小选择的变换对所述残留子块进行变换和量化来生成量化变换系数，并且当使用上下文自适应二进制算术编码(CABAC)时利用根据所述宏块和所述子块的不同的上下文表来生成编码视频数据，以对所述量化变换系数进行编码；以及通过利用CABAC和利用根据宏块和子块的不同的上下文表对编码数据进行解码来重构分割类型信息、运动信息、变换类型信息和量化变换系数，通过利用由所述变换类型信息识别的变换对所述量化变换系数进行逆量化和逆变换来重构残留子块，通过利用所述运动信息预测根据所述分割类型信息识别的子块来生成预测块，通过将所述预测子块与所述残留子块相加来重构所述子块，并且基于所述子块重构所述宏块。Another embodiment of the present invention provides a video encoding/decoding method, the video encoding/decoding method includes the following steps: generating a predicted sub-block by predicting the sub-blocks into which the macroblock is divided, and generating a predicted sub-block from the sub-block subtracting the predicted sub-block to generate a residual sub-block, generating quantized transform coefficients by transforming and quantizing the residual sub-block with a transform selected according to the size of the sub-block, and when using context-adaptive binary arithmetic coding (CABAC) using different context tables according to the macroblock and the subblock to generate encoded video data to encode the quantized transform coefficients; and by using CABAC and using different context tables according to the macroblock and subblock The context table decodes the encoded data to reconstruct partition type information, motion information, transform type information, and quantized transform coefficients by inverse quantizing and inverse transforming the quantized transform coefficients using the transform identified by the transform type information. constructing a residual sub-block, generating a predicted block by utilizing said motion information to predict a sub-block identified according to said partition type information, reconstructing said sub-block by adding said predicted sub-block to said residual sub-block, And reconstructing the macroblock based on the subblocks.
本发明的另一实施方式提供了一种视频编码方法，该视频编码方法包括以下步骤：通过预测宏块被划分成的子块来生成预测子块；通过从所述子块中减去所述预测子块来生成残留子块；通过利用根据所述子块的大小选择的变换对所述残留子块进行变换和量化来生成量化变换系数；以及当使用上下文自适应二进制算术编码(CABAC)时利用根据所述宏块和所述子块的不同的上下文表来生成编码视频数据，以对所述量化变换系数进行编码。Another embodiment of the present invention provides a video coding method, which includes the following steps: generating a predicted sub-block by predicting the sub-blocks into which the macroblock is divided; by subtracting the predicting a sub-block to generate a residual sub-block; generating quantized transform coefficients by transforming and quantizing the residual sub-block with a transform selected according to the size of the sub-block; and when using Context Adaptive Binary Arithmetic Coding (CABAC) Encoded video data is generated using different context tables from the macroblock and the subblock to encode the quantized transform coefficients.
这里，所述生成所述编码视频数据的处理可以根据相邻块的情况计算和改变要编码的当前块的二进制信号的出现概率。Here, the process of generating the encoded video data may calculate and change the occurrence probability of the binary signal of the current block to be encoded according to conditions of adjacent blocks.
所述生成所述编码视频数据的处理可以与要编码的当前块的二进制信号的出现概率对应地设置多个上下文表，并且利用根据相邻块和所述当前块的情况的所述多个上下文表当中的不同的上下文表来生成所述编码视频数据。The process of generating the encoded video data may set a plurality of context tables corresponding to occurrence probabilities of binary signals of the current block to be encoded, and utilize the plurality of context tables according to conditions of adjacent blocks and the current block different context tables among the tables to generate the encoded video data.
所述生成所述编码视频数据的处理可以通过基于图片类型、句法的类型和对所述句法进行编码时与所述相邻块对应的值中的至少一个选择上下文表来生成所述编码视频数据。The generating of the encoded video data may generate the encoded video data by selecting a context table based on at least one of a picture type, a type of syntax, and a value corresponding to the adjacent block when the syntax is encoded .
所述生成所述编码视频数据的处理可以根据要编码的当前块的上侧块和左侧块是否都是跳过模式或者所述上侧块和所述左侧块中的任一个是否是跳过模式来针对skipX_flag使用不同的上下文表。The process of generating the encoded video data may be based on whether the upper block and the left block of the current block to be encoded are both in skip mode or whether any of the upper block and the left block is a skip mode. Use a different context table for skipX_flag by mode.
所述生成所述编码视频数据的处理可以根据要编码的当前块的上侧块和左侧块是否是帧间模式还是帧内模式，或者所述上侧块和所述左侧块中的任一个是否是帧间模式来针对interX_flag使用不同的上下文表。The process of generating the encoded video data may be based on whether the upper block and the left block of the current block to be encoded are inter mode or intra mode, or any of the upper block and the left block Whether one is inter mode to use a different context table for interX_flag.
如果所述上侧块或所述左侧块大于16x16像素块，则所述生成所述编码视频数据的处理可以在帧内模式和帧间模式按照混合模式存在时以及仅存在帧间模式时针对interX_flag使用不同的上下文表。If the upper side block or the left side block is larger than a 16x16 pixel block, the process of generating the encoded video data may be for when intra mode and inter mode exist in mixed mode and when only inter mode exists interX_flag uses a different context table.
所述生成所述编码视频数据的处理可以根据要编码的当前块的上侧块和左侧块是否都是直接模式或者所述上侧块和所述左侧块中的任一个是否是直接模式来针对directX_flag使用不同的上下文表。The process of generating the encoded video data may be based on whether the upper block and the left block of the current block to be encoded are both in direct mode or whether any of the upper block and the left block is in direct mode to use a different context table for directX_flag.
所述生成所述编码视频数据的处理可以根据要编码的当前块的上侧块和左侧块是否都按照扩展宏块为单位进行编码或者所述上侧块和所述左侧块中的任一个是否按照扩展宏块为单位进行编码来针对extended_mb_flag使用不同的上下文表。The process of generating the encoded video data may be based on whether both the upper block and the left block of the current block to be encoded are encoded in units of extended macroblocks or whether any of the upper block and the left block Whether to encode in units of extended macroblocks to use different context tables for extended_mb_flag.
所述生成所述编码视频数据的处理可以根据要编码的当前块的上侧块和左侧块是否都被用作基准块或者所述上侧块和所述左侧块中的任一个是否被用作基准块来针对B_pdir使用不同的上下文表。The process of generating the encoded video data may be based on whether both the upper side block and the left side block of the current block to be encoded are used as reference blocks or whether any one of the upper side block and the left side block is used as a reference block. Used as a reference block to use a different context table for B_pdir.
所述生成所述编码视频数据的处理可以根据要编码的所述当前块的所述上侧块或所述左侧块是否假设L0预测、L1预测或双向预测来针对B_pdir使用不同的上下文表。The process of generating the encoded video data may use a different context table for B_pdir depending on whether the upper block or the left block of the current block to be encoded assumes L0 prediction, L1 prediction or bidirectional prediction.
针对MVD/ref_idx,a CBPX_flag和变换类型中的至少一个的不同的上下文表可以根据要编码的当前块的上侧块和下侧块来进行使用。Different context tables for at least one of MVD/ref_idx, a CBPX_flag, and transform type may be used according to upper and lower blocks of the current block to be encoded.
针对所述变换类型的所述上下文表可以根据要编码的所述当前块的可用变换类型而改变。The context table for the transform type may change according to available transform types of the current block to be encoded.
本发明的另一实施方式提供了一种视频解码方法，该视频解码方法包括以下步骤：通过利用CABAC和利用根据宏块和子块的不同的上下文表对编码数据进行解码来重构分割类型信息、运动信息、变换类型信息和量化变换系数；通过利用由所述变换类型信息识别的变换对所述量化变换系数进行逆量化和逆变换来重构残留子块；通过利用所述运动信息预测根据所述分割类型信息识别的子块来生成预测块；以及通过将所述预测子块与所述残留子块相加来重构所述子块，其中，所述宏块基于所述子块来进行重构。Another embodiment of the present invention provides a video decoding method including the steps of reconstructing partition type information by decoding encoded data using CABAC and using different context tables according to macroblocks and subblocks, motion information, transform type information, and quantized transform coefficients; reconstructing a residual sub-block by inverse quantizing and inverse transforming the quantized transform coefficients using a transform identified by the transform type information; generating a predicted block from a sub-block identified by the partition type information; and reconstructing the sub-block by adding the predicted sub-block to the residual sub-block, wherein the macroblock is based on the sub-block refactor.
这里，可以基于根据要解码的当前块的上侧块和左侧块而不同地设置了二进制信号的出现概率的上下文表来重构所述编码数据。Here, the encoded data may be reconstructed based on a context table in which occurrence probabilities of binary signals are set differently according to upper and left blocks of the current block to be decoded.
有益效果Beneficial effect
根据上述本发明，通过以下处理来提高视频压缩效率和视频重构效率：将扩展宏块划分为各种大小和形状的子块，以及根据这些子块的大小利用不同的预测单元和变换单元来进行视频编码/解码。According to the present invention described above, video compression efficiency and video reconstruction efficiency are improved by dividing an extended macroblock into subblocks of various sizes and shapes, and utilizing different prediction units and transform units according to the sizes of these subblocks to Perform video encoding/decoding.
附图说明Description of drawings
图1是例示根据本发明的实施方式的视频编码设备的示意性构造的框图；1 is a block diagram illustrating a schematic configuration of a video encoding device according to an embodiment of the present invention;
图2是用于描述在根据本发明的实施方式填充(pad)输入图像的情况下进行编码和解码的图，该图是例示输入图像和当前要编码的扩展宏块的位置的图；2 is a diagram for describing encoding and decoding in the case of padding an input image according to an embodiment of the present invention, which is a diagram illustrating an input image and positions of extended macroblocks currently to be encoded;
图3是图2中的当前要编码的扩展宏块的放大图；Fig. 3 is an enlarged view of the extended macroblock to be coded currently in Fig. 2;
图4是例示根据本发明的实施方式将宏块划分为针对帧内预测编码和帧间预测编码的各种大小的子块的处理的示例的图；4 is a diagram illustrating an example of a process of dividing a macroblock into subblocks of various sizes for intra-predictive encoding and inter-predictive encoding according to an embodiment of the present invention;
图5是例示根据本发明的另一实施方式将宏块划分为针对帧内预测编码和帧间预测编码的各种大小的子块的处理的示例的图；5 is a diagram illustrating an example of a process of dividing a macroblock into subblocks of various sizes for intra-prediction encoding and inter-prediction encoding according to another embodiment of the present invention;
图6是用于描述根据本发明的实施方式的帧内图片编码方法和要通过编码后数据发送的相关数据的例示图；6 is an illustration diagram for describing an intra picture encoding method and related data to be transmitted through encoded data according to an embodiment of the present invention;
图7是例示根据本发明的实施方式的亮度分量的帧内预测模式中的九个预测方向和预测模式编号的例示图；7 is an illustration diagram illustrating nine prediction directions and prediction mode numbers in an intra prediction mode of a luma component according to an embodiment of the present invention;
图8是例示根据本发明的实施方式的亮度分量的帧内预测模式中的四个预测方向和预测模式编号的例示图；8 is an illustration diagram illustrating four prediction directions and prediction mode numbers in an intra prediction mode of a luma component according to an embodiment of the present invention;
图9是例示根据本发明的实施方式的亮度分量的帧内预测模式中的三个预测方向和预测模式编号的例示图；9 is an illustration diagram illustrating three prediction directions and prediction mode numbers in an intra prediction mode of a luma component according to an embodiment of the present invention;
图10是例示根据本发明的实施方式的色度分量的帧内预测模式中的四个预测方向和预测模式编号的例示图；10 is an illustration diagram illustrating four prediction directions and prediction mode numbers in an intra prediction mode of a chroma component according to an embodiment of the present invention;
图11是例示根据本发明的实施方式的64x64像素扩展宏块的各种子块类型的例示图；11 is an illustration diagram illustrating various sub-block types of a 64x64 pixel extended macroblock according to an embodiment of the present invention;
图12是用于描述根据本发明的实施方式的帧间图片编码方法和要通过编码后数据发送的相关数据的例示图；12 is an illustration diagram for describing an inter picture encoding method and related data to be transmitted through encoded data according to an embodiment of the present invention;
图13是用于描述根据本发明的实施方式确定预测运动向量的处理的例示图；FIG. 13 is an illustration diagram for describing a process of determining a predicted motion vector according to an embodiment of the present invention;
图14是用于描述根据本发明的实施方式确定8x16像素块和16x8像素块的预测运动向量的处理的例示图；14 is an illustration diagram for describing a process of determining a prediction motion vector of an 8x16 pixel block and a 16x8 pixel block according to an embodiment of the present invention;
图15是例示根据本发明的实施方式的根据子块大小的可用变换类型和变换类型编码单元的例示图；15 is an illustration diagram illustrating available transformation types and transformation type coding units according to subblock sizes according to an embodiment of the present invention;
图16是例示根据本发明的实施方式要解块滤波的块的边界的例示图；16 is an illustration diagram illustrating boundaries of blocks to be deblocked and filtered according to an embodiment of the present invention;
图17是例示根据本发明的实施方式的帧内图片的句法的例示图；17 is an illustration diagram illustrating syntax of an intra picture according to an embodiment of the present invention;
图18是例示根据本发明的实施方式的各个分割类型的编号的例示图；FIG. 18 is an illustration diagram illustrating numbers of respective division types according to an embodiment of the present invention;
图19是用于描述根据本发明的实施方式将扩展宏块划分为各种大小的子块的情况下的分割类型编号编码序列的图；19 is a diagram for describing a division type number encoding sequence in a case where an extended macroblock is divided into subblocks of various sizes according to an embodiment of the present invention;
图20是顺序地例示根据本发明的实施方式逐个分隔层地划分宏块的处理的例示图；20 is an illustration diagram sequentially illustrating a process of dividing a macroblock by separation layer according to an embodiment of the present invention;
图21是用于描述顺序地对宏块的逐层分割类型编号进行编码的处理的例示图；21 is an illustration diagram for describing a process of sequentially encoding layer-by-layer partition type numbers of macroblocks;
图22是例示根据本发明的实施方式的帧内预测模式的编码序列的例示图；22 is an illustration diagram illustrating an encoding sequence of an intra prediction mode according to an embodiment of the present invention;
图23是用于描述根据本发明的实施方式对宏块的帧内预测模式进行编码的处理的示例的例示图；23 is an illustration diagram for describing an example of a process of encoding an intra prediction mode of a macroblock according to an embodiment of the present invention;
图24是用于描述根据本发明的实施方式针对帧内宏块对CBPX_flag和CBP进行编码的处理的例示图；24 is an illustration diagram for describing a process of encoding CBPX_flag and CBP for an intra macroblock according to an embodiment of the present invention;
图25是例示根据本发明的实施方式的CBP的结构的例示图；25 is an illustration diagram illustrating the structure of a CBP according to an embodiment of the present invention;
图26是例示根据本发明的实施方式对CBPX标记和CBP进行编码的序列的示例的例示图；Figure 26 is an illustration diagram illustrating an example of a sequence encoding a CBPX marker and a CBP according to an embodiment of the present invention;
图27是例示根据本发明的实施方式针对帧内宏块对CBPX标记和CBP进行编码的序列的另一示例的例示图；27 is an illustration diagram illustrating another example of a sequence for encoding a CBPX flag and a CBP for an intra macroblock according to an embodiment of the present invention;
图28是例示根据本发明的实施方式的逐个分割类型量化的变换系数的编码序列的例示图；28 is an illustration diagram illustrating an encoding sequence of transform coefficients quantized by partition type according to an embodiment of the present invention;
图29是例示根据本发明的实施方式的编码后数据的句法结构的例示图；29 is an illustration diagram illustrating a syntax structure of encoded data according to an embodiment of the present invention;
图30是用于描述根据本发明的实施方式在P图片(或P片段段)中使用的句法的示例的图；30 is a diagram for describing an example of syntax used in a P picture (or a P slice segment) according to an embodiment of the present invention;
图31是用于描述根据本发明的实施方式在B图片(或B片段段)中使用的句法的示例的图；31 is a diagram for describing an example of syntax used in a B-picture (or B-slice segment) according to an embodiment of the present invention;
图32是用于描述根据本发明的实施方式的SKIPX标记的编码处理的例示图；FIG. 32 is an illustration diagram for describing encoding processing of a SKIPX mark according to an embodiment of the present invention;
图33和图34是用于描述根据本发明的实施方式针对帧间宏块对CBPX_flag、变换类型和CBP进行编码的处理的例示图；33 and 34 are illustrations for describing a process of encoding CBPX_flag, transform type, and CBP for an inter macroblock according to an embodiment of the present invention;
图35是例示根据本发明的实施方式用于对skipX_flag进行编码/解码的ctx_inc的示例的图；35 is a diagram illustrating an example of ctx_inc for encoding/decoding skipX_flag according to an embodiment of the present invention;
图36是例示根据本发明的实施方式用于对interX_flag进行编码/解码的ctx_inc的示例的图；36 is a diagram illustrating an example of ctx_inc for encoding/decoding interX_flag according to an embodiment of the present invention;
图37是例示根据本发明的实施方式用于对directX_flag进行编码/解码的ctx_inc的示例的图；37 is a diagram illustrating an example of ctx_inc for encoding/decoding directX_flag according to an embodiment of the present invention;
图38和图39是用于描述根据本发明的实施方式利用树结构对分割类型信息进行编码的处理的图；38 and 39 are diagrams for describing a process of encoding division type information using a tree structure according to an embodiment of the present invention;
图40是例示根据本发明的实施方式对分割类型进行编码的示例的图；FIG. 40 is a diagram illustrating an example of encoding a division type according to an embodiment of the present invention;
图41是例示根据本发明的实施方式对分割类型进行编码的另一示例的图；41 is a diagram illustrating another example of encoding a division type according to an embodiment of the present invention;
图42是例示根据本发明的实施方式用于对B_pdir进行编码/解码的ctx_inc的示例的图；42 is a diagram illustrating an example of ctx_inc for encoding/decoding B_pdir according to an embodiment of the present invention;
图43是例示根据本发明的实施方式用于在两个可用变换类型的情况下对变换类型进行编码/解码的ctx_inc的示例的图；43 is a diagram illustrating an example of ctx_inc for encoding/decoding a transform type in the case of two available transform types according to an embodiment of the present invention;
图44是例示根据本发明的实施方式用于在三个可用变换类型的情况下对变换类型进行编码/解码的ctx_inc的示例的图；44 is a diagram illustrating an example of ctx_inc for encoding/decoding a transform type in the case of three available transform types according to an embodiment of the present invention;
图45是例示根据本发明的实施方式用于在三个可用变换类型的情况下对变换类型进行编码/解码的ctx_inc的示例的图；45 is a diagram illustrating an example of ctx_inc for encoding/decoding a transform type in the case of three available transform types according to an embodiment of the present invention;
图46是例示根据本发明的实施方式的视频解码设备的示意性构造的框图；46 is a block diagram illustrating a schematic configuration of a video decoding device according to an embodiment of the present invention;
图47是例示视频边界或片边界处的帧内预测的例示图；FIG. 47 is an illustration diagram illustrating intra prediction at a video boundary or a slice boundary;
图48是例示平面模式的示例的图；FIG. 48 is a diagram illustrating an example of a planar mode;
图49a-49b是例示根据本发明的实施方式对帧内图片的宏块和子块进行编码的流程图的例示图；49a-49b are illustrations illustrating a flowchart of encoding macroblocks and sub-blocks of an intra picture according to an embodiment of the present invention;
图50a-50b是例示根据本发明的实施方式对帧内图片的宏块和子块进行编码的流程图的另一例示图；50a-50b are another illustration diagram illustrating a flow chart for encoding macroblocks and sub-blocks of an intra picture according to an embodiment of the present invention;
图51a-51b是例示根据本发明的实施方式对变换子块的变换系数进行编码的序列的例示图；51a-51b are illustration diagrams illustrating a sequence for encoding transform coefficients of a transform sub-block according to an embodiment of the present invention;
图52a-52c是例示根据本发明的实施方式对帧间宏块和帧间图片的子块进行编码的序列的例示图；52a-52c are illustration diagrams illustrating sequences for encoding inter macroblocks and sub-blocks of inter pictures according to an embodiment of the present invention;
图53是针对批模式标记的ctx_inc的例示图；以及Figure 53 is an illustration of ctx_inc for batch mode marking; and
图54是针对差运动向量的x分量的ctx_inc的例示图。Fig. 54 is an illustration diagram of ctx_inc for the x-component of the difference motion vector.
具体实施方式Detailed ways
根据本发明的实施方式的视频编码设备和视频解码设备可以是个人计算机(PC)、笔记本或膝上型计算机、个人数字助理(PDA)、便携式多媒体播放器(PMP)、便携式游戏机(PSP)或无线通信终端、智能电话等，并且可以表示配备有例如诸如用于在各种装置或有线/无线通信网络之间进行通信的调制解调器的通信装置、用于存储用于对视频和相关数据进行编码或解码的各种程序的存储器以及用于执行这些程序以进行操作和控制的微处理器的各种设备。A video encoding device and a video decoding device according to an embodiment of the present invention may be a personal computer (PC), a notebook or laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a portable game console (PSP) or a wireless communication terminal, a smartphone, etc., and may mean a communication device equipped with, for example, a modem for communication between various devices or a wired/wireless communication network, for storing data for encoding video and related data Memory of various programs for decoding or decoding and various devices for microprocessors executing these programs for operation and control.
此外，由视频编码设备编码为比特流的视频可以实时或非实时地发送给用于在经由有线/无线通信网络发送视频之后对该视频进行解码以将其重构和再现为视频的视频解码设备，其中，所述有线/无线通信网络包括互联网、短距离无线通信网络、无线LAN网络、也称为WiMax网络的WiBro(无线宽带)以及移动通信网络或诸如电缆或USB(通用串行总线)的通信接口。Also, video encoded into a bit stream by a video encoding device may be transmitted in real time or non-real time to a video decoding device for decoding the video to reconstruct and reproduce it as a video after transmitting the video via a wired/wireless communication network , wherein the wired/wireless communication network includes the Internet, a short-distance wireless communication network, a wireless LAN network, WiBro (Wireless Broadband) also known as a WiMax network, and a mobile communication network or a network such as a cable or USB (Universal Serial Bus) Communication Interface.
图1是例示根据本发明的实施方式的视频编码设备的示意性构造的框图。FIG. 1 is a block diagram illustrating a schematic configuration of a video encoding device according to an embodiment of the present invention.
根据本发明的实施方式的视频编码设备100可以包括预测器110、减法器120、变换器/量化器130、编码器140、逆量化器和逆变换器150、加法器160、滤波器170和图片缓冲器180。A video encoding device 100 according to an embodiment of the present invention may include a predictor 110, a subtractor 120, a transformer/quantizer 130, an encoder 140, an inverse quantizer and inverse transformer 150, an adder 160, a filter 170, and a picture Buffer 180.
预测器110可以包括帧内预测器112和帧间预测器114。帧间预测器114可以包括运动估计器116和运动补偿器118。The predictor 110 may include an intra predictor 112 and an inter predictor 114 . Inter predictor 114 may include motion estimator 116 and motion compensator 118 .
输入视频可以按照帧或字段为单位输入到视频编码设备100，或者可以在被划分为具有NxN像素(N:大于或等于16的整数)的宏块之后输入到视频编码设备100。如果输入视频是诸如4Kx2K视频的高分辨率视频，则输入视频可以在被划分为具有NxN像素(N:大于16的整数)的扩展宏块之后被编码，从而实现视频压缩效率的提高。这里，具有NxN像素(N:大于16的整数)的宏块将被称作扩展宏块(EMB)。例如，扩展宏块可以包括诸如64x64和32x32的大小的正方形像素块。应当注意，下文描述的宏块可以包括扩展宏块和16x16像素块的普通宏块。The input video may be input to the video encoding device 100 in units of frames or fields, or may be input to the video encoding device 100 after being divided into macroblocks having NxN pixels (N: an integer greater than or equal to 16). If the input video is a high-resolution video such as 4Kx2K video, the input video may be encoded after being divided into extended macroblocks having NxN pixels (N: an integer greater than 16), thereby achieving improvement in video compression efficiency. Here, a macroblock having NxN pixels (N: an integer greater than 16) will be referred to as an extended macroblock (EMB). For example, extended macroblocks may include square pixel blocks of sizes such as 64x64 and 32x32. It should be noted that the macroblocks described below may include extended macroblocks and normal macroblocks of 16x16 pixel blocks.
当利用具有NxN像素(N：大于16的整数)的扩展宏块进行视频压缩时，如果输入视频不是16像素的倍数，则可以在输入视频被填充为16像素的倍数之后进行视频压缩。When performing video compression with an extended macroblock having NxN pixels (N: an integer greater than 16), if the input video is not a multiple of 16 pixels, video compression may be performed after the input video is padded to a multiple of 16 pixels.
图2是用于描述在根据本发明的实施方式填充输入图像的情况下进行视频编码和解码的图。图3是图2中的当前要编码的扩展宏块的放大图。如果被填充为16像素的倍数的输入视频和当前要编码的扩展宏块定位为如图2所示，则可以仅利用属于被填充为16像素的倍数的输入视频的像素值对当前的扩展宏块进行编码。即，在图3中，在对当前扩展宏块进行编码时，可能不希望使用当前扩展宏块中的填充区域。此外，编码在比特流中的数据还可以仅使用属于被填充为16像素的倍数的输入视频的像素值。FIG. 2 is a diagram for describing video encoding and decoding in the case of padding an input image according to an embodiment of the present invention. FIG. 3 is an enlarged view of the extended macroblock currently to be coded in FIG. 2 . If the input video padded to a multiple of 16 pixels and the current extended macroblock to be encoded are positioned as shown in Figure 2, the current extended macroblock can be encoded using only the pixel values belonging to the input video padded to a multiple of 16 pixels block to encode. That is, in FIG. 3 , when encoding the current extended macroblock, it may not be desirable to use the padding area in the current extended macroblock. Furthermore, the data encoded in the bitstream can also only use pixel values belonging to the input video that are padded to multiples of 16 pixels.
如果输入视频是4:2:0格式的，则宏块包括具有NxN像素的亮度块和具有两个像素的色度块。通过这种扩展宏块对高分辨率视频的压缩效率的提高可以通过以下描述的特定实施方式来实现。If the input video is in 4:2:0 format, the macroblock includes a luma block with NxN pixels and a chroma block with two pixels. The improvement of the compression efficiency of high-resolution video through such extended macroblocks can be realized through the specific implementation manners described below.
如图4所述，根据本发明的实施方式的扩展宏块可以在帧内预测编码或帧间预测编码之前被划分为子块。As described in FIG. 4 , an extended macroblock according to an embodiment of the present invention may be divided into subblocks before intra-predictive coding or inter-predictive coding.
即，如果具有NxN像素(N：大于或等于16的整数)被用于进行高分辨率视频编码，则扩展宏块可以在编码之前被划分为像素块，并且各个像素块可以在编码之前被划分为子块。此外，如图5所示，如果扩展宏块或者像素块的一边的长度大于16像素，则省略划分为具有诸如32x64,64x32,16x32或32x16的矩形形状的像素块，并且扩展宏块被划分为正方形像素块接着以16x16像素块为单位进行编码。图4和图5例示针对扩展宏块具有4x4的最小块大小的各个子块。例如，如图4所示，如果扩展宏块是64x64像素块，则64x64像素块、64x32像素块、32x64像素块和32x32像素块的子块可以属于宏块层0，并且32x32像素块、32x16像素块、16x32像素块和16x16像素块的子块可以属于宏块层1。此外，如图5所示，针对大于16x16像素块的子块，可以省略划分为诸如64x32像素块,32x64像素块,32x16像素块或16x32像素块的矩形子块。在该情况下，64x64像素块和32x32像素块的子块属于宏块层0，32x32像素块和16x16像素块的子块属于宏块层1。这里，宏块层K+1的子块仅在宏块层K(0≤K≤log2(N/4))的子块当中的最大子块被划分为四个块时才被使用。即，如同图5的情况，宏块层1的子块仅在宏块层0的64x64子块被划分为四个子块时才被使用，并且宏块层2的子块仅在宏块层1的32x32子块被划分为四个16x16子块时才被使用。That is, if NxN pixels (N: an integer greater than or equal to 16) are used for high-resolution video encoding, the extended macroblock can be divided into pixel blocks before encoding, and each pixel block can be divided before encoding for sub-blocks. In addition, as shown in FIG. 5, if the length of one side of an extended macroblock or a pixel block is greater than 16 pixels, division into pixel blocks having a rectangular shape such as 32x64, 64x32, 16x32, or 32x16 is omitted, and the extended macroblock is divided into The square pixel blocks are then encoded in units of 16x16 pixel blocks. 4 and 5 illustrate respective subblocks having a minimum block size of 4x4 for an extended macroblock. For example, as shown in Figure 4, if the extended macroblock is a 64x64 pixel block, subblocks of 64x64 pixel block, 64x32 pixel block, 32x64 pixel block, and 32x32 pixel block may belong to macroblock layer 0, and 32x32 pixel block, 32x16 pixel Blocks, 16x32 pixel blocks and sub-blocks of 16x16 pixel blocks may belong to macroblock layer 1 . In addition, as shown in FIG. 5 , for subblocks larger than 16x16 pixel blocks, rectangular subblocks divided into, for example, 64x32 pixel blocks, 32x64 pixel blocks, 32x16 pixel blocks or 16x32 pixel blocks may be omitted. In this case, subblocks of 64x64 pixel blocks and 32x32 pixel blocks belong to macroblock layer 0, and subblocks of 32x32 pixel blocks and 16x16 pixel blocks belong to macroblock layer 1. Here, the subblocks of the macroblock layer K+1 are used only when the largest subblock among the subblocks of the macroblock layer K (0≦K≦log2 (N/4)) is divided into four blocks. That is, as in the case of Figure 5, subblocks of macroblock layer 1 are used only when the 64x64 subblock of macroblock layer 0 is divided into four subblocks, and subblocks of macroblock layer 2 are used only when The 32x32 subblock is only used when it is divided into four 16x16 subblocks.
视频编码设备100在宏块在编码之前被划分为子块的情况下计算各个子块的编码效率，并且将具有最高编码效率的子块确定为最终的帧内预测块或帧间预测块。可以基于如式1所示的率失真优化(RDO)技术来测量编码效率。The video encoding apparatus 100 calculates encoding efficiency of each sub-block in a case where a macroblock is divided into sub-blocks before encoding, and determines a sub-block having the highest encoding efficiency as a final intra prediction block or an inter prediction block. Coding efficiency can be measured based on a rate-distortion optimization (RDO) technique as shown in Equation 1.
RDcost＝Distortion+λ·Bit 式1RDcost＝Distortion+λ·Bit Formula 1
在式1中，“Distortion”表示编码目标块与重构块之间的误差值，“λ”表示由量化参数确定的常数，并且“Bit”表示用于对编码目标块进行编码的比特的数量。In Equation 1, "Distortion" represents an error value between an encoding target block and a reconstructed block, "λ" represents a constant determined by a quantization parameter, and "Bit" represents the number of bits used to encode the encoding target block .
确定块模式可以包括针对所有类型的可用块确定各个块的最优预测模式(帧内预测模式和运动数据)的处理和确定块类型的另一处理。总体上，SAD(绝对差的和)或SATD(绝对变换差的和)用于减小(选择帧内预测模式或运动向量/基准图片时的)确定预测模式时的计算量，并且“Bit”中不考虑变换系数的比特。这可以总结并表述为式2。Determining the block mode may include a process of determining an optimal prediction mode (intra prediction mode and motion data) for each block for all types of available blocks and another process of determining a block type. In general, SAD (Sum of Absolute Differences) or SATD (Sum of Absolute Transform Differences) is used to reduce the calculation amount when determining the prediction mode (when selecting an intra prediction mode or a motion vector/reference picture), and "Bit" The bits of the transform coefficients are not considered in . This can be summarized and expressed as Equation 2.
RDcost＝SATD(or SAD)+λ·Bit[block type，prediction info，CBP] 式2RDcost＝SATD(or SAD)+λ·Bit[block type, prediction info, CBP] Formula 2
在式2中，“SAD”表示原始视频与预测视频之间的差值的绝对值的和，“SATD”表示残留块的变换值的绝对值的和。In Equation 2, 'SAD' represents the sum of absolute values of differences between the original video and the predicted video, and 'SATD' represents the sum of absolute values of transform values of residual blocks.
在确定块类型时，“RDcost”可以利用式3来严格计算。When determining the block type, "RDcost" can be strictly calculated using Equation 3.
RDcost＝Distortion+λ·Bit[block type，prediction info，CBP，coeficient] 式3RDcost＝Distortion+λ·Bit[block type, prediction info, CBP, coefficient] Formula 3
式3中的“Distortion”可以表示为式4。"Distortion" in Formula 3 can be expressed as Formula 4.
与式3比较，式2需要较小的计算量，但是在无运动区域或简单区域的情况下得到几乎类似的失真。因此，式2的使用造成难以确定最优模式。另一方面，与式2比较，式3可以计算更准确的编码效率，但是需要较大的计算量。因此，在本发明的实施方式中，式2或式3可以根据数据的优先(是否用作基准图片)或所希望的编码模式来选择性地使用。例如，式3可以用于确定帧内预测模式和确定块的运动向量/基准图片。然而，特殊的是，在确定B图片的16x16或更多像素块的预测方向(L0预测、L1预测和双向预测)和运动向量/基准图片的处理中，根据各个预测方向，可以利用式3确定运动向量和基准图片，并且可以利用式2确定预测方向。式3可以用于确定块类型。此外，如果子块的大小大于或等于预设的像素块的大小，则可以根据式3来确定编码成本，并且如果子块的大小小于预设的像素块的大小，则可以根据式2来确定编码成本。Compared with Equation 3, Equation 2 requires less computation, but results in almost similar distortions in the case of no motion regions or simple regions. Therefore, the use of Equation 2 makes it difficult to determine the optimal mode. On the other hand, compared with Equation 2, Equation 3 can calculate more accurate coding efficiency, but requires a larger amount of calculation. Therefore, in the embodiment of the present invention, Equation 2 or Equation 3 can be selectively used according to the priority of data (whether it is used as a reference picture) or a desired coding mode. For example, Equation 3 can be used to determine the intra prediction mode and determine the motion vector/reference picture of the block. However, it is special that in the process of determining the prediction direction (L0 prediction, L1 prediction and bidirectional prediction) and motion vector/reference picture of a 16x16 or more pixel block of a B picture, according to each prediction direction, Equation 3 can be used to determine Motion vectors and reference pictures, and can use formula 2 to determine the prediction direction. Equation 3 can be used to determine the block type. In addition, if the size of the sub-block is greater than or equal to the size of the preset pixel block, the encoding cost can be determined according to Equation 3, and if the size of the sub-block is smaller than the size of the preset pixel block, then it can be determined according to Equation 2 Coding costs.
例如，假设选择编码成本的标准被设置为16x16像素块。在该情况下，16x16像素块的可用编码模式可以包括使用16x16SKIP模式、16x16直接模式和L0预测模式(例如，运动信息和残留数据)的16x16块、使用L1预测模式(例如，运动信息和残留数据)的16x16块、使用L0和L1预测模式(例如，运动信息和残留数据)这两者的16x16块以及16x16帧内块。对于16x16或更多像素块可以使用精确的编码成本。此原因在于由于具有16x16或更多像素的大小的块类型主要对应于诸如背景的平坦区域，所以难以通过合适的方程式确定最优模式。因此，如果使用16x16或更多像素块，则可以在确定帧内宏块模式时根据块大小和模式来选择性使用编码成本。此外，在考虑编码成本计算式中所考虑的参数时，用于确定小块之间的模式的参数可以与用于确定大块之间的模式的参数不同。For example, assume that a criterion for selecting an encoding cost is set to a 16x16 pixel block. In this case, available encoding modes for a 16x16 pixel block may include 16x16 blocks using 16x16 SKIP mode, 16x16 direct mode and L0 prediction mode (e.g. motion information and residual data), 16x16 block using L1 prediction mode (e.g. motion information and residual data ), 16x16 blocks using both L0 and L1 prediction modes (eg, motion information and residual data), and 16x16 intra blocks. Exact coding costs may be used for 16x16 or more pixel blocks. The reason for this is that it is difficult to determine an optimum mode by an appropriate equation since a block type having a size of 16x16 pixels or more mainly corresponds to a flat area such as a background. Therefore, if 16x16 or more pixel blocks are used, the coding cost can be selectively used depending on the block size and mode when determining the intra macroblock mode. Also, when considering the parameters considered in the encoding cost calculation formula, the parameters for determining the mode between small blocks may be different from the parameters for determining the mode between large blocks.
根据作为最大可用层值的最大层值(MaxLayer)来确定最小块大小。例如，在NxN像素宏块的情况下，最小块大小被确定为N/(2MaxLayer)。The minimum block size is determined according to the maximum layer value (MaxLayer) which is the maximum available layer value. For example, in the case of an NxN pixel macroblock, the minimum block size is determined to be N/(2MaxLayer ).
下文将参照相关附图来描述视频编码设备100的各个元件。Respective elements of the video encoding apparatus 100 will be described below with reference to related drawings.
当进行视频编码时，视频编码设备100按照构成视频的图片为单位区分图片类型，并且根据图片类型不同地进行预测编码。例如，如果特定图片是帧内图片，则帧内图片中的宏块按照宏块类型全部是帧内宏块，并且针对各个宏块进行帧内预测编码。此外，如果特定图片是帧间图片，则帧间图片中的宏块按照宏块类型可以是帧间宏块或帧内宏块，并且可以根据宏块类型进行帧内预测编码或帧间预测编码。这种图片类型信息可以不经过编码或者在编码之后插入编码后数据的图片头或序列头中，并且稍后描述的视频解码设备可以从图片头或序列头中提取图片类型信息，以确定要解码的图片的图片类型。When performing video encoding, the video encoding apparatus 100 distinguishes picture types in units of pictures constituting a video, and differently performs predictive encoding according to picture types. For example, if a specific picture is an intra picture, the macroblocks in the intra picture are all intra macroblocks by macroblock type, and intra prediction encoding is performed for each macroblock. In addition, if the specific picture is an inter picture, the macroblocks in the inter picture can be inter macroblocks or intra macroblocks according to the macroblock type, and can be intra-prediction-encoded or inter-prediction-encoded according to the macroblock type . This picture type information may not be encoded or inserted into the picture header or sequence header of the encoded data after encoding, and the video decoding device described later may extract the picture type information from the picture header or sequence header to determine the decoding The image type of the image.
下文中，将根据图片类型来描述根据本发明的实施方式的编码方法和通过该编码方法生成的编码数据。Hereinafter, an encoding method according to an embodiment of the present invention and encoded data generated by the encoding method will be described according to picture types.
图6是用于描述根据本发明的实施方式的帧内图片编码方法和要通过编码后数据发送的相关数据的例示图。FIG. 6 is an illustration diagram for describing an intra picture encoding method and related data to be transmitted through encoded data according to an embodiment of the present invention.
预测器110的帧内预测器112利用当前要编码的块(下文称作“当前块”)的相邻像素生成基准块，并且通过将该当前块与基准块相比较来确定帧内预测模式。这里，相邻像素是指与当前块相邻的块中的像素，并包括与当前块相邻的块中并且与当前块相邻的块中所包含的相邻像素。The intra predictor 112 of the predictor 110 generates a reference block using neighboring pixels of a block currently to be encoded (hereinafter referred to as 'current block'), and determines an intra prediction mode by comparing the current block with the reference block. Here, adjacent pixels refer to pixels in blocks adjacent to the current block, and include adjacent pixels contained in blocks adjacent to the current block and in blocks adjacent to the current block.
假设扩展宏块针对亮度分量被划分为4x4像素的子块，以执行帧内预测。在该情况下，参照图7(例示根据亮度分量的帧内预测模式的九个预测方向和预测模式编号)，从九个预测方向(根据预测模式0至8的预测方向)当中选择针对各个4x4像素当前块的最合适的预测方向，并且所选择的预测方向被用于对当前块进行帧内预测编码。预测模式2表示要从八个相邻像素计算的平均值，该八个相邻像素包括当前块的左侧块的四个相邻像素和当前块的上侧块的四个相邻像素，以预测当前块的所有4x4像素。It is assumed that an extended macroblock is divided into sub-blocks of 4x4 pixels for a luma component to perform intra prediction. In this case, referring to FIG. 7 (illustrating nine prediction directions and prediction mode numbers according to intra prediction modes of luma components), a selection is made for each 4x4 from among nine prediction directions (prediction directions according to prediction modes 0 to 8). The most suitable prediction direction of the current block of pixels, and the selected prediction direction is used for intra-frame prediction coding of the current block. Prediction mode 2 indicates an average value to be calculated from eight neighboring pixels including four neighboring pixels of the left block of the current block and four neighboring pixels of the upper block of the current block, to Predict all 4x4 pixels of the current block.
在位于图片的左侧边界的块和位于图片的上侧边界的块的情况下，左侧块和上侧块位于图片的外部。在该情况下，由于不能指代偏离图片边界的块，所以预测方向的使用被限制。例如，指代上级块的像素的预测模式0、3、4、5、6和7的预测方向不能用在图片的最上级块中。尤其是，在DC模式的情况下，仅参照没有偏离图片边界的像素来预测当前块。如果与当前块相邻的左侧块和上侧块不可用，则将值128用作DC值。In the case of the block located at the left boundary of the picture and the block located at the upper boundary of the picture, the left block and the upper block are located outside the picture. In this case, the use of prediction directions is limited since it cannot refer to blocks off picture boundaries. For example, prediction directions of prediction modes 0, 3, 4, 5, 6, and 7 referring to pixels of a higher-level block cannot be used in the uppermost-level block of a picture. In particular, in the case of DC mode, the current block is predicted with reference only to pixels that do not deviate from the picture boundary. If the left and upper blocks adjacent to the current block are not available, a value of 128 is used as the DC value.
当前块的所选择的预测方向与当前块的左侧块和上侧块中的具有较小预测模式编号的一个的预测方向进行比较。如果该两个预测方向相同，则对指示从相邻块估计的当前块的预测方向与当前块的所选择的预测方向是否相同的预测模式标记(通过例如1比特来表示)进行编码，以指示所选择的预测方向与所估计的预测方向相同。The selected prediction direction of the current block is compared with the prediction direction of one of the left and upper blocks of the current block having a smaller prediction mode number. If the two prediction directions are the same, a prediction mode flag (represented by, for example, 1 bit) indicating whether the prediction direction of the current block estimated from the neighboring block is the same as the selected prediction direction of the current block is encoded to indicate The selected prediction direction is the same as the estimated prediction direction.
如果从相邻块估计的当前块的预测方向与当前块的所选择的预测方向不同，则预测模式标记被编码，以指示所选择的预测方向不同于所估计的预测方向。接着，对指示九个预测模式中除与当前块的所估计的预测方向相对应的预测模式以外的剩余八个预测模式中的哪一个是当前块的所选择的预测方向的预测模式信息(通过例如3比特来表示)进行编码。If the prediction direction of the current block estimated from the neighboring blocks is different from the selected prediction direction of the current block, a prediction mode flag is encoded to indicate that the selected prediction direction is different from the estimated prediction direction. Next, the prediction mode information (via For example, represented by 3 bits) for encoding.
假设扩展宏块针对亮度分量被划分为8x8像素的子块，以执行帧内预测。在该情况下，如同4x4像素为单位的帧内预测，针对亮度分量以8x8像素为单位的帧内预测使用如图7所示的九个预测方向。除了块大小的差异(4x4像素和8x8像素)以外，计算预测像素的方法与以4x4像素为单位的帧内预测编码的情况相同。It is assumed that an extended macroblock is divided into sub-blocks of 8x8 pixels for a luma component to perform intra prediction. In this case, like the intra prediction in units of 4x4 pixels, the intra prediction in units of 8x8 pixels for the luminance component uses nine prediction directions as shown in FIG. 7 . Except for the difference in block size (4x4 pixels and 8x8 pixels), the method of calculating predicted pixels is the same as in the case of intra prediction encoding in units of 4x4 pixels.
假设扩展宏块针对亮度分量被划分为16x16像素的子块，以执行帧内预测。在该情况下，在图8所示的四个预测方向当中，选择提供最高编码效率的预测方向。根据所选择的预测方向，根据包括16x16像素的当前块的上侧的16x16像素块中的16个相邻像素和16x16像素的当前块的左侧的16x16像素块中的16个相邻像素的32个相邻像素对16x16像素块进行预测编码。It is assumed that an extended macroblock is divided into sub-blocks of 16x16 pixels for a luma component to perform intra prediction. In this case, among the four prediction directions shown in FIG. 8 , the prediction direction that provides the highest encoding efficiency is selected. According to the selected prediction direction, according to 32 of 16 adjacent pixels in the 16x16 pixel block above the current block of 16x16 pixels and 16 adjacent pixels in the 16x16 pixel block to the left of the current block of 16x16 pixels Neighboring pixels predictively encode a 16x16 pixel block.
参照图8，在与预测模式3对应的平面预测的情况下，在预测之前按对角方向对当前块的上侧块中的相邻像素和当前块的左侧块中的相邻像素进行插值。在与预测模式2对应的平均值预测的情况下，通过当前块的所有16x16像素来预测当前块的上侧块中的16个相邻像素和当前块的左侧块中的16个相邻像素的平均值。这里，如果当前块位于图片的最上位置，则当前块的左侧块中的16个相邻像素的平均值被用作预测值。如果当前块位于图片中的最左位置，则当前块的上侧块中的16个相邻像素的平均值被用作预测值。如果当前块的左侧块和上侧块不可用，则值128被用作预测值。16x16像素块的预测模式不被预测编码，并且选择的预测模式的编号被简单地编码为预测模式信息(通过例如2比特来表示)。Referring to FIG. 8 , in the case of planar prediction corresponding to prediction mode 3, adjacent pixels in the upper block of the current block and adjacent pixels in the left block of the current block are interpolated in a diagonal direction before prediction . In the case of mean prediction corresponding to prediction mode 2, 16 adjacent pixels in the upper block of the current block and 16 adjacent pixels in the left block of the current block are predicted by all 16x16 pixels of the current block average of. Here, if the current block is located at the uppermost position of the picture, the average value of 16 adjacent pixels in the left block of the current block is used as a prediction value. If the current block is located at the leftmost position in the picture, the average value of 16 adjacent pixels in the upper side block of the current block is used as a prediction value. If the left and upper blocks of the current block are not available, a value of 128 is used as the prediction value. The prediction mode of the 16x16 pixel block is not predictively coded, and the number of the selected prediction mode is simply coded as prediction mode information (expressed by, for example, 2 bits).
如果除上述4x4像素块、8x8像素块或16x16像素块以外，扩展宏块被划分为子块以执行帧内预测，则可以通过如图9所示的三个预测方向和预测模式编号来表示亮度分量的帧内预测模式。如果除4x4像素块、8x8像素块或16x16像素块以外，扩展宏块被划分为剩余子块以执行帧内预测，并且如果相关子块的大小是m x n，则根据从图9所示的三个预测方向当中选择的提供最高编码效率的预测方向，从当前块的上侧块中的m个相邻像素和当前块的左侧块中的n个相邻像素对mxn像素块进行帧内预测编码。If the extended macroblock is divided into sub-blocks to perform intra prediction in addition to the above-mentioned 4x4 pixel block, 8x8 pixel block, or 16x16 pixel block, brightness can be represented by three prediction directions and prediction mode numbers as shown in FIG. 9 The intra prediction mode for the component. If the extended macroblock is divided into residual sub-blocks to perform intra prediction other than 4x4 pixel block, 8x8 pixel block or 16x16 pixel block, and if the size of the relevant sub-block is m x n, then according to the three The prediction direction that provides the highest coding efficiency is selected among the prediction directions, and the mxn pixel block is intra-predictively coded from m adjacent pixels in the upper block of the current block and n adjacent pixels in the left block of the current block .
在与预测模式2对应的平均值预测的情况下，当前块的上侧块中的m个相邻像素和当前块的左侧块中的n个相邻像素的平均值被用来对当前块进行预测编码。这里，如果当前块位于图片中的最上位置，则当前块的左侧块中的n个相邻像素的平均值被用作当前块的预测值。如果当前块位于图片中的最左位置，则当前块的上侧块中的m个相邻像素的平均值被用作当前块的预测值。如果当前块的左侧块和上侧块是不可用的，则值128被用作当前块的预测值。In the case of average value prediction corresponding to prediction mode 2, the average value of m adjacent pixels in the upper block of the current block and n adjacent pixels in the left block of the current block is used to calculate the current block Do predictive coding. Here, if the current block is located at the uppermost position in the picture, the average value of n adjacent pixels in the left block of the current block is used as the prediction value of the current block. If the current block is located at the leftmost position in the picture, the average value of m adjacent pixels in the upper block of the current block is used as the prediction value of the current block. If the left and upper blocks of the current block are unavailable, a value of 128 is used as the prediction value of the current block.
如果当前块的所选择的预测方向与当前块的左侧块和上侧块中的具有较小预测模式编码的一个的预测方向相同，则将指示从当前块的相邻块估计的当前块的预测方向与当前块的所选择的预测方向是否相同的预测模式标记(通过例如1比特来表示)编码为指示当前块的所选择的预测方向与当前块的所估计的预测方向相同的信息。由于mxn像素块的可用预测模式编号是0至2，所以如果利用当前块的左侧块和上侧块估计的当前块的预测模式编号大于或等于3，则当前块的预测方向可以使用预测模式编号2(DC模式)。例如，从相邻块估计的当前块的预测模式编号是4，当前块的预测模式编号被设置为2，以对指示当前块的所估计的预测方向是否与当前块的所选择的预测方向相同的预测模式标记(通过例如1比特来表示)进行编码。If the selected prediction direction of the current block is the same as that of the left block and the upper block of the current block with the smaller prediction mode coded, it will indicate the current block estimated from the neighboring blocks of the current block A prediction mode flag (represented by, for example, 1 bit) whether the prediction direction is the same as the selected prediction direction of the current block is encoded as information indicating that the selected prediction direction of the current block is the same as the estimated prediction direction of the current block. Since the available prediction mode number of the mxn pixel block is 0 to 2, if the prediction mode number of the current block estimated by using the left block and the upper side block of the current block is greater than or equal to 3, the prediction direction of the current block can use the prediction mode Number 2 (DC mode). For example, the prediction mode number of the current block estimated from the neighboring block is 4, and the prediction mode number of the current block is set to 2 to indicate whether the estimated prediction direction of the current block is the same as the selected prediction direction of the current block The prediction mode flag (represented by, for example, 1 bit) is encoded.
这里，扩展宏块的帧内预测可以按照16x16或更小像素块为单位，即，按照4x4像素块、8x8像素块、16x16像素块或mxn像素块(这里，m≠n，并且m和n是小于16的数)。Here, the intra-frame prediction of the extended macroblock can be in units of 16x16 or smaller pixel blocks, that is, in accordance with 4x4 pixel blocks, 8x8 pixel blocks, 16x16 pixel blocks or mxn pixel blocks (here, m≠n, and m and n are less than 16).
可以按照N/2x N/2像素块为单位执行色度分量的帧内预测。如图10所示，可以使用诸如平均值预测、水平预测、垂直预测和平面预测的四个预测方向。可以按照8x8像素块为单位执行色度分量的帧内预测。Intra prediction of chroma components may be performed in units of N/2×N/2 pixel blocks. As shown in FIG. 10, four prediction directions such as average prediction, horizontal prediction, vertical prediction, and planar prediction can be used. Intra prediction of chroma components may be performed in units of 8x8 pixel blocks.
图10例示根据本发明的实施方式与色度分量的四个帧内预测模式相对应的预测方向和预测模式编号。参照图10，除了块大小(16x16像素和8x8像素)的差异以外，针对与预测模式3相对应的平面预测的预测值计算方法和与预测模式0相对应的平均值预测与按照16x16像素为单位的亮度分量的帧内预测编码方法的情况相同。可以独立于亮度信号的预测模式来选择色度信号的预测模式。存在两种类型的色度信号，即，U和V，但是预测方法是相同的。色度信号的预测模式是针对U和V各一个。各个色度信号的一个预测模式不是预测编码的，并且选择的预测模式编号简单地利用2比特来表示。FIG. 10 illustrates prediction directions and prediction mode numbers corresponding to four intra prediction modes of a chroma component according to an embodiment of the present invention. Referring to FIG. 10 , in addition to the difference in block size (16x16 pixels and 8x8 pixels), the prediction value calculation method for planar prediction corresponding to prediction mode 3 and the average value prediction corresponding to prediction mode 0 are the same as in units of 16x16 pixels The same is true for the intra-frame prediction encoding method of the luminance component. The prediction mode for the chroma signal can be selected independently of the prediction mode for the luma signal. There are two types of chrominance signals, U and V, but the prediction method is the same. The prediction modes for chrominance signals are one for U and one for V. One prediction mode of each chroma signal is not predictively coded, and the selected prediction mode number is simply expressed with 2 bits.
再次参照图6，按照上文参照图7和图10所述的相同的方式，帧内预测器112利用通过针对4x4像素的单位按照九个预测方向对当前块进行预测和预测编码来按照RDO和类似方法确定的帧内预测模式、通过针对8x8像素的单位按照九个预测方向对当前块进行预测和预测编码来按照RDO和类似方法确定的帧内预测模式、通过针对16x16像素的单位按照九个预测方向对当前块进行预测和预测编码来按照RDO和类似方法确定的帧内预测模式和通过针对m ⅹ n像素的单位按照九个预测方向对当前块进行预测和预测编码来按照RDO和类似方法确定的帧内预测模式当中的由率失真优化确定的一个帧内预测模式，来确定当前块的帧内预测模式。这里使用的率失真优化方程可以表示为式3。当按照这种方法确定帧内预测模式时，还相应地确定了子块的大小。Referring again to FIG. 6, in the same manner as described above with reference to FIG. 7 and FIG. The intra-frame prediction mode determined by a similar method, the intra-frame prediction mode determined by RDO and similar methods by predicting and predictively encoding the current block according to nine prediction directions for a unit of 8x8 pixels, by using nine prediction directions for a unit of 16x16 pixels The prediction direction predicts and predictively encodes the current block according to the intra prediction mode determined by RDO and similar methods and by predicting and predictively encoding the current block according to nine prediction directions for units of m ⅹ n pixels according to RDO and similar methods An intra prediction mode determined by rate-distortion optimization among the determined intra prediction modes is used to determine the intra prediction mode of the current block. The rate-distortion optimization equation used here can be expressed as Equation 3. When the intra prediction mode is determined in this way, the size of the sub-block is also determined accordingly.
根据如上所述确定的帧内预测模式，帧内预测器112通过预测当前块生成预测块。减法器120通过从当前块中减去预测块来生成残留块。变换器/量化器130通过对残留块进行变换和量化来生成量化变换系数。编码器140通过对所述量化变换系数进行熵编码来生成编码后数据。The intra predictor 112 generates a prediction block by predicting a current block according to the intra prediction mode determined as described above. The subtracter 120 generates a residual block by subtracting the prediction block from the current block. The transformer/quantizer 130 generates quantized transform coefficients by transforming and quantizing the residual block. The encoder 140 generates encoded data by entropy encoding the quantized transform coefficients.
变换器/量化器130可以对选择了以8x8,8x16或16x8像素为单位的帧内预测的当前块的残留块进行4x4变换，并且对选择了以8x8,8x16或16x8像素为单位的帧内预测的当前块的残留块进行8x8变换。此外，变换器/量化器130可以对选择了以16x16或更多像素为单位的帧内预测的当前块的残留块进行16x16变换。在该情况下，由于帧内预测的单位和子块的大小相同，所以可以根据子块的块大小来确定变换类型。The transformer/quantizer 130 may perform 4x4 transformation on the residual block of the current block for which the intra prediction in the unit of 8x8, 8x16 or 16x8 pixels is selected, and the residual block in which the intra prediction in the unit of 8x8, 8x16 or 16x8 pixels is selected The residual block of the current block is transformed by 8x8. Also, the transformer/quantizer 130 may perform 16x16 transform on the residual block of the current block for which intra prediction in units of 16x16 or more pixels is selected. In this case, since the unit of intra prediction is the same as the size of the sub-block, the transform type can be determined according to the block size of the sub-block.
这里，进行了16x16变换的具有16x16像素的残留块(下文称为“16x16像素残留块”)被再次划分为十六个具有4x4像素的残留块(下文称为“4x4像素残留块”)，并且可以对4x4像素残留块进行4x4变换。此后，具有4x4变换系数的变换块(下文称作“DC分量块”)仅由通过对各个4x4像素残留块进行4x4变换而生成的各个具有4x4像素的变换块(下文称为“4x4像素变换块”)的十六个DC分量构成，并且再次对DC分量块进行4x4变换。这里，用于对DC分量块进行变换的变换类型可以与用于对残留块进行变换的变换类型不同。即，可以对4x4像素残留块进行4x4离散余弦变换(DCT)，并且可以对DC分量块进行4x4哈达玛变换。例如，选择了以16x32像素为单位的帧内预测的当前块的16x32像素残留块被划分为两个16x16像素残留块，并且各个16x16像素残留块以4x4像素为单位进行4x4变换。此后，DC分量块由16x16像素残留块中的各个4x4变换块的DC分量构成，并且对DC分量块再次进行4x4变换。Here, the 16x16-transformed residual block with 16x16 pixels (hereinafter referred to as "16x16 pixel residual block") is subdivided into sixteen residual blocks with 4x4 pixels (hereinafter referred to as "4x4 pixel residual block"), and A 4x4 transform may be performed on a 4x4 pixel residual block. Thereafter, a transform block having 4x4 transform coefficients (hereinafter referred to as a "DC component block") is composed only of transform blocks each having 4x4 pixels (hereinafter referred to as a "4x4 pixel transform block") generated by performing 4x4 transform on each 4x4 pixel residual block. ") of sixteen DC components, and again perform 4x4 transformation on the DC component block. Here, a transform type for transforming the DC component block may be different from a transform type for transforming the residual block. That is, 4x4 Discrete Cosine Transform (DCT) may be performed on the 4x4 pixel residual block, and 4x4 Hadamard Transform may be performed on the DC component block. For example, the 16x32 pixel residual block of the current block for which intra prediction in units of 16x32 pixels is selected is divided into two 16x16 pixel residual blocks, and each 16x16 pixel residual block is 4x4 transformed in units of 4x4 pixels. Thereafter, the DC component block is composed of the DC components of the respective 4x4 transform blocks in the 16x16 pixel residual block, and the 4x4 transform is performed on the DC component block again.
假设作为32x32像素块的扩展宏块位于整个被填充为16像素的倍数的图像和如图2所示填充的区域上，并且扩展宏块是帧内模式。在该情况下，如图3所示，仅针对实际图像区域(16x16像素块)确定编码模式，并且对相关区域的分割类型、帧内预测模式、CBP和变换系数进行编码。It is assumed that an extended macroblock which is a 32x32 pixel block is located on the entire image padded to a multiple of 16 pixels and a region padded as shown in FIG. 2 , and that the extended macroblock is an intra mode. In this case, as shown in FIG. 3 , the encoding mode is determined only for the actual image area (16x16 pixel block), and the division type, intra prediction mode, CBP, and transform coefficient of the relevant area are encoded.
图11是例示根据本发明的实施方式的64x64像素扩展宏块的子块的块类型的例示图。FIG. 11 is an illustration diagram illustrating block types of sub-blocks of a 64x64-pixel extended macroblock according to an embodiment of the present invention.
假设扩展宏块是64x64像素块，并且在如图11所示的帧内预测之后确定最优块类型。在该情况下，对块0至7进行4x4变换，对块8至12进行8x8变换，对16x16像素块中的块13至18进行4x4变换，并且对由各个16x16像素块中的各个4x4变换块的DC分量构成的DC分量块再次进行变换。It is assumed that an extended macroblock is a 64x64 pixel block, and an optimal block type is determined after intra prediction as shown in FIG. 11 . In this case, blocks 0 to 7 are 4x4 transformed, blocks 8 to 12 are 8x8 transformed, blocks 13 to 18 are 4x4 transformed in 16x16 pixel blocks, and each 4x4 transformed block in each 16x16 pixel block is transformed The DC component block formed by the DC component is transformed again.
在色度信号的情况下，如同以16x16或更多像素为单位的帧内预测的情况，进行4x4变换，接着再次对DC分量块进行4x4变换。然而，色度分量的宏块的残留块被划分为具有8x8像素的残留块(下文称为“8x8像素残留块”)，并且对各个8x8像素残留块中的四个4x4像素残留块进行4x4变换。具有2x2变换系数的DC分量块由8x8像素残留块中的四个4x4像素残留块的DC分量构成，并且对DC分量块进行2x2变换。在该情况下，同样地，对8x8像素残留块的4x4变换类型与对DC分量块的2x2变换类型可以不同。In the case of a chroma signal, as in the case of intra prediction in units of 16x16 or more pixels, 4x4 transform is performed, and then 4x4 transform is performed on the DC component block again. However, the residual block of the macroblock of the chroma component is divided into residual blocks having 8x8 pixels (hereinafter referred to as "8x8 pixel residual block"), and 4x4 transformation is performed on four 4x4 pixel residual blocks in each 8x8 pixel residual block . A DC component block having a 2x2 transform coefficient is composed of DC components of four 4x4 pixel residual blocks in an 8x8 pixel residual block, and 2x2 transform is performed on the DC component block. In this case, too, the 4x4 transform type for the 8x8 pixel residual block may be different from the 2x2 transform type for the DC component block.
取代针对扩展块确定优化块类型并且接着根据上述变换类型进行变换，可以根据以16x16像素块为单位的帧内预测中使用的子块的大小来选择并使用高效率变换类型。即，变换器/量化器130可以按照16x16像素块为单位进行一个或更多个变换和量化，选择最高效率的变换类型，并且根据所选择的变换类型生成量化变换系数。在该情况下，编码器140可以通过生成用于识别针对各个块选择的变换的类型的变换类型并且对其进行编码来生成变换类型数据。变换类型数据可以被包括在编码数据中。然而，不同的变换不能用在16x16像素块中。Instead of determining an optimized block type for an extended block and then performing transformation according to the above-mentioned transform type, a high-efficiency transform type may be selected and used according to the size of a sub-block used in intra prediction in units of 16x16 pixel blocks. That is, the transformer/quantizer 130 may perform one or more transforms and quantizations in units of 16x16 pixel blocks, select the most efficient transform type, and generate quantized transform coefficients according to the selected transform type. In this case, the encoder 140 may generate transform type data by generating a transform type for identifying a type of transform selected for each block and encoding it. Transform type data may be included in encoded data. However, different transforms cannot be used in 16x16 pixel blocks.
更具体地说，如果16x16像素块被划分为8x8像素块，并且如果四个8x8像素块中的一个或更多个被划分为小于8x8像素块的子块，则可以对16x16像素块仅进行4x4变换。在该情况下，不对变换类型进行编码。此外，如果16x16像素块被划分为四个8x8像素块，则可以对16x16像素块进行4x4变换和8x8变换，以选择要求最低编码成本的变换类型，并且可以对指示选择的变换的类型的变换类型(其可以通过例如1比特来表示)进行编码。此外，如果16x16像素块被划分为两个8x16像素子块，则可以对16x16像素子块的残留块进行4x4变换、8x8变换和8x16变换，以选择要求最低编码成本的变换类型，并且如果16x16像素块被划分为两个16x8像素子块，则可以对16x16像素子块的残留块进行4x4变换、8x8变换和16x8变换，以选择要求最低编码成本的变换类型，并且可以对指示选择的变换的类型的变换类型(其可以通过例如1比特或2比特来表示)进行编码。可以对16x16或更多的像素块的残留块进行4x4变换、8x8变换和16x16变换，以选择要求最低编码成本的变换类型，并且可以对指示选择的变换的类型的变换类型(其可以通过例如1比特或2比特来表示)进行编码。More specifically, a 16x16 pixel block can only be 4x4 if it is divided into 8x8 pixel blocks, and if one or more of the four 8x8 pixel blocks is divided into subblocks smaller than the 8x8 pixel block transform. In this case, the transform type is not encoded. In addition, if a 16x16 pixel block is divided into four 8x8 pixel blocks, 4x4 transformation and 8x8 transformation can be performed on the 16x16 pixel block to select the transformation type requiring the lowest coding cost, and the transformation type indicating the type of the selected transformation can be selected (which can be represented by, for example, 1 bit) is coded. Furthermore, if a 16x16 pixel block is divided into two 8x16 pixel subblocks, 4x4 transform, 8x8 transform, and 8x16 transform can be performed on the residual block of the 16x16 pixel subblock to select the transform type that requires the lowest encoding cost, and if the 16x16 pixel The block is divided into two 16x8 pixel sub-blocks, then 4x4 transformation, 8x8 transformation and 16x8 transformation can be performed on the residual block of the 16x16 pixel sub-block to select the transformation type that requires the lowest coding cost, and can indicate the type of transformation selected The transformation type (which can be represented by, for example, 1 bit or 2 bits) is encoded. 4x4 transformation, 8x8 transformation, and 16x16 transformation can be performed on a residual block of 16x16 or more pixel blocks to select the transformation type requiring the lowest encoding cost, and the transformation type indicating the type of the selected transformation (which can be passed, for example, 1 bit or 2 bits) for encoding.
图12是用于描述根据本发明的实施方式的帧间图片编码方法和要通过编码数据发送的相关数据的例示图。FIG. 12 is an illustration diagram for describing an inter picture encoding method and related data to be transmitted through encoded data according to an embodiment of the present invention.
在P图片或P片段段的情况下，图4中的所有块类型可以用于运动估计，并且可以按照16x16像素块为单位在扩展宏块中选择帧内模式或帧间模式。即，帧内模式和帧间模式这两者可以用在P图片或P片段段的扩展宏块中。In the case of a P picture or a P slice segment, all block types in FIG. 4 can be used for motion estimation, and intra mode or inter mode can be selected in extended macroblocks in units of 16x16 pixel blocks. That is, both intra mode and inter mode can be used in extended macroblocks of P pictures or P slice segments.
在B图片或B片段的情况下，可以选择多达两个基准图片，其分别称为L0预测和L1预测。两个基准图片执行不同的功能。总体上，L0预测(列表0预测，主要用作前向预测)用作前向预测，L1预测(列表1预测，主要用作反向预测)用作反向预测。稍后将详细描述P图片和B图片的帧间图片编码方法。In the case of B-pictures or B-slices, up to two reference pictures may be selected, which are called L0 prediction and L1 prediction, respectively. The two reference images perform different functions. In general, L0 prediction (list 0 prediction, mainly used for forward prediction) is used for forward prediction, and L1 prediction (list 1 prediction, mainly used for backward prediction) is used for backward prediction. The inter picture coding method of P pictures and B pictures will be described in detail later.
如果扩展宏块是32x32像素块，则可以通过扩展宏块标记(extended_mb_flag)指示是按照32x32像素块为单位还是按照16x16像素块为单位进行编码。例如，如果扩展宏块标记是1，则可以指示按照32x32像素块为单位进行编码，并且如果扩展宏块标记是0，则可以指示按照16x16像素块为单位进行编码。如果扩展宏块标记是0，则扩展宏块可以被划分为四个16x16像素块，使得可以按照16x16像素块为单位确定帧内模式或帧间模式。此外，各个16x16像素块可以在编码之前被划分为较小的子块。If the extended macroblock is a 32x32 pixel block, an extended macroblock flag (extended_mb_flag) may be used to indicate whether to encode in units of 32x32 pixel blocks or in units of 16x16 pixel blocks. For example, if the extended macroblock flag is 1, it may indicate that encoding is performed in units of 32x32 pixel blocks, and if the extended macroblock flag is 0, it may indicate that encoding is performed in units of 16x16 pixel blocks. If the extended macroblock flag is 0, the extended macroblock may be divided into four 16x16 pixel blocks, so that the intra mode or the inter mode may be determined in units of 16x16 pixel blocks. Furthermore, each 16x16 pixel block can be divided into smaller sub-blocks prior to encoding.
再次参照图1，预测器110的帧间预测器114的运动估计器116检测与作为当前图片中的当前编码目标的当前块最类似的块，即，基准图片中的基准块，并且输出代表基准块相对于当前块的相对位置的运动向量(MV)。该处理被称为运动估计。运动估计通过将宏块中的子块与图片缓冲器180中的一个或更多个基准图片进行比较来生成运动向量。8x8像素块可以使用不同的基准图片，但是8x8像素块中的子块使用相同的基准图片。Referring again to FIG. 1 , the motion estimator 116 of the inter predictor 114 of the predictor 110 detects the block most similar to the current block that is the current encoding target in the current picture, that is, the reference block in the reference picture, and outputs a representative reference block. The motion vector (MV) of the block's relative position with respect to the current block. This process is called motion estimation. Motion estimation generates motion vectors by comparing subblocks in a macroblock with one or more reference pictures in picture buffer 180 . The 8x8 pixel block may use different reference pictures, but the sub-blocks in the 8x8 pixel block use the same reference picture.
运动向量通过编码器140进行编码，并被包括在编码后数据中。编码器140使用当前块的相邻块的运动向量的平均值作为预测运动向量(PMV)。编码器140通过仅对作为预测运动向量与当前块的运动向量之间的差向量的差运动向量(DMV)进行编码来生成运动信息。这里，除了差运动向量以外，编码器140还可以对基准图片索引进行编码。即，运动信息可以包括差运动向量和基准图片索引。编码器140可以通过对运动信息进行编码来生成运动信息数据，并且将运动信息数据包括在编码数据中。The motion vector is encoded by the encoder 140 and included in the encoded data. The encoder 140 uses an average value of motion vectors of neighboring blocks of the current block as a predicted motion vector (PMV). The encoder 140 generates motion information by encoding only a difference motion vector (DMV), which is a difference vector between a predicted motion vector and a motion vector of a current block. Here, the encoder 140 may encode a reference picture index in addition to the difference motion vector. That is, the motion information may include a difference motion vector and a reference picture index. The encoder 140 may generate motion information data by encoding the motion information, and include the motion information data in the encoded data.
图13是用于描述根据本发明的实施方式确定预测运动向量的处理的例示图。FIG. 13 is an illustration diagram for describing a process of determining a predicted motion vector according to an embodiment of the present invention.
图13A例示在当前块和相邻块全部是64x64像素块时用于确定当前块的运动向量和预测运动向量的相邻块的运动向量。如果当前块和相邻块全部具有相同的大小，则相邻块的运动向量的均值可以如同以上方法一样确定为预测运动向量。FIG. 13A illustrates motion vectors of neighboring blocks used to determine a motion vector of a current block and a predicted motion vector when the current block and neighboring blocks are all 64x64 pixel blocks. If the current block and the neighboring blocks all have the same size, the mean value of the motion vectors of the neighboring blocks can be determined as the predicted motion vector as in the above method.
图13B例示在当前块和相邻块具有不同大小时用于确定当前块的运动向量和预测运动向量的相邻块的运动向量。如果当前块和相邻块具有不同大小，则必须选择这些相邻块中的一个，使用它来确定预测运动向量。FIG. 13B illustrates motion vectors of neighboring blocks used to determine a motion vector of a current block and a predicted motion vector when the current block and the neighboring blocks have different sizes. If the current block and neighboring blocks are of different sizes, one of these neighboring blocks must be selected and used to determine the predicted motion vector.
在本发明的实施方式中，如图13B所示，如果位于当前块的左侧的相邻块被划分为多个块，则它们当中的位于最上位置和最右位置的块A的运动向量mvA被用于估计。如果位于当前块的上侧的相邻块被划分为多个块，则它们当中的位于最下位置和最左位置的块B的运动向量mvB被用于估计。如果位于当前块的右上侧的相邻块被划分为多个块，则它们当中的位于最下位置和最左位置的块C的运动向量mvC被用于估计。然而，作为例外情况，如果块B和块C都位于图片的边界外部，则由于不能利用块B和块C确定预测运动向量，所以仅利用块A确定预测运动向量。此外，如果块A、B和C中的仅一个具有与当前块相同的基准图片编号，则相关块的运动向量被用于确定预测运动向量。In the embodiment of the present invention, as shown in FIG. 13B , if the adjacent block located on the left side of the current block is divided into multiple blocks, the motion vector mvA of the block A located at the uppermost position and the rightmost position among them is used for estimation. If the adjacent blocks located on the upper side of the current block are divided into a plurality of blocks, the motion vector mvB of the block B located at the lowermost and leftmost positions among them is used for estimation. If the adjacent blocks located on the upper right side of the current block are divided into a plurality of blocks, the motion vector mvC of the block C located at the lowermost and leftmost positions among them is used for estimation. However, as an exception, if both blocks B and C are located outside the boundary of the picture, since the predicted motion vector cannot be determined using the blocks B and C, only the predicted motion vector is determined using the block A. Also, if only one of the blocks A, B, and C has the same reference picture number as the current block, the motion vector of the relevant block is used to determine the prediction motion vector.
图14是用于描述根据本发明的实施方式确定8x16像素块和16x8像素块的预测运动向量的处理的例示图。FIG. 14 is an illustration diagram for describing a process of determining a prediction motion vector of an 8x16 pixel block and a 16x8 pixel block according to an embodiment of the present invention.
此外，如果当前块是8x16像素块或16x8像素块，则不使用平均值。即，如果当前块是8x16像素块，则作为左侧相邻块的块A和作为右侧相邻块的块C被用来确定预测运动向量。如果当前块是16x8像素块，则作为上侧相邻块的块B和作为下侧相邻块的块A被用来确定预测运动向量。Also, if the current block is an 8x16 pixel block or a 16x8 pixel block, the average value is not used. That is, if the current block is an 8x16 pixel block, block A as a left neighboring block and block C as a right neighboring block are used to determine a predicted motion vector. If the current block is a 16x8 pixel block, block B as the upper neighboring block and block A as the lower neighboring block are used to determine the prediction motion vector.
此外，在本发明的实施方式中，针对作为16x16或更多像素块的扩展宏块使用跳过模式。如果作为16x16或更多像素块的扩展宏块是SKIP宏块，则预测运动向量被用来进行运动估计和补偿。然而，如果用来确定预测运动向量的一个或更多个运动向量是0，则预测运动向量被确定为0。例如，如果相邻块A、B和C的运动向量分别为mvA(0,10),mvB(1,10)和mvC(2,10)，则当前块的预测运动向量是(0,10)。Furthermore, in embodiments of the present invention, skip mode is used for extended macroblocks that are 16x16 or more pixel blocks. If the extended macroblock, which is a 16x16 or more pixel block, is a SKIP macroblock, the predicted motion vector is used for motion estimation and compensation. However, if one or more motion vectors used to determine the predicted motion vector are 0, the predicted motion vector is determined to be 0. For example, if the motion vectors of neighboring blocks A, B and C are mvA(0,10), mvB(1,10) and mvC(2,10) respectively, then the predicted motion vector of the current block is (0,10) .
假设作为32x32像素块的扩展宏块位于整个被填充为16像素的倍数的图像和如图2所示填充的区域上，并且扩展宏块是跳过模式。在该情况下，如图3所示，实际图像块(16x16像素块)的左侧块和上侧块用于获得预测运动向量，并且预测运动向量用于对实际图像块进行运动预测和补偿。如果作为当前编码目标的扩展宏块的最优编码模式是SKIP模式，则skip32_flag被编码，并且下一个扩展宏块被编码。It is assumed that an extended macroblock which is a 32x32 pixel block is located on the entire image padded to a multiple of 16 pixels and an area padded as shown in FIG. 2 , and the extended macroblock is a skip mode. In this case, as shown in FIG. 3 , the left and upper blocks of the actual image block (16x16 pixel block) are used to obtain the predicted motion vector, and the predicted motion vector is used for motion prediction and compensation of the actual image block. If the optimal encoding mode of the extended macroblock that is the current encoding target is the SKIP mode, skip32_flag is encoded, and the next extended macroblock is encoded.
再次参照图1，减法器120通过从当前块中减去由在运动估计器116中估计的当前块运动向量指示的基准块来生成残留块。变换器/量化器130对由减法器120生成的残留块进行变换和量化。编码器140通过对量化变换系数进行熵编码来生成编码数据。这里，变换器/量化器130根据当前块的大小进行一个或更多个变换和量化，选择最高效率变换类型，并且根据所选择的变换类型生成量化变换系数。编码器140通过生成用于识别针对各个块选择的变换的类型的变换类型并对其进行编码来生成变换类型数据。变换类型数据可以被包括在编码数据中。然而，不同的变换不能用于16x16像素块。Referring again to FIG. 1 , the subtractor 120 generates a residual block by subtracting the reference block indicated by the motion vector of the current block estimated in the motion estimator 116 from the current block. The transformer/quantizer 130 transforms and quantizes the residual block generated by the subtractor 120 . The encoder 140 generates encoded data by entropy encoding the quantized transform coefficients. Here, the transformer/quantizer 130 performs one or more transform and quantization according to the size of the current block, selects the most efficient transform type, and generates quantized transform coefficients according to the selected transform type. The encoder 140 generates transform type data by generating a transform type for identifying a type of transform selected for each block and encoding it. Transform type data may be included in encoded data. However, different transforms cannot be used for 16x16 pixel blocks.
如同帧内预测快中描述的变换方法，如果16x16像素子块被划分为8x8像素子块，并且如果四个8x8像素子块中的一个或更多个被划分为小于8x8像素块的子块，则对16x16像素子块的残留块仅进行4x4变换。在该情况下，不对变换类型进行编码。如果16x16像素子块被划分为四个8x8像素子块，则对16x16像素子块的残留块进行4x4变换和8x8变换，以选择要求低编码成本的变换类型，并且对指示选择的变换的类型的变换类型(其可以通过例如1比特来表示)进行编码。As in the transformation method described in the intra prediction section, if a 16x16 pixel sub-block is divided into 8x8 pixel sub-blocks, and if one or more of the four 8x8 pixel sub-blocks is divided into sub-blocks smaller than the 8x8 pixel block, Then only 4x4 transformation is performed on the residual block of the 16x16 pixel sub-block. In this case, the transform type is not encoded. If a 16x16 pixel sub-block is divided into four 8x8 pixel sub-blocks, a 4x4 transform and an 8x8 transform are performed on the residual block of the 16x16 pixel sub-block to select a transform type requiring low coding cost, and to indicate the type of the selected transform The transform type (which can be represented by eg 1 bit) is coded.
如果16x16像素子块被划分为两个8x16像素子块，则对16x16像素子块的残留块进行4x4变换、8x8变换和8x16变换，以选择要求最低编码成本的变换类型，并且如果16x16像素子块被划分为两个16x8像素子块，则对16x16像素子块的残留块进行4x4变换、8x8变换和16x8变换，以选择要求最低编码成本的变换类型，并且对指示选择的变换的类型的变换类型(其可以通过例如1比特或2比特来表示)进行编码。对16x16或更多像素子块进行4x4变换、8x8变换和16x16变换，以选择要求最低编码成本的变换类型，并且对指示选择的变换的类型的变换类型(其可以通过例如1比特或2比特来表示)进行编码。If the 16x16 pixel sub-block is divided into two 8x16 pixel sub-blocks, perform 4x4 transform, 8x8 transform, and 8x16 transform on the residual block of the 16x16 pixel sub-block to select the transform type that requires the lowest coding cost, and if the 16x16 pixel sub-block is divided into two 16x8 pixel sub-blocks, then 4x4 transformation, 8x8 transformation and 16x8 transformation are performed on the residual block of the 16x16 pixel sub-block to select the transformation type that requires the lowest coding cost, and the transformation type indicating the type of transformation selected (which can be represented by, for example, 1 bit or 2 bits) is encoded. 4x4 transform, 8x8 transform, and 16x16 transform are performed on 16x16 or more pixel sub-blocks to select the transform type that requires the lowest coding cost, and the transform type indicating the type of the selected transform (which can be identified by, for example, 1 bit or 2 bits) representation) to encode.
如果如同16x16像素块被划分为四个8x8像素块一样存在两个可选择的变换类型，则可以如表1所示构造用来进行变换类型编码的编码比特。If there are two selectable transform types as if a 16x16 pixel block is divided into four 8x8 pixel blocks, the coding bits used for transform type coding can be constructed as shown in Table 1.
[表1][Table 1]
此外，如果16x16像素块被划分为两个8x16像素块或者两个16x8像素块，或者如果如同16x16或更多像素子块的情况一样存在三个可选择的变换类型，则可以如表2所示构造用来进行变换类型编码的编码比特。Furthermore, if a 16x16 pixel block is divided into two 8x16 pixel blocks or two 16x8 pixel blocks, or if there are three selectable transform types as in the case of 16x16 or more pixel sub-blocks, then it can be as shown in Table 2 Constructs coded bits for transform-type coding.
[表2][Table 2]
图15是例示根据本发明的实施方式的根据子块大小的可用变换类型和变换类型编码单元的例示图。FIG. 15 is an illustration diagram illustrating available transformation types and transformation type coding units according to subblock sizes according to an embodiment of the present invention.
图15例示了扩展宏块具有64x64像素并被划分为各种大小的子块的情况。如上所述，由于第一16x16像素块1510被划分为小于8x8像素块的4x4像素子块，所以可用变换仅为4x4变换，并且不对变换类型进行编码。由于第二16x16像素块1520被划分为四个8x8像素子块，所以从4x4变换和8x8变换当中选择需要较低编码成本的一个变换，并且指示所选择的变换的类型的变换类型被编码。由于第三16x16像素块1530被划分为两个8x16像素子块，所以从4x4变换、8x8变换和8x16变换当中选择需要最低编码成本的一个变换，并且指示所选择的变换的类型的变换类型被编码。由于第四16x16像素块1540被划分为两个16x8像素子块，所以从4x4变换、8x8变换和16x8变换当中选择需要最低编码成本的一个变换，并且指示所选择的变换的类型的变换类型被编码。由于第五16x32像素块1550、第六16x32像素块1560、第七32x32像素块1570和第八32x32像素块1580全部大于16x16像素块，所以从4x4变换、8x8变换和16x16变换当中选择需要最低编码成本的一个变换，并且指示所选择的变换的类型的变换类型被编码。针对大于16x16像素的块，可以按照16x16像素块为单位选择变换类型。在该情况下，针对包括在32x32像素块或大于16x16像素块的像素块中并具有非零编码块图案的16x16像素块，对多个变换类型进行编码。FIG. 15 illustrates a case where an extended macroblock has 64x64 pixels and is divided into subblocks of various sizes. As mentioned above, since the first 16x16 pixel block 1510 is divided into 4x4 pixel sub-blocks smaller than the 8x8 pixel block, the available transforms are only 4x4 transforms, and the transform type is not encoded. Since the second 16x16 pixel block 1520 is divided into four 8x8 pixel sub-blocks, one transform requiring a lower encoding cost is selected from among the 4x4 transform and the 8x8 transform, and a transform type indicating the type of the selected transform is encoded. Since the third 16x16 pixel block 1530 is divided into two 8x16 pixel sub-blocks, a transform requiring the lowest encoding cost is selected from among 4x4 transform, 8x8 transform, and 8x16 transform, and a transform type indicating the type of the selected transform is encoded . Since the fourth 16x16 pixel block 1540 is divided into two 16x8 pixel sub-blocks, a transform requiring the lowest encoding cost is selected from among the 4x4 transform, the 8x8 transform, and the 16x8 transform, and the transform type indicating the type of the selected transform is encoded . Since the fifth 16x32 pixel block 1550, the sixth 16x32 pixel block 1560, the seventh 32x32 pixel block 1570 and the eighth 32x32 pixel block 1580 are all larger than a 16x16 pixel block, choosing among the 4x4 transform, 8x8 transform and 16x16 transform requires the lowest coding cost A transform of , and a transform type indicating the type of the selected transform is encoded. For blocks larger than 16x16 pixels, the transformation type may be selected in units of 16x16 pixel blocks. In this case, multiple transform types are coded for a 16x16 pixel block included in a 32x32 pixel block or a pixel block larger than a 16x16 pixel block and having a non-zero coded block pattern.
再次参照图1，由变换器/量化器130变换/量化的残留块被逆变换器/逆量化器150逆变换/逆量化，以重构残留块。加法器160通过将所重构的残留块与由预测器110预测的预测块相加来重构当前块。Referring again to FIG. 1 , the residual block transformed/quantized by the transformer/quantizer 130 is inversely transformed/inverse quantized by the inverse transformer/inverse quantizer 150 to reconstruct the residual block. The adder 160 reconstructs the current block by adding the reconstructed residual block to the prediction block predicted by the predictor 110 .
滤波器170利用解块滤波器对通过积累由加法器160重构的当前块生成的图片进行滤波。经滤波的图片被存储在图片缓冲器180中，并被用作针对下一图片的基准图片。The filter 170 filters a picture generated by accumulating the current block reconstructed by the adder 160 with a deblocking filter. The filtered picture is stored in the picture buffer 180 and used as a reference picture for the next picture.
图16是例示根据本发明的实施方式要解块滤波的块的边界的例示图。FIG. 16 is an illustration diagram illustrating boundaries of blocks to be deblocking filtered according to an embodiment of the present invention.
图16A至图16E分别例示使用4x4变换、8x8变换、8x16变换、16x8变换和16x16变换的16x16像素块中要进行解块滤波的块的边界。图16A所示的使用4x4变换的16x16像素块在各个4x4像素块的左侧块和上侧块边界处进行解块滤波。图16B所示的使用8x8变换的16x16像素块在各个8x8像素块的左侧块和上侧块边界处进行解块滤波。图16C所示的使用8x16变换的16x16像素块在各个8x16像素块的左侧块和上侧块边界处进行解块滤波。图16D所示的使用16x8变换的16x16像素块在各个16x8像素块的左侧块和上侧块边界处进行解块滤波。图16E所示的使用16x16变换的16x16像素块在16x16像素块的左侧块和上侧块边界处进行解块滤波。16A to 16E respectively illustrate boundaries of blocks to be subjected to deblocking filtering among 16x16 pixel blocks using 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform. The 16x16 pixel blocks using 4x4 transform shown in FIG. 16A undergo deblocking filtering at the left block and upper block boundaries of each 4x4 pixel block. The 16x16 pixel blocks using 8x8 transform shown in FIG. 16B undergo deblocking filtering at the left block and upper block boundaries of each 8x8 pixel block. The 16x16 pixel blocks using the 8x16 transform shown in FIG. 16C undergo deblocking filtering at the left block and upper block boundaries of each 8x16 pixel block. The 16x16 pixel blocks using the 16x8 transform shown in FIG. 16D are subjected to deblocking filtering at the left block and upper block boundaries of each 16x8 pixel block. The 16x16 pixel block using the 16x16 transform shown in FIG. 16E performs deblocking filtering at the left and upper block boundaries of the 16x16 pixel block.
图17是例示根据本发明的实施方式的帧内图片的句法的例示图。FIG. 17 is an illustration diagram illustrating syntax of an intra picture according to an embodiment of the present invention.
如果扩展宏块的子块的类型包括如图4所示的矩形类型(例如，32x16像素块和16x32像素块)，则视频编码设备100应当将子块类型信息发送到视频解码设备。因此，帧内图片的编码数据可以包括诸如分割类型、帧内预测模式、CBPX标记/CBP(编码块图案X-flag/编码块图案)和变换系数的字段。视频编码设备100的编码器140通过对分割类型信息、帧内预测模式信息、CBPX标记/CBP信息和变换系数信息进行编码并将其分配给各个字段来生成编码后数据。If the type of the subblock of the extended macroblock includes a rectangular type (eg, 32x16 pixel block and 16x32 pixel block) as shown in FIG. 4 , the video encoding apparatus 100 should transmit the subblock type information to the video decoding apparatus. Accordingly, encoded data of an intra picture may include fields such as partition type, intra prediction mode, CBPX flag/CBP (Coding Block Pattern X-flag/Coding Block Pattern), and transform coefficients. The encoder 140 of the video encoding apparatus 100 generates encoded data by encoding partition type information, intra prediction mode information, CBPX flag/CBP information, and transformation coefficient information and assigning them to respective fields.
然而，图17所示的针对帧内图片的编码数据的句法仅是示例性的，并且不必构造为如图17所示。即，应当注意的是，针对帧内图片的编码数据的全部或者一部分句法的序列可以与图17所示不同地改变，并且可以省略一部分句法。However, the syntax of encoded data for an intra picture shown in FIG. 17 is only exemplary, and does not have to be configured as shown in FIG. 17 . That is, it should be noted that the sequence of all or part of the syntax for encoded data of an intra picture may be changed differently from that shown in FIG. 17 , and a part of the syntax may be omitted.
另一方面，如果扩展宏块被划分为如图5所示的正方形类型的像素块，接着按照16x16像素块为单位进行帧内预测编码，则可以省略CBPX标记的编码，并且可以对量化参数的递增/递减信息(Δpq)进行编码。这里，可以按照扩展宏块为单位对量化参数的递增/递减信息进行编码，并且可以按照16x16像素块为单位对CBP信息进行编码。On the other hand, if the extended macroblock is divided into square-type pixel blocks as shown in Figure 5, and then intra-frame prediction coding is performed in units of 16x16 pixel blocks, the coding of the CBPX flag can be omitted, and the quantization parameter Increment/decrement information (Δpq) is encoded. Here, the increment/decrement information of the quantization parameter may be encoded in units of extended macroblocks, and the CBP information may be encoded in units of 16x16 pixel blocks.
在图17中，分割类型字段被分配有作为扩展宏块的分割类型信息的编码后数据的分割类型数据，其表示宏块的子块的类型。分割类型信息可以利用图18所示的各个分割类型的编号来生成。In FIG. 17 , the division type field is assigned division type data which is coded data of division type information of an extended macroblock, which indicates the type of a subblock of the macroblock. The division type information can be generated using the number of each division type shown in FIG. 18 .
参照图18，如果宏块层K的N/2K x N/2K像素块没有被再次划分，则分割类型编号0被分配给宏块层K的N/2K x N/2K像素块。如果宏块层K的N/2K x N/2K像素块被划分为两个N/2K x N/2K+1像素块，则分割类型编号1被分配给宏块层K的N/2K x N/2K像素块。如果宏块层K的N/2K x N/2K像素块被划分为两个N/2K+1x N/2K像素块，则分割类型编号2被分配给宏块层K的N/2K x N/2K像素块。如果宏块层K的N/2K x N/2K像素块被划分为四个N/2K+1x N/2K+1像素块，则分割类型编号3被分配给宏块层K的N/2K x N/2K像素块。标记在宏块层K的N/2Kx N/2K像素块中的各个子块上的编号0、1、2和3是用于识别各个子块的分割编号。Referring to FIG. 18, if the N/2K x N/2K pixel block of the macroblock layer K is not subdivided, the partition type number 0 is assigned to the N/2K x N/2K pixel block of the macroblock layer K . If an N/2K x N/2K pixel block of macroblock layer K is divided into two N/2K x N/2K+1 pixel blocks, partition type number 1 is assigned to N of macroblock layer K /2K x N/2K pixel blocks. If an N/2K x N/2K pixel block of macroblock layer K is divided into two N/2K+1 x N/2K pixel blocks, partition type number 2 is assigned to N of macroblock layer K /2K x N/2K pixel blocks. If an N/2K x N/2K pixel block of macroblock layer K is divided into four N/2K+1 x N/2K+1 pixel blocks, partition type number 3 is assigned to macroblock layer K N/2K x N/2K pixel blocks. The numbers 0, 1, 2, and 3 marked on the respective sub-blocks in the N/2K x N/2K pixel block of the macroblock layer K are division numbers for identifying the respective sub-blocks.
尽管图18例示了按照编号的升序分配分割类型编号，但是图18中的分割类型编号仅是示例性的，分割类型编号可以按照编号的降序分配。此外，各个分割类型编号可以根据各个分割类型的出现概率来分配，并且分割类型编号可以自适应地改变。此外，各个划分块可以被划分为较小的块。例如，如果32x32像素块按照分割类型编号3被划分为四个16x16像素块，则各个16x16像素块可以利用图18的分割类型编号划分为较小的子块。此外，如果16x16像素子块被划分为较小的块，则可以使用上级像素块的分割类型编号(即，与32x32像素块的分割类型编号不同的分割类型编号)。如果扩展宏块在编码之前按照16x16像素块为单位被划分，则可以免除对大于16x16像素块的子块(例如，64x32像素块、32x64像素块、16x32像素块或32x16像素块)的分割类型编号的编码。Although FIG. 18 illustrates that division type numbers are assigned in ascending order of numbers, the division type numbers in FIG. 18 are merely exemplary, and division type numbers may be assigned in descending order of numbers. In addition, each division type number can be assigned according to the occurrence probability of each division type, and the division type number can be adaptively changed. Also, each divided block may be divided into smaller blocks. For example, if a 32x32 pixel block is divided into four 16x16 pixel blocks according to the division type number 3, each 16x16 pixel block can be divided into smaller sub-blocks using the division type number of FIG. 18 . Furthermore, if the 16x16 pixel sub-block is divided into smaller blocks, the division type number of the superior pixel block (ie, a division type number different from that of the 32x32 pixel block) may be used. If the extended macroblock is divided in units of 16x16 pixel blocks before encoding, the partition type number for subblocks larger than 16x16 pixel blocks (for example, 64x32 pixel blocks, 32x64 pixel blocks, 16x32 pixel blocks, or 32x16 pixel blocks) can be exempted encoding.
图19是用于描述根据本发明的实施方式将扩展宏块划分为各种大小的子块的情况下的分割类型编号编码序列的图。FIG. 19 is a diagram for describing a partition type number encoding sequence in a case where an extended macroblock is divided into subblocks of various sizes according to an embodiment of the present invention.
图19例示了扩展宏块是64x64像素块并且分割层的最大编号是4的情况。根据本发明的实施方式，当扩展宏块被划分为如图19所示时，各个分割层的分割类型编号可以根据分割类型编号编码序列来顺序地编码，以对宏块的分割类型信息进行编码。FIG. 19 exemplifies a case where an extended macroblock is a 64x64 pixel block and the maximum number of division layers is four. According to an embodiment of the present invention, when an extended macroblock is divided as shown in FIG. 19 , the partition type numbers of each partition layer can be sequentially coded according to the partition type code sequence to encode the partition type information of the macroblock .
图20是顺序地例示如图19所示的基于逐个分隔层划分宏块的处理的例示图。参照图20，在宏块层0(L0)中，64x64像素子块被划分为四个32x32像素块。在层1(L1)中，L1-P0(宏块层1的分割编号0)子块和L1-P3(宏块层1的分割编号3)子块被再次划分为四个16x16像素子块，L1-P1(宏块层1的分割编号1)子块和L1-P2(宏块层1的分割编号2)子块分别被再次划分为16x32像素子块和32x16像素子块。由于L1-P1和L1-P2子块在被划分为两个子块之后没有被再次划分，所以不例示逐层的分割编号。在层2(L2)中，L2-P0(宏块层2的分割编号0)子块被再次划分为四个8x8像素子块，L2-P3(宏块层2的分割编号3)子块被再次划分为两个16x8像素子块。在宏块层3(L3)中，L3-P0(宏块层3的分割编号0)子块和L3-P1(宏块层3的分割编号1)子块分别被再次划分为四个4x4像素子块。这里，如果扩展宏块按照16x16像素块为单位进行编码，则取代被分别划分为16x32像素子块和32x16像素子块，如同L1-P3，L1-P1块和L1-P2块按照四个16x16像素块为单位进行编码。FIG. 20 is an illustration diagram sequentially illustrating the process of dividing a macroblock on a separation layer-by-separation layer basis as shown in FIG. 19 . Referring to FIG. 20, in the macroblock layer 0 (L0), a 64x64 pixel sub-block is divided into four 32x32 pixel blocks. In layer 1 (L1), the subblocks L1-P0 (partition number 0 of macroblock layer 1) and L1-P3 (partition number 3 of macroblock layer 1) are subdivided into four 16x16 pixel subblocks again, The L1-P1 (division number 1 of the macroblock layer 1) sub-block and the L1-P2 (division number 2 of the macroblock layer 1) sub-block are subdivided into 16x32-pixel sub-blocks and 32x16-pixel sub-blocks, respectively. Since the L1-P1 and L1-P2 sub-blocks are not divided again after being divided into two sub-blocks, the layer-by-layer division numbers are not exemplified. In layer 2 (L2), the L2-P0 (partition number 0 of macroblock layer 2) subblock is subdivided into four 8x8 pixel subblocks again, and the L2-P3 (partition number 3 of macroblock layer 2) subblock is divided into Divided again into two 16x8 pixel sub-blocks. In macroblock layer 3 (L3), the subblocks L3-P0 (partition number 0 of macroblock layer 3) and L3-P1 (partition number 1 of macroblock layer 3) are divided again into four 4x4 pixels subblock. Here, if the extended macroblock is coded in units of 16x16 pixel blocks, it is instead divided into 16x32 pixel sub-blocks and 32x16 pixel sub-blocks, just like L1-P3, L1-P1 blocks and L1-P2 blocks are divided into four 16x16 pixel blocks Encoding is performed in units of blocks.
图21是用于描述顺序地对宏块的逐层分割类型编号进行编码的处理的例示图。FIG. 21 is an illustration diagram for describing a process of sequentially encoding layer-by-layer partition type numbers of macroblocks.
当图19所示的宏块的分割类型信息被编码时，可以如图21所示进行编码。在图19中，正方形“□”中的数字表示各个子块的分割类型编号编码的顺序。当扩展宏块的逐层分割类型编号根据分割类型编号编码序列顺序地编码时，逐层分割类型编号可以根据图21所示的序列顺序地编码。When the division type information of the macroblock shown in FIG. 19 is coded, it can be coded as shown in FIG. 21 . In FIG. 19 , the numbers in the squares "□" indicate the order in which the division type numbers of the respective sub-blocks are coded. When the layer-by-layer partition type numbers of extended macroblocks are sequentially encoded according to the partition type number coding sequence, the layer-by-layer partition type numbers may be sequentially coded according to the sequence shown in FIG. 21 .
首先，由于宏块层0的64x64像素子块L0-P0被划分为四个32x32像素子块，所以分割类型编号3被编码。由于64x64像素子块中的四个32x32像素子块当中的第一32x32像素子块L1-P0也被划分为四个16x16像素子块，所以分割类型编号3被编码。由于层1的第一32x32像素子块中的四个16x16像素子块当中的第一16x16像素子块L2-P0也被划分为四个8x8像素子块，所以分割类型编号3被编码。由于相关像素子块L2-P0中的四个8x8像素块L3-P0,L3-P1,L3-P2和L3-P3没有被再次划分为较小的子块，所以分割类型编号{3,3,0,0}分别被编码。由于层3的子块不能被划分为较小的子块，所以层3的子块的分割类型编号没有被编码。First, since the 64x64 pixel sub-block L0-P0 of macroblock layer 0 is divided into four 32x32 pixel sub-blocks, partition type number 3 is coded. Since the first 32x32 pixel sub-block L1-P0 among the four 32x32 pixel sub-blocks in the 64x64 pixel sub-block is also divided into four 16x16 pixel sub-blocks, division type number 3 is coded. Since the first 16x16 pixel sub-block L2-P0 among the four 16x16 pixel sub-blocks in the first 32x32 pixel sub-block of layer 1 is also divided into four 8x8 pixel sub-blocks, partition type number 3 is coded. Since the four 8x8 pixel blocks L3-P0, L3-P1, L3-P2 and L3-P3 in the associated pixel sub-block L2-P0 are not subdivided into smaller sub-blocks again, the partition type numbers {3,3, 0,0} are encoded respectively. Since a sub-block of layer 3 cannot be divided into smaller sub-blocks, the partition type number of the sub-block of layer 3 is not coded.
由于宏块层3中的子块的全部分割类型编号都已经被编码，所以宏块层2的第二16x16像素子块L2-P1和第三16x16像素子块L2-P2的分割类型编号被编码。这里，由于它们没有被再次划分为较小的块，所以分割类型编号0被编码。由于第四16x16像素子块L2-P3被划分为16x8像素子块，但是分割类型编号不是3，所以仅分割类型编号1被编码。由于层2中的四个子块的所有分割类型编号都已经被编码，所以层1的第二32x32像素子块L1-P1的分割类型编号被编码。这里，由于层1的第二32x32像素子块L1-P1已经被划分为16x32像素子块，并且各个划分子块没有再被划分为较小的块，所以分割类型编号2被编码。这样，当对宏块层1的第三32x32像素子块L1-P2进行分割类型编号的编码时，宏块层1的第四32x32像素子块L1-P3和它们的下属的四个16x16像素子块L2-P0,L2-P1,L2-P2和L2-P3(顺序地，为{1,3,0,0,0,0})被编码。Since all partition type numbers of sub-blocks in macroblock layer 3 have already been coded, the partition type numbers of the second 16x16 pixel sub-block L2-P1 and the third 16x16 pixel sub-block L2-P2 of macroblock layer 2 are coded . Here, partition type number 0 is coded since they are not subdivided into smaller blocks again. Since the fourth 16x16 pixel sub-block L2-P3 is divided into 16x8 pixel sub-blocks, but the partition type number is not 3, only partition type number 1 is coded. Since all partition type numbers of the four sub-blocks in layer 2 have already been coded, the partition type numbers of the second 32x32 pixel sub-block L1-P1 of layer 1 are coded. Here, since the second 32x32 pixel sub-block L1-P1 of layer 1 has been divided into 16x32 pixel sub-blocks, and each divided sub-block is not further divided into smaller blocks, division type number 2 is coded. Thus, when the third 32x32 pixel sub-block L1-P2 of the macroblock layer 1 is encoded with the partition type number, the fourth 32x32 pixel sub-block L1-P3 of the macroblock layer 1 and their subordinate four 16x16 pixel sub-blocks Blocks L2-P0, L2-P1, L2-P2 and L2-P3 (in order, {1,3,0,0,0,0}) are encoded.
当图19所示的宏块的分割类型信息根据上述编码方法被编码时，分割类型编号{3,3,3,3,3,0,0,0,0,1,2,1,3,0,0,0,0}如图21所示进行编码。这里，可以利用诸如二进制算术编码或哈夫曼编码的无损压缩编码将分割类型编号编码为二进制比特串。When the division type information of the macroblock shown in FIG. 0,0,0,0} are encoded as shown in Figure 21. Here, the division type number can be encoded as a binary bit string using lossless compression coding such as binary arithmetic coding or Huffman coding.
例如，如果使用二进制算术编码，则各个分割类型编号可以根据当前要编码的分割类型编号的层编号来使用不同的二进制值。如果层编号小于或等于log2(n/16)，则表3可以用于编码，并且如果层编号大于log2(n/16)，则表4可以用于编码。例如，参照图3，由于图11的子块L1-P0的分割类型编号可以通过二进制数字“01”来表示，所以可以通过对二进制数字“0”和“1”进行算术编码来对分割类型编号3进行编码。参照图4，由于属于子块L2-P0的子块L3-P2的分割类型编号可以通过二进制数字“1”来表示，所以可以通过对二进制数字“1”进行算术编码来对分割类型编号3进行编码。For example, if binary arithmetic coding is used, each division type number may use a different binary value according to the layer number of the division type number currently to be encoded. Table 3 may be used for encoding if the layer number is less than or equal to log2 (n/16), and Table 4 may be used for encoding if the layer number is greater than log2 (n/16). For example, referring to FIG. 3, since the division type number of the sub-block L1-P0 of FIG. 11 can be represented by the binary number "01", the division type number can be expressed by arithmetically encoding the binary numbers "0" and "1". 3 to encode. Referring to FIG. 4, since the division type number of the sub-block L3-P2 belonging to the sub-block L2-P0 can be represented by the binary number "1", the division type number 3 can be arithmetically encoded by the binary number "1". coding.
[表3][table 3]
[表4][Table 4]
再次参照图17，当帧内图片的宏块的分割类型信息被编码并且分割类型数据被分配给分割类型字段时，通过对与帧内预测模式有关的信息进行编码而获得的帧内预测模式数据被分配给帧内预测模式字段。Referring again to FIG. 17 , when the division type information of the macroblock of the intra picture is encoded and the division type data is assigned to the division type field, the intra prediction mode data obtained by encoding the information on the intra prediction mode is assigned to the intra prediction mode field.
图22是例示根据本发明的实施方式的帧内预测模式的编码序列的例示图。FIG. 22 is an illustration diagram illustrating an encoding sequence of an intra prediction mode according to an embodiment of the present invention.
类似于图19，图22例示了扩展宏块是64x64像素块并且分割层的最大编号是4的情况。当扩展宏块被划分为如图22所示时，扩展宏块的各个子块的帧内预测模式可以根据图22所示的帧内预测模式的编码序列顺序地编码，以生成帧内预测模式数据。即，各个子块中标记的数字表示用于进行帧内预测模式编码的帧内预测模式的编码序列。如果64x64像素扩展宏块按照16x16像素块为单位进行编码，则与第十五块和第十六块或者第十七块和第十八块对应的32x32像素块可以在编码之前按照与第十九到第二十二块类似的四个像素块为单位进行划分，而不是被划分为16x32像素块或32x16像素块。Similar to FIG. 19 , FIG. 22 exemplifies the case where the extended macroblock is a 64x64 pixel block and the maximum number of division layers is 4. Referring to FIG. When the extended macroblock is divided as shown in Figure 22, the intra prediction modes of the respective sub-blocks of the extended macroblock can be sequentially coded according to the encoding sequence of the intra prediction modes shown in Figure 22 to generate the intra prediction mode data. That is, the numbers marked in the respective sub-blocks represent the encoding sequence of the intra prediction mode used for encoding the intra prediction mode. If the 64x64 pixel extended macroblock is coded in units of 16x16 pixel blocks, the 32x32 pixel blocks corresponding to the fifteenth block and the sixteenth block or the seventeenth block and the eighteenth block can be coded in accordance with the nineteenth pixel block before encoding Four pixel blocks similar to the twenty-second block are divided into units instead of being divided into 16x32 pixel blocks or 32x16 pixel blocks.
如果扩展宏块的亮度分量的子块不是16x16像素块，则指示从相邻块估计的当前块的预测方向与选择的当前块的预测方向是否相同的预测模式标记被编码。如果预测模式标记指示作为当前编码目标的块的预测方向与从相邻块估计的当前块的预测方向不相同，则指示当前块的预测方向与从相邻块估计的当前块的预测方向不同的预测模式标记以及指示根据相关子块的大小的多个帧内预测方向当中除了从相邻块估计的当前块的预测方向以外的剩余预测方向中的哪一个对应于预测方向的预测模式的预测模式编号被编码为相关块的帧内预测模式，以生成帧内预测模式数据。这里，如果扩展宏块被划分为除了4x4像素块、8x8像素块或16x16像素块以外的子块以执行帧内预测，并且如果利用当前块的左侧块和上侧块估计的当前块的预测模式编号大于或等于3，则当前块的预测方向可以使用图9的预测模式编号2(DC模式)。If the subblock of the luminance component of the extended macroblock is not a 16x16 pixel block, a prediction mode flag indicating whether the prediction direction of the current block estimated from the neighboring block is the same as the prediction direction of the selected current block is coded. If the prediction mode flag indicates that the prediction direction of the block that is the current encoding target is different from the prediction direction of the current block estimated from the neighboring block, it indicates that the prediction direction of the current block is different from the prediction direction of the current block estimated from the neighboring block A prediction mode flag and a prediction mode indicating which of the remaining prediction directions other than the prediction direction of the current block estimated from the neighboring block among the plurality of intra prediction directions according to the size of the relevant sub-block corresponds to the prediction mode of the prediction direction The number is encoded as the intra prediction mode of the associated block to generate intra prediction mode data. Here, if the extended macroblock is divided into sub-blocks other than 4x4 pixel blocks, 8x8 pixel blocks, or 16x16 pixel blocks to perform intra prediction, and if the prediction of the current block estimated using the left and upper blocks of the current block If the mode number is greater than or equal to 3, the prediction direction of the current block can use the prediction mode number 2 (DC mode) in FIG. 9 .
如果从相邻块估计的当前块的预测方向与当前块的预测方向相同，则指示从相邻块估计的当前块的预测方向与当前块的预测方向相同的预测模式标记被编码为帧内预测模式，以生成帧内预测模式数据。如果亮度分量的子块是16x16像素块，则色度分量的宏块的预测模式将相关块的预测模式编号本身编码为帧内预测模式，以生成帧内预测模式数据。按照这种方式，通过按照图22所示的序列进行编码生成的宏块的各个子块的帧内预测模式数据被分配给如图17所示的帧内预测模式字段。If the prediction direction of the current block estimated from the neighboring block is the same as the prediction direction of the current block, the prediction mode flag indicating that the prediction direction of the current block estimated from the neighboring block is the same as the prediction direction of the current block is encoded as intra prediction mode to generate intra prediction mode data. If the sub-block of the luma component is a 16x16 pixel block, the prediction mode of the macroblock of the chroma component encodes the prediction mode number itself of the relevant block as an intra prediction mode to generate intra prediction mode data. In this way, the intra prediction mode data of each sub-block of the macroblock generated by encoding in the sequence shown in FIG. 22 is assigned to the intra prediction mode field as shown in FIG. 17 .
图23是用于描述根据本发明的实施方式对宏块的帧内预测模式进行编码的处理的示例的例示图。FIG. 23 is an illustration diagram for describing an example of a process of encoding an intra prediction mode of a macroblock according to an embodiment of the present invention.
图23A例示当前要编码的当前宏块是64x64像素块并被划分为各种大小的子块并且形状如图所示的情况；当前宏块的左侧相邻块是按照64x64像素为单元帧内预测的，因而左侧相邻块的预测模式编号是0(垂直模式)；并且当前宏块的上侧相邻块是按照64x64像素为单位帧内预测的，因而左侧相邻块的预测模式编号是1(水平模式)。当前扩展宏块的各个子块中标记的数字表示各个子块的预测模式编号。Figure 23A exemplifies the situation that the current macroblock to be encoded is a 64x64 pixel block and is divided into sub-blocks of various sizes and the shape is as shown in the figure; Predicted, so the prediction mode number of the left adjacent block is 0 (vertical mode); and the upper side adjacent block of the current macroblock is predicted according to the unit of 64x64 pixels, so the prediction mode of the left adjacent block The number is 1 (horizontal mode). The number marked in each sub-block of the current extended macroblock indicates the prediction mode number of each sub-block.
当前扩展宏块的第一4x4像素子块的预测模式编号是2(DC模式)，并且利用左侧相邻块和上侧相邻块估计的相关子块的预测模式编号是0。即，由于左侧相邻块的预测模式编号0小于上侧相邻块的预测模式编号1，所以第一4x4像素子块的预测模式编号被估计为0。由于第一4x4像素子块的预测模式不同于从相邻块估计的相关子块的预测模式，所以指示相关子块的预测方向与从相邻块估计的相关子块的预测模式不同的预测模式标记以及作为相关子块的预测模式编号的预测模式编号2被编码为帧内预测模式。第二4x4像素子块的预测模式编号是1，从相关块估计的相关子块的预测模式编号是1。因而，由于相关子块的预测方向与从相邻块估计的相关子块的预测方向相同，所以仅指示这些预测方向相同的预测模式标记被编码为相关子块的帧内预测模式。The prediction mode number of the first 4x4 pixel sub-block of the current extended macroblock is 2 (DC mode), and the prediction mode number of the related sub-block estimated using the left neighboring block and the upper side neighboring block is 0. That is, since the prediction mode number 0 of the left neighboring block is smaller than the prediction mode number 1 of the upper neighboring block, the prediction mode number of the first 4x4 pixel sub-block is estimated to be 0. Since the prediction mode of the first 4x4 pixel sub-block is different from the prediction mode of the related sub-block estimated from the neighboring block, the prediction mode indicating that the prediction direction of the related sub-block is different from the prediction mode of the related sub-block estimated from the neighboring block The flag and the prediction mode number 2 which is the prediction mode number of the relevant sub-block are coded as intra prediction modes. The prediction mode number of the second 4x4 pixel sub-block is 1, and the prediction mode number of the related sub-block estimated from the related block is 1. Thus, since the prediction direction of the relevant subblock is the same as that of the relevant subblock estimated from the neighboring block, only the prediction mode flags indicating that these prediction directions are the same are encoded as the intra prediction modes of the relevant subblock.
接着，下一个子块的帧内预测模式根据图22所示的帧内预测模式的编码序列被编码。在该情况下，第三4x4像素子块的预测模式编号是0，并且从相邻块估计的相关子块的预测模式编号是0。因而，由于相关子块的预测方向与从相邻块估计的相关子块的预测方向相同，所以仅指示这些预测方向相同的预测模式标记被编码为相关子块的帧内预测模式。在第十七16x16像素子块的情况下，预测标记不被编码，并且相关子块的预测模式编号1被编码为相关子块的帧内预测模式。图23例示了第十五块和第十六块被划分为16x32像素块并被编码为帧内预测模式；并且第十七块和第十八块被划分为32x16像素块并被编码为帧内预测模式的情况。然而，如果按照16x16像素块为单元对扩展宏块进行帧内预测编码，则针对大于16x16像素块的子块省略诸如第十五块和第十六块或者第十七块和第十八块的矩形像素块的划分。Next, the intra prediction mode of the next sub-block is coded according to the coding sequence of intra prediction modes shown in FIG. 22 . In this case, the prediction mode number of the third 4x4 pixel sub-block is 0, and the prediction mode number of the related sub-block estimated from the neighboring block is 0. Thus, since the prediction direction of the relevant subblock is the same as that of the relevant subblock estimated from the neighboring block, only the prediction mode flags indicating that these prediction directions are the same are encoded as the intra prediction modes of the relevant subblock. In the case of the seventeenth 16x16 pixel sub-block, the prediction flag is not coded, and the prediction mode number 1 of the relevant sub-block is coded as the intra prediction mode of the relevant sub-block. 23 illustrates that the fifteenth block and the sixteenth block are divided into 16x32 pixel blocks and encoded as intra prediction mode; and the seventeenth and eighteenth blocks are divided into 32x16 pixel blocks and encoded as intra prediction mode; The case of the forecast model. However, if the extended macroblock is intra-predictively encoded in units of 16x16 pixel blocks, subblocks such as the fifteenth and sixteenth blocks or the seventeenth and eighteenth blocks are omitted for subblocks larger than the 16x16 pixel block. Division of rectangular pixel blocks.
类似于图23A，图23B例示了在当前宏块的帧内预测模式被编码时被包括在帧内预测模式字段中的帧内预测模式数据。如这里所示，针对第一4x4子块对预测模式标记和预测模式编号进行编码，针对第二、第三和第四4x4子块，仅对预测模式标记进行编码。Similar to FIG. 23A , FIG. 23B illustrates intra prediction mode data included in the intra prediction mode field when the intra prediction mode of the current macroblock is encoded. As shown here, the prediction mode flag and the prediction mode number are coded for the first 4x4 sub-block, and only the prediction mode flag is coded for the second, third and fourth 4x4 sub-blocks.
当扩展宏块是帧内预测编码的时，如果其被划分为大于16x16像素块的矩形子块(例如，16x32和32x16)以进行帧内预测编码，则考虑子块的大小的编码块图案标记(CBPX_flag：编码块图案X_flag，下文称作“CBPX标记”)和编码块图案(CBP)可以被编码并分配给CBPX_flag/CBP字段。分配给CBPX_flag/CBP字段的CBP数据指示非零量化变换系数是否被包括在宏块的子块中。然而，如图5所示，如果扩展宏块按照16x16像素块为单元进行划分，以进行帧内预测编码，则可以省略块图案标记的编码，并且可以对量化参数的的递增/递减信息(Δpq)进行编码。这里，可以按照扩展宏块为单位对量化参数的递增/递减信息进行编码，并且可以按照16x16像素块为单元对CBP数据进行编码。When an extended macroblock is intra-prediction coded, if it is divided into rectangular sub-blocks larger than 16x16 pixel blocks (for example, 16x32 and 32x16) for intra-prediction coding, the coded block pattern flag considering the size of the sub-block (CBPX_flag: coded block pattern X_flag, hereinafter referred to as 'CBPX flag') and coded block pattern (CBP) may be coded and assigned to the CBPX_flag/CBP field. CBP data allocated to the CBPX_flag/CBP field indicates whether non-zero quantized transform coefficients are included in subblocks of a macroblock. However, as shown in Fig. 5, if the extended macroblock is divided into units of 16x16 pixel blocks for intra-frame prediction coding, the coding of the block pattern flag can be omitted, and the increment/decrement information of the quantization parameter (Δpq ) to encode. Here, the increment/decrement information of the quantization parameter may be encoded in units of extended macroblocks, and the CBP data may be encoded in units of 16x16 pixel blocks.
图24是用于描述根据本发明的实施方式针对帧内宏块对CBPX_flag和CBP进行编码的处理的例示图。FIG. 24 is an illustration diagram for describing a process of encoding CBPX_flag and CBP for an intra macroblock according to an embodiment of the present invention.
图24例示了在扩展宏块是64x64像素块的情况下对CBPX_flag和CBP进行编码的处理。视频编码设备100的编码器140确定扩展宏块的子块的大小是否为64x64,64x32和32x64中的一个(S2410)。如果扩展宏块的子块的大小不是64x64,64x32和32x64中的一个，则编码器140确定子块的大小是否是32x32,32x16和16x32中的一个(S2420)。这里，如果扩展宏块按照16x16像素为单元进行划分，以进行帧内预测编码，则编码器140可以在步骤S2410中确定子块的大小是否为64x64像素块，即，是否按照扩展宏块为单位进行帧内预测编码。如果子块的大小不是64x64像素块，则编码器140可以在步骤S2420中确定子块的大小是否为32x32像素块。如果在步骤S2420中确定子块的大小是32x32,32x16和16x32中的一个，或者是32x32像素块，则编码器140确定子块中是否存在非零变换系数(S2430)。如果子块中存在非零变换系数，则编码器140对指示子块中存在非零变换系数的CBP32标记(其可以通过例如诸如“1”的1比特来表示)进行编码(S2440)，并且按照16x16像素块为单位对子块中的CBP进行编码(S2450)。如果子块中不存在非零变换系数，则编码器140对指示子块中不存在非零变换系数的CBP32标记(其可以通过例如诸如“0”的1比特来表示)进行编码(S2460)。FIG. 24 illustrates the process of encoding CBPX_flag and CBP in the case where the extended macroblock is a 64x64 pixel block. The encoder 140 of the video encoding apparatus 100 determines whether the size of the subblock of the extended macroblock is one of 64x64, 64x32, and 32x64 (S2410). If the size of the subblock of the extended macroblock is not one of 64x64, 64x32, and 32x64, the encoder 140 determines whether the size of the subblock is one of 32x32, 32x16, and 16x32 (S2420). Here, if the extended macroblock is divided in units of 16x16 pixels for intra-frame prediction encoding, the encoder 140 may determine in step S2410 whether the size of the sub-block is a 64x64 pixel block, that is, whether the extended macroblock is used as a unit Perform intra-frame predictive coding. If the size of the sub-block is not a 64x64 pixel block, the encoder 140 may determine whether the size of the sub-block is a 32x32 pixel block in step S2420. If it is determined in step S2420 that the size of the sub-block is one of 32x32, 32x16, and 16x32, or a 32x32 pixel block, the encoder 140 determines whether a non-zero transform coefficient exists in the sub-block (S2430). If there is a non-zero transform coefficient in the sub-block, the encoder 140 encodes a CBP32 flag (which may be represented by, for example, 1 bit such as "1") indicating that there is a non-zero transform coefficient in the sub-block (S2440), and follows The CBP in the sub-block is encoded in units of 16x16 pixel blocks (S2450). If there is no non-zero transform coefficient in the sub-block, the encoder 140 encodes a CBP32 flag (which may be represented by, for example, 1 bit such as '0') indicating that there is no non-zero transform coefficient in the sub-block (S2460).
这里，CBP64标记指示64x64像素块中是否存在非零编码系数。类似地，CBP32标记指示32x32像素块中是否存在非零编码系数。如果亮度分量的残留块不具有非零变换系数，则由于不必发送变换类型，所以这种CBPX标记(X:32,64等)可以用来指示亮度分量是否具有非零变换系数。CBP按照16x16像素块为单位进行编码。在该情况下，每8x8像素块使用1比特，以指示16x16像素块中是否每8x8像素块存在非零变换系数。针对色度分量，1比特用来指示两个2x2色度分量DC块中是否存在非零变换系数，并且1比特用来指示两个8x8色度分量AC块中是否存在非零变换系数。Here, the CBP64 flag indicates whether there are non-zero coding coefficients in the 64x64 pixel block. Similarly, the CBP32 flag indicates whether there are non-zero coded coefficients in the 32x32 pixel block. Such a CBPX flag (X:32, 64, etc.) can be used to indicate whether the luma component has non-zero transform coefficients, since the transform type does not have to be sent if the residual block of the luma component does not have non-zero transform coefficients. CBP is coded in units of 16x16 pixel blocks. In this case, 1 bit is used every 8x8 pixel block to indicate whether there are non-zero transform coefficients every 8x8 pixel block in the 16x16 pixel block. For chroma components, 1 bit is used to indicate whether there are non-zero transform coefficients in the two 2x2 chroma component DC blocks, and 1 bit is used to indicate whether there are non-zero transform coefficients in the two 8x8 chroma component AC blocks.
另一方面，如果在步骤S2420中确定子块的大小既不是32x32,32x16和16x32中的一个也不是32x32像素块，则编码器140进行到步骤S2450，并且对CBP进行编码。这里，如果子块的大小既不是32x32,32x16和16x32中的一个也不是16x32像素块，则编码器140可以确定子块的大小是否为16x16,16x8或8x16像素块。如果子块的大小是16x16,16x8或8x16像素块，并且如果相关子块中存在非零编码系数，则编码器140可以对CBP16标记进行编码(其可以通过例如诸如“1”的1比特来表示)。如果不是，则编码器140可以在不使用CBP标记的情况下按照16x16像素块为单位对CBP进行编码。On the other hand, if it is determined in step S2420 that the size of the sub-block is neither one of 32x32, 32x16, and 16x32 nor a 32x32 pixel block, the encoder 140 proceeds to step S2450 and encodes the CBP. Here, if the size of the sub-block is neither one of 32x32, 32x16, and 16x32 nor a 16x32 pixel block, the encoder 140 may determine whether the size of the sub-block is a 16x16, 16x8, or 8x16 pixel block. If the size of the sub-block is a 16x16, 16x8 or 8x16 pixel block, and if there are non-zero coding coefficients in the associated sub-block, the encoder 140 may encode a CBP16 flag (which may be represented by, for example, 1 bit such as "1") ). If not, the encoder 140 may encode the CBP in units of 16x16 pixel blocks without using the CBP flag.
此外，如果在步骤S2410中确定子块的大小是64x64,64x32和32x64中的一个或者是64x64像素块，则编码器140确定子块中是否存在非零变换系数(S2470)。如果子块中存在非零变换系数，则编码器140对指示子块中存在非零变换系数的CBP64标记(其可以通过例如诸如“1”的1比特来表示)CBP64标记进行编码(S2480)；并且如果子块中不存在非零变换系数，则编码器140对指示子块中不存在非零变换系数的CBP64标记(其可以通过例如诸如“0”的1比特来表示)CBP64标记进行编码(S2490)。这里，CBP64标记指示64x64像素块中是否存在非零量化变换系数。Also, if it is determined in step S2410 that the size of the sub-block is one of 64x64, 64x32, and 32x64 or is a 64x64 pixel block, the encoder 140 determines whether a non-zero transform coefficient exists in the sub-block (S2470). If there is a non-zero transform coefficient in the sub-block, the encoder 140 encodes a CBP64 flag (which may be represented by, for example, 1 bit such as "1") CBP64 flag indicating that there is a non-zero transform coefficient in the sub-block (S2480); And if there is no non-zero transform coefficient in the sub-block, the encoder 140 encodes a CBP64 flag (which may be represented by, for example, 1 bit such as "0") indicating that there is no non-zero transform coefficient in the sub-block ( S2490). Here, the CBP64 flag indicates whether there are non-zero quantized transform coefficients in a 64x64 pixel block.
如果按照16x16像素块为单位对64x64像素扩展宏块进行帧内预测编码，则可以省略CBP64标记和CBP32标记，并且可以如上所述对量化参数的递增/递减信息(Δpq)进行编码。这里，可以按照扩展宏块为单位对量化参数的递增/递减信息进行编码。If the 64x64-pixel extended macroblock is intra-predictively encoded in units of 16x16-pixel blocks, the CBP64 flag and the CBP32 flag can be omitted, and quantization parameter increment/decrement information (Δpq) can be encoded as described above. Here, the increment/decrement information of the quantization parameter may be encoded in units of extended macroblocks.
如果CBP标记被编码，并且如果CBP标记不是0，则变换类型被编码。尤其是，如果16x16像素块被划分为四个8x8像素块，并且如果编码后CBP不是0，则对按照16x16像素块为单位应用的变换类型进行编码。If the CBP flag is coded, and if the CBP flag is not 0, the transform type is coded. In particular, if a 16x16 pixel block is divided into four 8x8 pixel blocks, and if the encoded CBP is not 0, the transform type applied in units of 16x16 pixel blocks is encoded.
在变换类型被编码之后，根据变换类型对CBP进行编码。如果使用16x16变换，则色度分量的仅两个CBP比特被编码；并且如果使用8x16变换或16x8变换，则指示16x16像素块中的两个8x16或16x8像素块是否具有非零变换系数的两个CBP比特被编码。然而，例外的是，如果两个分割块当中的第一分割块的CBP比特是0，则第二分割块的CBP比特不被编码。After the transform type is encoded, the CBP is encoded according to the transform type. If a 16x16 transform is used, only two CBP bits of the chrominance components are coded; and if an 8x16 or 16x8 transform is used, indicates whether two 8x16 or 16x8 pixel blocks in a 16x16 pixel block have two non-zero transform coefficients CBP bits are encoded. However, as an exception, if the CBP bit of the first partition block among the two partition blocks is 0, the CBP bit of the second partition block is not encoded.
图25是例示根据本发明的实施方式的CBP的结构的例示图。FIG. 25 is an illustration diagram illustrating the structure of a CBP according to an embodiment of the present invention.
图25例示了输入视频是4:2:0格式的情况下CBP的结构。如果亮度分量的第一8x8像素块Y0中不存在非零变换系数，则CBP的最低比特(LSB)是0；并且如果第二8x8像素块Y1中存在非零变换系数，则CBP的第二LSB是1。这样，CBP被分配给第三8x8像素块Y2和第四8x8像素块Y3。由于两个2x2色度分量DC块U-DC和V-DC中存在非零量化变换系数，所以针对色度DC(色度DC)的比特是1；并且由于两个8x8色度分量AC块U-AC和V-AC中不存在非零量化变换系数，所以针对色度DC(色度AC)的比特是0。Fig. 25 illustrates the structure of CBP in the case where the input video is in 4:2:0 format. If there are no non-zero transform coefficients in the first 8x8 pixel block Y0 of the luma component, the least significant bit (LSB) of the CBP is 0; and if there are non-zero transform coefficients in the second 8x8 pixel block Y1, the second LSB of the CBP it's 1. Thus, CBPs are allocated to the third 8x8 pixel block Y2 and the fourth 8x8 pixel block Y3. The bit for chroma DC (chroma-DC) is 1 due to the presence of non-zero quantized transform coefficients in the two 2x2 chroma component DC blocks U-DC and V-DC; and since the two 8x8 chroma component AC blocks U - There are no non-zero quantized transform coefficients in AC and V-AC, so the bit for chroma DC (chroma AC) is 0.
图26是例示根据本发明的实施方式对CBPX标记和CBP进行编码的序列的示例的例示图。FIG. 26 is an illustration diagram illustrating an example of a sequence encoding a CBPX flag and a CBP according to an embodiment of the present invention.
图26例示了在扩展宏块是64x64像素块并且作为当前编码目标的子块的大小是64x64,64x32或32x64的情况下CBP32标记和CBP的编码序列。在图26中，点虚线箭头指示CBPX标记为0的情况下的处理序列，虚线箭头指示CBPX标记为1的情况下的处理序列，并且实线箭头指示无条件地出现的处理序列。FIG. 26 illustrates an encoding sequence of a CBP32 flag and a CBP in a case where an extended macroblock is a 64x64 pixel block and the size of a subblock that is a current encoding target is 64x64, 64x32, or 32x64. In FIG. 26 , dotted-line arrows indicate a processing sequence in the case where the CBPX flag is 0, broken-line arrows indicate a processing sequence in the case where the CBPX flag is 1, and solid-line arrows indicate a processing sequence that occurs unconditionally.
如果作为扩展宏块的64x64像素块中的量化变换系数全部为0，则CBP64标记被编码为0；如果一个或更多个量化变换系数不是0，则CBP64标记被编码为1。如果CBP64标记为0，则由于没有更多要编码的CBP，下一个宏块的CBP32标记和CBP被编码。如果CBP64标记是1，则64x64像素块中的各个32x32像素子块的CBP32标记被编码；并且如果CBP32标记是1，则32x32像素子块中的四个16x16像素块的CBP被编码。The CBP64 flag is coded as 0 if the quantized transform coefficients in a 64x64 pixel block that is an extended macroblock are all 0; if one or more quantized transform coefficients are not 0, the CBP64 flag is coded as 1. If the CBP64 flag is 0, the CBP32 flag and CBP of the next macroblock are coded since there is no more CBP to code. If the CBP64 flag is 1, then the CBP32 flags are coded for each 32x32 pixel sub-block in the 64x64 pixel block; and if the CBP32 flag is 1, then the CBPs for the four 16x16 pixel blocks in the 32x32 pixel sub-block are coded.
图27是例示根据本发明的实施方式针对帧内宏块对CBPX标记和CBP进行编码的序列的另一示例的例示图。FIG. 27 is an illustration diagram illustrating another example of a sequence of encoding a CBPX flag and a CBP for an intra macroblock according to an embodiment of the present invention.
图27例示了在编码目标是32x32像素块并且作为当前编码目标的子块的大小是32x32,32x16或16x32的情况下CBP32标记和CBP的编码序列。在图27中，点虚线箭头指示在CBPX标记为0的情况下的处理序列，虚线箭头指示在CBPX标记为1的情况下的处理序列，并且实线箭头指示无条件地出现的处理序列。FIG. 27 illustrates an encoding sequence of a CBP32 flag and a CBP in a case where the encoding target is a 32x32 pixel block and the size of a subblock that is the current encoding target is 32x32, 32x16, or 16x32. In FIG. 27 , dotted-line arrows indicate a processing sequence in a case where the CBPX flag is 0, broken-line arrows indicate a processing sequence in a case where the CBPX flag is 1, and solid-line arrows indicate a processing sequence that occurs unconditionally.
再次参照图1，视频编码设备100的编码器140对CBPX标记(X:大于或等于32的整数)和CBP进行编码，接着仅针对具有非零量化变换系数的块对量化变换系数进行编码，从而生成编码后的视频数据。如果编码目标块的大小大于变换的大小，则编码器140按照光栅扫描顺序对块中的量化变换系数进行编码。Referring again to FIG. 1 , the encoder 140 of the video encoding apparatus 100 encodes a CBPX flag (X: an integer greater than or equal to 32) and a CBP, and then encodes quantized transform coefficients only for blocks with non-zero quantized transform coefficients, thereby Generate encoded video data. If the size of the encoding target block is larger than the size of the transform, the encoder 140 encodes the quantized transform coefficients in the block in raster scan order.
图28是例示根据本发明的实施方式的逐个分割类型量化的变换系数的编码序列的例示图。FIG. 28 is an illustration diagram illustrating an encoding sequence of transform coefficients quantized by partition type according to an embodiment of the present invention.
图28例示了在编码目标块是32x32像素块的情况下量化变化系数的编码序列。由于对16x16或更多像素块进行16x16变换(当16x16块中的十六个4x4块是4x4变换的并且接着对十六个DC系数进行4x4变换时，可以说，对16x16像素单元进行了变换，为便于描述，其将被称作16x16变换)，对32x32像素块进行四个16x16变换。对各个32x16像素块进行两个16x16变换，并且对各个16x32像素块进行两个16x16变换。对各个16x16像素块进行一个16x16变换。FIG. 28 illustrates an encoding sequence of quantization variation coefficients in the case where an encoding target block is a 32×32 pixel block. Since a 16x16 transform is performed on a 16x16 or more pixel block (when sixteen 4x4 blocks in a 16x16 block are 4x4 transformed and then sixteen DC coefficients are 4x4 transformed, so to speak, a 16x16 pixel unit is transformed, For ease of description, it will be referred to as a 16x16 transform), four 16x16 transforms are performed on a 32x32 pixel block. Two 16x16 transforms are performed on each 32x16 pixel block, and two 16x16 transforms are performed on each 16x32 pixel block. A 16x16 transform is performed on each 16x16 pixel block.
图29是例示根据本发明的实施方式的针对帧内图片的编码数据的句法结构的例示图。FIG. 29 is an illustration diagram illustrating a syntax structure of encoded data for an intra picture according to an embodiment of the present invention.
图29例示了当根据本发明的实施方式对作为帧间图片的输入视频进行编码时生成的编码数据的句法结构。如这里所示，根据本发明的实施方式的帧间图片的编码后数据可以包括SKIP_N标记字段、宏块类型字段和分割类型字段中的一个或更多个。在扩展宏块是帧内宏块的情况下，编码后数据还可以包括帧内预测模式字段、CBPX标记/CBP字段和变换系数字段中的一个或更多个。在扩展宏块是帧间宏块的情况下，编码后数据还可以包括帧内预测模式字段、SKIPX标记字段、差运动向量/基准图片索引字段、CBPX标记字段、变换类型字段、CBP字段和变换系数字段中的一个或更多个。FIG. 29 illustrates a syntax structure of encoded data generated when encoding an input video as an inter picture according to an embodiment of the present invention. As shown here, encoded data of an inter picture according to an embodiment of the present invention may include one or more of a SKIP_N flag field, a macroblock type field, and a partition type field. In case the extended macroblock is an intra macroblock, the encoded data may further include one or more of an intra prediction mode field, a CBPX flag/CBP field, and a transform coefficient field. In the case where the extended macroblock is an inter macroblock, the encoded data may also include an intra prediction mode field, a SKIPX flag field, a difference motion vector/reference picture index field, a CBPX flag field, a transform type field, a CBP field, and a transform One or more of the coefficient fields.
然而，如图29所示的针对帧间图片的编码数据的句法仅是示例性的，不必构造为如图29所示。即，应当注意的是，针对帧间图片的编码数据的句法中的全部或者一部分的序列可以与图29所示不同地改变，并且可以省略一部分句法。However, the syntax of encoded data for an inter picture as shown in FIG. 29 is only exemplary, and does not have to be configured as shown in FIG. 29 . That is, it should be noted that all or part of the sequence in the syntax of encoded data for an inter picture may be changed differently from that shown in FIG. 29 , and a part of the syntax may be omitted.
再次参照图1，视频编码设备100的编码器140对指示作为当前编码目标的宏块是否为SKIP宏块的SKIPN标记进行编码。这里，“N”表示宏块的大小。例如，当宏块是64x64像素块时，指示64x64像素宏块是否为SKIP宏块的SKIP64标记被编码。当宏块是32x32像素块时，指示32x32像素宏块是否为SKIP宏块的SKIP64标记被编码。类似地，指示16x16像素宏块是否为SKIP宏块的SKIP16标记被编码。Referring again to FIG. 1 , the encoder 140 of the video encoding apparatus 100 encodes a SKIPN flag indicating whether a macroblock that is a current encoding target is a SKIP macroblock. Here, "N" represents the size of a macroblock. For example, when the macroblock is a 64x64 pixel block, a SKIP64 flag indicating whether the 64x64 pixel macroblock is a SKIP macroblock is coded. When the macroblock is a 32x32 pixel block, a SKIP64 flag indicating whether the 32x32 pixel macroblock is a SKIP macroblock is coded. Similarly, a SKIP16 flag indicating whether a 16x16 pixel macroblock is a SKIP macroblock is coded.
如果宏块是SKIP宏块，则由于当前宏块没有要编码的数据，所以对下一个宏块进行编码。如果宏块不是SKIP宏块，则发送指示宏块是帧内宏块还是帧间宏块的宏块类型标记。例如，如果当前要编码的输入视频是P图片，则可用的宏块类型是I宏块和P宏块；并且如果当前要编码的输入视频是B图片，则可用的宏块类型是I宏块和B宏块。If the macroblock is a SKIP macroblock, since the current macroblock has no data to encode, the next macroblock is encoded. If the macroblock is not a SKIP macroblock, a macroblock type flag is sent indicating whether the macroblock is an intra macroblock or an inter macroblock. For example, if the input video currently to be encoded is a P picture, the available macroblock types are I macroblock and P macroblock; and if the input video currently to be encoded is a B picture, the available macroblock types are I macroblock and B macroblocks.
由于已经参照图17描述了分割类型字段、针对帧内图片的帧内预测模式字段、CBPX标记/CBP字段和变换系数字段，所以下文将描述针对帧间图片的SKIPX标记字段、差运动向量/基准图片索引字段、CBPX标记字段、变换类型字段、CBP字段和变换系数字段。Since the partition type field, the intra prediction mode field for an intra picture, the CBPX flag/CBP field, and the transform coefficient field have been described with reference to FIG. 17 , the SKIPX flag field for an inter picture, difference motion vector/reference will be described below A picture index field, a CBPX flag field, a transform type field, a CBP field, and a transform coefficient field.
在P图片或P片段的情况下，图4中的所有块类型可以用于运动估计，可以按照16x16像素块为单位在扩展宏块中选择帧内模式或帧间模式。即，帧内模式和帧间模式这两者可以用于P图片或P片段的扩展宏块。In the case of P pictures or P slices, all block types in Figure 4 can be used for motion estimation, and intra mode or inter mode can be selected in extended macroblocks in units of 16x16 pixel blocks. That is, both intra mode and inter mode can be used for extended macroblocks of P pictures or P slices.
如果编码目标是32x32像素块，则可以通过扩展宏块标记(extended_mb_flag)指示是按照32x32像素块为单位还是按照16x16像素块为单元进行编码。例如，如果扩展宏块标记是1，则可以指示按照32x32像素块为单位进行编码，并且如果扩展宏块标记是0，则可以指示按照16x16像素块为单位进行编码。如果扩展宏块标记是0，则扩展宏块可以被划分为四个16x16像素块，使得可以按照16x16像素块为单位确定帧内模式或帧间模式。此外，各个16x16像素块可以在编码之前被划分为较小的子块。If the encoding target is a 32x32 pixel block, an extended macroblock flag (extended_mb_flag) may be used to indicate whether to encode in units of 32x32 pixel blocks or 16x16 pixel blocks. For example, if the extended macroblock flag is 1, it may indicate that encoding is performed in units of 32x32 pixel blocks, and if the extended macroblock flag is 0, it may indicate that encoding is performed in units of 16x16 pixel blocks. If the extended macroblock flag is 0, the extended macroblock may be divided into four 16x16 pixel blocks, so that the intra mode or the inter mode may be determined in units of 16x16 pixel blocks. Furthermore, each 16x16 pixel block can be divided into smaller sub-blocks prior to encoding.
图30是用于描述根据本发明的实施方式在P图片(或P片段)中使用的句法的示例的图。为便于描述，假设扩展宏块是32x32像素块。FIG. 30 is a diagram for describing an example of syntax used in a P picture (or P slice) according to an embodiment of the present invention. For ease of description, it is assumed that an extended macroblock is a 32x32 pixel block.
在熵编码的情况下，如果进行上下文自适应二进制数算术码(CABAC)编码，则skipX_flag和interX_flag可以具有根据X值不同的上下文表。例如，skip32_flag和skip16_flag可以具有不同的上下文表。稍后将对此进行详细描述。In the case of entropy coding, if context-adaptive binary arithmetic code (CABAC) coding is performed, skipX_flag and interX_flag may have different context tables according to X values. For example, skip32_flag and skip16_flag can have different context tables. This will be described in detail later.
skipX_flag指示要编码的当前块是SKIP模式(跳过模式)，并且可以使用1比特来指示当前块是否为跳过模式，如图5所示。例如，skipX_flag的编码比特，其可以指示当前块是跳过模式。这里，“X”指示大于或等于16的数字，其在扩展宏块的范围内。skipX_flag indicates that the current block to be encoded is in SKIP mode (skip mode), and 1 bit may be used to indicate whether the current block is in skip mode, as shown in FIG. 5 . For example, the coded bits of skipX_flag, which may indicate that the current block is in skip mode. Here, "X" indicates a number greater than or equal to 16, which is within the range of an extended macroblock.
[表5][table 5]
如果当前块不是跳过模式，则skipX_flag被编码。这里，skipX_flag指示当前块是帧内模式还是帧间模式，并且可以指示1比特或2比特扩展宏块中的像素块是帧内模式还是帧间模式，如图6所示。skipX_flag is encoded if the current block is not in skip mode. Here, skipX_flag indicates whether the current block is an intra mode or an inter mode, and may indicate whether a pixel block in a 1-bit or 2-bit extended macroblock is an intra mode or an inter mode, as shown in FIG. 6 .
[表6][Table 6]
例如，如果是interX_flag是“00”，则表示扩展宏块中的所有子块都是帧内模式。在该情况下，作为扩展宏块的32x32像素块被划分为四个16x16像素块，并且各个16x16像素块通过上述帧内预测编码方法进行编码。因而，如果扩展宏块中的所有子块都是帧内模式，则子块的块类型、亮度分量和色度分量的帧内预测模式、CBP、量化参数和变换系数可以被编码。For example, if interX_flag is "00", it means that all sub-blocks in the extended macroblock are in intra mode. In this case, a 32x32 pixel block as an extended macroblock is divided into four 16x16 pixel blocks, and each 16x16 pixel block is encoded by the above-described intra prediction encoding method. Thus, if all subblocks in an extended macroblock are intra mode, block types, intra prediction modes of luma components and chroma components, CBP, quantization parameters, and transform coefficients of the subblocks can be encoded.
如图30所示，针对作为扩展宏块的32x32像素块，如果skip32_flag是0并且inter32_flag是1，则表示扩展宏块中的所有子块都是帧间模式，并且发送作为1比特标记的“extended_mb_flag”。这里，如果extended_mb_flag是1，则表示按照扩展宏块为单位进行运动补偿；并且由于存在一个运动向量，所以一个MVD/ref_idx被编码。这里，MVD/ref_idx表示包括在扩展宏块中的运动向量信息/基准索引。如果extended_mb_flag是0，则表示扩展宏块在编码之前已经被划分为16x16像素块，并且指示各个16x16像素块是否为跳过模式的skip16_flag被编码。在该情况下，仅针对不是跳过模式的16x16像素块对块类型和MVD/ref_idx进行编码。利用表5来对skip16_flag进行编码。这里，如果skip32_flag和skip16_flag是CABAC编码的，则各个句法可以具有如上所述不同的上下文表。As shown in FIG. 30, for a 32x32 pixel block as an extended macroblock, if skip32_flag is 0 and inter32_flag is 1, it means that all subblocks in the extended macroblock are in inter mode, and "extended_mb_flag" is sent as a 1-bit flag ". Here, if extended_mb_flag is 1, it means that motion compensation is performed in units of extended macroblocks; and since there is one motion vector, one MVD/ref_idx is coded. Here, MVD/ref_idx indicates motion vector information/reference index included in the extended macroblock. If extended_mb_flag is 0, it indicates that the extended macroblock has been divided into 16x16 pixel blocks before encoding, and skip16_flag indicating whether each 16x16 pixel block is a skip mode is encoded. In this case, the block type and MVD/ref_idx are encoded only for 16x16 pixel blocks that are not in skip mode. Use Table 5 to encode skip16_flag. Here, if skip32_flag and skip16_flag are CABAC-coded, each syntax may have a different context table as described above.
针对扩展宏块，如果skip32_flag是0并且inter32_flag是01，则表示帧间模式和帧内模式这两者被用于扩展宏块。在该情况下，“extended_mb_flag”不是必需的，并且由于其表示扩展宏块在编码之前已经被划分为16x16像素块，所以可以利用指示各个16x16像素块是帧内模式还是帧间模式的表7对其进行编码。For an extended macroblock, if skip32_flag is 0 and inter32_flag is 01, it means that both the inter mode and the intra mode are used for the extended macroblock. In this case, "extended_mb_flag" is not necessary, and since it indicates that the extended macroblock has been divided into 16x16 pixel blocks before encoding, you can use Table 7 to indicate whether each 16x16 pixel block is an intra mode or an inter mode. It encodes.
[表7][Table 7]
这里，如果帧内模式和帧间模式在扩展宏块中共存，则各个16x16像素块中的所有子块主要是帧间模式或者帧内模式，并且很少出现帧内模式子块和帧间模式子块在16x16像素块中共存的情况。因而，在各个16x16像素块的分割类型被编码之后，具有inter16_flag 0或者是帧内模式的16x16像素块中的子块的亮度分量和色度分量被编码为预测模式。此外，针对具有inter16_flag 1或者是帧间模式的16x16像素块，利用表5对inter16_flag进行编码。在进行分割类型编码之后对skip16_flag进行编码的原因在于，由于帧内模式和帧间模式在扩展宏块中的共存表示具有一些纹理的区域，所以存在与帧内模式相邻的帧间模式被划分为小块的较高概率，进而分割类型可以被首先更好地发送，接着可以仅针对16x16像素块对skip16_flag进行编码。然而，编码序列不限于此。例如，可以在对skip16_flag进行编码之后对分割类型进行编码。Here, if intra-mode and inter-mode coexist in an extended macroblock, all sub-blocks in each 16x16 pixel block are mainly inter-mode or intra-mode, and intra-mode sub-blocks and inter-mode rarely occur The case where subblocks co-exist in 16x16 pixel blocks. Thus, after the partition type of each 16x16 pixel block is encoded, the luma component and chrominance component of the subblock in the 16x16 pixel block having inter16_flag 0 or intra mode are encoded as the prediction mode. In addition, for a 16x16 pixel block with inter16_flag 1 or inter mode, use Table 5 to encode inter16_flag. The reason for encoding skip16_flag after split type encoding is that since the coexistence of intra mode and inter mode in extended macroblocks indicates an area with some texture, there is an inter mode adjacent to the intra mode that is divided Higher probability for small blocks, and thus the partition type can be better sent first, then the skip16_flag can be encoded only for 16x16 pixel blocks. However, coding sequences are not limited thereto. For example, the split type may be encoded after encoding the skip16_flag.
CBPX_flag、变换类型、CBP/量化参数和变换系数的后续编码与如上所述相同，因而可以省略对其的描述。然而，在帧内图片/片或者帧内EMB的情况下，可以不对CBPX_flag进行编码。Subsequent encoding of CBPX_flag, transform type, CBP/quantization parameters, and transform coefficients is the same as described above, and thus descriptions thereof can be omitted. However, in case of intra picture/slice or intra EMB, CBPX_flag may not be coded.
如同扩展宏块是32x32像素块的情况，如果扩展宏块是64x64像素块，仅针对16x16或更多像素子块对当前块的分割类型信息接着对skipX_flag进行编码。这里，“X”表示诸如16或32的子块大小。例如，如果宏块是64x64像素块并且不是跳过模式，并且如果扩展宏块的子块的大小是64x32,32x64和32x32中的一个，则针对各个子块对skip32_flag进行编码。如果64x64像素扩展宏块的子块的大小是64x32或32x64，则对两个skip32_flag进行编码。如果子块的大小小于32x32，则不对skip32_flag进行编码。此外，如果子块的大小是32x16,16x32和16x16中的一个，则对各个子块的skip16_flag进行编码。As in the case where the extended macroblock is a 32x32 pixel block, if the extended macroblock is a 64x64 pixel block, the partition type information of the current block is then encoded only for 16x16 or more pixel subblocks followed by skipX_flag. Here, "X" represents a sub-block size such as 16 or 32. For example, if the macroblock is a 64x64 pixel block and is not skip mode, and if the size of the subblocks of the extended macroblock is one of 64x32, 32x64, and 32x32, skip32_flag is encoded for each subblock. If the size of the subblock of the 64x64 pixel extended macroblock is 64x32 or 32x64, two skip32_flags are encoded. If the size of the subblock is less than 32x32, skip32_flag is not encoded. Also, if the size of the sub-block is one of 32x16, 16x32, and 16x16, the skip16_flag of each sub-block is encoded.
图31是用于描述根据本发明的实施方式在B图片(或B片段)中使用的句法的示例的图。FIG. 31 is a diagram for describing an example of syntax used in a B picture (or B slice) according to an embodiment of the present invention.
在B图片的情况下，可以选择多达两个基准图片，其分别被称作L0预测和L1预测。各个基准图片执行不同的功能。总体上，L0预测(列表0预测)被用作前向预测，L1预测被用作反向预测。即，B图片可以使用随后是当前编码目标图片的图片和跟随当前编码目标图片的图片作为基准图片，并且可以同时使用两个基准图片(双向预测)来进行运动估计和补偿。图2的所有块类型可以用作用于运动预测的块类型，并且如同P图片的情况，可以按照16x16像素块为单位在扩展宏块中选择帧内模式或帧间模式。在本发明的实施方式中，两个图片不限于随后是当前图片的图片和跟随当前图片的图片，两个随机图片可以独立于其序列被用作基准图片。这两个基准图片可以用来进行运动估计/补偿(双向预测)。In the case of B-pictures, up to two reference pictures can be selected, which are called L0 prediction and L1 prediction, respectively. Each reference picture performs a different function. In general, L0 prediction (list 0 prediction) is used as forward prediction and L1 prediction is used as backward prediction. That is, a B picture can use a picture subsequent to the current encoding target picture and a picture following the current encoding target picture as reference pictures, and can simultaneously use two reference pictures (bidirectional prediction) for motion estimation and compensation. All block types of FIG. 2 can be used as a block type for motion prediction, and as in the case of a P picture, an intra mode or an inter mode can be selected in an extended macroblock in units of 16x16 pixel blocks. In an embodiment of the present invention, the two pictures are not limited to the picture following the current picture and the picture following the current picture, and two random pictures can be used as reference pictures independently of their sequence. These two reference pictures can be used for motion estimation/compensation (bidirectional prediction).
根据本发明的实施方式在B图片中可用的编码模式的示例包括帧内模式、跳过模式、直接模式、L0预测、L1预测和双向预测。Examples of encoding modes available in a B picture according to an embodiment of the present invention include intra mode, skip mode, direct mode, L0 prediction, L1 prediction, and bidirectional prediction.
帧内模式可以利用与上述相同的帧内编码方法进行编码。帧内模式可以按照扩展宏块为单位或者按照包括在扩展宏块中的16x16像素块为单位进行选择。Intra mode can be coded using the same intra coding method as described above. The intra mode may be selected in units of extended macroblocks or in units of 16x16 pixel blocks included in the extended macroblocks.
跳过模式仅发送指示要编码的当前块是跳过模式的标记。这里，跳过模式可以用于诸如16x16像素块或32x32像素块的16x16或更多像素块。例如，预测B_pred_dir、预测运动向量MVD和预测ref_idx被用来进行运动补偿，通过运动补偿获得的基准块是重构块。这里，B_pred_dir是B图片的预测方向，其表示两个方向L0和L1。Skip mode only sends a flag indicating that the current block to be encoded is in skip mode. Here, skip mode may be used for 16x16 or more pixel blocks such as 16x16 pixel blocks or 32x32 pixel blocks. For example, prediction B_pred_dir, prediction motion vector MVD, and prediction ref_idx are used for motion compensation, and a reference block obtained through motion compensation is a reconstructed block. Here, B_pred_dir is a prediction direction of a B picture, which indicates two directions L0 and L1.
直接模式仅对指示直接模式的标记以及与变换系数编码相关的数据(例如，CBPX_flag/CBP、量化参数、变换类型和变换系数)进行编码。即，直接模式是省略运动信息编码的编码模式，并且可以被用于诸如8x8,16x16和32x32的8x8或更多像素块。在NxN直接模式的情况下，按照N/2x N/2像素块为单位进行运动估计和补偿。例如，在8x8直接模式的情况下，按照4x4像素块为单位进行运动估计和补偿。The direct mode encodes only a flag indicating the direct mode and data related to transform coefficient encoding (for example, CBPX_flag/CBP, quantization parameter, transform type, and transform coefficient). That is, the direct mode is an encoding mode omitting motion information encoding, and may be used for 8x8 or more pixel blocks such as 8x8, 16x16, and 32x32. In the case of the NxN direct mode, motion estimation and compensation are performed in units of N/2xN/2 pixel blocks. For example, in the case of the 8x8 direct mode, motion estimation and compensation are performed in units of 4x4 pixel blocks.
L0预测是仅使用L0运动信息的单向预测，L1预测是仅使用L1运动信息的单向预测。双向预测是使用L0运动信息和L1运动信息的预测，并且使用两个图片的平均值作为基准图片。L0 prediction is unidirectional prediction using only L0 motion information, and L1 prediction is unidirectional prediction using only L1 motion information. Bidirectional prediction is prediction using L0 motion information and L1 motion information, and uses an average value of two pictures as a reference picture.
针对B图片的运动估计方法的示例包括直接预测模式的运动估计/补偿方法、跳过模式的运动估计/补偿方法、单向预测(L0预测或L1预测)的运动估计/补偿方法和双向预测的运动估计/补偿方法。Examples of motion estimation methods for B pictures include a motion estimation/compensation method for direct prediction mode, a motion estimation/compensation method for skip mode, a motion estimation/compensation method for unidirectional prediction (L0 prediction or L1 prediction), and a motion estimation/compensation method for bidirectional prediction. Motion estimation/compensation methods.
首先，关于直接模式的运动估计/补偿方法，直接模式包括时间直接预测模式的运动估计/补偿方法和使用空间方向的运动信息相关(图片中的垂直/水平两维空间)的空间直接预测模式的运动估计/补偿方法。时间直接预测模式和空间直接预测模式中的一个按照片为单位而不是按照宏块或块为单位进行选择。在直接预测模式中，L1预测中具有最小标号的基准图片是重要的，其被称作“锚定图片”。按照显示顺序的目标图片的反向方向中的最近的基准图片是锚定图片，锚定图片中位于与目标块相同的空间位置的块被称作“锚定块”。First, regarding the motion estimation/compensation method of the direct mode, the direct mode includes the motion estimation/compensation method of the temporal direct prediction mode and the spatial direct prediction mode using the correlation of motion information in the spatial direction (vertical/horizontal two-dimensional space in the picture) Motion estimation/compensation methods. One of the temporal direct prediction mode and the spatial direct prediction mode is selected in units of pictures instead of units of macroblocks or blocks. In direct prediction mode, the reference picture with the smallest index in L1 prediction is important, which is called "anchor picture". The nearest reference picture in the reverse direction of the target picture in display order is an anchor picture, and a block located at the same spatial position as the target block in the anchor picture is called an "anchor block".
在时间直接预测模式中，检查锚定块的运动信息，并且mvCol(MV或共同位置块、相同块的运动向量)被用作锚定块的L0运动向量。如果不存在锚定块的L0运动向量并且存在L1运动向量，则L1运动向量被用作mvCol。时间直接预测模式的L0基准图片是mvCol的基准图片，并且时间直接预测模式的L1基准图片是锚定图片。如果锚定图片在图片中没有编码运动信息，则运动向量的幅度是0，并且时间直接预测模式的L0基准图片是基准图片编号0。此外，从运动向量mvCol获得L0运动向量mvL0和L1运动向量mvL1。如果基准图片、目标图片和锚定图片之间的运动速度是恒定的，则mvL0和mvL1的幅度与基准图片、目标图片和锚定图片之间的距离成比例，进而通过比例分布来进行计算。即，mvL0可以利用基准图片与目标图片之间的距离tb以及基准图片与锚定图片之间的距离td通过式5来进行计算。In temporal direct prediction mode, the motion information of the anchor block is checked, and mvCol (MV or co-located block, motion vector of the same block) is used as the L0 motion vector of the anchor block. If there is no L0 motion vector of the anchor block and there is an L1 motion vector, the L1 motion vector is used as mvCol. The L0 reference picture of the temporal direct prediction mode is the reference picture of mvCol, and the L1 reference picture of the temporal direct prediction mode is the anchor picture. If the anchor picture has no encoded motion information in the picture, the magnitude of the motion vector is 0, and the L0 reference picture of temporal direct prediction mode is reference picture number 0. Also, the L0 motion vector mvL0 and the L1 motion vector mvL1 are obtained from the motion vector mvCol. If the motion speed between the reference picture, the target picture and the anchor picture is constant, the magnitudes of mvL0 and mvL1 are proportional to the distances between the reference picture, the target picture and the anchor picture, and then calculated through the proportional distribution. That is, mvL0 can be calculated by formula 5 by using the distance tb between the reference picture and the target picture and the distance td between the reference picture and the anchor picture.
此外，由于mvL1与mvCol在相同的方向上，所以它们可以通过式6来计算。In addition, since mvL1 and mvCol are in the same direction, they can be calculated by Equation 6.
类似地，在空间直接预测模式中，作为在L1预测中具有最小标号的基准图片的锚定图片以及位于与锚定图片相同的空间位置的锚定块被用来计算运动向量。在空间直接预测模式中，根据编码在目标图片中的运动向量来预测目标块的运动向量。时间直接预测模式的一部分概念也被引入到空间直接预测模式中。当锚定块的运动信息满足以下全部条件时，L0运动向量mvL0和L1运动向量mvL1被设定为0：i)锚定块的运动向量的幅度在水平方向和垂直方向上小于或等于±1像素，ii)锚定块的基准图片的标号是0，和iii)L0中的最小标号的基准图片是短期基准图片。此外，L0预测中具有最小标号的图片被设定为L0预测图片，并且L1预测中具有最小标号的图片被设定为L1预测图片。当满足这些条件时，根据编码在目标图片中的运动向量来计算目标块以及基准图片L0和L1的运动向量mvL0和mvL1。Similarly, in the spatial direct prediction mode, an anchor picture which is a reference picture having the smallest number in L1 prediction and an anchor block located at the same spatial position as the anchor picture are used to calculate a motion vector. In the spatial direct prediction mode, the motion vector of the target block is predicted from the motion vector encoded in the target picture. A part of the concept of the temporal direct prediction mode is also introduced into the spatial direct prediction mode. The L0 motion vector mvL0 and the L1 motion vector mvL1 are set to 0 when the motion information of the anchor block satisfies all of the following conditions: i) the magnitude of the motion vector of the anchor block is less than or equal to ±1 in the horizontal and vertical directions Pixel, ii) the index of the anchor block's reference picture is 0, and iii) the lowest indexed reference picture in L0 is the short-term reference picture. Also, the picture with the smallest number in L0 prediction is set as the L0 prediction picture, and the picture with the smallest number in L1 prediction is set as the L1 prediction picture. When these conditions are satisfied, the motion vectors mvL0 and mvL1 of the target block and the reference pictures L0 and L1 are calculated from the motion vector encoded in the target picture.
在L0预测中，位于左侧、上侧和左上侧的L0预测运动向量mvA、mvB和mvC被用来计算水平分量和垂直分量的均值，并且被设定为mvL0。类似地，在L1预测中，位于左侧、上侧和右上侧的L1预测运动向量mvA、mvB和mvC被用来计算水平分量和垂直分量的均值，并且被设定为mvL1。如果相邻块假设单向预测或者编码在图片中的L0预测，或者如果不存在L1预测的运动向量，则用0向量来代替。计算位于左侧、上侧和右上侧的运动向量mvA、mvB和mvC的标号的最小值，并将其设定为NxN块的标号。独立于L0预测和L1预测进行该标号计算。此外，如果相邻块假设单向预测，或者如果不存在图片中预测编码的L0或L1预测的运动向量，则在计算标号时排除该块。这里，应当注意的是，当进行NxN像素块的直接预测模式时，按照NxN像素块为单位进行计算mvL0、mvL1、L0基准图片和L1基准图片的处理，但是按照块为单位确定是使用0向量还是从相邻块计算的运动向量(mvL0,mvL1)。In the L0 prediction, the L0 prediction motion vectors mvA, mvB, and mvC located on the left, upper, and upper left sides are used to calculate the mean value of the horizontal component and the vertical component, and are set as mvL0. Similarly, in L1 prediction, L1 prediction motion vectors mvA, mvB, and mvC located on the left, upper, and upper right sides are used to calculate the mean value of the horizontal component and the vertical component, and are set as mvL1. If the neighboring block assumes unidirectional prediction or L0 prediction coded in the picture, or if there is no motion vector for L1 prediction, a 0 vector is used instead. The minimum value of the labels of the motion vectors mvA, mvB, and mvC located on the left, upper, and upper right sides is calculated and set as the label of the NxN block. This index calculation is done independently of L0 prediction and L1 prediction. Furthermore, if a neighboring block assumes unidirectional prediction, or if there is no predictively coded L0 or L1 predicted motion vector in the picture, this block is excluded from the calculation of the label. Here, it should be noted that when the direct prediction mode of NxN pixel blocks is performed, the processing of calculating mvL0, mvL1, L0 reference pictures and L1 reference pictures is performed in units of NxN pixel blocks, but it is determined in units of blocks that 0 vectors are used Again motion vectors (mvL0, mvL1) calculated from neighboring blocks.
下文将描述跳过模式的运动估计/补偿方法。The motion estimation/compensation method of skip mode will be described below.
跳过模式与P图片的跳过模式的相同之处在于预测运动向量被用来进行运动预测和补偿。然而，用于B图片中的跳过模式的运动向量与基准图片的直接模式的预测运动向量对于基准图片是相同的。即，跳过模式的运动估计/补偿方法与上述的直接预测模式的运动估计/补偿方法相同。直接模式与跳过的不同之处在于跳过模式不具有运动信息和变换系数，而直接模式不是仅具有运动信息。Skip mode is the same as that of P pictures in that the predicted motion vector is used for motion prediction and compensation. However, the motion vector for skip mode in a B picture is the same as the predicted motion vector for direct mode of a reference picture for a reference picture. That is, the motion estimation/compensation method in the skip mode is the same as the motion estimation/compensation method in the direct prediction mode described above. Direct mode differs from skip in that skip mode does not have motion information and transform coefficients, whereas direct mode does not have motion information only.
除了基准图片是L0和L1中的一个以外，单向预测(L0预测或L1预测)的运动估计/补偿方法与P图片的运动估计/补偿方法相同。因而，将省略对其的描述。The motion estimation/compensation method of the unidirectional prediction (L0 prediction or L1 prediction) is the same as that of the P picture except that the reference picture is one of L0 and L1. Therefore, description thereof will be omitted.
最后，下文将描述双向模式的运动估计/补偿方法。Finally, the motion estimation/compensation method for bidirectional mode will be described below.
针对B图片中具有运动信息的8x8或更多像素块，利用上述的P图片的运动估计/补偿方法，L0基准图片和mvL0被计算以进行L0预测，并且L1基准图片和mvL1被计算以进行L1预测。此外，通过L0预测处理计算的L0基准图片中的由mvL0指示的基准块和通过L0预测处理计算的L1基准图片中的由mvL1指示的基准块的平均块可以用作对mvL0和mvL1进行双向预测的基准块。此外，可以进行关于L0预测的运动信息的L1预测，以选择针对mvL0/0基准图片最优的mvL1、L1基准图片，并且所选择的L1基准图片中的由mvL1指示的基准块和通过L0预测处理计算的L0基准图片中的由mvL0指示的基准块的平均块可以用作进行双向预测的基准块。此外，可以进行关于L1预测的运动信息的L0预测，以选择针对mvL1/1基准图片最优的mvL0、L0基准图片，并且所选择的L0基准图片中的由mvL0指示的基准块和通过L1预测处理计算的L1基准图片中的由mvL1指示的基准块的平均块可以用作进行双向预测的基准块。可以对针对B图片中具有运动信息的8x8或更多像素块的双向预测的编码效率进行比较，并且可以选择提供较好编码效率的一个双向预测模式。针对16x16或更多像素块，当上述双向预测模式的运动信息被获得以确定最优编码模式时，可以进行变换、量化和熵变换系数编码，并且预测误差(失真)和相关模式中需要的实际编码比特可以用来确定双向预测模式。For an 8x8 or more pixel block with motion information in a B-picture, using the motion estimation/compensation method for a P-picture described above, the L0 reference picture and mvL0 are computed for L0 prediction, and the L1 reference picture and mvL1 are computed for L1 predict. In addition, the average block of the reference block indicated by mvL0 in the L0 reference picture calculated by the L0 prediction process and the reference block indicated by mvL1 in the L1 reference picture calculated by the L0 prediction process can be used as the basis for bidirectional prediction of mvL0 and mvL1 base block. In addition, L1 prediction on motion information for L0 prediction can be performed to select mvL1, L1 reference picture optimal for mvL0/0 reference picture, and the reference block indicated by mvL1 in the selected L1 reference picture and the reference block indicated by L0 prediction The average block of the reference blocks indicated by mvL0 in the calculated L0 reference picture may be used as a reference block for bidirectional prediction. In addition, L0 prediction on motion information for L1 prediction can be performed to select mvL0, L0 reference pictures optimal for mvL1/1 reference pictures, and the reference block indicated by mvL0 in the selected L0 reference picture and the reference block indicated by L1 prediction The average block of the reference block indicated by mvL1 in the L1 reference picture calculated by processing may be used as a reference block for bidirectional prediction. Coding efficiencies of bi-prediction for 8x8 or more pixel blocks with motion information in a B-picture can be compared, and one bi-prediction mode that provides better coding efficiency can be selected. For 16x16 or more pixel blocks, when the motion information of the above-mentioned bidirectional prediction mode is obtained to determine the optimal coding mode, transformation, quantization and entropy transformation coefficient coding can be performed, and the prediction error (distortion) and the actual The coded bits can be used to determine the bidirectional prediction mode.
针对L0预测、L1预测和双向预测，8x8或更多像素块可以使用不同的基准图片，但是8x8像素块中的子块使用相同的基准图片。For L0 prediction, L1 prediction and bidirectional prediction, 8x8 or more pixel blocks may use different reference pictures, but the sub-blocks in the 8x8 pixel block use the same reference picture.
参照图1，减法器120通过从当前块中减去由运动估计器116估计的当前块的由运动向量指示的基准块来生成残留块。残留块被变换器/量化器130变换和量化，并且量化变化系数被编码器140编码，从而生成编码后数据。这里，B图片的变换方法与P图片的上述变换方法相同，因而省略对其的描述。Referring to FIG. 1 , the subtractor 120 generates a residual block by subtracting a reference block indicated by a motion vector of a current block estimated by the motion estimator 116 from the current block. The residual block is transformed and quantized by the transformer/quantizer 130, and the quantized variation coefficient is encoded by the encoder 140, thereby generating encoded data. Here, the transformation method of the B picture is the same as the above-mentioned transformation method of the P picture, and thus its description is omitted.
图31例示了基于扩展宏块是32x32像素块的假设要通过B图片(或B片段)的比特流发送的数据。总体上，B图片(或B片段)的发送句法类似于参照图30所述的P图片(或B片段)的发送句法。这里，将仅描述添加到P图片的句法的部分。FIG. 31 illustrates data to be transmitted through the bit stream of a B picture (or B slice) based on the assumption that an extended macroblock is a 32x32 pixel block. In general, the transmission syntax of a B picture (or B slice) is similar to the transmission syntax of a P picture (or B slice) described with reference to FIG. 30 . Here, only a part of syntax added to a P picture will be described.
directX_flag是指示仅针对帧间块当中的扩展宏块、16x16或更多像素块和8x8或更多像素块的直接模式不是跳过模式的标记，其可以由表8所示的1比特构成。directX_flag is a flag indicating that the direct mode only for extended macroblocks, 16x16 or more pixel blocks, and 8x8 or more pixel blocks among inter blocks is not a skip mode, and may be composed of 1 bit shown in Table 8.
[表8][Table 8]
在直接模式的情况下，运动信息不被编码，并且与变换系数相关的信息被编码。此外，如果扩展宏块不是直接模式，则运动信息的数据(例如，B_pdir,MVD和ref_idx)被编码。这里，B_pdir是表示当前块的预测方向(L0预测、L1预测和双向预测)的信息。仅仅不是跳过模式或直接模式的帧间块被编码，并且存在表9所示的三个模式。In case of direct mode, motion information is not encoded, and information related to transform coefficients is encoded. Also, if the extended macroblock is not direct mode, data of motion information (for example, B_pdir, MVD, and ref_idx) is encoded. Here, B_pdir is information indicating the prediction direction (L0 prediction, L1 prediction, and bidirectional prediction) of the current block. Only an inter block that is not a skip mode or a direct mode is coded, and there are three modes shown in Table 9.
[表9][Table 9]
在双向预测的情况下，L0预测的运动信息(mvL0和ref_idx_L0)和L1预测的运动信息(mvL1和ref_idx_L1)都被编码。在L0预测的情况下，仅L0预测的运动信息(mvL0和ref_idx_L0)被编码。在L1预测的情况下，仅L1预测的运动信息(mvL1和ref_idx_L1)被编码。In the case of bidirectional prediction, both L0 predicted motion information (mvL0 and ref_idx_L0) and L1 predicted motion information (mvL1 and ref_idx_L1) are encoded. In case of L0 prediction, only motion information (mvL0 and ref_idx_L0) of L0 prediction is encoded. In case of L1 prediction, only L1 predicted motion information (mvL1 and ref_idx_L1) is encoded.
图32是用于描述根据本发明的实施方式的SKIPX标记的编码处理的例示图。FIG. 32 is an illustration diagram for describing encoding processing of a SKIPX mark according to an embodiment of the present invention.
图32例示了宏块是64x64像素块并且被划分为各种大小和形状的子块的情况下宏块的分割类型和各个子块的编码序列。如果宏块被划分为如图32所示，则十一16x16像素子块的SKIP16标记、十二16x16像素子块的SKIP16标记、十五16x32像素子块的SKIP16标记、十六16x32像素子块的SKIP16标记、十七32x32像素子块的SKIP32标记和十八32x32像素子块的SKIP32标记按照所列出的顺序被依次进行编码。FIG. 32 exemplifies division types of macroblocks and encoding sequences of respective subblocks in the case where a macroblock is a 64x64 pixel block and is divided into subblocks of various sizes and shapes. If the macroblock is divided as shown in Figure 32, eleven 16x16 pixel sub-blocks with SKIP16 marks, twelve 16x16 pixel sub-blocks with SKIP16 marks, fifteen 16x32 pixel sub-blocks with SKIP16 marks, sixteen 16x32 pixel sub-blocks with SKIP16 marks, The SKIP16 flag, the SKIP32 flag of the seventeenth 32x32 pixel sub-block, and the SKIP32 flag of the eighteenth 32x32 pixel sub-block are sequentially encoded in the order listed.
针对不是SKIP宏块的子块，包括作为相关子块的运动向量与利用相关子块的相邻块估计的预测运动向量之间的差向量的差运动向量(DMV)的运动信息被编码。For a subblock that is not a SKIP macroblock, motion information including a difference motion vector (DMV) that is a difference vector between a motion vector of a relevant subblock and a predicted motion vector estimated using a neighboring block of the relevant subblock is encoded.
图33和图34是用于描述根据本发明的实施方式针对帧间宏块对CBPX_flag、变换类型和CBP进行编码的处理的例示图。33 and 34 are illustration diagrams for describing a process of encoding CBPX_flag, transform type, and CBP for an inter macroblock according to an embodiment of the present invention.
图33和图34例示了宏块是64x64像素块的情况下对CBPX_flag、变换类型和CBP进行编码的处理。视频编码设备100的编码器140确定扩展宏块的子块的大小是否为64x64,64x32和32x64中的一个(S3302)。如果扩展宏块的子块的大小不是64x64,64x32和32x64中的一个，则编码器140确定子块的大小是否为32x32,32x16和16x32中的一个(S3304)。33 and 34 illustrate the process of encoding CBPX_flag, transform type, and CBP in the case where the macroblock is a 64x64 pixel block. The encoder 140 of the video encoding apparatus 100 determines whether the size of the subblock of the extended macroblock is one of 64x64, 64x32, and 32x64 (S3302). If the size of the subblock of the extended macroblock is not one of 64x64, 64x32, and 32x64, the encoder 140 determines whether the size of the subblock is one of 32x32, 32x16, and 16x32 (S3304).
如果在步骤S3304中确定子块的大小是32x32,32x16和16x32中的一个，则编码器140确定子块中是否存在非零量化变换系数(S3306)。如果子块中存在非零量化变换系数，则编码器140对指示子块中存在非零量化变换系数的CBP32标记(其可以通过例如诸如“1”的1比特来表示)进行编码(S3308)。If it is determined in step S3304 that the size of the sub-block is one of 32x32, 32x16, and 16x32, the encoder 140 determines whether non-zero quantized transform coefficients exist in the sub-block (S3306). If a non-zero quantized transform coefficient exists in the sub-block, the encoder 140 encodes a CBP32 flag (which may be represented by, for example, 1 bit such as '1') indicating the presence of a non-zero quantized transform coefficient in the sub-block (S3308).
在步骤S3308中对CBP32标记进行了编码之后，编码器140确定16x16像素块中是否存在非零量化变换系数(S3310)。如果16x16像素块中存在非零量化变换系数，则编码器140对指示16x16像素块中存在非零量化变换系数的CBP16标记进行编码(S3312)。After encoding the CBP32 flag in step S3308, the encoder 140 determines whether non-zero quantized transform coefficients exist in the 16x16 pixel block (S3310). If non-zero quantized transform coefficients exist in the 16x16 pixel block, the encoder 140 encodes a CBP16 flag indicating that non-zero quantized transform coefficients exist in the 16x16 pixel block (S3312).
在如上对CBPX标记进行了编码之后，编码器140对仅针对具有非零量化变换系数的块的变换类型进行编码(S3316)。这里，如果子块的大小大于或等于16x16，则按照子块为单位进行变换类型编码；并且如果子块的大小小于16x16，则按照16x16为单位进行变换标记编码。这样做的原因在于变换是按照16x16像素为单位而选择的。例如，如果64x64像素宏块被划分为两个64x32像素子块并且CBP64标记不是0，则针对各个64x32像素子块中具有非零CBP32标记的两个64x32像素子块进行变换类型编码。After encoding the CBPX flag as above, the encoder 140 encodes the transform type only for blocks with non-zero quantized transform coefficients (S3316). Here, if the size of the sub-block is greater than or equal to 16x16, transform type coding is performed in units of sub-blocks; and if the size of the sub-block is smaller than 16x16, transform flag coding is performed in units of 16x16. The reason for this is that the transform is chosen in units of 16x16 pixels. For example, if a 64x64 pixel macroblock is divided into two 64x32 pixel sub-blocks and the CBP64 flag is not 0, transform-type coding is performed on the two 64x32 pixel sub-blocks with non-zero CBP32 flags in each 64x32 pixel sub-block.
这里，如果子块的大小大于或等于16x16并且属于该子块的所有CBPX标记都不为0，则利用表3对各个子块的变换类型进行编码。Here, if the size of the sub-block is greater than or equal to 16x16 and all the CBPX flags belonging to the sub-block are not 0, use Table 3 to encode the transformation type of each sub-block.
[表10][Table 10]
此外，如果CBP16标记不是0并且16x16像素块被划分为两个8x16或16x8像素块，则利用表4按照16x16像素块为单位对变换类型进行编码。In addition, if the CBP16 flag is not 0 and the 16x16 pixel block is divided into two 8x16 or 16x8 pixel blocks, the transform type is encoded in units of 16x16 pixel blocks using Table 4.
[表11][Table 11]
如果CBP16标记不是0；16x16像素块被划分为四个8x8像素块；并且选择了4x4变换；则0被编码为变换类型；并且如果选择了8x8变换，则1被编码为变换类型。在其它情况下，不对变换类型进行编码。If the CBP16 flag is not 0; a 16x16 pixel block is divided into four 8x8 pixel blocks; and a 4x4 transform is selected; a 0 is coded as the transform type; and a 1 is coded as the transform type if an 8x8 transform is selected. In other cases, the transform type is not encoded.
在步骤S3316中对变换类型进行编码之后，编码器140根据变换类型对CBP进行编码。为此，编码器140确定变换类型是否为16x16变换(S3318)。如果变换类型是16x16变换，则编码器140仅对色度分量的两个CBP比特进行编码(S3320)。如果变换类型不是16x16变换，则编码器140确定变换类型是否为8x16变换或16x8变换(S3322)。如果变换类型是8x16变换或16x8变换，则编码器140对亮度分量的两个CBP比特和色度分量的两个CBP比特进行编码(S3324)。如果变换类型不是8x16变换和16x8变换，则编码器140确定变换类型是否为8x8变换或4x4变换(S3326)。如果变换类型是8x8变换或4x4变换，则编码器140对亮度分量的四个CBP比特和色度分量的两个CBP比特进行编码(S3328)。在步骤S320、S3324和S3328中，给出在输入视频是4:2:0格式的情况下对亮度分量的CBP和色度分量的CBP进行编码的示例性方法的描述。因而，如果输入视频的格式改变，则亮度分量的CBP和色度分量的CBP的编码方法可以相应地改变。After encoding the transform type in step S3316, the encoder 140 encodes the CBP according to the transform type. For this, the encoder 140 determines whether the transform type is 16x16 transform (S3318). If the transform type is 16x16 transform, the encoder 140 encodes only two CBP bits of the chroma component (S3320). If the transform type is not 16x16 transform, the encoder 140 determines whether the transform type is 8x16 transform or 16x8 transform (S3322). If the transform type is 8x16 transform or 16x8 transform, the encoder 140 encodes two CBP bits of the luma component and two CBP bits of the chroma component (S3324). If the transform type is not 8x16 transform and 16x8 transform, the encoder 140 determines whether the transform type is 8x8 transform or 4x4 transform (S3326). If the transform type is 8x8 transform or 4x4 transform, the encoder 140 encodes four CBP bits of the luma component and two CBP bits of the chroma component (S3328). In steps S320, S3324, and S3328, a description is given of an exemplary method of encoding the CBP of the luma component and the CBP of the chroma component in the case that the input video is in the 4:2:0 format. Thus, if the format of the input video is changed, the encoding method of the CBP of the luma component and the CBP of the chroma component may be changed accordingly.
另一方面，如果在步骤S3306中确定子块中不存在非零量化变换系数，则编码器140对指示子块中不存在非零量化变换系数的CBP32标记进行编码(S3330)。由于子块中不存在非零量化变换系数，所以变换类型和CBP不被编码。On the other hand, if it is determined in step S3306 that there is no non-zero quantized transform coefficient in the sub-block, the encoder 140 encodes a CBP32 flag indicating that there is no non-zero quantized transform coefficient in the sub-block (S3330). Since there are no non-zero quantized transform coefficients in the sub-block, the transform type and CBP are not coded.
此外，如果在步骤S3302中确定子块的大小是64x64,64x32和32x64中的一个，则编码器140确定子块中是否存在非零量化变换系数(S3330)。如果子块中存在非零量化变换系数，则编码器140对指示子块中存在非零量化变换系数的CBP64标记进行编码(S3334)。处理进行至步骤S3306。如果子块中不存在非零量化变换系数，则编码器140对指示子块中不存在非零量化变换系数的CBP64标记进行编码(S3336)。由于子块中不存在非零量化变换系数，所以变换类型和CBP不被编码。Also, if it is determined in step S3302 that the size of the sub-block is one of 64x64, 64x32, and 32x64, the encoder 140 determines whether non-zero quantized transform coefficients exist in the sub-block (S3330). If non-zero quantized transform coefficients exist in the sub-block, the encoder 140 encodes a CBP64 flag indicating that non-zero quantized transform coefficients exist in the sub-block (S3334). The process proceeds to step S3306. If there is no non-zero quantized transform coefficient in the sub-block, the encoder 140 encodes a CBP64 flag indicating that there is no non-zero quantized transform coefficient in the sub-block (S3336). Since there are no non-zero quantized transform coefficients in the sub-block, the transform type and CBP are not coded.
这样，在对CBPX(X：大于或等于32的整数)和CBP进行编码之后，视频编码设备100的编码器140对仅针对具有非零量化变换系数的块的量化变换系数进行编码，以生成编码视频数据，并且将其分配给变换系数字段。对量化变换系数进行编码的方法与上述对帧内宏块的量化变换系数进行编码的方法相同。In this way, after encoding CBPX (X: an integer greater than or equal to 32) and CBP, the encoder 140 of the video encoding apparatus 100 encodes only quantized transform coefficients for blocks with non-zero quantized transform coefficients to generate encoded video data and assign it to the transform coefficient field. The method of encoding the quantized transform coefficients is the same as the above-mentioned method of encoding the quantized transform coefficients of the intra macroblock.
在本发明的实施方式中，CABAC(基于上下文的自适应可变长度码)可以用作可变长度编码方法，并且CABAC编码包括二进制算术编码单元和上下文计算单元。In an embodiment of the present invention, CABAC (Context-Based Adaptive Variable Length Code) may be used as a variable length coding method, and CABAC coding includes a binary arithmetic coding unit and a context computing unit.
二进制算术编码单元是用于转换为二进制信号(0或1信号)的二进制化单元。上下文计算单元保留要编码的二进制信号的多个出现概率(上下文表)，并且在根据当前的编码目标和周围情况改变二进制信号的出现概率的同时将二进制信号的出现概率传送到二进制算术编码单元。这里，上下文表在片编码/解码的开始进行初始化，并且在片中进行更新。各个句法具有根据ctxIdx(上下文编号)不同的表(出现概率)，并且通过式7计算ctxIdx。The binary arithmetic coding unit is a binarization unit for conversion into a binary signal (0 or 1 signal). The context calculation unit holds a plurality of occurrence probabilities (context table) of binary signals to be encoded, and transfers the occurrence probabilities of binary signals to the binary arithmetic encoding unit while changing the occurrence probabilities of binary signals according to current encoding targets and surrounding conditions. Here, the context table is initialized at the start of slice encoding/decoding, and is updated within a slice. Each syntax has a different table (appearance probability) according to ctxIdx (context number), and ctxIdx is calculated by Equation 7.
ctxldx＝ctxIdx_offset+ctx_inc 式7ctxldx＝ctxIdx_offset+ctx_inc Formula 7
这里，ctx_offset是通过图片类型和句法类型确定的。在该情况下，interX_flag可以更好的使用根据X值变化的表，或者可以使用相同的表。当对目标句法进行编码时，根据周围条件来确定ctx_inc，并且按照句法的ctx_inc的示例被例示在图35至图37中。例如，在skipX_flag的情况下，如图35所示，ctx_inc＝0可以指示上侧相邻块和左侧相邻块全都不是跳过模式。ctx_inc＝1可以指示上侧相邻块或左侧相邻块是跳过模式。ctx_inc＝2可以指示上侧相邻块和左侧相邻块全都是跳过模式。上下文表可以根据各个ctx_inc来不同地使用。类似地，在interX_flag的情况下，如图36所示，ctx_inc 0至5指示针对作为32x32像素块的扩展宏块的上下文表的预测类型的示例，ctx_inc 7至9指示针对16x16像素块的上下文表的预测类型的示例，并且上下文表可以根据ctx_inc来不同地使用。类似地，图37中例示了针对对directX_flag进行编码/解码的ctx_inc的示例。如以上表8所示，可以通过1比特来表示directX_flag的编码比特，并且可以根据编码目标块的上侧块和左侧块不是直接模式的情况、上侧块和左侧块都是直接模式的情况以及上侧块和左侧块中仅一个是直接模式的情况来不同地确定上下文表。此外，类似图37所示，在针对对extended_flag进行编码/解码的ctx_inc的情况下，上下文表可以根据编码目标块的相邻块是已经按照扩展宏块为单位进行了编码(在该情况下，extended_mb_flag可以被设定为二进制比特1)还是已经按照像素块为单位进行了编码(在该情况下，extended_mb_flag可以被设定为二进制比特1)来不同地使用。这样，在interX_flag的情况下，不同的上下文表可以根据X值、图片类型、相邻块的数据以及是否将其用作基准图片来进行使用。在本发明的实施方式中，上下文表可以根据宏块中的划分子块当中的要编码的当前块的相邻块的上下文来选择性地使用。在该情况下，可以考虑要编码的当前块的相邻块是否为跳过模式、它们是否为按照宏块为单位的跳过模式以及它们是否为按照子块为单位的跳过模式，并且可以根据这些周围块的情况来选择性地使用要编码的块的skipX_flag,an interX_flag和extended_mb_flag的上下文表。Here, ctx_offset is determined by the picture type and syntax type. In that case, interX_flag could better use a table that changes according to the X value, or could use the same table. When encoding the target syntax, ctx_inc is determined according to surrounding conditions, and examples of ctx_inc by syntax are illustrated in FIGS. 35 to 37 . For example, in the case of skipX_flag, as shown in FIG. 35 , ctx_inc=0 may indicate that neither the upper side neighboring block nor the left side neighboring block is in skip mode. ctx_inc=1 may indicate that the upper neighboring block or the left neighboring block is skip mode. ctx_inc=2 may indicate that both the upper and left neighboring blocks are in skip mode. The context table can be used differently according to each ctx_inc. Similarly, in the case of interX_flag, as shown in FIG. 36 , ctx_inc 0 to 5 indicate an example of the prediction type for a context table of an extended macroblock which is a 32x32 pixel block, and ctx_inc 7 to 9 indicate a context table for a 16x16 pixel block An example of the type of prediction for , and the context table can be used differently depending on ctx_inc. Similarly, an example for ctx_inc encoding/decoding directX_flag is illustrated in FIG. 37 . As shown in Table 8 above, the coding bit of directX_flag can be represented by 1 bit, and according to the fact that the upper side block and the left side block of the encoding target block are not direct mode, the upper side block and the left side block are both direct mode The context table is determined differently for the case and the case where only one of the upper and left blocks is direct mode. In addition, as shown in FIG. 37 , in the case of ctx_inc for encoding/decoding extended_flag, the context table may be based on whether the adjacent blocks of the encoding target block have been encoded in units of extended macroblocks (in this case, The extended_mb_flag may be set to binary bit 1) or has been encoded in units of pixel blocks (in this case, extended_mb_flag may be set to binary bit 1) to be used differently. Thus, in the case of interX_flag, different context tables can be used depending on the X value, picture type, data of neighboring blocks and whether it is used as a reference picture. In an embodiment of the present invention, the context table may be selectively used according to the context of adjacent blocks of a current block to be encoded among divided sub-blocks in a macroblock. In this case, it may be considered whether the adjacent blocks of the current block to be encoded are in skip mode, whether they are in skip mode in units of macroblocks, and whether they are in skip mode in units of subblocks, and may The context table of skipX_flag, an interX_flag and extended_mb_flag of the block to be encoded is selectively used according to the conditions of these surrounding blocks.
图38和图39是用于描述根据本发明的实施方式对分割类型信息进行编码的处理的例示图。38 and 39 are illustration diagrams for describing a process of encoding division type information according to an embodiment of the present invention.
图38例示了用于利用树结构对分割类型信息进行编码的要编码的扩展宏块中的各个16x16像素块的分割类型值。Fig. 38 illustrates partition type values for respective 16x16 pixel blocks in an extended macroblock to be encoded for encoding partition type information using a tree structure.
在图38A中，Mx(a,b)表示与要编码的扩展宏块中的位置(a,b)对应的16x16像素块的分割类型值。即，Mx(0,0)表示与要编码的扩展宏块中的位置(0,0)对应的16x16像素块(即，扩展宏块中的按照光栅扫描方向的第一16x16像素块)的分割类型值，并且Mx(0,1)表示与要编码的扩展宏块中的位置(0,1)对应的16x16像素块(即，扩展宏块中的按照光栅扫描方向的第二16x16像素块)的分割类型值。In FIG. 38A , Mx(a,b) represents the division type value of the 16x16 pixel block corresponding to the position (a,b) in the extended macroblock to be encoded. That is, Mx(0,0) represents the division of the 16x16 pixel block corresponding to position (0,0) in the extended macroblock to be coded (i.e., the first 16x16 pixel block in the extended macroblock in the raster scan direction) type value, and Mx(0,1) represents the 16x16 pixel block corresponding to position (0,1) in the extended macroblock to be encoded (i.e., the second 16x16 pixel block in the raster scan direction in the extended macroblock) The split type value for .
图38A所示的16x16像素块的分割类型值按照预定区域为单位进行分组，例如，按照包括KxL分割类型值的区域为单位进行分组，并且分组区域中的分割类型值当中的最小值被选择并分配为针对分组区域的分割类型值。当分组区域中的分割类型值是Mx-1(c,d)时，c＝a/K并且d＝b/L。这里，Mx-1是Mx的上级节点，并且可以通过选择上级节点的值的方法来选择最大值。此外，其可以通过对选择上级节点的值的方法进行编码来自适应地进行使用。The division type values of the 16x16 pixel block shown in FIG. 38A are grouped in units of predetermined areas, for example, grouped in units of areas including KxL division type values, and the minimum value among the division type values in the grouping area is selected and Assigned as the split type value for the grouped region. When the division type value in the grouping area is Mx-1(c,d), c=a/K and d=b/L. Here, Mx-1 is the upper-level node of Mx, and the maximum value can be selected by selecting the value of the upper-level node. In addition, it can be used adaptively by encoding a method of selecting a value of a superior node.
图38B例示了按照包括2x2分割类型值的区域为单位对图38A所示的16x16像素块的分割类型值进行分组，接着将各个分组中的分割类型值当中的最小值分配作为分组区域的分割类型值的结果。包括图38A所示的16x16像素块(0,0),(0,1),(1,0)和(1,1)的像素块的分割类型值按照2x2区域为单位进行分组，并且16x16像素块的分割类型值MX(0,0),MX(0,1),MX(1,0)和MX(1,1)当中的最小值被选择并分配作为针对分组区域的分割类型值MX-1(0,0)。此外，包括16x16像素块(0,2),(0,3),(1,2)和(1,3)的像素块的分割类型值按照2x2区域为单位进行分组，并且16x16像素块的分割类型值MX(0,2),MX(0,3),MX(1,2)和MX(1,3)当中的最小值被选择并分配作为针对分组区域的分割类型值MX-1(0,1)。这样，对剩余的16x16像素块进行相同的处理。当直到层MX-2,MX-3…M1为止重复将图38A所示的16x16像素块的分割类型值分配作为图38B所示的分组区域的分割类型值的处理时，分组区域的分割类型值可以分配为如图38C所示。当直到层M0为止再次重复该处理时，进行分组以使得具有如图38D所示的一个分组区域的分割类型值。FIG. 38B illustrates grouping the division type values of the 16x16 pixel block shown in FIG. 38A in units of areas including 2x2 division type values, and then assigning the minimum value among the division type values in each group as the division type of the grouped area value result. The division type values of the pixel blocks including the 16x16 pixel blocks (0,0), (0,1), (1,0) and (1,1) shown in FIG. 38A are grouped in units of 2x2 regions, and the 16x16 pixel The smallest value among the partition type values MX (0,0), MX (0,1), MX (1,0) and MX (1,1) of the block is selected and assigned as the partition for the grouping area Type value MX-1 (0,0). In addition, the division type values of pixel blocks including 16x16 pixel blocks (0,2), (0,3), (1,2) and (1,3) are grouped in units of 2x2 regions, and the division of 16x16 pixel blocks The smallest value among the type values MX (0,2), MX (0,3), MX (1,2) and MX (1,3) is selected and assigned as the segmentation type value M for the grouping areaX-1 (0,1). This way, the same process is done for the remaining 16x16 pixel blocks. When the process of assigning the division type value of the 16×16 pixel block shown in FIG. 38A as the division type value of the grouped area shown in FIG. 38B is repeated up to layers MX-2 , M X-3 . . . Split type values can be assigned as shown in Figure 38C. When the process is repeated again up to layerM0 , grouping is performed so as to have a division type value of one grouping area as shown in FIG. 38D.
尽管图38例示了按照2x2区域为单位对相邻16x16像素块的分割类型值进行分组，但是本发明不限于此。例如，针对相邻16x16像素块的分割类型值可以按照诸如4x2和2x4的各种大小的区域为单位进行分组，或者非相邻16x16像素块的分割类型值可以按照预定区域为单位进行分组，以分配分割类型值。Although FIG. 38 exemplifies grouping partition type values of adjacent 16x16 pixel blocks in units of 2x2 regions, the present invention is not limited thereto. For example, partition type values for adjacent 16x16 pixel blocks may be grouped in units of regions of various sizes such as 4x2 and 2x4, or partition type values for non-adjacent 16x16 pixel blocks may be grouped in units of predetermined regions to Assign a split type value.
将要编码的16x16像素块的分割类型值分配作为如图38所示的分组区域的分割类型值可以按照如图39所示的树结构来表示。Assigning the division type value of the 16x16 pixel block to be coded as the division type value of the grouping area as shown in FIG. 38 can be expressed in a tree structure as shown in FIG. 39 .
图39例示了按照树结构的逐层分组区域的分割类型值。如果逐层分组区域的分割类型值是节点值，则可以通过对相对于上级节点的节点值的差进行编码来对按照图39中的树结构的节点值进行编码。Fig. 39 exemplifies division type values of layer-by-layer grouping regions in a tree structure. If the division type value of the layer-by-layer grouping area is a node value, the node value in the tree structure in FIG. 39 can be encoded by encoding the difference from the node value of the upper node.
对节点值和上级节点的节点值之间的差值进行编码的方法对与该差值对应的二进制比特0进行编码并且对最后一个二进制比特1进行编码。如果上级节点的节点值与要编码的节点的节点值之间的差是0，则对二进制比特1进行编码。例如，算术编码方法可以用于对二进制比特0和1进行解码。在该情况下，在各个层中可以使用不同的上下文。The method of encoding a difference between a node value and a node value of a superior node encodes a binary bit 0 corresponding to the difference and encodes the last binary bit 1 . Binary bit 1 is encoded if the difference between the node value of the parent node and the node value of the node to be encoded is 0. For example, an arithmetic coding method can be used to decode binary bits 0 and 1. In this case, different contexts can be used in each layer.
这样，当树结构用于对分割类型值(即，节点值)进行编码时，可以如上所述利用二进制比特0和1对0和最上级节点的节点值(下文称作“最上级节点值”)之间的差值进行编码。在参照图18所述分配分割类型编号时，如果按照出现频率的降序分配大分割类型编号，则可以利用二进制比特0和1对最上级节点值与最大分割类型编号之间的差值进行编码。另一方面，在参照图18所述分配分割类型编号时，如果按照出现频率的降序分配小分割类型编号，则可以利用二进制比特0和1对最上级节点值与最小分割类型编号之间的差值进行编码。Thus, when a tree structure is used to encode a partition type value (i.e., a node value), the binary bits 0 and 1 can be used for 0 and the node value of the uppermost node (hereinafter referred to as "the uppermost node value") as described above. ) to encode the difference between them. When assigning division type numbers as described with reference to FIG. 18 , if large division type numbers are allocated in descending order of appearance frequency, the difference between the uppermost node value and the largest division type number can be encoded with binary bits 0 and 1. On the other hand, when assigning the division type numbers as described with reference to FIG. 18, if the small division type numbers are allocated in descending order of appearance frequency, the difference between the uppermost node value and the smallest division type number can be utilized by binary bits 0 and 1. value is encoded.
在除了最上级节点值的剩余节点值的情况下，可以利用二进制比特0和1对要编码的节点的节点值与该节点的上级节点的节点值之间的差值进行编码。即，可以通过对与差值一样多的二进制比特0进行编码并且对最后一个二进制比特1进行编码来对各个节点值进行编码。在该情况下，如果要编码的节点的节点值与该节点的上级节点的节点值之间的差值是0，则对二进制比特1进行编码。此外，相反，可以通过对与差值一样多的二进制比特1进行编码并且对最后一个二进制比特0进行编码来对各个节点值进行编码。在该情况下，如果要编码的节点的节点值与该节点的上级节点的节点值之间的差值是0，则对二进制比特0进行编码。In the case of the remaining node values except for the uppermost node value, the difference between the node value of the node to be encoded and the node value of the node's upper node may be encoded with binary bits 0 and 1. That is, each node value may be encoded by encoding as many binary bits of 0 as the difference value and encoding the last binary bit of 1 . In this case, a binary bit 1 is encoded if the difference between the node value of the node to be encoded and the node value of the node's superior node is 0. Also, instead, each node value may be encoded by encoding as many binary ones as the difference value and encoding the last binary bit zero. In this case, if the difference between the node value of the node to be encoded and the node value of the node's superior node is 0, a binary bit 0 is encoded.
然而，当对各个节点值进行编码时，如果上级节点值是针对分割类型编号可用的最大值，则相关上级节点的下级节点的节点值不被编码。例如，如果节点M1(0,0)的节点值是3，则作为节点M1(0,0)的下级节点M2(0,0),M2(0,1),M2(1,0)和M2(1,1)的节点的节点值不被编码。即，由于M1(0,0)是M2(0,0),M2(0,1),M2(1,0)和M2(1,1)中的最小值，所以M2(0,0),M2(0,1),M2(1,0)和M2(1,1)全都具有大于或等于3的值。然而，在图18中，由于分割类型编号的最大值是3，所以M2(0,0),M2(0,1),M2(1,0)和M2(1,1)无需被编码，因为它们不能具有除3以外的值。However, when each node value is encoded, if the upper node value is the maximum value available for the division type number, the node values of the lower nodes of the relevant upper node are not encoded. For example, if the node value of node M1(0,0) is 3, then the subordinate nodes M2 (0,0),M2 (0,1),M2 (1,0 ) and nodes of M2 (1,1) are not coded. That is, since M1 (0,0) is the smallest value among M2 (0,0), M2 (0,1), M2 (1,0) and M2 (1,1), M2 (0,0), M2 (0,1), M2 (1,0), and M2 (1,1) all have values greater than or equal to 3. However, in Figure 18, since the maximum value of the division type number is 3, M2 (0,0), M2 (0,1), M2 (1,0) and M2 (1,1) need not are encoded because they cannot have a value other than 3.
此外，当要编码的节点与该节点的上级节点之间的差值，如果要编码的当前节点的节点值是对于分割类型编号可用的最大值，则仅对与差值一样多的二进制比特0进行编码，并且不对最后一个二进制比特1进行编码。例如，如果要编码的当前节点的上级节点的节点值M1(0,0)和要编码的当前节点的节点值M2(0,0),M2(0,1),M2(1,0)和M2(1,1)分别是1,2,3,3和2时，节点值M2(0,0)和M2(1,1)对二进制比特01进行编码，并且节点值M2(0,1)和M2(1,0)对二进制比特00而不对二进制比特001进行编码。Also, when the difference between the node to be encoded and the node's parent node, if the node value of the current node to be encoded is the maximum value available for the split type number, only as many binary bits as the difference are 0 is encoded, and the last binary bit 1 is not encoded. For example, if the node value M1 (0,0) of the parent node of the current node to be encoded and the node value M2 (0,0) of the current node to be encoded,M2 (0,1),M2 (1 ,0) and M2 (1,1) are 1,2,3,3 and 2 respectively, the node values M2 (0,0) and M2 (1,1) encode the binary bit 01, and the node The values M2 (0,1) and M2 (1,0) encode binary bit 00 and not binary bit 001 .
此外，具有相同上级节点的节点当中的最后一个节点的节点值被编码时，如果除了最后一个节点的节点的节点值大于上级节点的节点值，则最后一个节点的节点值不被编码。例如，如果要编码的节点的上级节点的节点值M1(0,0)和要编码的节点的节点值M2(0,0),M2(0,1),M2(1,0)和M2(1,1)分别是1,2,3,3和1，则由于节点值M2(0,0),M2(0,1)和M2(1,0)全都大于节点值M1(0,0)，所以最后一个节点的节点值M2(1,1)不被编码。Also, when the node value of the last node among nodes having the same upper node is encoded, if the node value of the nodes other than the last node is larger than the node value of the upper node, the node value of the last node is not encoded. For example, if the node value of the parent node of the node to be coded is M1 (0,0) and the node value of the node to be coded is M2 (0,0),M2 (0,1),M2 (1,0 ) and M2 (1,1) are 1, 2, 3, 3 and 1 respectively, then since the node values M2 (0,0), M2 (0,1) and M2 (1,0) are all greater than The node value M1 (0,0), so the node value M2 (1,1) of the last node is not encoded.
如上所述，可以通过利用二进制比特0和1对要编码的节点的节点值与该节点的上级节点的节点值之间的差值进行编码来对除了最上级节点的剩余节点的节点值进行编码。然而，与此不同，可以通过对各个节点的节点值与分割类型的出现频率最高的分割类型值之间的差值进行编码来对除了最上级节点的剩余节点的节点值进行编码。这里，分割类型的出现频率最高的分割类型值可以是固定值，也可以不是固定值。如果分割类型的出现频率最高的分割类型值不是固定值，则其可以在编码之后发送到相关解码器，也可以不发送到相关解码器。如果分割类型的出现频率最高的分割类型值不被发送到相关解码器，则在当前块之前编码的块的统计可以积累，并且可以使用直到现在为止出现频率最高的模式。在利用树结构对分割类型信息进行编码的另一实施方式中，当图38A所示的分割类型值被分组以确定图38B所示的分组区域的分割类型值时，如果按照出现频率的降序分配大分割类型编号，则分组区域的值当中的最大值可以用作分组区域的分割类型值。As described above, the node values of the remaining nodes except the uppermost node can be encoded by encoding the difference between the node value of the node to be encoded and the node value of the node's upper node with binary bits 0 and 1 . However, unlike this, the node values of the remaining nodes other than the uppermost node may be encoded by encoding the difference between the node values of the respective nodes and the division type value whose appearance frequency is the highest for the division type. Here, the value of the segmentation type with the highest occurrence frequency of the segmentation type may or may not be a fixed value. If the division type value with the highest occurrence frequency of the division type is not a fixed value, it may or may not be sent to the relevant decoder after encoding. If the most frequently occurring value of the partition type for the partition type is not sent to the relevant decoder, the statistics of the blocks encoded before the current block can be accumulated and the most frequent pattern up to now can be used. In another embodiment in which the division type information is encoded using a tree structure, when the division type values shown in FIG. 38A are grouped to determine the division type values of the grouped regions shown in FIG. If the division type number is large, the maximum value among the values of the grouping area can be used as the division type value of the grouping area.
在本发明的实施方式中，如果图18所示的分割类型值被用来将扩展宏块划分为16x16像素块(即，extended_mb_flag＝0)，则利用上述方法对分割类型进行编码/解码。在该情况下，最低节点的值可以是扩展宏块中的16x16像素块的分割类型值，并且下级节点的值中的最大值可以用作上级节点的代表性值。例如，如图40所示，如果32x32像素块的扩展宏块属于被填充为16的图像，并且被划分为16x16像素块，则由于最上级节点的代表性值是0，则通过表示代表性值“0”与分割类型的最大值“3”之间的差值“3”的二进制比特“000”来对最上级节点进行编码。作为另一示例，如图41所示，如果32x32像素块的扩展宏块属于被填充为16的图像；并且16x16像素块被划分为8x8或更少像素块接着被编码；并且一个16x16像素块被编码为16x16像素块，则由于最上级节点的代表性值是3，所以对表示最上级节点的代表性值“3”与分割类型的最大值“3”之间的差值的二进制比特“1”进行编码。这里，如果下级节点的值与上级节点的值之间的差值为0，则二进制比特“000”被编码。In an embodiment of the present invention, if the partition type value shown in FIG. 18 is used to divide the extended macroblock into 16x16 pixel blocks (ie, extended_mb_flag=0), the above method is used to encode/decode the partition type. In this case, the value of the lowest node may be a division type value of a 16x16 pixel block in the extended macroblock, and the maximum value among the values of the lower nodes may be used as a representative value of the upper node. For example, as shown in Figure 40, if an extended macroblock of a 32x32 pixel block belongs to an image padded with 16 and is divided into 16x16 pixel blocks, since the representative value of the uppermost node is 0, the representative value is represented by Binary bits "000" of the difference "3" between "0" and the maximum value "3" of the division type are used to encode the uppermost node. As another example, as shown in Figure 41, if an extended macroblock of 32x32 pixel blocks belongs to an image that is padded to 16; and a 16x16 pixel block is divided into 8x8 or less pixel blocks and then encoded; and a 16x16 pixel block is Encoded as a 16x16 pixel block, since the representative value of the uppermost node is 3, the binary bit "1" representing the difference between the representative value "3" of the uppermost node and the maximum value "3" of the division type " to encode. Here, if the difference between the value of the lower node and the value of the upper node is 0, binary bits '000' are encoded.
如参照图6至图10所述，帧内预测模式的编码对指示亮度分量的4x4和8x8像素块的帧内预测模式是否与利用相邻块预测的预测帧内模式相同的1比特标记进行编码，并且如果帧内预测模式与预测的帧内模式不同，则使用3比特来指示帧内预测模式对应于九个预测模式当中除预测的预测方向以外的八个预测模式中的哪一个。此外，2比特被用来指示16x16像素块使用四个预测模式中的哪一个。此外，帧内预测模式的编码对指示除4x4像素块、8x8像素块和16x16像素块以外的剩余mxn像素块的帧内预测模式是否与利用相邻块预测的预测帧内模式相同的1比特标记进行编码，并且如果帧内预测模式与预测的帧内模式不同，则使用1比特来指示帧内预测模式对应于三个预测模式当中除预测的预测方向以外的两个预测模式中的哪一个。此外，2比特被用来指示色度分量的16x16像素块使用四个预测模式中的哪一个。这里，亮度分量和色度分量的4x4,8x8,16x16和mxn像素块可以使用不同的上下文表。As described with reference to FIGS. 6 to 10 , the encoding of the intra prediction mode encodes a 1-bit flag indicating whether the intra prediction mode of the 4x4 and 8x8 pixel blocks of the luma component is the same as the predicted intra mode predicted with the neighboring block , and if the intra prediction mode is different from the predicted intra mode, 3 bits are used to indicate which of the eight prediction modes other than the predicted prediction direction among the nine prediction modes the intra prediction mode corresponds to. Also, 2 bits are used to indicate which of the four prediction modes is used for the 16x16 pixel block. In addition, the encoding of the intra prediction mode is a 1-bit flag indicating whether the intra prediction mode of the remaining mxn pixel blocks other than the 4x4 pixel block, the 8x8 pixel block, and the 16x16 pixel block is the same as the predicted intra mode predicted with the neighboring block Encoding is performed, and if the intra prediction mode is different from the predicted intra mode, 1 bit is used to indicate which of the two prediction modes other than the prediction direction of prediction among the three prediction modes the intra prediction mode corresponds to. In addition, 2 bits are used to indicate which of the four prediction modes is used for the 16x16 pixel block of the chroma component. Here, different context tables may be used for 4x4, 8x8, 16x16 and mxn pixel blocks of luma and chroma components.
图42是例示用于对双向预测进行编码/解码的ctx_inc的示例的图。如上所述，根据当对目标句法进行编码时的周围情况来确定ctx_inc。表示双向预测的B_pdir的编码比特与表9所示相同。如果作为L0预测的相邻块是0；作为L1预测的相邻块是1；并且假设双向预测的相邻块是2，则可以根据如图42所示的周围情况来确定目标句法。在该情况下，如果左侧块和上侧块不可用，则L0预测(0)可以用作不可用预测方向。不同的上下文表可以用于图43所示的各个ctx_inc。Fig. 42 is a diagram illustrating an example of ctx_inc used for encoding/decoding bidirectional prediction. As described above, ctx_inc is determined according to the surrounding conditions when encoding the target syntax. The coded bits of B_pdir representing bidirectional prediction are the same as those shown in Table 9. If the adjacent block predicted as L0 is 0; the adjacent block predicted as L1 is 1; and assuming that the adjacent block predicted bidirectionally is 2, the target syntax can be determined according to the surrounding conditions as shown in FIG. 42 . In this case, if the left and upper blocks are unavailable, L0 prediction (0) may be used as an unavailable prediction direction. Different context tables can be used for each ctx_inc shown in Figure 43.
在用于对CBPX_flag进行编码/解码的ctx_inc的情况下，可以按照与图37的情况类似的方式确定上下文表。例如，如果扩展宏块或帧内图片是帧内模式，则不对CBPX_flag进行编码。如果要编码的当前块的上侧块或左侧块是帧内模式的情况被设定为0；并且上侧块或左侧块是帧间模式的情况被设定为1，则可以通过根据周围情况的上下文表来确定目标句法。In the case of ctx_inc for encoding/decoding CBPX_flag, the context table can be determined in a similar manner to the case of FIG. 37 . For example, if the extended macroblock or intra picture is Intra mode, then the CBPX_flag is not encoded. If the upper block or the left block of the current block to be encoded is set to 0 in the intra mode; and the upper block or the left block is set to 1 in the inter mode, you can pass according to A context table of the surrounding situations is used to determine the target syntax.
图43是例示用于对变换类型进行编码/解码的ctx_inc的示例的图。图43例示了在两个可用变换类型的情况下的示例。如表1所示，如果16x16像素块被划分为四个8x8像素子块，则存在诸如4x4变换和8x8变换的两个可用变换类型。因此，可以根据当前块的左侧块和上侧块这两者是4x4变换的情况、上侧块和左侧块这两者是8x8变换的情况以及上侧块和左侧块中的一个是4x4变换而另一个是8x8变换的情况来通过不同的上下文表确定目标句法。FIG. 43 is a diagram illustrating an example of ctx_inc for encoding/decoding a transform type. Fig. 43 illustrates an example in the case of two available transform types. As shown in Table 1, if a 16x16 pixel block is divided into four 8x8 pixel sub-blocks, there are two available transform types such as 4x4 transform and 8x8 transform. Therefore, according to the case where both the left block and the upper block of the current block are 4x4 transforms, the case where both the upper block and the left block are 8x8 transforms, and one of the upper block and the left block is 4x4 transformation and another 8x8 transformation to determine the target syntax through different context tables.
图44和图45是例示用于在三个可用变换类型的情况下对变换类型进行编码/解码的ctx_inc的示例的图。图44例示了用于选择用于对第一比特进行编码的上下文表的ctx_inc的示例。图45例示了用于选择用于对第二比特进行编码的上下文表的ctx_inc的示例。44 and 45 are diagrams illustrating an example of ctx_inc for encoding/decoding a transform type in the case of three available transform types. FIG. 44 illustrates an example of ctx_inc for selecting a context table for encoding the first bit. FIG. 45 illustrates an example of ctx_inc for selecting a context table for encoding the second bit.
如上所述，如果当前块被划分为诸如4x4像素块、4x8像素块或8x4像素块的8x8或更少像素块，则对其进行4x4变换。此外，对8x8像素块进行4x4变换和8x8变换当中编码成本较低的变换；对8x16像素块进行4x4变换、8x8变换和8x16变换当中编码成本较低的变换；对16x8像素块进行4x4变换、8x8变换和16x8变换当中编码成本较低的变换；并且对16x16或更多像素块进行4x4变换、8x8变换和16x16变换当中编码成本较低的变换。因此，在对第一比特进行编码时，如图44所示，可以根据当前块的左侧块和上侧块这两者是否为4x4变换块、左侧块和上侧块中是否仅一个为4x4变换块或者左侧块和上侧块这两者是否都不是4x4变换块来不同地使用用于对变换类型进行编码/解码的ctx_inc。类似地，在对第二比特进行编码时，如图45所示，可以根据左侧块和上侧块中的任一个是否为8x8变换块来不同地使用用于对变换类型进行编码/解码的ctx_inc。As described above, if the current block is divided into 8x8 or less pixel blocks such as 4x4 pixel blocks, 4x8 pixel blocks, or 8x4 pixel blocks, it is 4x4 transformed. In addition, 4x4 transformation and 8x8 transformation are performed on 8x8 pixel blocks with lower coding costs; 8x16 pixel blocks are transformed with lower coding costs among 4x4 transformation, 8x8 transformation and 8x16 transformation; 16x8 pixel blocks are 4x4 transformation, 8x8 Transform and 16x8 transform with lower encoding cost; and 4x4 transform, 8x8 transform, and 16x16 transform for 16x16 or more pixel blocks with lower encoding cost. Therefore, when encoding the first bit, as shown in FIG. 44 , it may be based on whether the left block and the upper block of the current block are both 4x4 transform blocks, and whether only one of the left block and the upper block is Whether a 4x4 transform block or both the left block and the upper side block are not 4x4 transform blocks to use ctx_inc for encoding/decoding the transform type differently. Similarly, when encoding the second bit, as shown in FIG. 45 , the values for encoding/decoding the transform type can be used differently depending on whether any of the left block and the upper block is an 8x8 transform block. ctx_inc.
图46是例示根据本发明的实施方式的视频解码设备的示意性构造的框图。FIG. 46 is a block diagram illustrating a schematic configuration of a video decoding device according to an embodiment of the present invention.
根据本发明的实施方式的视频解码设备4600可以包括解码器4610、逆量化器/逆变换器4620、预测器4630、加法器4640、滤波器4650和图片缓冲器4660。A video decoding apparatus 4600 according to an embodiment of the present invention may include a decoder 4610 , an inverse quantizer/inverse transformer 4620 , a predictor 4630 , an adder 4640 , a filter 4650 and a picture buffer 4660 .
编码器4610使用诸如熵解码的各种编码/解码技术来对由视频编码设备100编码的编码数据进行解码。由解码器4610解码的数据根据要解码的图片的类型而变化。关于这种图片类型的信息可以从图片头或编码数据的序列头中提取，或者可以从其提取并解码以进行重构。这里，如果当前要解码的扩展宏块位于如图2所示的整个图像边界上(在序列参数中发送输入图像的实际大小)，则找到属于扩展宏块中的图像的部分，并且通过熵解码获得相关区域的预测模式和文本数据，并且进行帧内预测或运动补偿以及逆变换/逆量化以对属于实际图像的部分进行解码。The encoder 4610 decodes encoded data encoded by the video encoding apparatus 100 using various encoding/decoding techniques such as entropy decoding. Data decoded by the decoder 4610 varies according to the type of picture to be decoded. Information about this picture type can be extracted from the picture header or the sequence header of encoded data, or can be extracted from it and decoded for reconstruction. Here, if the currently extended macroblock to be decoded is located on the boundary of the entire image as shown in Figure 2 (the actual size of the input image is sent in the sequence parameters), then find the part of the image that belongs to the extended macroblock, and pass entropy decoding The prediction mode and text data of the relevant area are obtained, and intra prediction or motion compensation and inverse transformation/inverse quantization are performed to decode the part belonging to the actual image.
如果要解码的图片是帧内图片，则视频编码数据中的要由解码器4610解码的数据的类型和序列与如图17所示相同。即，解码器4610通过对从输入比特流提取的分割类型数据进行解码来重构分割类型信息，利用由重构的分割类型信息识别的分割类型来划分宏块，通过对帧内预测模式数据进行解码来重构与宏块中划分的子块的数量一样多的帧内预测模式，并且通过根据各个子块的大小按照与图24所示的序列相同或相似的序列对分配给CBPX标记/CBP的CBP数据进行解码来重构CBPX标记或CBP。此后，解码器4610通过对仅针对由CBP指示的具有非零变换系数的子块的变换系数数据进行解码来重构变换系数。这里，如果视频编码设备100已经按照16x16像素块为单位对扩展宏块进行了帧内预测编码，则可以省略对CBPX标记的解码。在该情况下，可以按照扩展宏块为单位对编码的量化参数进行解码。If the picture to be decoded is an intra picture, the type and sequence of data to be decoded by the decoder 4610 in video encoded data are the same as shown in FIG. 17 . That is, the decoder 4610 reconstructs the partition type information by decoding the partition type data extracted from the input bitstream, divides the macroblock using the partition type identified by the reconstructed partition type information, and divides the macroblock by performing the intra prediction mode data Decoding to reconstruct as many intra prediction modes as the number of sub-blocks divided in the macroblock, and assigning to CBPX flag/CBP by the same or similar sequence pair as the sequence shown in Fig. 24 according to the size of each sub-block The CBP data is decoded to reconstruct the CBPX token or CBP. Thereafter, the decoder 4610 reconstructs transform coefficients by decoding transform coefficient data for only subblocks indicated by the CBP having non-zero transform coefficients. Here, if the video encoding apparatus 100 has intra-predictively encoded the extended macroblock in units of 16x16 pixel blocks, decoding of the CBPX flag may be omitted. In this case, encoded quantization parameters may be decoded in units of extended macroblocks.
然而，图17所示的视频编码数据的句法结构仅是示例性的，并且根据本发明的实施方式的视频编码数据的句法结构不限于图17所示的结构。此外，图24所示的CBPX标记或CBP的编码序列仅是示例性的，并且根据本发明的实施方式的CBPX标记或CBP的重构序列不限于图24所示的序列。However, the syntax structure of video encoding data shown in FIG. 17 is only exemplary, and the syntax structure of video encoding data according to an embodiment of the present invention is not limited to the structure shown in FIG. 17 . In addition, the coding sequence of the CBPX marker or CBP shown in FIG. 24 is only exemplary, and the reconstructed sequence of the CBPX marker or CBP according to the embodiment of the present invention is not limited to the sequence shown in FIG. 24 .
这里，由解码器4610进行的帧内预测模式解码方法根据子块的大小而变化。即，如果子块的大小不是16x16而是4x4或8x8，则解码器4610通过对从比特流中提取的帧内预测模式数据当中的1比特进行解码来重构预测模式标记。如果预测模式标记指示与相关子块的帧内预测模式对应的预测方向和与从相邻块估计的相关子块的帧内预测模式对应的预测方向相同，则解码器4610将从相邻块估计的相关子块的帧内预测模式重构为当前块的帧内预测模式，并且对下一子块的帧内预测模式进行解码。如果预测模式标记指示与相关子块的帧内预测模式对应的预测方向和与从相邻块估计的相关子块的帧内预测模式对应的预测方向不同，则解码器4610将通过对帧内预测模式数据的下三个比特进行解码重构的帧内预测模式重构为当前块的帧内预测模式。此外，如果子块的大小是16x16，则解码器4610通过对从比特流中提取帧内预测模式数据当中的2比特进行解码来重构当前块的帧内预测模式。重构的帧内预测模式被输入到预测器4630的帧内预测器4632。Here, the intra prediction mode decoding method by the decoder 4610 varies according to the size of the subblock. That is, if the size of the subblock is not 16x16 but 4x4 or 8x8, the decoder 4610 reconstructs the prediction mode flag by decoding 1 bit among the intra prediction mode data extracted from the bitstream. If the prediction mode flag indicates that the prediction direction corresponding to the intra prediction mode of the relevant sub-block is the same as the prediction direction corresponding to the intra prediction mode of the relevant sub-block estimated from the neighboring block, the decoder 4610 will estimate from the neighboring block The intra prediction mode of the relevant sub-block is reconstructed as the intra prediction mode of the current block, and the intra prediction mode of the next sub-block is decoded. If the prediction mode flag indicates that the prediction direction corresponding to the intra prediction mode of the relevant sub-block is different from the prediction direction corresponding to the intra prediction mode of the relevant sub-block estimated from the neighboring block, the decoder 4610 will pass the intra prediction The next three bits of the mode data are decoded and the reconstructed intra-frame prediction mode is reconstructed as the intra-frame prediction mode of the current block. Also, if the size of the subblock is 16x16, the decoder 4610 reconstructs the intra prediction mode of the current block by decoding 2 bits among the intra prediction mode data extracted from the bitstream. The reconstructed intra prediction mode is input to the intra predictor 4632 of the predictor 4630 .
此外，如果作为32x32像素块的扩展宏块是帧内模式，则属于被填充为16像素的倍数的扩展宏块中的图像的块被确定并且接着该块的分割类型被熵解码。在图3的情况下，属于作为32x32像素块的扩展宏块中的填充图像的块仅对应于第一16x16像素块，并且第一16x16像素块的分割类型是熵解码的。如果相关16x16像素块被划分为两个16x8像素块，则两个帧内预测模式被熵解码，并且针对16x16像素块的CBP和16x16变换系数被解码。利用通过熵解码获得的子块类型、帧内预测模式和变换系数来对相关16x16像素块进行解码。Also, if an extended macroblock that is a 32x32 pixel block is Intra mode, a block belonging to an image in the extended macroblock padded to a multiple of 16 pixels is determined and then the partition type of the block is entropy decoded. In the case of FIG. 3 , the blocks belonging to the padding image in the extended macroblock which is a 32x32 pixel block correspond only to the first 16x16 pixel block, and the partition type of the first 16x16 pixel block is entropy decoding. If the relevant 16x16 pixel block is divided into two 16x8 pixel blocks, the two intra prediction modes are entropy decoded, and the CBP and 16x16 transform coefficients for the 16x16 pixel block are decoded. The associated 16x16 pixel block is decoded using the subblock type, intra prediction mode and transform coefficients obtained by entropy decoding.
如果要解码的图片是帧间图片，则视频编码数据中要由解码器4610进行解码的数据的类型和序列与如图29所示相同。即，解码器4610通过对从比特流提取的SKIPN标记数据进行解码来重构指示当前要解码的宏块是SKIP宏块的SKIPN标记。如果由重构的SKIPN标记识别的宏块不是SKIP宏块，则解码器4610通过对从比特流提取的宏块类型标记数据和分割类型数据进行解码来重构宏块类型标记和分割类型信息。If the picture to be decoded is an inter picture, the type and sequence of data to be decoded by the decoder 4610 in video encoded data are the same as shown in FIG. 29 . That is, the decoder 4610 reconstructs the SKIPN flag indicating that the macroblock to be currently decoded is a SKIP macroblock by decoding the SKIPN flag data extracted from the bitstream. If the macroblock identified by the reconstructed SKIPN flag is not a SKIP macroblock, the decoder 4610 reconstructs the macroblock type flag and partition type information by decoding the macroblock type flag data and partition type data extracted from the bitstream.
此外，如果扩展宏块是32x32像素块，则属于被填充为16像素的倍数的扩展宏块中的图像的块被确定，并且接着skip32_flag被解码。如果当前要解码的扩展宏块是跳过模式，则左侧相邻块和上侧相邻块被用来寻找预测运动向量并进行运动补偿。在图3的情况下，属于作为32x32像素块的扩展宏块中的填充图像的块仅与第一16x16像素块对应，并且skip32_flag被解码。如果当前要解码的扩展宏块是跳过模式，则左侧相邻块和上侧相邻块被用来寻找预测运动向量并进行运动补偿。Also, if the extended macroblock is a 32x32 pixel block, a block belonging to an image in the extended macroblock padded to a multiple of 16 pixels is determined, and then skip32_flag is decoded. If the extended macroblock currently to be decoded is in the skip mode, the left adjacent block and the upper adjacent block are used to find the predicted motion vector and perform motion compensation. In the case of FIG. 3 , the blocks belonging to the padding image in the extended macroblock which is a 32x32 pixel block correspond to only the first 16x16 pixel block, and skip32_flag is decoded. If the extended macroblock currently to be decoded is in the skip mode, the left adjacent block and the upper adjacent block are used to find the predicted motion vector and perform motion compensation.
如果由重构的宏块类型标记识别的宏块类型是帧内宏块，则其与如上所述相同。如果宏块类型是帧间宏块，则解码器4610通过仅针对16x16或更多像素子块对SKIPX标记进行解码来重构SKIPX标记，并且通过针对不作为由重构的SKIPX标记识别的跳过块的子块或者针对小于16x16像素子块的子块(指示非跳过)对MVD/ref_idx数据进行解码来重构作为运动信息的运动向量和基准图片索引。此外，解码器4610通过按照与图33所示的序列相同或相似的序列对非跳过块进行解码来重构CBPX标记、变换类型和CBP，并且仅针对由重构的CBP识别的具有非零系数的子块重构量化变换系数。重构的运动信息被输入到帧间预测器4634(其可以通过运动估计器来实现)，并且重构的CBPX标记、CBP、变换类型和量化变换系数被输入到逆量化器/逆变换器4620，以便重构要解码的目标宏块或目标块的子块的残留块。If the macroblock type identified by the reconstructed macroblock type flag is an intra macroblock, it is the same as above. If the macroblock type is an inter macroblock, the decoder 4610 reconstructs the SKIPX flag by decoding the SKIPX flag only for 16x16 or more pixel sub-blocks, and The MVD/ref_idx data is decoded for sub-blocks of a block or for sub-blocks smaller than a 16x16 pixel sub-block (indicating non-skipping) to reconstruct motion vectors and reference picture indices as motion information. In addition, the decoder 4610 reconstructs the CBPX flag, transform type and CBP by decoding non-skipped blocks in the same or similar sequence as shown in FIG. The sub-block of coefficients reconstructs the quantized transform coefficients. The reconstructed motion information is input to the inter predictor 4634 (which may be implemented by a motion estimator), and the reconstructed CBPX flag, CBP, transform type and quantized transform coefficients are input to the inverse quantizer/inverse transformer 4620 , in order to reconstruct the residual block of the target macroblock or sub-block of the target block to be decoded.
逆量化器/逆变换器4620通过对从解码器4610接收到的量化变换系数进行逆量化/逆变换来重构残留块。这里，逆量化器/逆变换器4620可以利用从解码器4610接收到的CBPX标记、CBP和变换类型对量化变换系数进行逆量化/逆变换。The inverse quantizer/inverse transformer 4620 reconstructs a residual block by inverse quantizing/inverse transforming the quantized transform coefficient received from the decoder 4610 . Here, the inverse quantizer/inverse transformer 4620 may inverse quantize/inverse transform the quantized transform coefficients using the CBPX flag, CBP, and transform type received from the decoder 4610 .
预测器4630通过预测要解码的目标宏块或目标宏块的子块来生成预测块。为此，帧内预测器4632通过利用由解码器4610重构的帧内预测模式对要解码的目标宏块或目标宏块的子块进行帧内预测来生成目标宏块或目标宏块的子块的预测块。帧间预测器4634通过利用由解码器4610重构的运动信息对要解码的目标宏块或目标宏块的子块进行帧间预测来生成目标宏块或目标宏块的子块的预测块。The predictor 4630 generates a prediction block by predicting a target macroblock to be decoded or a subblock of the target macroblock. To this end, the intra predictor 4632 generates a target macroblock or a subblock of the target macroblock by intra-predicting the target macroblock or a subblock of the target macroblock to be decoded using the intra prediction mode reconstructed by the decoder 4610. The predicted block for the block. The inter predictor 4634 generates a predicted block of a target macroblock or a subblock of a target macroblock by inter-predicting a target macroblock or a subblock of the target macroblock to be decoded using the motion information reconstructed by the decoder 4610 .
加法器4640通过将由预测器4630生成的预测块与由逆量化器/逆变换器4620重构的残留块相加来重构要解码的目标宏块。The adder 4640 reconstructs a target macroblock to be decoded by adding the prediction block generated by the predictor 4630 and the residual block reconstructed by the inverse quantizer/inverse transformer 4620 .
由加法器4640重构的目标宏块由滤波器4640进行解块滤波，按照图片为单位积累接着输出作为重构的视频。所重构的视频存储在图片缓冲器4660中，使得其稍后可用于帧间预测器4634中的运动补偿预测。The target macroblock reconstructed by the adder 4640 is subjected to deblocking filtering by the filter 4640, accumulated in units of pictures, and then output as a reconstructed video. The reconstructed video is stored in picture buffer 4660 so that it can be used later for motion compensated prediction in inter predictor 4634 .
帧内预测器4632、帧间预测器4634和逆量化器/逆变换器4620的操作与参照图1所述的视频编码设备100的帧内预测器223、帧间预测器114和逆量化器/逆变换器150的操作相同或相似，因而将省略对其的详细描述。The operations of the intra predictor 4632, the inter predictor 4634, and the inverse quantizer/inverse transformer 4620 are similar to those of the intra predictor 223, the inter predictor 114, and the inverse quantizer/inverse transformer 100 of the video encoding apparatus 100 described with reference to FIG. The operation of the inverse converter 150 is the same or similar, and thus a detailed description thereof will be omitted.
视频解码设备4600可以根据扩展宏块是帧间宏块还是帧内宏块来如下进行构造。The video decoding apparatus 4600 may be configured as follows according to whether an extended macroblock is an inter macroblock or an intra macroblock.
例如，如果宏块是帧内宏块，则解码器4610通过对编码数据进行解码来重构分割类型信息、帧内预测模式和量化变换系数；逆量化器/逆变换器4620通过利用根据由分割类型信息识别的子块的大小而选择的变换对量化变换系数进行逆量化/逆变换来重构残留子块；预测器4630通过利用帧内预测模式预测根据分割类型信息识别的子块来生成预测子块；并且加法器4640通过将预测子块与残留子块相加来重构子块。For example, if the macroblock is an intra macroblock, the decoder 4610 reconstructs partition type information, intra prediction mode, and quantized transform coefficients by decoding the encoded data; The transform selected by the size of the sub-block identified by the type information performs inverse quantization/inverse transform on the quantized transform coefficients to reconstruct the residual sub-block; the predictor 4630 generates a prediction by predicting the sub-block identified according to the partition type information using the intra prediction mode sub-block; and adder 4640 reconstructs the sub-block by adding the predicted sub-block to the residual sub-block.
作为另一示例，如果宏块是帧间宏块，则解码器4610通过对编码后数据进行解码来重构分割类型信息、运动信息、变换类型信息和量化变换系数；逆量化器/逆变换器4620通过利用由变换类型信息识别的变换对量化变换系数进行逆量化/逆变换来重构残留子块；预测器4630通过利用运动信息预测根据分割类型信息而识别的子块来生成预测子块；并且加法器4640通过将预测子块与残留子块相加来重构子块。As another example, if the macroblock is an inter macroblock, the decoder 4610 reconstructs partition type information, motion information, transform type information, and quantized transform coefficients by decoding encoded data; the inverse quantizer/inverse transformer 4620 reconstructs the residual subblock by inverse quantizing/inverse transforming the quantized transform coefficients using the transform identified by the transform type information; the predictor 4630 generates a predicted subblock by predicting the subblock identified according to the partition type information using the motion information; And the adder 4640 reconstructs the sub-block by adding the predicted sub-block to the residual sub-block.
在根据本发明的实施方式的视频解码方法中，如果要解码的宏块是帧内宏块，则视频解码设备4600通过对编码后数据进行解码来重构分割类型信息、帧内预测模式和量化变换系数；通过利用根据由分割类型信息识别的子块的大小而选择的变换对量化变换系数进行逆量化/逆变换来重构残留子块；通过利用帧内预测模式预测根据分割类型信息而识别的子块来生成预测子块；并且通过将预测子块与残留子块相加来重构子块。In the video decoding method according to the embodiment of the present invention, if the macroblock to be decoded is an intra macroblock, the video decoding device 4600 reconstructs the division type information, intra prediction mode and quantization by decoding encoded data Transform coefficients; reconstruct residual sub-blocks by inverse quantizing/inverse transforming the quantized transform coefficients with a transform selected according to the size of the sub-block identified by the partition type information; predicted by utilizing the intra prediction mode identified according to the partition type information sub-blocks to generate predicted sub-blocks; and reconstruct sub-blocks by adding predicted sub-blocks to residual sub-blocks.
这里，如果子块的大小是4x4,4x8和8x4中的一个，则根据子块的大小选择的变换可以是4x4变换。如果子块的大小是8x8,8x16和16x8中的一个，则根据子块的大小选择的变换可以是8x8变换。如果子块的大小大于或等于16x16，则根据子块的大小选择的变换可以是16x16变换。Here, if the size of the sub-block is one of 4x4, 4x8, and 8x4, the transform selected according to the size of the sub-block may be 4x4 transform. If the size of the sub-block is one of 8x8, 8x16, and 16x8, the transform selected according to the size of the sub-block may be an 8x8 transform. If the size of the sub-block is greater than or equal to 16x16, the transform selected according to the size of the sub-block may be a 16x16 transform.
此外，在根据本发明的实施方式的视频解码方法中，如果要解码的宏块是帧间宏块，则视频解码设备4600通过对编码后数据进行解码来重构分割类型信息、运动信息、变换类型信息和量化变换系数；通过利用由变换类型信息识别的变换对量化变换系数进行逆量化/逆变换来重构残留子块；通过利用运动信息预测根据分割类型信息识别的子块来生成预测子块；并且通过将预测子块与残留子块相加来重构子块。Furthermore, in the video decoding method according to the embodiment of the present invention, if the macroblock to be decoded is an inter macroblock, the video decoding apparatus 4600 reconstructs partition type information, motion information, transform type information and quantized transform coefficients; reconstruct residual subblocks by inverse quantizing/inverse transforming quantized transform coefficients using the transform identified by the transform type information; generate predictors by predicting subblocks identified according to the partition type information using motion information block; and reconstructing the sub-block by adding the predicted sub-block to the residual sub-block.
这里，由变换类型信息识别的变换可以是4x4变换、8x8变换、8x16变换、16x8变换和16x16变换中的一个。此外，视频解码设备4600可以对重构的子块进行滤波。例如，视频解码设备4600可以在重构的子块中根据选择的变换对块的左侧界和右侧界进行解块滤波。Here, the transform identified by the transform type information may be one of 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform. Also, the video decoding apparatus 4600 may filter the reconstructed sub-block. For example, the video decoding apparatus 4600 may perform deblocking filtering on the left boundary and the right boundary of the block according to the selected transform in the reconstructed sub-block.
此外，视频解码设备4600还可以通过对从编码后数据中提取的SKIP_N标记数据进行解码来重构指示宏块是否为SKIP宏块的SKIP_N标记。视频解码设备4600可以仅对由SKIP_N标记识别为不是SKIP宏块的宏块进行解码处理。In addition, the video decoding apparatus 4600 may also reconstruct a SKIP_N flag indicating whether a macroblock is a SKIP macroblock by decoding SKIP_N flag data extracted from encoded data. The video decoding apparatus 4600 may only perform a decoding process on a macroblock identified by the SKIP_N flag as not being a SKIP macroblock.
此外，视频解码设备4600还可以通过对从编码后数据中提取的宏块类型标记数据进行解码来重构指示宏块是帧内宏块还是帧间宏块的宏块类型标记。视频解码设备4600可以通过对从编码后数据中提取的宏块类型标记数据进行解码来从宏块类型比较中检测要解码的宏块是帧内宏块还是帧间宏块，并且确定是进行帧内预测解码还是帧间预测解码。In addition, the video decoding apparatus 4600 may also reconstruct a macroblock type flag indicating whether a macroblock is an intra macroblock or an inter macroblock by decoding macroblock type flag data extracted from encoded data. The video decoding device 4600 can detect whether the macroblock to be decoded is an intra macroblock or an inter macroblock from the macroblock type comparison by decoding the macroblock type flag data extracted from the encoded data, and determine whether to perform frame Intra prediction decoding or inter prediction decoding.
与宏块的类型无关，可以由逐分割层分割类型编号来识别分割类型编号。在该情况下，视频解码设备4600还可以通过对从编码后数据中提取的CBP数据进行解码来重构与子块的大小对应的CBPX标记和CBP。这里，CBPX标记可以是指示XxX像素块中是否存在非零量化变换系数的标记，CBP可以是指示16x16像素块中是否存在非零量化变换系数的标记。Regardless of the type of macroblock, the partition type number can be identified by the partition type number by partition layer. In this case, the video decoding apparatus 4600 may also reconstruct the CBPX flag and CBP corresponding to the size of the sub-block by decoding the CBP data extracted from the encoded data. Here, the CBPX flag may be a flag indicating whether non-zero quantized transform coefficients exist in the XxX pixel block, and CBP may be a flag indicating whether non-zero quantized transform coefficients exist in the 16x16 pixel block.
第二实施方式：Second embodiment:
下文将描述本发明的第二实施方式。具体地说，本发明的第二实施方式描述根据句法的CABAC编码、变换子块的句法结构和预测子块。A second embodiment of the present invention will be described below. Specifically, the second embodiment of the present invention describes CABAC encoding according to syntax, a syntax structure of a transform sub-block, and a predictor sub-block.
此外，当利用CABAC对相关编码信息进行编码时，根据宏块、子块、预测子块和变换子块利用不同的上下文表对其进行编码和解码。Furthermore, when encoding related encoding information using CABAC, it is encoded and decoded using different context tables according to macroblocks, subblocks, predicted subblocks, and transformed subblocks.
在本发明的第二实施方式中，输入视频可以按照帧为单位或者按照字段为单位输入到视频编码设备100，或者可以在被划分为NxN像素宏块(N：大于或等于16的整数)之后输入到视频编码设备100。下文描述的宏块可以被划分为正方形子块。此外，各个宏块和字块可以针对预测的目的划分为预测子块。宏块和子块可以可变地划分为具有诸如MxM,(M/2)x(M/2),Mx(M/2),(M/2)xM,1xM和Mx1(M＝N)的各种大小的预测子块。此外，可以根据变换子块来确定变换类型。这里，可以通过将宏块或宏块的子块可变地划分为类似的预测子块来获得变换子块。In the second embodiment of the present invention, the input video may be input to the video encoding device 100 in units of frames or in units of fields, or may be divided into NxN pixel macroblocks (N: an integer greater than or equal to 16) input to the video encoding device 100. The macroblocks described below may be divided into square sub-blocks. Furthermore, individual macroblocks and subblocks may be divided into predictive subblocks for prediction purposes. Macroblocks and subblocks can be variably divided into individual blocks such as MxM, (M/2)x(M/2), Mx(M/2), (M/2)xM, 1xM, and Mx1 (M=N). predictor blocks of various sizes. Also, the transform type can be determined according to the transform sub-block. Here, the transformed subblock may be obtained by variably dividing a macroblock or a subblock of a macroblock into similar predictive subblocks.
如果不是N的倍数，则输入视频可以被填充为N的倍数，接着可以进行视频压缩。If not a multiple of N, the input video may be padded to a multiple of N, and then video compression may be performed.
视频编码设备100在宏块被划分为子块的情况下计算各个子块的编码效率，并且将提供最高编码效率的子块确定为最终的帧内块或帧间块。编码效率可以基于式1来测量。The video encoding apparatus 100 calculates encoding efficiency of each sub-block in a case where a macroblock is divided into sub-blocks, and determines a sub-block providing the highest encoding efficiency as a final intra block or an inter block. Coding efficiency can be measured based on Equation 1.
块模式确定处理可以包括针对所有类型的可用块来确定各个预测子块的最优预测模式(帧内预测模式和运动数据)，并且确定块类型。通常，在确定预测模式时(在选择帧内预测模式或运动向量/基准图片时)，为了减小计算量，式2被用来确定最优预测模式。The block mode determination process may include determining an optimal prediction mode (intra prediction mode and motion data) for each prediction sub-block for all types of available blocks, and determining a block type. Usually, when determining the prediction mode (when selecting the intra prediction mode or the motion vector/reference picture), in order to reduce the calculation amount, Equation 2 is used to determine the optimal prediction mode.
在确定子块的类型时，式8可以用来严格地计算RDcost。When determining the type of sub-block, Equation 8 can be used to strictly calculate RDcost.
RDcost＝Distortion+λ·Bit[block type，predlction infor，transform info，CBP，coefflclent] 式8RDcost＝Distortion+λ·Bit[block type, predlction infor, transform info, CBP, coefflclent] Formula 8
式8中的“Distortion”可以表示为式4。"Distortion" in Formula 8 can be expressed as Formula 4.
在确定子块的预测模式时，式9可以用来计算RDcost。When determining the prediction mode of a sub-block, Equation 9 can be used to calculate RDcost.
RDcost＝Dlstortion+λ·Bit[predlction info] 式9RDcost＝Dlstortion+λ·Bit[predlction info] Formula 9
将参照图1所示的视频编码设备100来描述根据本实施方式的视频编码方法。然而，从以下描述可以容易地理解，视频编码设备100中的一些元件进行与第一实施方式不同的操作。A video encoding method according to the present embodiment will be described with reference to the video encoding apparatus 100 shown in FIG. 1 . However, as can be easily understood from the following description, some elements in the video encoding apparatus 100 operate differently from the first embodiment.
下文中，将根据图片类型描述来根据本发明的第二实施方式的视频编码方法和通过该视频编码方法生成的编码后数据。Hereinafter, a video encoding method according to a second embodiment of the present invention and encoded data generated by the video encoding method will be described according to picture types.
预测器110的帧内预测器112利用要编码的块(下文称作“当前块”)的相邻像素生成基准块，并且通过将当前块与基准块相比较来确定帧内预测模式。这里，相邻像素包括与当前块相邻的像素，并且还包括包括在与当前块相邻的块中的像素。图47是例示视频边界或片段边界处的帧内预测的例示图。The intra predictor 112 of the predictor 110 generates a reference block using neighboring pixels of a block to be encoded (hereinafter referred to as 'current block'), and determines an intra prediction mode by comparing the current block with the reference block. Here, the adjacent pixels include pixels adjacent to the current block, and also include pixels included in blocks adjacent to the current block. Fig. 47 is an illustration diagram illustrating intra prediction at a video boundary or slice boundary.
参照图7，如果当前块是A，则由于基准像素仅是作为左侧块的块C的像素，所以预测模式被限制。因而，块C的像素被复制到块A的顶部。类似地，在块B的情况下，由于基准像素仅是作为上侧块的块C的像素，所以块C的像素被复制到块B的左侧。Referring to FIG. 7, if a current block is A, since reference pixels are only pixels of block C which is a left block, a prediction mode is limited. Thus, the pixels of block C are copied on top of block A. Similarly, in the case of block B, since the reference pixels are only the pixels of block C which is the upper block, the pixels of block C are copied to the left of block B.
并且，不具有值的像素可以假设为128，以供使用。Also, pixels with no value can be assumed to be 128 for use.
在帧内预测模式当中，利用平面标记对平面模式进行编码。例如，如果当前预测模式是平面模式，则利用1对平面标记进行编码。Among the intra prediction modes, the planar mode is coded using a planar flag. For example, if the current prediction mode is planar mode, the planar flag is encoded with a 1.
图48是例示平面模式的示例的图。FIG. 48 is a diagram illustrating an example of a planar mode.
参照图48，在平面模式的情况下，生成块48-1和48-2，并且两个块的平均值被用作预测像素。在块48-1的情况下，如这里所示，像素A被复制到其向下的方向，并且利用复制的值和与当前块的左侧相邻的像素在多个位置处生成剩余像素。例如，参照图8，当生成像素X时，由于像素X的位置是(2,4)，所以通过式Ax(2/8)+Cx(6/8)来计算像素X的值。按照与块48-1相同的方式生成块48-2。Referring to FIG. 48, in the case of the planar mode, blocks 48-1 and 48-2 are generated, and an average value of the two blocks is used as a predicted pixel. In the case of block 48-1, as shown here, pixel A is copied to its downward direction, and the remaining pixels are generated at multiple locations using the copied value and the pixels adjacent to the left of the current block. For example, referring to FIG. 8 , when pixel X is generated, since the position of pixel X is (2,4), the value of pixel X is calculated by the formula Ax(2/8)+Cx(6/8). Block 48-2 is generated in the same manner as block 48-1.
当对帧内预测模式进行编码时，指示当前块的预测模式是否与相邻帧内块的预测模式相同的标记被编码。这称作预测模式标记。如果预测模式标记是1，则指示相邻块的帧内预测模式与当前块的帧内预测模式相同。在该情况下，对预测模式指示索引进行编码。预测模式指示索引指示相邻块中的哪一个具有与当前块相同的帧内模式。When encoding the intra prediction mode, a flag indicating whether the prediction mode of the current block is the same as that of the neighboring intra blocks is encoded. This is called predictive mode flagging. If the prediction mode flag is 1, it indicates that the intra prediction mode of the neighboring block is the same as the intra prediction mode of the current block. In this case, the prediction mode indication index is encoded. The prediction mode indication index indicates which of neighboring blocks has the same intra mode as the current block.
如果相邻块的帧内预测模式与当前块的帧内预测模式彼此不同，则利用0对预测模式标记进行编码，并且对剩余的预测模式当中的除预测模式当中的相邻块的预测模式以外的当前块的预测模式进行编码。另选地，对剩余的预测模式当中的仅除相邻块的预测模式当中的分配有小编号的预测模式以外的当前块的预测模式进行编码。If the intra prediction mode of the adjacent block and the intra prediction mode of the current block are different from each other, the prediction mode flag is encoded with 0, and the prediction mode of the adjacent block other than the prediction mode among the remaining prediction modes The prediction mode of the current block is encoded. Alternatively, only the prediction mode of the current block other than the prediction mode assigned a small number among the prediction modes of the neighboring blocks among the remaining prediction modes is encoded.
可以按照N/2x N/2像素块为单位进行色度分量的帧内预测。预测方向可以使用独立于亮度分量的预测模式的平均值预测、水平预测、垂直预测、平面预测、根据亮度分量的帧内预测模式的模式以及诸如利用亮度分量的像素特性的预测模式的预测模式。The intra prediction of the chrominance component can be performed in units of N/2×N/2 pixel blocks. The prediction direction can use average prediction, horizontal prediction, vertical prediction, planar prediction, mode of intra prediction mode according to luma component, and prediction mode such as a prediction mode using pixel characteristics of luma component independently of the luma component's prediction mode.
变换器/量化器130确定各个宏块或子块中的变换类型，并且根据变换类型进行变换/量化。变换类型是指子块的大小和变换模式。变换模式不仅是指传统的DCT，还指变型的DCT、DST(离散正弦变换)和KLT(Karhunen-Loeve变换)。变换模式可以每变换子块、每宏块、每子块或者根据帧内预测模式自适应地选择。并且，变换模式可以根据变换子块的大小自适应地选择。The transformer/quantizer 130 determines a transform type in each macroblock or subblock, and performs transform/quantization according to the transform type. The transform type refers to the size and transform mode of the sub-block. The transformation mode refers not only to the traditional DCT, but also to the modified DCT, DST (Discrete Sine Transform) and KLT (Karhunen-Loeve Transform). The transform mode can be selected adaptively per transform sub-block, per macroblock, per sub-block or according to intra prediction mode. And, the transformation mode can be adaptively selected according to the size of the transformation sub-block.
这里，变换类型可以通过变换子块分割标记来表示。例如，如果TxT子块的变换子块分割标记是1，则变换子块的大小是(T/2)x(T/2)。在该情况下，DST可以用作水平变换，KLT可以用作垂直变换。Here, the transform type may be represented by transforming sub-block division flags. For example, if the transform subblock split flag of a TxT subblock is 1, the size of the transform subblock is (T/2)x(T/2). In this case, DST can be used as a horizontal transform and KLT can be used as a vertical transform.
此外，CBP每宏块、子块或变换子块进行编码和解码。Furthermore, CBP encodes and decodes per macroblock, subblock or transform subblock.
在P图片或P片段的情况下，可以按照宏块或子块为单位选择帧内模式或帧间模式。即，帧内模式和帧间模式这两者可以用于P图片或P片段的扩展宏块。In the case of a P picture or a P slice, an intra mode or an inter mode may be selected in units of macroblocks or subblocks. That is, both intra mode and inter mode can be used for extended macroblocks of P pictures or P slices.
在B图片或B片段的情况下，可以选择多达两个基准图片，其分别称为L0预测和L1预测。两个基准图片执行不同的功能。总体上，L0预测(列表0预测，主要用作前向预测)用作前向预测，L1预测(列表1预测，主要用作反向预测)用作反向预测。稍后将详细描述P图片和B图片的帧间图片编码方法。In the case of B-pictures or B-slices, up to two reference pictures may be selected, which are called L0 prediction and L1 prediction, respectively. The two reference images perform different functions. In general, L0 prediction (list 0 prediction, mainly used for forward prediction) is used for forward prediction, and L1 prediction (list 1 prediction, mainly used for backward prediction) is used for backward prediction. The inter picture coding method of P pictures and B pictures will be described in detail later.
再次参照图1，预测器110的帧间预测器114的运动估计器116检测与当前预测子块最类似的块，即，基准图片中的基准块，并且输出代表基准块相对于当前块的相对位置的运动向量(MV)。运动估计通过将宏块中的子块与图片缓冲器180中的一个或更多个基准图片进行比较来生成运动向量。Referring again to FIG. 1 , the motion estimator 116 of the inter predictor 114 of the predictor 110 detects the block most similar to the current prediction subblock, that is, the reference block in the reference picture, and outputs The motion vector (MV) of the position. Motion estimation generates motion vectors by comparing subblocks in a macroblock with one or more reference pictures in picture buffer 180 .
运动向量由编码器140进行编码，并被包括在编码后数据中。编码器140选择当前块的相邻块的运动向量和共同位置块(与当前块位于相同位置的基准图片块)的运动向量当中的预测运动向量。编码器140通过仅对作为预测运动向量与当前块的运动向量之间的差向量的差运动向量(DMV)进行编码来生成运动信息。这里，除了差运动向量以外，编码器140还可以对基准图片索引进行编码。The motion vectors are encoded by the encoder 140 and included in the encoded data. The encoder 140 selects a predicted motion vector among motion vectors of neighboring blocks of the current block and motion vectors of co-located blocks (reference picture blocks co-located with the current block). The encoder 140 generates motion information by encoding only a difference motion vector (DMV), which is a difference vector between a predicted motion vector and a motion vector of a current block. Here, the encoder 140 may encode a reference picture index in addition to the difference motion vector.
此外，在本发明的实施方式中，针对宏块、子块、预测子块使用批模式。批模式是使用相邻运动块或者共同位置块的运动信息(预测方向、运动向量和基准图片)作为当前块的运动信息的模式。因而，在批模式的情况下，可以对批模式标记和预测运动信息索引进行编码。Furthermore, in the embodiments of the present invention, batch mode is used for macroblocks, subblocks, and predicted subblocks. The batch mode is a mode that uses motion information (prediction direction, motion vector, and reference picture) of adjacent motion blocks or co-located blocks as motion information of a current block. Thus, in case of batch mode, a batch mode flag and a predicted motion information index can be encoded.
此外，在本发明的实施方式中，针对宏块或子块使用跳过模式。在跳过块的情况下，预测运动向量被用来进行运动估计和补偿。这里，编码器对跳过标记和预测运动向量索引进行编码。作为另一示例，不是预测运动向量，而是预测运动信息可以用来进行跳过模式的运动估计和补偿。在该情况下，可以对跳过标记和预测运动信息索引进行编码。Furthermore, in embodiments of the present invention, skip mode is used for macroblocks or subblocks. In the case of skipped blocks, the predicted motion vector is used for motion estimation and compensation. Here, the encoder encodes a skip flag and a predicted motion vector index. As another example, instead of predicting motion vectors, predictive motion information may be used for motion estimation and compensation in skip mode. In this case, a skip flag and a predicted motion information index may be encoded.
帧间预测块的变换方法与上述帧内预测块的变换方法相同。The transformation method of the inter prediction block is the same as the transformation method of the intra prediction block described above.
图49a-49b是例示根据本发明的实施方式对帧内图片的宏块和子块进行编码的流程图的例示图。49a-49b are illustrations illustrating a flowchart of encoding macroblocks and sub-blocks of an intra picture according to an embodiment of the present invention.
图50a-50b是例示根据本发明的实施方式对帧内图片的宏块和子块进行编码的流程图的另一例示图。Figures 50a-50b are another illustration diagram illustrating a flowchart of encoding macroblocks and sub-blocks of an intra picture according to an embodiment of the present invention.
参照图49a-49b和图50a-50b，帧内图片的子块和宏块包括分割标记、差量QP、预测模式标记、预测模式指示索引、预测模式、变换子块分割标记、亮度分量CBP标记、色度分量CBP标记和变换系数。然而，图49a-49b和图50a-50b所示的帧内图片的句法流程图仅是示例性的。即，帧内图片的句法的一部分或全部可以改变，并且可以省略一部分句法。Referring to Figures 49a-49b and Figures 50a-50b, sub-blocks and macroblocks of intra pictures include partition flag, difference QP, prediction mode flag, prediction mode indication index, prediction mode, transformed sub-block partition flag, luminance component CBP flag, Chroma component CBP flags and transform coefficients. However, the syntax flow diagrams for intra pictures shown in Figures 49a-49b and Figures 50a-50b are merely exemplary. That is, a part or all of the syntax of an intra picture may be changed, and a part of the syntax may be omitted.
在本实施方式中，如同本发明的其它实施方式，CABAC被用来对各个句法进行编码和解码。将省略对CABAC的详细描述，以避免重复较早的说明。In this embodiment, as in other embodiments of the invention, CABAC is used to encode and decode individual syntaxes. A detailed description of CABAC will be omitted to avoid repetition of earlier descriptions.
图49a-49b和图50a-50b中的分割标记的ctx_inc、预测模式标记、预测模式指示索引、平面模式标记、变换子块分割标记和CBP标记可以按照与确定参照图35在第一实施方式中描述的确定skipX_flag的ctx_inc的方法相似的方式进行确定。因而，将省略对其的详细描述。The ctx_inc, prediction mode flag, prediction mode indication index, plane mode flag, transform subblock partition flag, and CBP flag of the partition flag in Figures 49a-49b and Figures 50a-50b can be determined according to and with reference to Figure 35 in the first embodiment The determination is made in a similar manner to the method described for determining ctx_inc of skipX_flag. Thus, a detailed description thereof will be omitted.
图51a-51b是例示根据本发明的实施方式对变换子块的变换系数进行编码的序列的例示图。51a-51b are illustration diagrams illustrating a sequence for encoding transform coefficients of a transform sub-block according to an embodiment of the present invention.
参照图51a-51b，包括了变换系数标记、最后的变换系数标记、Abs_value_greater1_flag、Abs_value_greater2_flag、Absolute(变换系数)-3和变换系数符号(+或-)。然而，图51a-51b所示的变换系数的句法流程图仅是示例性的。即，变换系数的句法的一部分或者全部可以被改变，并且可以省略一部分句法。Referring to Figures 51a-51b, transform coefficient flag, last transform coefficient flag, Abs_value_greater1_flag, Abs_value_greater2_flag, Absolute (transform coefficient)-3 and transform coefficient sign (+ or -) are included. However, the syntax flow diagrams for transform coefficients shown in Figures 51a-51b are merely exemplary. That is, part or all of the syntax of transform coefficients may be changed, and a part of the syntax may be omitted.
图52a-52c是例示根据本发明的实施方式对帧间宏块和帧间图片的子块进行编码的序列的例示图。Figures 52a-52c are diagrams illustrating a sequence for encoding an inter macroblock and a sub-block of an inter picture according to an embodiment of the present invention.
参照图52a-52c，包括了分割标记、跳过标记、批模式标记、预测运动信息索引、预测方向标记、差量QP、预测子块类型、基准图片索引、差运动向量、预测运动向量索引、变换子块分割标记和CBP标记。然而，图52a-52c所示的帧间图片的句法流程图仅是示例性的。即，帧间图片的句法的一部分或者全部可以被改变，并且可以省略一部分句法。Referring to Figures 52a-52c, it includes segmentation flag, skip flag, batch mode flag, prediction motion information index, prediction direction flag, difference QP, prediction sub-block type, reference picture index, difference motion vector, prediction motion vector index, Transform subblock split flags and CBP flags. However, the syntax flow diagrams for inter pictures shown in Figures 52a-52c are merely exemplary. That is, part or all of the syntax of the inter picture may be changed, and a part of the syntax may be omitted.
图52a-52c所示的预测子块类型表示子块或当前宏块中的预测子块的大小以及是帧间预测还是帧内预测。这里，例示了帧间预测。如果是帧内预测，则可以参照图49a-49b和图50a-50b类似地对预测子块类型之后的句法进行编码。The predicted subblock type shown in Figures 52a-52c indicates the size of the subblock or the predicted subblock in the current macroblock and whether it is inter prediction or intra prediction. Here, inter prediction is exemplified. In case of intra prediction, the syntax following the prediction sub-block type can be similarly encoded with reference to Figures 49a-49b and Figures 50a-50b.
此外，可以参照图51a-51b类似地对帧间预测的变换系数编码进行编码。因而将省略对其的详细描述。Furthermore, the encoding of transform coefficients for inter prediction can be similarly encoded with reference to Figures 51a-51b. A detailed description thereof will thus be omitted.
如同本发明的第一实施方式，各个句法可以参照式7使用CABAC编码。然而，针对各个句法的ctx_inc可以根据周围情况而变化。Like the first embodiment of the present invention, each syntax can refer to Equation 7 using CABAC encoding. However, ctx_inc for each syntax may vary according to surrounding circumstances.
图53是针对批模式标记的ctx_inc的例示图。Figure 53 is an illustration of ctx_inc for batch mode flags.
例如，批模式标记可以参照图37和图53进行CABAC编码。For example, batch mode flags can be CABAC encoded with reference to Figure 37 and Figure 53 .
并且，可以参照图54对差运动向量进行CABAC编码。And, the difference motion vector may be CABAC-coded with reference to FIG. 54 .
图54是针对差运动向量的x分量的ctx_inc的例示图。Fig. 54 is an illustration diagram of ctx_inc for the x-component of the difference motion vector.
差运动向量的x分量的ctx_inc是利用如图54所示的相邻块的阈值X标记来确定的。相邻块的阈值X标记的值是根据预定阈值和差运动向量的x分量的值来确定的。例如，假设阈值是9。在该情况下，如果差运动向量x分量的绝对值小于阈值“9”，则阈值X标记的值是“1”；并且如果该绝对值大于9，则阈值X标记的值是0。这里，可以由用户来选择阈值。在差运动向量的y分量的情况下，按照与差运动向量的x分量的情况相同的方式确定ctx_inc。The ctx_inc of the x-component of the difference motion vector is determined using the threshold X-flag of the neighboring blocks as shown in FIG. 54 . The value of the threshold Xflag for neighboring blocks is determined based on the predetermined threshold and the value of the x-component of the difference motion vector. For example, suppose the threshold is 9. In this case, if the absolute value of the difference motion vector x component is smaller than the threshold "9", the value of the threshold X flag is "1"; and if the absolute value is greater than 9, the value of the threshold X flag is 0. Here, the threshold may be selected by the user. In the case of the y component of the difference motion vector, ctx_inc is determined in the same manner as in the case of the x component of the difference motion vector.
图54中的句法的ctx_inc可以与如第一实施方式所述类似地确定，因而将省略对其的详细描述。ctx_inc of the syntax in FIG. 54 can be determined similarly as described in the first embodiment, and thus a detailed description thereof will be omitted.
尽管已经针对例示性目的描述了本发明的示例性实施方式，但是本领域技术人员将理解，无需脱离本发明的实质特性，可以进行各种修改、添加和替换。因此，本发明的示例性实施方式不是针对限制目的而描述的。因此，本发明的范围不由以上实施方式而由权利要求及其等同物限定。Although the exemplary embodiments of the present invention have been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions can be made without departing from the essential characteristics of the invention. Accordingly, the exemplary embodiments of the present invention are not described for limiting purposes. Therefore, the scope of the present invention is defined not by the above embodiments but by the claims and their equivalents.
工业实用性Industrial Applicability
如上所述，本发明的实施方式对于本领域的应用非常有用，因为它们可以通过以下处理来提高视频压缩效率和视频重构效率：将扩展宏块划分为各种大小和形状的子块，以及利用根据这些子块的大小来使用不同的预测单元和变换单元进行视频编码/解码。As described above, embodiments of the present invention are very useful for applications in this field because they can improve video compression efficiency and video reconstruction efficiency by dividing an extended macroblock into subblocks of various sizes and shapes, and Video encoding/decoding is performed using different prediction units and transformation units according to the sizes of these sub-blocks.
相关申请的交叉引用Cross References to Related Applications
如果适用，本申请要求在韩国于2010年4月16提交的专利申请No.10-2010-0035450以及于2011年4月16日提交的专利申请No.10-2011-0035425在35U.S.C§119(a)下的优先权，通过引用将其全部内容并入本文。此外，该非临时申请以基于韩国专利申请相同的原因要求在除美国以外的国家的优先权，并且通过引用将其全部内容并入本文。This application claims that Patent Application No. 10-2010-0035450 filed on April 16, 2010 in Korea and Patent Application No. 10-2011-0035425 filed on April 16, 2011 in 35 U.S.C § 119 Priority under (a), the entire contents of which are incorporated herein by reference. Furthermore, this non-provisional application claims priority in countries other than the United States for the same reasons based on the Korean patent application, and its entire content is incorporated herein by reference.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR20100035450 | 2010-04-16 | ||
| KR10-2010-0035450 | 2010-04-16 | ||
| KR1020110035425AKR101791242B1 (en) | 2010-04-16 | 2011-04-16 | Video Coding and Decoding Method and Apparatus |
| KR10-2011-0035425 | 2011-04-16 | ||
| CN201180029771.8ACN102939756B (en) | 2010-04-16 | 2011-04-18 | Video encoding/decoding apparatus and method |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201180029771.8ADivisionCN102939756B (en) | 2010-04-16 | 2011-04-18 | Video encoding/decoding apparatus and method |
| Publication Number | Publication Date |
|---|---|
| CN105704490A CN105704490A (en) | 2016-06-22 |
| CN105704490Btrue CN105704490B (en) | 2018-07-31 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610065599.6AActiveCN105704490B (en) | 2010-04-16 | 2011-04-18 | Video encoder |
| Country | Link |
|---|---|
| CN (1) | CN105704490B (en) |
| WO (1) | WO2011129673A2 (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9807401B2 (en)* | 2011-11-01 | 2017-10-31 | Qualcomm Incorporated | Transform unit partitioning for chroma components in video coding |
| KR20250099275A (en)* | 2018-03-05 | 2025-07-01 | 파나소닉 인텔렉츄얼 프로퍼티 코포레이션 오브 아메리카 | Coding device, decoding device, coding method and decoding method |
| MX2020013802A (en)* | 2018-06-25 | 2021-05-27 | Electronics & Telecommunications Res Inst | Method and apparatus for encoding/decoding image using quantization parameter, and recording medium storing bitstream. |
| WO2020094050A1 (en) | 2018-11-06 | 2020-05-14 | Beijing Bytedance Network Technology Co., Ltd. | Using inter prediction with geometric partitioning for video processing |
| CN112106368B (en)* | 2018-12-29 | 2022-07-22 | 富士通株式会社 | An image intra-block encoding or decoding method, data processing device and electronic device |
| CN113170166B (en) | 2018-12-30 | 2023-06-09 | 北京字节跳动网络技术有限公司 | Use of inter prediction with geometric partitioning in video processing |
| CN113170139B (en)* | 2019-01-10 | 2023-12-05 | 北京字节跳动网络技术有限公司 | Simplified context modeling for context-adaptive binary arithmetic coding |
| WO2020244658A1 (en)* | 2019-06-06 | 2020-12-10 | Beijing Bytedance Network Technology Co., Ltd. | Sub-block based intra block copy |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1589023A (en)* | 2004-08-06 | 2005-03-02 | 联合信源数字音视频技术（北京）有限公司 | Coding and decoding method and device for multiple coded list lengthening based on context |
| CN1917647A (en)* | 2005-04-19 | 2007-02-21 | 三星电子株式会社 | Method and apparatus for adaptively selecting context model for entropy coding |
| CN101034891A (en)* | 2006-02-22 | 2007-09-12 | 三星电子株式会社 | Cabac encoding method and apparatus and cabac decoding method and apparatus |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090129926A (en)* | 2008-06-13 | 2009-12-17 | 삼성전자주식회사 | Image encoding method and apparatus, image decoding method and apparatus |
| KR101359496B1 (en)* | 2008-08-06 | 2014-02-11 | 에스케이 텔레콤주식회사 | Encoding Mode Determination Method and Apparatus and Video Encoding Apparatus Using Same |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1589023A (en)* | 2004-08-06 | 2005-03-02 | 联合信源数字音视频技术（北京）有限公司 | Coding and decoding method and device for multiple coded list lengthening based on context |
| CN1917647A (en)* | 2005-04-19 | 2007-02-21 | 三星电子株式会社 | Method and apparatus for adaptively selecting context model for entropy coding |
| CN101034891A (en)* | 2006-02-22 | 2007-09-12 | 三星电子株式会社 | Cabac encoding method and apparatus and cabac decoding method and apparatus |
| Publication number | Publication date |
|---|---|
| WO2011129673A3 (en) | 2012-03-15 |
| CN105704490A (en) | 2016-06-22 |
| WO2011129673A2 (en) | 2011-10-20 |
| Publication | Publication Date | Title |
|---|---|---|
| CN105306943B (en) | Method for video coding | |
| CN102948149B (en) | Video encoding/decoding apparatus and method | |
| KR101874840B1 (en) | Video coding/decoding apparatus and method | |
| KR101984764B1 (en) | Video Coding and Decoding Method and Apparatus | |
| KR101479129B1 (en) | Video Coding Method and Apparatus Using Variable size Macroblock | |
| CN105704490B (en) | Video encoder | |
| KR20170058838A (en) | Method and apparatus for encoding/decoding of improved inter prediction | |
| CN107071458A (en) | Equipment for encoding motion pictures | |
| CN117615128A (en) | Method of encoding/decoding image and method of transmitting bitstream | |
| KR20110043407A (en) | Image encoding / decoding apparatus and method |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |