CN102474617B - Image processing device and method - Google Patents

Abstract

The present invention relates to an image processing apparatus and method capable of performing noise removal according to an image and a bit rate. The low-pass filter setting unit 93 sets filter coefficients corresponding to the intra prediction mode information and the quantization parameters from the filter coefficients stored in the built-in filter coefficient memory 94. The neighboring image setting unit 81 performs filter processing on neighboring pixel values of the current block from the frame memory 72 using the filter coefficients set by the low-pass filter setting unit 93. The prediction image generation unit 82 performs intra prediction using the neighboring pixel values subjected to the filtering processing from the neighboring image setting unit 81, and generates a prediction image. The present invention can be applied to an image encoding device that encodes in, for example, the H.264/AVC format.

Description

Translated fromChinese

图像处理装置和方法Image processing device and method

技术领域technical field

本发明涉及一种图像处理装置和方法，具体地涉及一种能够提高预测效率的图像处理装置和方法。The present invention relates to an image processing device and method, in particular to an image processing device and method capable of improving prediction efficiency.

背景技术Background technique

近年来，广泛地使用通过下述方式对图像进行压缩编码的装置：利用用于将图像信息作为数字信号处理的编码格式，并且，利用图像信息特有的冗余性(redundancy)，其中，此时以发送和存储高效信息作为目标，通过诸如离散余弦变换等的正交变换和运动补偿来压缩图像。该编码方法的例子包括MPEG(运动图片专家组)等。In recent years, apparatuses for compressing and encoding images by using an encoding format for processing image information as a digital signal and utilizing redundancy specific to image information have been widely used, wherein at this time With the aim of transmitting and storing highly efficient information, images are compressed by orthogonal transform such as discrete cosine transform and motion compensation. Examples of the encoding method include MPEG (Moving Picture Experts Group) and the like.

特别地，MPEG2(ISO/IEC 13818-2)被定义为通用图像编码格式，并且，是包括隔行扫描图像和顺序扫描图像二者、以及标准分辨率图像和高清晰图像的标准。例如，MPEG2现在已经被用于专业用途和消费者用途的广范围的应用广泛地利用。例如，通过利用MPEG2压缩格式，在具有720×480个像素的标准分辨率的隔行扫描图像的情况中，分配4至8Mbps的代码量(比特率)。此外，例如，通过利用MPEG2压缩格式，在具有1920×1088个像素的高分辨率的隔行扫描图像的情况中，分配18至22Mbps的代码量(比特率)。因此，可以实现高压缩率和优异的图像质量。In particular, MPEG2 (ISO/IEC 13818-2) is defined as a general-purpose image encoding format, and is a standard including both interlaced images and sequential images, as well as standard-resolution images and high-definition images. For example, MPEG2 is now widely utilized by a wide range of applications for professional use and consumer use. For example, by utilizing the MPEG2 compression format, in the case of an interlaced image having a standard resolution of 720×480 pixels, a code amount (bit rate) of 4 to 8 Mbps is allocated. Furthermore, for example, by utilizing the MPEG2 compression format, in the case of an interlaced image having a high resolution of 1920×1088 pixels, a code amount (bit rate) of 18 to 22 Mbps is allocated. Therefore, high compression ratio and excellent image quality can be realized.

对于MPEG2，主要以适合广播用途的高图像质量作为目标，但是，不处理比MPEG1的代码量低的代码量(比特率)，即，具有较高的压缩率的编码格式。随着个人数字助理的普及，已经期望从现在开始将要增加对这种编码格式的需求，并且，响应于此，已经执行了MPEG4编码格式的标准化。关于图像编码格式，其规范在1998年12月被确认为国际标准ISO/IEC 14496-2。For MPEG2, high image quality suitable for broadcasting is mainly aimed at, however, a coding format with a lower coding amount (bit rate) than that of MPEG1, ie, a higher compression rate, is not handled. With the spread of personal digital assistants, it has been expected that the demand for such an encoding format will increase from now on, and in response to this, standardization of the MPEG4 encoding format has been performed. Regarding the image encoding format, its specification was confirmed as the international standard ISO/IEC 14496-2 in December 1998.

此外，近年来，已经通过原本预计用于电视会议用途的图像编码进行了用作H.26L(ITU-T Q6/16 VCEG)的标准的标准化。对于H.26L，已经知道，与诸如MPEG2或MPEG4的传统编码格式相比，尽管对于其编码和解码要求更大的计算量，但是实现了更高的编码效率。此外，当前，作为MPEG4的活动的一部分，为了实现更高的编码效率，已经执行了也利用不被H.26L支持但以该H.26L作为基础的功能的标准化，作为增强压缩视频编码的联合模型。作为标准化的时间表(schedule)，H.264和MPEG-4 Part10(高级视频编码，下文中被称为H.264/AVC)在2003年3月成为了国际标准。In addition, in recent years, standardization as a standard of H.26L (ITU-T Q6/16 VCEG) has been performed by image coding originally expected for videoconferencing use. As for H.26L, it is known that higher encoding efficiency is achieved, although a larger calculation amount is required for encoding and decoding thereof, compared with conventional encoding formats such as MPEG2 or MPEG4. In addition, currently, as part of the activities of MPEG4, in order to achieve higher coding efficiency, standardization using functions that are not supported by H.26L but based on this H.26L has been performed as joint enhancement compression video coding Model. As a schedule for standardization, H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as H.264/AVC) became an international standard in March 2003.

此外，作为其扩展，在2005年2月已经完成了包括诸如RGB、4:2:2或4:4:4的业务使用所需的编码工具、由MPEG-2规定的8x8DCT和量化矩阵的FRExt(保真度范围扩展，Fidelity Range Extension)的标准化。因此，H.264/AVC已经成为甚至能够合适地表达电影中包含的影片噪声的编码格式，并且，已经被用于诸如蓝光盘(Blu-Ray Disc，注册商标)等的广泛的应用。In addition, as an extension of it, FRExt including encoding tools required for business use such as RGB, 4:2:2 or 4:4:4, 8x8DCT and quantization matrix specified by MPEG-2 has been completed in February 2005 (Fidelity Range Extension) standardization. Therefore, H.264/AVC has become an encoding format capable of appropriately expressing even film noise contained in movies, and has been used for a wide range of applications such as Blu-Ray Disc (registered trademark) and the like.

但是，现今，已经增加了对于进一步的高压缩编码的需求，例如，预计压缩具有约4000×2000个像素的图像，即，高视觉图像的四倍。或者，已经增加了对于进一步的高压缩编码的需求，例如，预计在如互联网一样的具有受限的传输容量的环境内发布高视觉图像。因此，对于在ITU-T的控制下的上述VCEG(＝视觉编码专家组)，已经连续地执行了与提高编码效率有关的研究。Nowadays, however, the demand for further high-compression encoding has increased, for example, it is expected to compress an image having about 4000×2000 pixels, ie, four times that of a high-vision image. Or, demand for further high-compression coding has increased, for example, distribution of high-visual images is expected within an environment with limited transmission capacity like the Internet. Therefore, for the above-mentioned VCEG (=Visual Coding Experts Group) under the control of ITU-T, studies related to improving coding efficiency have been continuously performed.

现在，可以给出的为什么H.264/AVC格式与传统的MPEG2格式等相比实现了高的编码效率的一种因素是利用了帧内预测方法(intra prediction method)。Now, one factor that can be given why the H.264/AVC format achieves high encoding efficiency compared with the conventional MPEG2 format and the like is the use of an intra prediction method.

对于帧内预测方法，关于亮度信号，确定了四种16×16像素宏块单位以及九种4×4像素和8×8像素块单位的帧内预测模式。关于颜色差信号，确定了四种8×8像素块单位的帧内预测模式。用于颜色差信号的帧内预测模式可以与用于亮度信号的帧内预测模式独立地被设置。For the intra prediction method, with respect to the luminance signal, four types of intra prediction modes of 16×16 pixel macroblock units and nine 4×4 pixel and 8×8 pixel block units are determined. Regarding the color difference signal, four types of intra prediction modes in units of 8×8 pixel blocks are determined. The intra prediction mode for the color difference signal can be set independently from the intra prediction mode for the luma signal.

关于如何表现在这种帧内预测之后的残差(residual)，针对每一种帧内预测模式存在特定的图案(pattern)。Regarding how to represent the residual after such intra prediction, there is a specific pattern for each intra prediction mode.

作为消除这种冗余性并进一步提高编码效率的方法，非专利文献1提出了下面的方法。As a method of eliminating such redundancy and further improving encoding efficiency, Non-Patent Document 1 proposes the following method.

也就是说，预先使用离线处理的训练信号通过正常的H.264/AVC格式来执行帧内图像编码处理(intra image encoding processing)，关于每一块，针对每一种帧内预测模式执行诸如Karhunen-Loéve变换等的正交变换，并且，计算最佳的变换系数。That is, intra image encoding processing (intra image encoding processing) is performed by the normal H.264/AVC format using the training signal processed offline in advance, and for each block, such as Karhunen- Orthogonal transform such as Loéve transform, and calculate the optimum transform coefficient.

然后，在实际的编码处理中，使用下述处理：使用通过前述的Karhunen-Loéve变换针对每一种模式优化的正交变换系数，而不是由H.264/AVC格式规定的正交变换。Then, in actual encoding processing, a process of using orthogonal transform coefficients optimized for each mode by the aforementioned Karhunen-Loéve transform instead of orthogonal transform prescribed by the H.264/AVC format is used.

此外，非专利文献2提出了将前述的帧内预测和帧间预测(inter prediction)组合的方法。Furthermore, Non-Patent Document 2 proposes a method of combining the aforementioned intra prediction and inter prediction.

也就是说，对于非专利文献2，关于在帧间预测中获得的运动向量信息，不仅对于当前块，而且对于当前块周围的邻近像素值，产生了差分信息。在以这样方式产生的关于当前块的差分信息和关于邻近像素的差分信息之间执行帧内预测，产生二次差分信息。然后，对产生的二次差分信息进行正交变换和量化，并且将其在下游与压缩图像一起输出。That is, with Non-Patent Document 2, regarding motion vector information obtained in inter prediction, difference information is generated not only for the current block but also for neighboring pixel values around the current block. Intra prediction is performed between the differential information on the current block and the differential information on neighboring pixels generated in this manner, generating secondary differential information. Then, the resulting quadratic difference information is subjected to orthogonal transformation and quantization, and is output downstream together with the compressed image.

因此，进一步提高了编码效率。Therefore, coding efficiency is further improved.

此外，如上所述，宏块尺寸为具有H.264/AVC格式的16×16个像素。但是，对于将通过下一代编码方法处理的诸如UHD(超高清晰度：4000×2000个像素)的大图像帧，16×16个像素的宏块尺寸不是最佳的。Also, as described above, the macroblock size is 16×16 pixels in the H.264/AVC format. However, the macroblock size of 16×16 pixels is not optimal for a large image frame such as UHD (Ultra High Definition: 4000×2000 pixels) to be processed by a next-generation encoding method.

因此，例如，非专利文献3等提出了将宏块尺寸放大为32×32个像素的尺寸。Therefore, for example, Non-Patent Document 3 and the like propose enlarging the macroblock size to a size of 32×32 pixels.

引文列表Citation list

非专利文献non-patent literature

非专利文献1：″Improved Intra Coding″，VCEG-AF15，ITU-TelecommunicationsStandardization Sector STUDY GROUPQuestion 6 Video coding Experts Group(VCEG)，20-21 April 2007Non-Patent Document 1: "Improved Intra Coding", VCEG-AF15, ITU-Telecommunications Standardization Sector STUDY GROUP Question 6 Video coding Experts Group (VCEG), 20-21 April 2007

非专利文献2：″Second Order Prediction(SOP)in P Slice″，Sijia Chen，Jinpeng Wang，Shangwen Li and，Lu Yu，VCEG-AD09，ITU-TelecommunicationsStandardization Sector STUDY GROUPQuestion 6 Video coding Experts Group(VCEG)，16-18 July 2008Non-Patent Document 2: "Second Order Prediction (SOP) in P Slice", Sijia Chen, Jinpeng Wang, Shangwen Li and, Lu Yu, VCEG-AD09, ITU-Telecommunications Standardization Sector STUDY GROUP Question 6 Video coding Experts Group (VCEG), 16 -18 July 2008

非专利文献3：″Video Coding Using Extended Block Sizes″，VCEG-AD09，ITU-Telecommunications Standardization Sector STUDY GROUP Question 16-Contribution 123，Jan 2009Non-Patent Document 3: "Video Coding Using Extended Block Sizes", VCEG-AD09, ITU-Telecommunications Standardization Sector STUDY GROUP Question 16-Contribution 123, Jan 2009

发明内容Contents of the invention

技术问题technical problem

现在，对于H.264/AVC格式，在以上述的8×8个像素的块为增量执行帧内预测之前，对邻近像素的像素值执行低通滤波处理。因此，去除了邻近像素中包含的噪声，并且，提高了相关性，从而可以实现更高的编码效率。Now, with the H.264/AVC format, low-pass filter processing is performed on pixel values of adjacent pixels before intra prediction is performed in increments of blocks of 8×8 pixels as described above. Therefore, noise contained in adjacent pixels is removed, and correlation is improved, so that higher encoding efficiency can be achieved.

但是，不管所包含的噪声的程度随着输入图像、量化参数值、帧内预测模式等如何不同，都已经用H.264/AVC格式固定了用于去除噪声的低通滤波器。也就是说，根据输入图像、量化参数值、帧内预测模式等，该低通滤波器仍然不是最佳的。However, a low-pass filter for removing noise has been fixed with the H.264/AVC format regardless of how the degree of noise included differs depending on the input image, quantization parameter value, intra prediction mode, and the like. That is, the low-pass filter is still not optimal according to the input image, quantization parameter value, intra prediction mode, etc.

此外，对于H.264/AVC格式，仅仅以8×8个像素的块为增量用帧内预测模式对上述的邻近像素的像素值执行了噪声去除，并且，对于其它模式，这仍然不适用。Furthermore, for the H.264/AVC format, noise removal is performed on the pixel values of neighboring pixels as described above with the intra prediction mode only in blocks of 8×8 pixels, and this still does not apply for other modes .

鉴于该情形而作出了本发明，并且，本发明实现了与图像和比特率相应的噪声去除，从而提高了预测效率。The present invention has been made in view of this situation, and realizes noise removal corresponding to images and bit rates, thereby improving prediction efficiency.

解决问题的方案solution to the problem

根据本发明的第一方面的图像处理装置包括：解码部件，被配置为对要作为解码处理的对象的当前块的图像进行解码；滤波设置部件，被配置为根据编码参数设置与所述当前块相应的、用于所述当前块的帧内预测的、且用于要使所述当前块的邻近像素经过的滤波处理的系数；以及帧内预测部件，被配置为：使用由所述滤波设置部件设置的所述系数，使所述邻近像素经过所述滤波处理，并且对所述当前块执行帧内预测。The image processing apparatus according to the first aspect of the present invention includes: decoding means configured to decode an image of a current block to be the target of decoding processing; Corresponding coefficients used for intra prediction of the current block and used for filter processing to be subjected to neighboring pixels of the current block; and intra prediction means configured to: use The coefficients set by means, subject the neighboring pixels to the filtering process, and perform intra prediction on the current block.

编码参数可以包括当前块的帧内预测的模式或者当前块的量化参数；解码部件可以对所述当前块的帧内预测的模式或者所述当前块的量化参数进行解码；并且所述滤波设置部件可以根据由所述解码部件解码的帧内预测的模式或者由所述解码部件解码的量化参数设置所述系数。The encoding parameter may include a mode of intra prediction of the current block or a quantization parameter of the current block; the decoding unit may decode the mode of intra prediction of the current block or the quantization parameter of the current block; and the filter setting unit The coefficient may be set according to a mode of intra prediction decoded by the decoding section or a quantization parameter decoded by the decoding section.

图像处理装置还可以包括：滤波系数存储部件，被配置为存储所述系数；其中，所述系数被计算以用学习图像获得作为编码处理的对象的学习块与预测图像之间的最小残差，并且，所述系数被存储在所述滤波系数存储部件中，所述预测图像是通过对所述学习块执行与所述学习块的帧内预测的模式或者所述学习块的量化参数相对应的帧内预测而获得的；并且所述滤波设置部件将存储于所述滤波系数存储部件中的所述系数当中的、与所述当前块的帧内预测的模式或者所述当前块的量化参数相对应的系数设置作为所述系数。The image processing apparatus may further include: a filter coefficient storage section configured to store the coefficient; wherein the coefficient is calculated to obtain a minimum residual between the learning block that is the target of the encoding process and the prediction image using the learning image, And, the coefficient is stored in the filter coefficient storage part, and the prediction image is obtained by performing a mode corresponding to intra prediction of the learning block or a quantization parameter of the learning block on the learning block. obtained by intra-frame prediction; and the filter setting section sets the mode of intra-frame prediction of the current block or the quantization parameter of the current block among the coefficients stored in the filter coefficient storage section The corresponding coefficient is set as the coefficient.

所述滤波系数存储部件可以根据处理器的寄存器长度将所述系数保持为n(其中，n是整数)比特值。The filter coefficient storage section may hold the coefficient as an n (where n is an integer) bit value according to a register length of the processor.

所述解码部件可以对所述系数进行解码，在编码侧处使用在经过所述滤波处理之前的所述邻近像素计算了该系数，使得关于通过对于所述当前块执行的帧内预测而获得的预测图像的残差是最小的，并且，与所述当前块和所述当前块的帧内预测的模式或者所述当前块的量化参数相对应地设置了该系数；并且所述滤波设置部件可以将由所述解码部件解码的所述系数当中的、与所述当前块的帧内预测的模式或者所述当前块的量化参数相对应的系数设置作为所述系数。The decoding section may decode the coefficient, which was calculated at the encoding side using the neighboring pixels before being subjected to the filtering process, so that with respect to The residual error of the predicted image is the smallest, and the coefficient is set corresponding to the current block and the intra prediction mode of the current block or the quantization parameter of the current block; and the filter setting part may Among the coefficients decoded by the decoding section, a coefficient corresponding to a mode of intra prediction of the current block or a quantization parameter of the current block is set as the coefficient.

所述系数可以由滤波系数和偏移值构成。The coefficients may consist of filter coefficients and offset values.

所述滤波系数可以由三个抽头构成。The filter coefficients may consist of three taps.

所述滤波系数可以具有以对应于0相位的系数为中心的对称性。The filter coefficients may have symmetry centered on a coefficient corresponding to 0 phase.

所述解码部件可以对所述当前块的帧内预测的模式进行解码；并且said decoding means may decode a mode of intra prediction of said current block; and

所述滤波设置部件可以将帧内预测的模式中的垂直模式或水平模式取为第一类别，并且可以将其它模式取为第二类别，在所述当前块的帧内预测的模式属于所述第一类别的情况中，所述滤波设置部件可以设置与所述第一类别相对应的所述系数，在已经被解码的所述当前块的帧内预测的模式属于所述第二类别的情况中，所述滤波设置部件可以设置与所述第二类别相对应的所述系数。The filter setting section may take a vertical mode or a horizontal mode among intra-predicted modes as a first category, and may take other modes as a second category, and the intra-predicted mode of the current block belongs to the In the case of the first category, the filter setting section may set the coefficient corresponding to the first category, and in the case where the intra prediction mode of the current block that has been decoded belongs to the second category In, the filter setting means may set the coefficient corresponding to the second category.

图像处理装置还可以包括：接收部件，被配置为接收指示对于所述邻近像素是否执行所述滤波处理的标志信息；其中，基于由所述接收部件接收到的标志信息，所述滤波设置部件设置对于所述邻近像素是否执行所述滤波处理。The image processing apparatus may further include: receiving means configured to receive flag information indicating whether to perform the filtering process for the neighboring pixels; wherein, based on the flag information received by the receiving means, the filter setting means sets Whether to perform the filtering process for the neighboring pixels.

所述接收部件可以以宏块为增量接收所述标志信息；并且基于由所述接收部件接收到的标志信息，所述滤波设置部件可以设置对于所述邻近像素是否执行所述滤波处理。The receiving section may receive the flag information in increments of macroblocks; and based on the flag information received by the receiving section, the filter setting section may set whether to perform the filtering process for the neighboring pixels.

所述接收部件可以以块为增量接收所述标志信息；并且基于由所述接收部件接收到的标志信息，所述滤波设置部件可以设置是否以块为增量对所述邻近像素执行所述滤波处理。The receiving section may receive the flag information in block increments; and based on the flag information received by the receiving section, the filter setting section may set whether to perform the filter processing.

根据本发明的第一方面的图像处理方法包括下述步骤：图像处理装置执行对要作为解码处理的对象的当前块的图像进行解码；根据编码参数设置与所述当前块相应的用于要对所述当前块的邻近像素进行的滤波处理的系数，所述邻近像素用于所述当前块的帧内预测；以及使用已经设置的所述系数，对所述邻近像素进行所述滤波处理，并且对所述当前块执行帧内预测。The image processing method according to the first aspect of the present invention includes the following steps: the image processing device performs decoding of the image of the current block to be decoded; coefficients of filtering processing performed on neighboring pixels of the current block, the neighboring pixels being used for intra prediction of the current block; and performing the filtering processing on the neighboring pixels using the coefficients that have been set, and Intra prediction is performed on the current block.

根据本发明的第二方面的图像处理装置包括：滤波设置部件，被配置为根据编码参数设置用于要对要作为解码处理的对象的当前块的邻近像素进行的滤波处理的系数，所述邻近像素用于所述当前块的帧内预测；帧内预测部件，被配置为：使用由所述滤波设置部件设置的所述系数，使所述邻近像素经过所述滤波处理，并且对所述当前块执行帧内预测；以及编码部件，被配置为对所述当前块的图像进行编码。An image processing apparatus according to a second aspect of the present invention includes: a filter setting section configured to set coefficients for filter processing to be performed on neighboring pixels of a current block to be a target of decoding processing according to encoding parameters, the neighboring pixels A pixel is used for intra-frame prediction of the current block; an intra-frame prediction unit configured to: use the coefficient set by the filter setting unit, subject the neighboring pixels to the filtering process, and perform the filtering process on the current block. a block performing intra prediction; and an encoding unit configured to encode an image of the current block.

编码参数可以包括当前块的帧内预测的模式或者当前块的量化参数；并且所述滤波设置部件可以根据所述当前块的帧内预测的模式或者所述当前块的量化参数设置所述系数；并且编码部件可以对所述当前块的帧内预测的相应模式或者所述当前块的量化参数进行编码。The encoding parameter may include a mode of intra prediction of the current block or a quantization parameter of the current block; and the filter setting part may set the coefficient according to a mode of intra prediction of the current block or the quantization parameter of the current block; And the encoding component may encode a corresponding mode of intra prediction of the current block or a quantization parameter of the current block.

图像处理装置还可以包括：滤波系数存储部件，被配置为存储所述系数；其中，所述系数被计算以用学习图像获得作为编码处理的对象的学习块与预测图像之间的最小残差，并且，所述系数被存储在所述滤波系数存储部件中，所述预测图像是通过对所述学习块执行与所述学习块的帧内预测的模式或者所述学习块的量化参数相对应的帧内预测而获得的；并且所述滤波设置部件将存储于所述滤波系数存储部件中的所述系数当中的、与所述当前块的帧内预测的模式或者所述当前块的量化参数相对应的系数设置作为所述系数。The image processing apparatus may further include: a filter coefficient storage section configured to store the coefficient; wherein the coefficient is calculated to obtain a minimum residual between the learning block that is the target of the encoding process and the prediction image using the learning image, And, the coefficient is stored in the filter coefficient storage part, and the prediction image is obtained by performing a mode corresponding to intra prediction of the learning block or a quantization parameter of the learning block on the learning block. obtained by intra-frame prediction; and the filter setting section sets the mode of intra-frame prediction of the current block or the quantization parameter of the current block among the coefficients stored in the filter coefficient storage section The corresponding coefficient is set as the coefficient.

图像处理装置还可以包括：滤波系数计算部件，被配置为计算所述系数，使得所述当前块和预测图像之间的残差最小，所述预测图像是通过根据所述当前块的帧内预测的模式或者所述当前块的量化参数，使用在进行所述滤波处理之前的所述邻近像素对所述学习块执行帧内预测而获得的；所述滤波设置部件将由所述滤波系数计算部件计算出的所述系数当中的、与所述当前块的帧内预测的模式或者所述当前块的量化参数相对应的系数设置作为所述系数。The image processing apparatus may further include: a filter coefficient calculation section configured to calculate the coefficient so that a residual error between the current block and a predicted image obtained through intra prediction based on the current block is minimized mode of the current block or the quantization parameter of the current block, which is obtained by performing intra-frame prediction on the learning block using the neighboring pixels before performing the filtering process; the filter setting part will be calculated by the filter coefficient calculation part Among the obtained coefficients, the coefficient corresponding to the intra prediction mode of the current block or the quantization parameter of the current block is set as the coefficient.

所述系数可以由滤波系数和偏移值构成。The coefficients may consist of filter coefficients and offset values.

所述滤波设置部件可以设置是否对邻近像素执行所述滤波处理；并且所述编码部件可以对指示是否执行由所述滤波设置部件设置的所述滤波处理的标志信息进行编码。The filter setting section may set whether to perform the filter processing on adjacent pixels; and the encoding section may encode flag information indicating whether to perform the filter process set by the filter setting section.

根据本发明的第二方面的图像处理方法包括下述步骤：图像处理装置执行根据编码参数设置用于要对要作为解码处理的对象的当前块的邻近像素进行的滤波处理的系数，所述邻近像素用于所述当前块的帧内预测；使用已经设置的所述系数，对所述邻近像素进行所述滤波处理，并且对所述当前块执行帧内预测；以及对所述当前块的图像进行编码。An image processing method according to a second aspect of the present invention includes the step of: an image processing device performs a coefficient for filter processing to be performed on adjacent pixels of a current block to be an object of decoding processing set according to an encoding parameter, the adjacent pixels The pixels are used for intra-frame prediction of the current block; using the coefficients that have been set, the filtering process is performed on the adjacent pixels, and intra-frame prediction is performed on the current block; and the image of the current block to encode.

根据本发明的第一方面，对要作为解码处理的对象的当前块的图像进行解码，根据编码参数设置系数，该系数用于要对当前块的邻近像素进行的滤波处理，其中，所述邻近像素用于当前块的帧内预测。然后，使用已经设置的系数对邻近像素进行滤波处理，并且，对当前块执行帧内预测。According to the first aspect of the present invention, the image of the current block to be decoded is decoded, and coefficients are set according to encoding parameters, and the coefficients are used for filtering processing to be performed on adjacent pixels of the current block, wherein the adjacent Pixels are used for intra prediction of the current block. Then, filter processing is performed on adjacent pixels using the coefficients that have been set, and intra prediction is performed on the current block.

根据本发明的第二方面，根据编码参数设置用于要对要作为解码处理的对象的当前块的邻近像素进行的滤波处理的系数，所述邻近像素用于当前块的帧内预测，使用已经设置的系数对邻近像素进行滤波处理，对当前块执行帧内预测，并且，对当前块的图像进行编码。According to the second aspect of the present invention, coefficients for filtering processing to be performed on adjacent pixels of a current block to be subjected to decoding processing, which are used for intra prediction of the current block, are set according to encoding parameters using The set coefficient performs filtering processing on adjacent pixels, performs intra prediction on the current block, and encodes an image of the current block.

请注意，上述的图像处理装置可以是独立的装置，或者，可以是构成一个图像编码装置或图像解码装置的内部块。Note that the image processing device described above may be an independent device, or may be an internal block constituting one image encoding device or image decoding device.

本发明的有益效果Beneficial effects of the present invention

根据第一发明，可以对图像进行解码。此外，根据第二发明，可以根据图像和比特率执行噪声去除。According to the first invention, an image can be decoded. Furthermore, according to the second invention, noise removal can be performed according to images and bit rates.

根据第二发明，可以对图像进行编码。此外，根据第一发明，可以根据图像和比特率执行噪声去除。According to the second invention, an image can be encoded. Furthermore, according to the first invention, noise removal can be performed according to images and bit rates.

附图说明Description of drawings

[图1]图1是图示应用了本发明的图像编码装置的实施例的配置的框图。[ Fig. 1] Fig. 1 is a block diagram illustrating a configuration of an embodiment of an image encoding device to which the present invention is applied.

[图2]图2是用于描述在16×16像素帧内预测模式的情况中的处理序列的示图。[ Fig. 2] Fig. 2 is a diagram for describing a processing sequence in the case of a 16×16 pixel intra prediction mode.

[图3]图3是图示亮度信号的4×4像素帧内预测模式的种类的示图。[ Fig. 3] Fig. 3 is a diagram illustrating kinds of 4x4-pixel intra prediction modes of a luminance signal.

[图4]图4是图示亮度信号的4×4像素帧内预测模式的种类的示图。[ Fig. 4] Fig. 4 is a diagram illustrating kinds of 4x4-pixel intra prediction modes of a luminance signal.

[图5]图5是用于描述4×4像素帧内预测的方向的示图。[ Fig. 5] Fig. 5 is a diagram for describing directions of 4×4 pixel intra prediction.

[图6]图6是用于描述4×4像素帧内预测的示图。[ Fig. 6] Fig. 6 is a diagram for describing 4×4 pixel intra prediction.

[图7]图7是用于描述亮度信号的4×4像素帧内预测模式的编码的示图。[ Fig. 7] Fig. 7 is a diagram for describing encoding of a 4×4 pixel intra prediction mode of a luminance signal.

[图8]图8是图示亮度信号的8×8像素帧内预测模式的种类的示图。[ Fig. 8] Fig. 8 is a diagram illustrating kinds of 8×8 pixel intra prediction modes of a luminance signal.

[图9]图9是图示亮度信号的8×8像素帧内预测模式的种类的示图。[ Fig. 9] Fig. 9 is a diagram illustrating kinds of 8x8-pixel intra prediction modes of a luminance signal.

[图10]图10是图示亮度信号的16×16像素帧内预测模式的种类的示图。[ Fig. 10] Fig. 10 is a diagram illustrating kinds of 16×16 pixel intra prediction modes of a luminance signal.

[图11]图11是图示亮度信号的16×16像素帧内预测模式的种类的示图。[ Fig. 11] Fig. 11 is a diagram illustrating kinds of 16×16 pixel intra prediction modes of a luminance signal.

[图12]图12是用于描述16×16像素帧内预测的示图。[ Fig. 12] Fig. 12 is a diagram for describing 16×16 pixel intra prediction.

[图13]图13是图示颜色差信号的帧内预测模式的种类的示图。[ Fig. 13] Fig. 13 is a diagram illustrating kinds of intra prediction modes of a color difference signal.

[图14]图14是图示图1中的邻近像素内插滤波切换单元和帧内预测单元的配置例子的框图。[ Fig. 14] Fig. 14 is a block diagram illustrating a configuration example of a neighboring pixel interpolation filtering switching unit and an intra prediction unit in Fig. 1 .

[图15]图15是用于描述滤波系数的计算的示图。[ Fig. 15] Fig. 15 is a diagram for describing calculation of filter coefficients.

[图16]图16是用于描述图1中的图像编码装置的编码处理的流程图。[ Fig. 16] Fig. 16 is a flowchart for describing encoding processing of the image encoding device in Fig. 1 .

[图17]图17是用于描述图16中的步骤S21中的预测处理的流程图。[ Fig. 17] Fig. 17 is a flowchart for describing prediction processing in step S21 in Fig. 16 .

[图18]图18是用于描述图17中的步骤S31中的帧内预测处理的流程图。[ Fig. 18] Fig. 18 is a flowchart for describing intra prediction processing in step S31 in Fig. 17 .

[图19]图19是用于描述图17中的步骤S32中的帧内运动预测处理的流程图。[ Fig. 19] Fig. 19 is a flowchart for describing intra motion prediction processing in step S32 in Fig. 17 .

[图20]图20是图示图1中的邻近像素内插滤波切换单元和帧内预测单元的另一配置例子的框图。[ Fig. 20] Fig. 20 is a block diagram illustrating another configuration example of the adjacent pixel interpolation filtering switching unit and the intra prediction unit in Fig. 1 .

[图21]图21是用于描述图17中的步骤S31中的帧内预测处理的另一例子的流程图。[ Fig. 21] Fig. 21 is a flowchart for describing another example of intra prediction processing in step S31 in Fig. 17 .

[图22]图22是图示应用了本发明的图像解码装置的实施例的配置的框图。[ Fig. 22] Fig. 22 is a block diagram illustrating a configuration of an embodiment of an image decoding device to which the present invention is applied.

[图23]图23是图示图22中的邻近像素内插滤波切换单元和帧内预测单元的配置例子的框图。[ Fig. 23] Fig. 23 is a block diagram illustrating a configuration example of a neighboring pixel interpolation filtering switching unit and an intra prediction unit in Fig. 22 .

[图24]图24是用于描述图22中的图像解码装置的解码处理的流程图。[ Fig. 24] Fig. 24 is a flowchart for describing decoding processing of the image decoding device in Fig. 22 .

[图25]图25是用于描述图24中的步骤S138中的预测处理的流程图。[ Fig. 25] Fig. 25 is a flowchart for describing prediction processing in step S138 in Fig. 24 .

[图26]图26是图示图22中的邻近像素内插滤波切换单元和帧内预测单元的另一配置例子的框图。[ Fig. 26] Fig. 26 is a block diagram illustrating another configuration example of the adjacent pixel interpolation filtering switching unit and the intra prediction unit in Fig. 22 .

[图27]图27是用于描述图24中的步骤S138中的预测处理的另一例子的流程图。[ Fig. 27] Fig. 27 is a flowchart for describing another example of prediction processing in step S138 in Fig. 24 .

[图28]图28是图示应用了本发明的学习装置的实施例的配置的框图。[ Fig. 28] Fig. 28 is a block diagram illustrating a configuration of an embodiment of a learning device to which the present invention is applied.

[图29]图29是图示图28中的邻近像素内插滤波计算单元和帧内预测单元的配置例子的框图。[ Fig. 29] Fig. 29 is a block diagram illustrating a configuration example of a neighboring pixel interpolation filtering calculation unit and an intra prediction unit in Fig. 28 .

[图30]图30是用于描述图28中的采用学习装置的帧内预测处理的流程图。[ Fig. 30] Fig. 30 is a flowchart for describing intra prediction processing using a learning device in Fig. 28 .

[图31]图31是图示应用了本发明的图像编码装置的另一实施例的配置的框图。[ Fig. 31] Fig. 31 is a block diagram illustrating a configuration of another embodiment of an image encoding device to which the present invention is applied.

[图32]图32是用于描述二次预测处理的示图。[ Fig. 32] Fig. 32 is a diagram for describing secondary prediction processing.

[图33]图33是图示应用了本发明的图像解码装置的另一实施例的配置的框图。[ Fig. 33] Fig. 33 is a block diagram illustrating a configuration of another embodiment of an image decoding device to which the present invention is applied.

[图34]图34是图示应用了本发明的图像编码装置的又一实施例的配置的框图。[ Fig. 34] Fig. 34 is a block diagram illustrating a configuration of still another embodiment of an image encoding device to which the present invention is applied.

[图35]图35是图示图34中的帧内预测单元的配置例子的框图。[ Fig. 35] Fig. 35 is a block diagram illustrating a configuration example of an intra prediction unit in Fig. 34 .

[图36]图36是用于描述图17中的步骤S31中的帧内预测处理的另一例子的流程图。[ Fig. 36] Fig. 36 is a flowchart for describing another example of intra prediction processing in step S31 in Fig. 17 .

[图37]图37是用于描述图17中的步骤S31中的帧内预测处理的又一例子的流程图。[ Fig. 37] Fig. 37 is a flowchart for describing still another example of intra prediction processing in step S31 in Fig. 17 .

[图38]图38是用于描述图17中的步骤S31中的帧内预测处理的另一例子的流程图。[ Fig. 38] Fig. 38 is a flowchart for describing another example of intra prediction processing in step S31 in Fig. 17 .

[图39]图39是图示应用了本发明的图像解码装置的又一实施例的配置的框图。[ Fig. 39] Fig. 39 is a block diagram illustrating a configuration of still another embodiment of an image decoding device to which the present invention is applied.

[图40]图40是图示图39中的邻近像素内插滤波控制单元和帧内预测单元的配置例子的框图。[ Fig. 40] Fig. 40 is a block diagram illustrating a configuration example of a neighboring pixel interpolation filtering control unit and an intra prediction unit in Fig. 39 .

[图41]图41是用于描述图24中的步骤S138中的预测处理的又一例子的流程图。[ Fig. 41] Fig. 41 is a flowchart for describing still another example of prediction processing in step S138 in Fig. 24 .

[图42]图42是图示应用了本发明的图像编码装置的另一实施例的配置的框图。[ Fig. 42] Fig. 42 is a block diagram illustrating a configuration of another embodiment of an image encoding device to which the present invention is applied.

[图43]图43是图示应用了本发明的图像解码装置的另一实施例的配置的框图。[ Fig. 43] Fig. 43 is a block diagram illustrating a configuration of another embodiment of an image decoding device to which the present invention is applied.

[图44]图44是图示扩展块尺寸的例子的示图。[ Fig. 44] Fig. 44 is a diagram illustrating an example of an extended block size.

[图45]图45是图示计算机的硬件的配置例子的框图。[ Fig. 45] Fig. 45 is a block diagram illustrating a configuration example of hardware of a computer.

[图46]图46是图示应用了本发明的电视接收器的主要配置例子的框图。[ Fig. 46] Fig. 46 is a block diagram illustrating a main configuration example of a television receiver to which the present invention is applied.

[图47]图47是图示应用了本发明的蜂窝电话的主要配置例子的框图。[ Fig. 47] Fig. 47 is a block diagram illustrating a main configuration example of a cellular phone to which the present invention is applied.

[图48]图48是图示应用了本发明的硬盘记录器的主要配置例子的框图。[ Fig. 48] Fig. 48 is a block diagram illustrating a main configuration example of a hard disk recorder to which the present invention is applied.

[图49]图49是图示应用了本发明的照相机的主要配置例子的框图。[ Fig. 49] Fig. 49 is a block diagram illustrating a main configuration example of a camera to which the present invention is applied.

具体实施方式detailed description

下文中将参照附图描述本发明的实施例。请注意，将按照以下顺序进行描述。Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that description will be made in the following order.

1.第一实施例(邻近像素内插滤波切换：帧内预测的例子)1. First Embodiment (Nearby Pixel Interpolation Filter Switching: Example of Intra Prediction)

2.第二实施例(邻近像素内插滤波切换：二次预测的例子)2. Second Embodiment (Nearby Pixel Interpolation Filter Switching: Example of Secondary Prediction)

3.第三实施例(邻近像素内插滤波on/off(开/关)控制：帧内预测的例子)3. Third embodiment (adjacent pixel interpolation filter on/off (on/off) control: example of intra prediction)

4.第四实施例(邻近像素内插滤波打开/关闭控制：二次预测的例子)4. Fourth Embodiment (Nearby Pixel Interpolation Filter On/Off Control: Example of Secondary Prediction)

<1.第一实施例><1. First embodiment>

[图像编码装置的配置例子][Configuration Example of Image Coding Device]

图1表示用作应用了本发明的图像处理装置的图像编码装置的实施例的配置。FIG. 1 shows the configuration of an embodiment of an image encoding device serving as an image processing device to which the present invention is applied.

该图像编码装置51使用例如H.264和MPEG-4 Part10(高级视频编码)(下文中被描述为H.264/AVC)格式来对图像进行压缩编码。The image encoding device 51 compression-encodes images using, for example, H.264 and MPEG-4 Part 10 (Advanced Video Coding) (hereinafter described as H.264/AVC) formats.

对于图1中的例子，图像编码装置51由A/D转换单元61、画面重新排列缓冲器62、计算单元63、正交变换单元64、量化单元65、无损耗编码单元66、存储缓冲器67、逆量化单元68、逆正交变换单元69、计算单元70、去块滤波器71、帧存储器72、开关73、帧内预测单元74、邻近像素内插滤波切换单元75、运动预测/补偿单元76、预测图像选择单元77和速率控制单元78构成。For the example in Fig. 1, the image coding device 51 is composed of an A/D conversion unit 61, a picture rearrangement buffer 62, a calculation unit 63, an orthogonal transformation unit 64, a quantization unit 65, a lossless coding unit 66, and a storage buffer 67 , inverse quantization unit 68, inverse orthogonal transformation unit 69, calculation unit 70, deblocking filter 71, frame memory 72, switch 73, intra prediction unit 74, adjacent pixel interpolation filtering switching unit 75, motion prediction/compensation unit 76. A predictive image selection unit 77 and a rate control unit 78 are formed.

A/D转换单元61对输入图像进行模拟数字转换，并且将该输入图像输出到画面重新排列缓冲器62以便存储。画面重新排列缓冲器62根据GOP(图片组)将用于显示的按照存储的顺序的帧的图像重新排列为用于编码的帧的顺序。The A/D conversion unit 61 performs analog-digital conversion on an input image, and outputs the input image to the screen rearrangement buffer 62 for storage. The screen rearrangement buffer 62 rearranges images of frames in the stored order for display into the order of frames for encoding according to GOP (Group of Pictures).

计算单元63从由画面重新排列缓冲器62读出的图像当中减去由预测图像选择单元77选择的来自帧内预测单元74的预测图像或者来自运动预测/补偿单元76的预测图像，并且将其差分信息输出到正交变换单元64。正交变换单元64对来自计算单元63的差分信息进行诸如离散余弦变换、Karhunen-Loéve变换等之类的正交变换，并且输出其变换系数。量化单元65对正交变换单元64输出的变换系数进行量化。The calculation unit 63 subtracts the predicted image from the intra prediction unit 74 or the predicted image from the motion prediction/compensation unit 76 selected by the predicted image selection unit 77 from the images read out by the screen rearrangement buffer 62, and converts it to The difference information is output to the orthogonal transform unit 64 . The orthogonal transform unit 64 performs orthogonal transform such as discrete cosine transform, Karhunen-Loéve transform, etc. on the difference information from the calculation unit 63 and outputs transform coefficients thereof. The quantization unit 65 quantizes the transform coefficient output from the orthogonal transform unit 64 .

作为量化单元65的输出的量化变换系数被输入到无损耗编码单元66，在无损耗编码单元66中对它进行诸如可变长度编码、算术编码等之类的无损耗编码和压缩。The quantized transform coefficient as an output of the quantization unit 65 is input to a lossless encoding unit 66 where it is subjected to lossless encoding and compression such as variable length encoding, arithmetic encoding, and the like.

无损耗编码单元66从帧内预测单元74获得指示帧内预测等的信息，并且从运动预测/补偿单元76获取指示帧间预测模式等的信息。请注意，在下文中，指示帧内预测的信息将被称为帧内预测模式信息。同样地，在下文中，指示帧间预测的信息将被称为帧间预测模式信息。The lossless encoding unit 66 acquires information indicating intra prediction and the like from the intra prediction unit 74 , and acquires information indicating an inter prediction mode and the like from the motion prediction/compensation unit 76 . Note that hereinafter, information indicating intra prediction will be referred to as intra prediction mode information. Also, hereinafter, information indicating inter prediction will be referred to as inter prediction mode information.

无损耗编码单元66对量化的变换系数进行编码，还对指示帧内预测的信息、指示帧间预测模式的信息、量化参数等进行编码，并且取这些信息作为压缩图像中的头信息的一部分。无损耗编码单元66将编码数据供应给存储缓冲器67以便存储。The lossless encoding unit 66 encodes quantized transform coefficients, also encodes information indicating intra prediction, information indicating inter prediction mode, quantization parameters, and the like, and takes these as part of header information in compressed images. The lossless encoding unit 66 supplies the encoded data to the storage buffer 67 for storage.

例如，对于无损耗编码单元66，执行诸如可变长度编码、算术编码等之类的无损耗编码处理。可变长度编码的例子包括由H.264/AVC格式确定的CAVLC(上下文自适应可变长度编码)。算术编码的例子包括CABAC(上下文自适应二进制算术编码)。For example, with the lossless encoding unit 66, lossless encoding processing such as variable length encoding, arithmetic encoding, and the like is performed. Examples of variable length coding include CAVLC (Context Adaptive Variable Length Coding) specified by the H.264/AVC format. Examples of arithmetic coding include CABAC (Context Adaptive Binary Arithmetic Coding).

存储缓冲器67将从无损耗编码单元66供应的数据作为由H.264/AVC格式编码的压缩图像输出到在该图中未示出的下游的存储装置或传送路径等。The storage buffer 67 outputs the data supplied from the lossless encoding unit 66 as a compressed image encoded by the H.264/AVC format to a downstream storage device or transmission path or the like not shown in the figure.

此外，从量化单元65输出的量化变换系数还被输入到逆量化单元68，进行了逆量化，然后，在逆正交变换单元69处进一步进行了逆正交变换。由计算单元70将进行了逆正交变换的输出与从预测图像选择单元77供应的预测图像相加，并且将其改变为局部解码的图像。去块滤波器71从解码图像当中去除块失真(block distortion)，然后将其供应到帧存储器72，以便存储。在去块滤波器71执行去块滤波处理之前的图像也被供应给帧存储器72，以便存储。Furthermore, the quantized transform coefficients output from the quantization unit 65 are also input to the inverse quantization unit 68 , subjected to inverse quantization, and then further subjected to inverse orthogonal transformation at the inverse orthogonal transformation unit 69 . The output subjected to the inverse orthogonal transformation is added to the predicted image supplied from the predicted image selection unit 77 by the calculation unit 70 and changed into a locally decoded image. The deblocking filter 71 removes block distortion from the decoded image, which is then supplied to the frame memory 72 for storage. The image before the deblocking filter 71 performs deblocking filter processing is also supplied to the frame memory 72 to be stored.

开关73将存储于帧存储器72中的参考图像输出到运动预测/补偿单元76或帧内预测单元74。The switch 73 outputs the reference image stored in the frame memory 72 to the motion prediction/compensation unit 76 or the intra prediction unit 74 .

例如，对于该图像编码装置51，来自画面重新排列缓冲器62的I图片、B图片和P图片作为要进行帧内预测(也被称为帧内处理)的图像被供应给帧内预测单元74。此外，从画面重新排列缓冲器62读出的B图片和P图片作为要进行帧间预测(也被称为帧间处理)的图像被供应给运动预测/补偿单元76。For example, with this image encoding device 51, I pictures, B pictures, and P pictures from the screen rearranging buffer 62 are supplied to the intra prediction unit 74 as images to be subjected to intra prediction (also referred to as intra processing). . Furthermore, the B pictures and P pictures read out from the screen rearranging buffer 62 are supplied to the motion prediction/compensation unit 76 as images to be subjected to inter prediction (also referred to as inter processing).

帧内预测单元74基于从画面重新排列缓冲器62读出的要进行帧内预测的图像和从帧存储器72供应的参考图像执行所有的候选帧内预测模式的帧内预测处理，以产生预测图像。The intra prediction unit 74 performs intra prediction processing for all the candidate intra prediction modes based on the image to be intra predicted read out from the screen rearrangement buffer 62 and the reference image supplied from the frame memory 72 to generate a predicted image .

在帧内预测处理之前，帧内预测单元74对邻近像素执行滤波处理，所述邻近像素是用于每一个当前块的帧内预测且以预定的位置关系与当前块邻近的像素。根据从帧内预测单元74供应的帧内预测模式等，该滤波处理使用由邻近像素内插滤波切换单元75设置的滤波系数。也就是说，对于所有的候选帧内预测模式的帧内预测处理，帧内预测单元74使用进行了采用由邻近像素内插滤波切换单元75设置的滤波系数的滤波处理的邻近像素。Before the intra prediction process, the intra prediction unit 74 performs filter processing on adjacent pixels, which are pixels used for intra prediction of each current block and adjacent to the current block in a predetermined positional relationship. This filter processing uses the filter coefficient set by the adjacent pixel interpolation filter switching unit 75 according to the intra prediction mode and the like supplied from the intra prediction unit 74 . That is, for intra prediction processing of all candidate intra prediction modes, the intra prediction unit 74 uses adjacent pixels subjected to filter processing with filter coefficients set by the adjacent pixel interpolation filter switching unit 75 .

帧内预测单元74计算关于产生了预测图像的帧内预测模式的成本函数值(costfunction value)，并且，选择其中计算出的成本函数值给出了最小值的帧内预测模式作为最佳帧内预测模式。帧内预测单元74将在最佳帧内预测模式中产生的预测图像和针对相应的最佳帧内预测模式计算出的成本函数值供应给预测图像选择单元77。The intra prediction unit 74 calculates a cost function value (costfunction value) with respect to the intra prediction mode that generated the predicted image, and selects the intra prediction mode in which the calculated cost function value gives the smallest value as the optimum intra prediction mode. predictive mode. The intra prediction unit 74 supplies the predicted image generated in the optimal intra prediction mode and the cost function value calculated for the corresponding optimal intra prediction mode to the predicted image selection unit 77 .

在预测图像选择单元77选择了在最佳帧内预测模式中产生的预测图像的情况中，帧内预测单元74将指示最佳帧内预测模式的信息供应给无损耗编码单元66。在从帧内预测单元74传送了信息的情况中，无损耗编码单元66对该信息进行编码，并且取该信息作为压缩图像中的头信息的一部分。In a case where the predicted image selection unit 77 has selected a predicted image generated in the optimal intra prediction mode, the intra prediction unit 74 supplies information indicating the optimal intra prediction mode to the lossless encoding unit 66 . In the case where information is transmitted from the intra prediction unit 74, the lossless encoding unit 66 encodes the information, and takes the information as part of the header information in the compressed image.

邻近像素内插滤波切换单元75存储通过使用训练图像在稍后描述的图28中的学习装置251处执行学习而获得的、与量化参数和帧内预测模式相对应的滤波系数。The adjacent pixel interpolation filter switching unit 75 stores filter coefficients corresponding to quantization parameters and intra prediction modes obtained by performing learning at the learning means 251 in FIG. 28 described later using training images.

向邻近像素内插滤波切换单元75供应来自速率控制单元78的量化参数和来自帧内预测单元74的帧内预测模式信息。邻近像素内插滤波切换单元75设置与来自速率控制单元78的量化参数和来自帧内预测单元74的帧内预测模式相对应的滤波系数。邻近像素内插滤波切换单元75将设置的滤波系数供应给帧内预测单元74。The quantization parameter from the rate control unit 78 and the intra prediction mode information from the intra prediction unit 74 are supplied to the adjacent pixel interpolation filter switching unit 75 . The adjacent pixel interpolation filter switching unit 75 sets a filter coefficient corresponding to the quantization parameter from the rate control unit 78 and the intra prediction mode from the intra prediction unit 74 . The adjacent pixel interpolation filter switching unit 75 supplies the set filter coefficient to the intra prediction unit 74 .

请注意，邻近像素内插滤波切换单元75可以执行与量化系数和帧内预测模式中的仅仅一个而不是两个相对应的滤波系数的学习和存储。Note that the adjacent pixel interpolation filter switching unit 75 may perform learning and storage of filter coefficients corresponding to only one of quantization coefficients and intra prediction modes, not both.

此外，虽然邻近像素内插滤波切换单元75存储预先离线学习的滤波系数，但是可以代替地在线计算滤波系数。在这种情况下，由邻近像素内插滤波切换单元75设置的滤波系数被输出到无损耗编码单元66以发送给解码侧，如点线箭头所示。Furthermore, although the adjacent pixel interpolation filter switching unit 75 stores filter coefficients learned offline in advance, the filter coefficients may be calculated online instead. In this case, the filter coefficient set by the adjacent pixel interpolation filter switching unit 75 is output to the lossless encoding unit 66 to be sent to the decoding side, as indicated by the dotted arrow.

运动预测/补偿单元76针对所有的候选帧间预测模式执行运动预测和补偿处理。具体地，经由开关73向运动预测/补偿单元76供应从画面重新排列缓冲器62读出的要进行帧间处理的图像和来自帧存储器72的参考图像。运动预测/补偿单元76基于要进行帧间处理的图像和参考图像检测所有的候选帧间预测模式的运动向量，基于运动向量对参考图像进行补偿处理，并且产生预测图像。The motion prediction/compensation unit 76 performs motion prediction and compensation processing for all candidate inter prediction modes. Specifically, the image to be subjected to inter-frame processing read out from the screen rearranging buffer 62 and the reference image from the frame memory 72 are supplied to the motion prediction/compensation unit 76 via the switch 73 . The motion prediction/compensation unit 76 detects motion vectors of all candidate inter prediction modes based on the image to be inter-processed and the reference image, performs compensation processing on the reference image based on the motion vector, and generates a predicted image.

此外，运动预测/补偿单元76针对所有的候选帧间预测模式计算成本函数值。运动预测/补偿单元76确定计算出的成本函数值当中的提供最小值的预测模式作为最佳帧间预测模式。Furthermore, motion prediction/compensation unit 76 calculates cost function values for all candidate inter prediction modes. The motion prediction/compensation unit 76 determines the prediction mode providing the smallest value among the calculated cost function values as the optimal inter prediction mode.

运动预测/补偿单元76将在最佳帧间预测模式中产生的预测图像及其成本函数值供应给预测图像选择单元77。在预测图像选择单元77选择了在最佳帧间预测模式中产生的预测图像的情况中，运动预测/补偿单元76将指示最佳帧间预测模式的信息(帧间预测模式信息)输出给无损耗编码单元66。The motion prediction/compensation unit 76 supplies the predicted image generated in the optimum inter prediction mode and its cost function value to the predicted image selection unit 77 . In a case where the predicted image selection unit 77 has selected the predicted image generated in the optimum inter prediction mode, the motion prediction/compensation unit 76 outputs information indicating the optimum inter prediction mode (inter prediction mode information) to none Loss coding unit 66 .

请注意，运动向量信息、标志信息、参考帧信息等根据需要被输出到无损耗编码单元66。无损耗编码单元66也对来自运动预测/补偿单元76的信息进行诸如可变长度编码、算术编码等的无损耗编码处理，并且将其插入压缩图像的头部中。Note that motion vector information, flag information, reference frame information, and the like are output to the lossless encoding unit 66 as necessary. The lossless encoding unit 66 also performs lossless encoding processing such as variable length encoding, arithmetic encoding, etc. on the information from the motion prediction/compensation unit 76, and inserts it into the header of the compressed image.

预测图像选择单元77基于从帧内预测单元74或运动预测/补偿单元76输出的成本函数值从最佳帧内预测模式和最佳帧间预测模式中确定最佳预测模式。预测图像选择单元77然后选择确定的最佳预测模式中的预测图像，并且将其供应给计算单元63和70。此时，预测图像选择单元77将预测图像的选择信息供应给帧内预测单元74或运动预测/补偿单元76。The predicted image selection unit 77 determines the optimum prediction mode from the optimum intra prediction mode and the optimum inter prediction mode based on the cost function value output from the intra prediction unit 74 or the motion prediction/compensation unit 76 . The predicted image selection unit 77 then selects the predicted image in the determined optimum prediction mode, and supplies it to the calculation units 63 and 70 . At this time, the predicted image selection unit 77 supplies selection information of the predicted image to the intra prediction unit 74 or the motion prediction/compensation unit 76 .

速率控制单元78基于存储于存储缓冲器67中的压缩图像用量化参数控制量化单元65的量化操作的速率，以不会导致上溢或下溢。The rate control unit 78 controls the rate of the quantization operation of the quantization unit 65 based on the quantization parameters for compressed images stored in the storage buffer 67 so as not to cause overflow or underflow.

将用于在量化单元65处的速率控制的量化参数供应给无损耗编码单元66，对该量化参数进行无损耗编码处理，并且将其插入到压缩图像的头部。该量化参数被供应给邻近像素内插滤波切换单元75，并且被用于设置用于要对邻近像素施加的滤波处理的滤波系数。The quantization parameter used for rate control at the quantization unit 65 is supplied to the lossless encoding unit 66, is subjected to lossless encoding processing, and is inserted into the header of the compressed image. This quantization parameter is supplied to the adjacent pixel interpolation filter switching unit 75, and is used to set a filter coefficient for filter processing to be applied to the adjacent pixels.

[根据H.264/AVC格式的帧内预测处理的描述][Description of intra prediction processing according to H.264/AVC format]

首先，将描述由H.264/AVC格式确定的帧内预测模式。First, the intra prediction mode determined by the H.264/AVC format will be described.

首先，将描述亮度信号的帧内预测模式。对于亮度信号的帧内预测模式，确定帧内4×4预测模式、帧内8×8预测模式和帧内16×16预测模式的三个系统。存在用于确定块单位的模式，针对每一个宏块设置模式。此外，针对每一个宏块，可以与亮度信号独立地给颜色差信号设置帧内预测模式。First, an intra prediction mode of a luma signal will be described. For the intra prediction mode of the luma signal, three systems of intra 4×4 prediction mode, intra 8×8 prediction mode, and intra 16×16 prediction mode are determined. There is a mode for determining a block unit, and the mode is set for each macroblock. Furthermore, for each macroblock, an intra prediction mode can be set for the color difference signal independently of the luma signal.

此外，在帧内4×4预测模式的情况中，对于每一个4×4像素当前块，可以设置九种预测模式当中的一种预测模式。此外，在帧内8×8预测模式的情况中，对于每一个8×8像素当前块，可以设置九种预测模式当中的一种预测模式。此外，在帧内16×16预测模式的情况中，可以给16×16像素当前宏块设置四种预测模式当中的一种预测模式。Also, in the case of the intra 4×4 prediction mode, for each current block of 4×4 pixels, one prediction mode among nine kinds of prediction modes can be set. Furthermore, in the case of the intra 8×8 prediction mode, for each current block of 8×8 pixels, one prediction mode among nine kinds of prediction modes can be set. Furthermore, in the case of the intra 16×16 prediction mode, one prediction mode among four prediction modes may be set for a current macroblock of 16×16 pixels.

请注意，在下文中，帧内4×4预测模式、帧内8×8预测模式和帧内16×16预测模式在适当的时候也将分别被称为4×4像素帧内预测模式、8×8像素帧内预测模式和16×16像素帧内预测模式。Please note that in the following, Intra 4×4 prediction mode, Intra 8×8 prediction mode and Intra 16×16 prediction mode will also be referred to as 4×4 pixel intra prediction mode, 8× 8 pixel intra prediction mode and 16×16 pixel intra prediction mode.

对于图2中的例子，附于各块的数字1至25表示其块的比特流序列(解码侧的处理序列)。请注意，关于亮度信号，宏块被分割为4×4像素，并且，执行4×4像素的DCT。仅仅在帧内16×16预测模式的情况中，如在1的块中所示，聚集块的DC成分，产生4×4矩阵，并且，对该矩阵还进行正交变换。With the example in FIG. 2, numbers 1 to 25 attached to each block indicate the bit stream sequence (processing sequence on the decoding side) of the block thereof. Note that, regarding the luminance signal, a macroblock is divided into 4×4 pixels, and DCT of 4×4 pixels is performed. Only in the case of the intra 16×16 prediction mode, as shown in the block of 1, the DC components of the block are aggregated, a 4×4 matrix is generated, and the matrix is also subjected to orthogonal transformation.

另一方面，关于颜色差信号，在宏块被分割为4×4像素并且执行4×4像素的DCT之后，如块16和17中所示，聚集块的DC成分，产生2×2矩阵，并且对该矩阵还进行正交变换。On the other hand, regarding the color difference signal, after the macroblock is divided into 4x4 pixels and DCT of 4x4 pixels is performed, as shown in blocks 16 and 17, the DC components of the blocks are aggregated, resulting in a 2x2 matrix, And an orthogonal transformation is performed on the matrix.

请注意，关于帧内8×8预测模式，这可以仅仅适用于下述情况：其中，以高画质或更高的画质(high profile or a profile beyond this)对当前宏块进行8×8正交变换。Note that with respect to the intra 8x8 prediction mode, this can only be applied in cases where the current macroblock is 8x8 at high profile or a profile beyond this. Orthogonal transformation.

图3和图4是示出亮度信号的九种4×4像素帧内预测模式(Intra_4×4_pred_mode)的示图。除了示出平均值(DC)预测的模式2以外的八种模式分别对应于图5中的用数字0、1、3至8指示的方向。3 and 4 are diagrams illustrating nine 4×4 pixel intra prediction modes (Intra_4×4_pred_mode) of a luma signal. The eight modes other than mode 2 which shows average (DC) prediction correspond to the directions indicated by numbers 0, 1, 3 to 8 in FIG. 5, respectively.

将参照图6描述九种intra_4×4_pred_mode。对于图6中的例子，像素a至p表示要进行帧内处理的当前块的像素，像素值A至M表示属于邻近块的像素的像素值。具体地说，像素a至p是从画面重新排列缓冲器62读出的要处理的图像，并且，像素值A至M是要从帧存储器72读出且被参考的解码图像的像素值。Nine types of intra_4×4_pred_mode will be described with reference to FIG. 6 . For the example in FIG. 6 , pixels a to p represent pixels of the current block to be intra-processed, and pixel values A to M represent pixel values of pixels belonging to neighboring blocks. Specifically, pixels a to p are images to be processed read out from the screen rearrangement buffer 62 , and pixel values A to M are pixel values of decoded images to be read out from the frame memory 72 and referred to.

在图3和图4中示出的帧内预测模式的情况中，使用属于邻近块的像素的像素值A至M，如下产生像素a至p的预测像素值。这里，像素值是“可用的”表示该像素值是可用的而没有这样的原因：该像素位于图像帧的边缘中或者尚未被编码。另一方面，像素值是“不可用的”表示由于这样的原因而导致该像素值是不可用的：该像素位于图像帧的边缘中或者尚未被编码。In the case of the intra prediction modes shown in FIGS. 3 and 4 , using pixel values A to M of pixels belonging to adjacent blocks, predicted pixel values of pixels a to p are generated as follows. Here, a pixel value being "available" means that the pixel value is available without a reason that the pixel is located in the edge of the image frame or has not been coded. On the other hand, a pixel value being "unavailable" means that the pixel value is not available because the pixel is located in the edge of the image frame or has not been coded.

模式0是垂直预测模式，并且仅仅适用于像素值A至D是“可用的”的情况。在这种情况下，如下述表达式(1)一样产生像素a至p的预测像素值：Mode 0 is a vertical prediction mode, and is only applicable when pixel values A to D are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (1):

像素a、e、i和m的预测像素值＝APredicted pixel values for pixels a, e, i, and m = A

像素b、f、j和n的预测像素值＝BPredicted pixel values for pixels b, f, j and n = B

像素c、g、k和o的预测像素值＝CPredicted pixel values for pixels c, g, k, and o = C

像素d、h、l和p的预测像素值＝D。..(1)Predicted pixel value = D for pixels d, h, l and p. ..(1)

模式1是水平预测模式，并且仅仅适用于像素值I至L是“可用的”的情况。在这种情况下，如下述表达式(2)一样产生像素a至p的预测像素值：Mode 1 is a horizontal prediction mode, and is only applicable when pixel values I to L are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (2):

像素a、b、c和d的预测像素值＝IPredicted pixel values for pixels a, b, c and d=1

像素e、f、g和h的预测像素值＝JPredicted pixel values for pixels e, f, g, and h = J

像素i、j、k和l的预测像素值＝KPredicted pixel values for pixels i, j, k and l = K

像素m、n、o和p的预测像素值＝L。..(2)Predicted pixel value = L for pixels m, n, o, and p. ..(2)

模式2是DC预测模式，并且，当像素值A、B、C、D、I、J、K和L全部是“可用的”时，如表达式(3)一样产生预测像素值。Mode 2 is a DC prediction mode, and when pixel values A, B, C, D, I, J, K, and L are all "available", a predicted pixel value is generated like Expression (3).

(A+B+C+D+I+J+K+L+4)＞＞3 ...(3)(A+B+C+D+I+J+K+L+4)＞＞3 ...(3)

此外，当像素值A、B、C和D全部是“不可用的”时，如表达式(4)一样产生预测像素值。Also, when the pixel values A, B, C, and D are all "unusable", predicted pixel values are generated as in Expression (4).

(I+J+K+L+2)＞＞2 ...(4)(I+J+K+L+2)>>2 ... (4)

此外，当像素值I、J、K和L全部是“不可用的”时，如表达式(5)一样产生预测像素值。Also, when the pixel values I, J, K, and L are all "unavailable", predicted pixel values are generated as in Expression (5).

(A+B+C+D+2)＞＞2 ...(5)(A+B+C+D+2)>>2 ... (5)

请注意，当像素值A、B、C、D、I、J、K和L全部是“不可用的”时，128用作预测像素值。Note that 128 is used as the predicted pixel value when pixel values A, B, C, D, I, J, K, and L are all "not available".

模式3是Diagonal_Down_Left(左下对角)预测模式，并且仅仅适用于像素值A、B、C、D、I、J、K、L和M是“可用的”的情况。在这种情况下，如下述表达式(6)一样产生像素a至p的预测像素值：Mode 3 is the Diagonal_Down_Left (lower left diagonal) prediction mode and is only applicable when pixel values A, B, C, D, I, J, K, L and M are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (6):

像素a的预测像素值＝(A+2B+C+2)＞＞2Predicted pixel value of pixel a = (A+2B+C+2)>>2

像素b和e的预测像素值＝(B+2C+D+2)＞＞2Predicted pixel values of pixels b and e = (B+2C+D+2)>>2

像素c、f和i的预测像素值＝(C+2D+E+2)＞＞2Predicted pixel values of pixels c, f and i = (C+2D+E+2)>>2

像素d、g、j和m的预测像素值＝(D+2E+F+2)＞＞2Predicted pixel values of pixels d, g, j and m = (D+2E+F+2)>>2

像素h、k和n的预测像素值＝(E+2F+G+2)＞＞2Predicted pixel values of pixels h, k and n = (E+2F+G+2)>>2

像素l和o的预测像素值＝(F+2G+H+2)＞＞2Predicted pixel values of pixels l and o = (F+2G+H+2)>>2

像素p的预测像素值＝(G+3H+2)＞＞2。..(6)Predicted pixel value of pixel p=(G+3H+2)>>2. ..(6)

模式4是Diagonal_Down_Right(右下对角)预测模式，并且仅仅适用于像素值A、B、C、D、I、J、K、L和M是“可用的”的情况。在这种情况下，如下述表达式(7)一样产生像素a至p的预测像素值：Mode 4 is the Diagonal_Down_Right (lower right diagonal) prediction mode, and only applies if pixel values A, B, C, D, I, J, K, L, and M are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (7):

像素m的预测像素值＝(J+2K+L+2)＞＞2Predicted pixel value of pixel m=(J+2K+L+2)>>2

像素i和n的预测像素值＝(I+2J+K+2)＞＞2Predicted pixel values of pixels i and n = (I+2J+K+2)>>2

像素e、j和o的预测像素值＝(M+2I+J+2)＞＞2Predicted pixel values of pixels e, j and o = (M+2I+J+2)>>2

像素a、f、k和p的预测像素值＝(A+2M+I+2)＞＞2Predicted pixel values of pixels a, f, k and p = (A+2M+I+2)>>2

像素b、g和l的预测像素值＝(M+2A+B+2)＞＞2Predicted pixel values of pixels b, g and l = (M+2A+B+2)>>2

像素c和h的预测像素值a＝(A+2B+C+2)＞＞2Predicted pixel values a=(A+2B+C+2)>>2 for pixels c and h

像素d的预测像素值＝(B+2C+D+2)＞＞2。..(7)Predicted pixel value of pixel d=(B+2C+D+2)>>2. ..(7)

模式5是Diagonal_Vertical_Right(右垂直对角)预测模式，并且仅仅适用于像素值A、B、C、D、I、J、K、L和M是“可用的”的情况。在这种情况下，如下述表达式(8)一样产生像素a至p的预测像素值：Mode 5 is the Diagonal_Vertical_Right (right vertical diagonal) prediction mode, and is only applicable when pixel values A, B, C, D, I, J, K, L and M are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (8):

像素a和j的预测像素值＝(M+A+1)＞＞1Predicted pixel values of pixels a and j = (M+A+1)>>1

像素b和k的预测像素值＝(A+B+1)＞＞1Predicted pixel values of pixels b and k = (A+B+1)>>1

像素c和l的预测像素值＝(B+C+1)＞＞1Predicted pixel values of pixels c and l = (B+C+1)>>1

像素d的预测像素值＝(C+D+1)＞＞1Predicted pixel value of pixel d=(C+D+1)>>1

像素e和n的预测像素值＝(I+2M+A+2)＞＞2Predicted pixel values of pixels e and n = (I+2M+A+2)>>2

像素f和o的预测像素值＝(M+2A+B+2)＞＞2Predicted pixel values of pixels f and o = (M+2A+B+2)>>2

像素g和p的预测像素值＝(A+2B+C+2)＞＞2Predicted pixel values of pixels g and p = (A+2B+C+2)>>2

像素h的预测像素值＝(B+2C+D+2)＞＞2Predicted pixel value of pixel h=(B+2C+D+2)>>2

像素i的预测像素值＝(M+2I+J+2)＞＞2Predicted pixel value of pixel i=(M+2I+J+2)>>2

像素m的预测像素值＝(I+2J+K+2)＞＞2..(8)Predicted pixel value of pixel m=(I+2J+K+2)>>2..(8)

模式6是Horizontal_Down(下水平)预测模式，并且仅仅适用于像素值A、B、C、D、I、J、K、L和M是“可用的”的情况。在这种情况下，如下述表达式(9)一样产生像素a至p的预测像素值：Mode 6 is the Horizontal_Down (lower horizontal) prediction mode, and only applies if pixel values A, B, C, D, I, J, K, L, and M are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (9):

像素a和g的预测像素值＝(M+I+1)＞＞1Predicted pixel values of pixels a and g = (M+I+1)>>1

像素b和h的预测像素值＝(I+2M+A+2)＞＞2Predicted pixel values of pixels b and h = (I+2M+A+2)>>2

像素c的预测像素值＝(M+2A+B+2)＞＞2Predicted pixel value of pixel c=(M+2A+B+2)>>2

像素d的预测像素值＝(A+2B+C+2)＞＞2Predicted pixel value of pixel d=(A+2B+C+2)>>2

像素e和k的预测像素值＝(I+J+1)＞＞1Predicted pixel values of pixels e and k = (I+J+1)>>1

像素f和l的预测像素值＝(M+2I+J+2)＞＞2Predicted pixel values of pixels f and l = (M+2I+J+2)>>2

像素i和o的预测像素值＝(J+K+1)＞＞1Predicted pixel values of pixels i and o = (J+K+1)>>1

像素j和p的预测像素值＝(I+2J+K+2)＞＞2Predicted pixel values of pixels j and p = (I+2J+K+2)>>2

像素m的预测像素值＝(K+L+1)＞＞1Predicted pixel value of pixel m=(K+L+1)>>1

像素n的预测像素值＝(J+2K+L+2)＞＞2。..(9)Predicted pixel value of pixel n=(J+2K+L+2)>>2. ..(9)

模式7是Vertical_Left(下垂直)预测模式，并且仅仅适用于像素值A、B、C、D、I、J、K、L和M是“可用的”的情况。在这种情况下，如下述表达式(10)一样产生像素a至p的预测像素值：Mode 7 is the Vertical_Left (vertical down) prediction mode, and only applies if pixel values A, B, C, D, I, J, K, L, and M are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (10):

像素a的预测像素值＝(A+B+1)＞＞1Predicted pixel value of pixel a = (A+B+1)>>1

像素b和i的预测像素值＝(B+C+1)＞＞1Predicted pixel values of pixels b and i = (B+C+1)>>1

像素c和j的预测像素值＝(C+D+1)＞＞1Predicted pixel values of pixels c and j = (C+D+1)>>1

像素d和k的预测像素值＝(D+E+1)＞＞1Predicted pixel values of pixels d and k = (D+E+1)>>1

像素l的预测像素值＝(E+F+1)＞＞1Predicted pixel value of pixel l=(E+F+1)>>1

像素e的预测像素值＝(A+2B+C+2)＞＞2Predicted pixel value of pixel e=(A+2B+C+2)>>2

像素f和m的预测像素值＝(B+2C+D+2)＞＞2Predicted pixel values of pixels f and m = (B+2C+D+2)>>2

像素g和n的预测像素值＝(C+2D+E+2)＞＞2Predicted pixel values of pixels g and n = (C+2D+E+2)>>2

像素h和o的预测像素值＝(D+2E+F+2)＞＞2Predicted pixel values of pixels h and o = (D+2E+F+2)>>2

像素p的预测像素值＝(E+2F+G+2)＞＞2。..(10)Predicted pixel value of pixel p=(E+2F+G+2)>>2. ..(10)

模式8是Horizontal_Up(上水平)预测模式，并且仅仅适用于像素值A、B、C、D、I、J、K、L和M是“可用的”的情况。在这种情况下，如下述表达式(11)一样产生像素a至p的预测像素值：Mode 8 is the Horizontal_Up (horizontal upper) prediction mode, and only applies if pixel values A, B, C, D, I, J, K, L, and M are "available". In this case, predicted pixel values of pixels a to p are generated as in the following expression (11):

像素a的预测像素值＝(I+J+1)＞＞1Predicted pixel value of pixel a = (I+J+1)>>1

像素b的预测像素值＝(I+2J+K+2)＞＞2Predicted pixel value of pixel b=(I+2J+K+2)>>2

像素c和e的预测像素值＝(J+K+1)＞＞1Predicted pixel values of pixels c and e = (J+K+1)>>1

像素d和f的预测像素值＝(J+2K+L+2)＞＞2Predicted pixel values of pixels d and f = (J+2K+L+2)>>2

像素g和i的预测像素值＝(K+L+1)＞＞1Predicted pixel values of pixels g and i = (K+L+1)>>1

像素h和j的预测像素值＝(K+3L+2)＞＞2Predicted pixel values of pixels h and j = (K+3L+2)>>2

像素k、l、m、n、o和p的预测像素值＝L。..(11)Predicted pixel value = L for pixels k, l, m, n, o and p. ..(11)

接下来，将参照图7描述亮度信号的4×4像素帧内预测模式(Intra_4×4_pred_mode)的编码格式。对于图7的例子，示出构成4×4像素的用作编码目标的当前块C，并且，示出构成4×4像素的与当前块邻近的块A和块B。Next, the encoding format of the 4×4 pixel intra prediction mode (Intra_4×4_pred_mode) of the luminance signal will be described with reference to FIG. 7 . With the example of FIG. 7 , a current block C serving as an encoding target constituting 4×4 pixels is shown, and blocks A and B adjacent to the current block constituting 4×4 pixels are shown.

在这种情况下，可以想到，当前块C中的Intra_4×4_pred_mode与块A和块B中的Intra_4×4_pred_mode具有高相关性。使用该相关性按照如下执行编码处理，从而可以实现更高的编码效率。In this case, it is conceivable that the Intra_4×4_pred_mode in the current block C has a high correlation with the Intra_4×4_pred_mode in the blocks A and B. Encoding processing is performed as follows using this correlation, so that higher encoding efficiency can be achieved.

具体地说，对于图7中的例子，块A和块B中的Intra_4×4_pred_mode分别被取为Intra_4×4_pred_modeA和Intra_4×4_pred_modeB，并且，MostProbableMode被定义为以下表达式(12)：Specifically, for the example in Fig. 7, Intra_4×4_pred_mode in block A and block B are taken as Intra_4×4_pred_modeA and Intra_4×4_pred_modeB respectively, and MostProbableMode is defined as the following expression (12):

MostProbableMode＝Min(Intra_4×4_pred_modeA，Intra_4×4_pred_modeB) ...(12)。MostProbableMode=Min(Intra_4×4_pred_modeA, Intra_4×4_pred_modeB) . . . (12).

也就是说，块A和块B中的被分配较小的mode_number的块被取为MostProbableMode。That is, a block assigned a smaller mode_number among the blocks A and B is taken as MostProbableMode.

被称为prev_intra4×4_pred_mode_flag[luma4×4Blkldx]和rem_intra4x4_pred_mode[luma4x4Blkldx]的两个值在比特流内被定义为关于当前块C的参数，并且，通过基于下述表达式(13)中示出的伪码的处理来执行解码处理，从而可以获得关于块C的Intra_4×4_pred_mode和Intra4×4PredMode[luma4×4Blkldx]的值。Two values called prev_intra4x4_pred_mode_flag[luma4x4Blkldx] and rem_intra4x4_pred_mode[luma4x4Blkldx] are defined in the bitstream as parameters for the current block C, and, by The decoding process is performed by processing the code, so that the values of Intra_4×4_pred_mode and Intra4×4PredMode[luma4×4Blkldx] about the block C can be obtained.

接下来，将描述8×8像素帧内预测模式。图8和图9是示出亮度信号的九种8×8像素帧内预测模式(intra_8×8_pred_mode)的示图。Next, the 8×8 pixel intra prediction mode will be described. 8 and 9 are diagrams showing nine 8×8-pixel intra prediction modes (intra_8×8_pred_mode) of a luma signal.

假设说，当前8×8块中的像素值被取为p[x，y](0≤x≤7；0≤y≤7)，并且，如p[-1，-1]、......、p[-1，15]、p[-1，0]、......、[p-1，7]一样表示邻近块的像素值。Assume that the pixel value in the current 8×8 block is taken as p[x, y] (0≤x≤7; 0≤y≤7), and, for example, p[-1,-1],... ..., p[-1, 15], p[-1, 0], ..., [p-1, 7] also represent the pixel values of adjacent blocks.

关于8×8像素帧内预测模式，在产生预测值之前对邻近像素进行低通滤波。现在，假设说，在低通滤波处理之前的像素值用p[-1，-1]、......、p[-1，15]、p[-1，0]、......、p[-1，7]表示，并且，在该处理之后的像素值用p′[-1，-1]、......、p′[-1，15]、p′[-1，0]、......、p′[-1，7]表示。Regarding the 8x8 pixel intra prediction mode, low-pass filtering is performed on neighboring pixels before generating the predicted value. Now, suppose that the pixel values before the low-pass filtering process are represented by p[-1, -1], ..., p[-1, 15], p[-1, 0], ... ..., p[-1, 7], and the pixel values after this processing are denoted by p'[-1,-1], ..., p'[-1, 15], p '[-1,0],..., p'[-1,7] represent.

首先，在p[-1，-1]是“可用的”的情况中，如下述表达式(14)一样计算p′[0，-1]，在“不可用的”的情况中，如下述表达式(15)一样计算p′[0，-1]。First, in the case where p[-1, -1] is "available", p'[0, -1] is calculated as in the following expression (14), and in the case of "unavailable", as in the following Calculate p'[0, -1] as in expression (15).

p′[0，-1]＝(p[-1，-1]+2^＊p[0，-1]+p[1，-1]+2)＞＞2 ...(14)p'[0,-1]=(p[-1,-1]+2^* p[0,-1]+p[1,-1]+2)>>2 ... (14)

p′[0，-1]＝(3^＊p[0，-1]+p[1，-1]+2)＞＞2 ...(15)p'[0,-1]=(3^* p[0,-1]+p[1,-1]+2)>>2...(15)

如下述表达式(16)一样计算p′[x，-1](x＝0、......、7)。p'[x, -1] (x=0, . . . , 7) is calculated as in Expression (16) below.

p′[x，-1]＝(p[x-1，-1]+2^＊p[x，-1]+p[x+1，-1]+2)＞＞2 ...(16)p'[x,-1]=(p[x-1,-1]+2^* p[x,-1]+p[x+1,-1]+2)>>2 ... (16 )

在p[x，-1](x＝8、......、15)是“可用的”的情况中，如下述表达式(17)一样计算p′[x，-1](x＝8、......、15)。In the case where p[x, -1] (x=8, . . . , 15) is "available", p'[x, -1] (x =8,...,15).

p′[x，-1]＝(p[x-1，-1]+2^＊p[x，-1]+p[x+1，-1]+2)＞＞2p'[x,-1]=(p[x-1,-1]+2^* p[x,-1]+p[x+1,-1]+2)>>2

p′[15，-1]＝(p[14，-1]+3^＊p[15，-1]+2)＞＞2 ...(17)p'[15,-1]=(p[14,-1]+3^* p[15,-1]+2)>>2...(17)

在p[-1，-1]是“可用的”的情况中，按照如下计算p′[-1，-1]。具体地说，在p[0，-1]和p[-1，0]二者都是“可用的”的情况中，如表达式(18)一样计算p′[-1，-1]，并且，在p[-1，0]是“不可用的”的情况中，如表达式(19)一样计算p′[-1，-1]。此外，在p[0，-1]是“不可用的”的情况中，如表达式(20)一样计算p′[-1，-1]。In case p[-1, -1] is "available", p'[-1, -1] is calculated as follows. Specifically, in the case where both p[0, -1] and p[-1, 0] are "available", p'[-1, -1] is calculated as in expression (18), And, in the case where p[-1, 0] is "not available", p'[-1, -1] is calculated like Expression (19). Also, in the case where p[0, -1] is "unavailable", p'[-1, -1] is calculated as in Expression (20).

p′[-1，-1]＝(p[0，-1]+2^＊p[-1，-1]+p[-1，0]+2)＞＞2 ...(18)p'[-1,-1]=(p[0,-1]+2^* p[-1,-1]+p[-1,0]+2)>>2 ... (18)

p′[-1，-1]＝(3^＊p[-1，-1]+p[0，-1]+2)＞＞2 ...(19)p'[-1,-1]=(3^* p[-1,-1]+p[0,-1]+2)>>2...(19)

p′[-1，-1]＝(3^＊p[-1，-1]+p[-1，0]+2)＞＞2 ...(20)p'[-1,-1]=(3^* p[-1,-1]+p[-1,0]+2)>>2 ... (20)

当p[-1，y](y＝0、......、7)是“可用的”时，按照如下计算p′[-1，y](y＝0、......、7)。具体地说，首先，在p[-1，-1]是“可用的”的情况中，如表达式(21)一样计算p′[-1，0]，并且在p[-1，-1]是“不可用的”的情况中，如表达式(22)一样计算p′[-1，0]When p[-1, y] (y=0, ..., 7) is "available", p'[-1, y] (y = 0, ..., 7) is calculated as follows ., 7). Specifically, first, in the case where p[-1, -1] is "available", p'[-1, 0] is calculated as in expression (21), and in p[-1, -1 ] is "unavailable", calculate p′[-1, 0] as in expression (22)

p′[-1，0]＝(p[-1，-1]+2^＊p[-1，0]+p[-1，1]+2)＞＞2 ...(21)p'[-1,0]=(p[-1,-1]+2^* p[-1,0]+p[-1,1]+2)>>2 ... (21)

p′[-1，0]＝(3^＊p[-1，0]+p[-1，1]+2)＞＞2 ...(22)p'[-1,0]=(3^* p[-1,0]+p[-1,1]+2)>>2...(22)

此外，如下述表达式(23)一样计算p′[-1，y](y＝1、......、6)，并且，如表达式(24)一样计算p′[-1，7]。Furthermore, p'[-1, y] (y=1, . . . , 6) is calculated as in the following expression (23), and p'[-1 is calculated as in expression (24), 7].

p[-1，y]＝(p[-1，y-1]+2^＊p[-1，y]+p[-1，y+1]+2)＞＞2 ...(23)p[-1, y]=(p[-1, y-1]+2^* p[-1, y]+p[-1, y+1]+2)>>2 ... (23)

p′[-1，7]＝(p[-1，6]+3^＊p[-1，7]+2)＞＞2 ...(24)p'[-1,7]=(p[-1,6]+3^* p[-1,7]+2)>>2...(24)

使用这样计算出的p′，按照如下产生在图8和图9中示出的帧内预测模式中的预测值。Using p' thus calculated, prediction values in the intra prediction modes shown in FIGS. 8 and 9 are generated as follows.

模式0是垂直预测模式，并且，仅仅在p[x，-1](x＝0、......、7)是“可用的”时适用。如下述表达式(25)一样产生预测值pred8×8_L[x，y]。Mode 0 is a vertical prediction mode, and is only applicable when p[x, -1] (x=0, . . . , 7) is "available". The predicted value pred8×8_L [x, y] is generated as in Expression (25) below.

pred8×8_L[x，y]＝p′[x，-1]x，y＝0、......、7 ...(25)_pred8 ×8L[x,y]=p'[x,-1]x,y=0,...,7...(25)

模式1是水平预测模式，并且，仅仅在p[-1，y](y＝0、......、7)是“可用的”时适用。如下述表达式(26)一样产生预测值pred8×8_L[x，y]。Mode 1 is a horizontal prediction mode and is only applicable when p[-1,y] (y=0,...,7) is "available". The predicted value pred8×8_L [x, y] is generated as in Expression (26) below.

pred8×8_L[x，y]＝p′[-1，y]x，y＝0、......、7 ...(26)pred8×8_L [x, y] = p'[-1, y] x, y = 0, ..., 7 ... (26)

模式2是DC预测模式，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，在p[x，-1](x＝0、......、7)和p[-1，y](y＝0、......、7)二者都是“可用的”的情况中，如表达式(27)一样产生预测值pred8×8_L[x，y]。Mode 2 is a DC prediction mode, and a prediction value pred8×8_L [x, y] is generated as follows. Specifically, in both p[x,-1] (x=0,...,7) and p[-1,y] (y=0,...,7) In the case of all "available", the predicted value pred8×8_L [x, y] is generated as in expression (27).

[数学式1][mathematical formula 1]

$\mathrm{<span><span>Pred</span>Pred</span>}\mathrm{<span><span>8</span>8</span>}\mathrm{<span><span>x</span>x</span>}{\mathrm{<span><span>8</span>8</span>}}_{\mathrm{<span><span>L</span>L</span>}}<span><span>[</span>[</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>]</span>]</span><span><span>=</span>=</span><span><span>(</span>(</span>\underset{{\mathrm{<span><span>x</span>x</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>7</span>7</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}{\mathrm{<span><span>P</span>P</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>[</span>[</span>{\mathrm{<span><span>x</span>x</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>]</span>]</span><span><span>+</span>+</span>\underset{{\mathrm{<span><span>y</span>the\; y</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>7</span>7</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}{\mathrm{<span><span>P</span>P</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>[</span>[</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>]</span>]</span><span><span>+</span>+</span>\mathrm{<span><span>8</span>8</span>}<span><span>)</span>)</span><span><span>></span>></span><span><span>></span>></span>\mathrm{<span><span>4</span>4</span>}<span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>27</span>27</span>}<span><span>)</span>)</span>$

在p[x，-1](x＝0、......、7)是“可用的”而p[-1，y](y＝0、......、7)是“不可用的”的情况中，如表达式(28)一样产生预测值pred8×8_L[x，y]。where p[x,-1] (x=0,...,7) is "available" and p[-1,y] (y=0,...,7) is In the case of "unavailable", the predicted value pred8×8_L [x, y] is generated as in Expression (28).

[数学式2][mathematical formula 2]

$\mathrm{<span><span>Pred</span>Pred</span>}\mathrm{<span><span>8</span>8</span>}\mathrm{<span><span>x</span>x</span>}{\mathrm{<span><span>8</span>8</span>}}_{\mathrm{<span><span>L</span>L</span>}}<span><span>[</span>[</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>]</span>]</span><span><span>=</span>=</span><span><span>(</span>(</span>\underset{{\mathrm{<span><span>x</span>x</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>7</span>7</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}{\mathrm{<span><span>P</span>P</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>[</span>[</span>{\mathrm{<span><span>x</span>x</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>]</span>]</span><span><span>+</span>+</span>\mathrm{<span><span>4</span>4</span>}<span><span>)</span>)</span><span><span>></span>></span><span><span>></span>></span>\mathrm{<span><span>3</span>3</span>}<span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>28</span>28</span>}<span><span>)</span>)</span>$

在p[x，-1](x＝0、......、7)是“不可用的”而p[-1，y](y＝0、......、7)是“可用的”的情况中，如表达式(29)一样产生预测值pred8×8_L[x，y]。At p[x,-1] (x=0,...,7) is "not available" and p[-1,y] (y=0,...,7) In the case where is "available", the predicted value pred8×8_L [x, y] is generated as in Expression (29).

[数学式3][mathematical formula 3]

$\mathrm{<span><span>Pred</span>Pred</span>}\mathrm{<span><span>8</span>8</span>}\mathrm{<span><span>x</span>x</span>}{\mathrm{<span><span>8</span>8</span>}}_{\mathrm{<span><span>L</span>L</span>}}<span><span>[</span>[</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>]</span>]</span><span><span>=</span>=</span><span><span>(</span>(</span>\underset{{\mathrm{<span><span>y</span>the\; y</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>7</span>7</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}{\mathrm{<span><span>P</span>P</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>[</span>[</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>]</span>]</span><span><span>+</span>+</span>\mathrm{<span><span>4</span>4</span>}<span><span>)</span>)</span><span><span>></span>></span><span><span>></span>></span>\mathrm{<span><span>3</span>3</span>}<span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>29</span>29</span>}<span><span>)</span>)</span>$

在p[x，-1](x＝0、......、7)和p[-1，y](y＝0、......、7)二者都是“不可用的”的情况中，如表达式(30)一样产生预测值pred8×8_L[x，y]。Both p[x, -1] (x=0,...,7) and p[-1,y] (y=0,...,7) are "impossible In the case of "used", the predicted value pred8×8_L [x, y] is generated as in Expression (30).

pred8×8_L[x，y]＝128 ...(30)_pred8 ×8L[x,y]=128...(30)

这里，表达式(30)表示8比特输入的情况。Here, Expression (30) represents the case of 8-bit input.

模式3是Diagonal_Down_Left_prediction模式(左下对角预测模式)，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，Diagonal_Down_Left_prediction模式仅仅在p[x，-1]，x＝0、......、15是“可用的”时适用，并且，如下述表达式(31)一样产生在x＝7且y＝7的情况中的预测像素值，并且，如下述表达式(32)一样产生其他预测像素值。Mode 3 is a Diagonal_Down_Left_prediction mode (lower left diagonal prediction mode), and a predicted value pred8×8_L [x, y] is generated as follows. Specifically, the Diagonal_Down_Left_prediction mode applies only when p[x,-1], x=0, . 7 and y=7, and other predicted pixel values are generated as in Expression (32) below.

pred8×8_L[x，y]＝(p′[14，-1]+3^＊p[15，-1]+2)＞＞2 ...(31)pred8×8_L [x, y]=(p'[14,-1]+3^* p[15,-1]+2)>>2 ... (31)

pred8×8_L[x，y]＝(p′[x+y，-1]+2^＊p′[x+y+1，-1]+p′[x+y+2，-1]+2)＞＞2 ...(32)pred8×8_L [x, y]=(p′[x+y,-1]+2^* p′[x+y+1,-1]+p′[x+y+2,-1]+ 2)>>2...(32)

模式4是Diagnonal_Down_Right_prediction模式(右下对角预测模式)，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，Diagonal_Down_Right_prediction模式仅仅在p[x，-1]，x＝0、......、7和p[-1，y]，y＝0、......、7是“可用的”时适用，并且，如下述表达式(33)一样产生在x＞y的情况中的预测像素值，并且，如下述表达式(34)一样产生在x＜y的情况中的预测像素值。此外，如下述表达式(35)一样产生在x＝y的情况中的预测像素值。Mode 4 is a Diagnonal_Down_Right_prediction mode (lower right diagonal prediction mode), and a predicted value pred8×8_L [x, y] is generated as follows. Specifically, the Diagonal_Down_Right_prediction mode is only in p[x,-1], x=0,...,7 and p[-1,y], y=0,...,7 is Applies when "available", and produces a predicted pixel value in the case of x>y as in the following expression (33), and produces a prediction in the case of x<y as in the following expression (34) Pixel values. Also, the predicted pixel value in the case of x=y is generated as in the following expression (35).

pred8×8_L[x，y]＝(p′[x-y-2，-1]+2^＊p′[x-y-1，-1]+p′[x-y，-1]+2)＞＞2 ...(33)pred8×8_L [x, y]=(p'[xy-2,-1]+2^* p'[xy-1,-1]+p'[xy,-1]+2)>>2. ..(33)

pred8×8_L[x，y]＝(p′[-1，y-x-2]+2^＊p′[-1，y-x-1]+p′[-1，y-x]+2)＞＞2 ...(34)pred8×8_L [x, y] = (p'[-1, yx-2]+2^* p'[-1, yx-1]+p'[-1, yx]+2)>>2. ..(34)

pred8×8_L[x，y]＝(p′[0，-1]+2^＊p′[-1，-1]+p′[-1，0]+2)＞＞2 ...(35)pred8×8_L [x, y]=(p'[0,-1]+2^* p'[-1,-1]+p'[-1,0]+2)>>2 ...( 35)

模式5是Vertical_Right_prediction模式(右垂直预测模式)，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，Vertical_Right_prediction模式仅仅在p[x，-1]，x＝0、......、7和p[-1，y]，y＝-1、......、7是“可用的”时适用。现在，如下述表达式(36)一样定义zVR。Mode 5 is a Vertical_Right_prediction mode (right vertical prediction mode), and a predicted value pred8×8_L [x, y] is generated as follows. Specifically, the Vertical_Right_prediction mode is only in p[x,-1], x=0,...,7 and p[-1,y], y=-1,...,7 is applicable when "available". Now, zVR is defined as in Expression (36) below.

zVR＝2^＊x-y...(36)zVR=2^* xy...(36)

此时，在zVR是0、2、4、6、8、10、12或14的情况中，如下述表达式(37)一样产生预测像素值，并且，在zVR是1、3、5、7、9、11或13的情况中，如下述表达式(38)一样产生预测像素值。At this time, in the case where zVR is 0, 2, 4, 6, 8, 10, 12, or 14, the predicted pixel value is generated as in the following expression (37), and, when zVR is 1, 3, 5, 7 , 9, 11, or 13, the predicted pixel value is generated as in Expression (38) below.

pred8×8_L[x，y]＝(p′[x-(y＞＞1)-1，-1]+p′[x-(y＞＞1)，-1]+1)＞＞1 ...(37)pred8×8_L [x, y]=(p'[x-(y>>1)-1,-1]+p'[x-(y>>1),-1]+1)>>1 ...(37)

pred8×8_L[x，y]＝(p′[x-(y＞＞1)-2，-1]+2^＊p′[x-(y＞＞1)-1，-1]+p′[x-(y＞＞1)，-1]+2)＞＞2 ...(38)pred8×8_L [x, y]=(p'[x-(y>>1)-2,-1]+2^* p'[x-(y>>1)-1,-1]+p '[x-(y>>1),-1]+2)>>2...(38)

此外，在zVR是-1的情况中，如下述表达式(39)一样产生预测像素值，并且，在除此之外的情况中，具体地说，在zVR是-2、-3、-4、-5、-6或-7的情况中，如下述表达式(40)一样产生预测像素值。Furthermore, in the case where zVR is -1, the predicted pixel value is generated as in the following expression (39), and, in other cases, specifically, when zVR is -2, -3, -4 , -5, -6, or -7, the predicted pixel value is generated as in Expression (40) below.

pred8×8_L[x，y]＝(p′[-1，0]+2^＊p′[-1，-1]+p′[0，-1]+2)＞＞2 ...(39)pred8×8_L [x, y]=(p'[-1,0]+2^* p'[-1,-1]+p'[0,-1]+2)>>2 ...( 39)

pred8×8_L[x，y]＝(p′[-1，y-2^＊x-1]+2^＊p′[-1，y-2^＊x-2]+p′[-1，y-2^＊x-3]+2)＞＞2 ...(40)pred8×8_L [x,y]=(p′[-1,y-2^* x-1]+2^* p′[-1,y-2^* x-2]+p′[-1,y -2^＊ x-3]+2)＞＞2...(40)

模式6是Horizontal_Down_prediction模式(下水平预测模式)，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，Horizontal_Down_prediction模式仅仅在p[x，-1]，x＝0、......、7和p[-1，y]，y＝-1、......、7是“可用的”时适用。现在，如下述表达式(41)一样定义zHD。Mode 6 is a Horizontal_Down_prediction mode (down horizontal prediction mode), and a predicted value pred8×8_L [x, y] is generated as follows. Specifically, the Horizontal_Down_prediction mode is only in p[x,-1], x=0,...,7 and p[-1,y], y=-1,...,7 is applicable when "available". Now, zHD is defined as in Expression (41) below.

zHD＝2^＊y-x...(41)zHD=2^* yx...(41)

此时，在zHD是0、2、4、6、8、10、12或14的情况中，如下述表达式(42)一样产生预测像素值，并且，在zHD是1、3、5、7、9、11或13的情况中，如下述表达式(43)一样产生预测像素值。At this time, in the case where zHD is 0, 2, 4, 6, 8, 10, 12, or 14, the predicted pixel value is generated as in the following expression (42), and, when zHD is 1, 3, 5, 7 , 9, 11, or 13, the predicted pixel value is generated as in Expression (43) below.

pred8×8_L[x，y]＝(p′[-1，y-(x＞＞1)-1]+p′[-1，y-(x＞＞1)+1]＞＞1 ...(42)pred8×8_L [x, y]=(p'[-1, y-(x>>1)-1]+p'[-1, y-(x>>1)+1]>>1 . ..(42)

pred8×8_L[x，y]＝(p′[-1，y-(x＞＞1)-2]+2^＊p′[-1，y-(x＞＞1)-1]+p′[-1，y-(x＞＞1)]+2)＞＞2 ...(43)pred8×8_L [x, y]=(p'[-1, y-(x>>1)-2]+2^* p'[-1, y-(x>>1)-1]+p '[-1,y-(x>>1)]+2)>>2...(43)

此外，在zHD是-1的情况中，如下述表达式(44)一样产生预测像素值，并且，在除此之外的情况中，具体地说，在zHD是-2、-3、-4、-5、-6或-7的情况中，如下述表达式(45)一样产生预测像素值。Also, in the case where zHD is -1, the predicted pixel value is generated as in the following expression (44), and, in other cases, specifically, when zHD is -2, -3, -4 In the case of , -5, -6, or -7, the predicted pixel value is generated as in Expression (45) below.

pred8×8_L[x，y]＝(p′[-1，0]+2^＊p′[-1，-1]+p′[0，-1]+2)＞＞2 ...(44)pred8×8_L [x, y]=(p'[-1,0]+2^* p'[-1,-1]+p'[0,-1]+2)>>2 ...( 44)

pred8×8_L[x，y]＝(p′[x-2^＊Y-1，-1]+2^＊p′[x-2^＊y-2，-1]+p′[x-2^＊y-3，-1]+2)＞＞2 ...(45)pred8×8_L [x,y]=(p′[x-2^* Y-1,-1]+2^* p′[x-2^* y-2,-1]+p′[x-2^* y-3,-1]+2)>>2...(45)

模式7是Vertical_Left_prediction模式(左垂直预测模式)，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，Vertical_Left_prediction模式仅仅在p[x，-1]，x＝0、......、15是“可用的”时适用，在y＝0、2、4或6的情况中，如下述表达式(46)一样产生预测像素值，并且，在除此之外的情况中，即，在y＝1、3、5或7的情况中，如下述表达式(47)一样产生预测像素值。Mode 7 is a Vertical_Left_prediction mode (left vertical prediction mode), and a predicted value pred8×8_L [x, y] is generated as follows. Specifically, the Vertical_Left_prediction mode is only applicable when p[x,-1], x=0,...,15 are "available", in the case of y=0, 2, 4 or 6, The predicted pixel value is generated as in the following expression (46), and, in other cases, that is, in the case of y=1, 3, 5, or 7, the prediction is generated as in the following expression (47) Pixel values.

pred8×8_L[x，y]＝(p′[x+(y＞＞1)，-1]+p′[x+(y＞＞1)+1，-1]+1)＞＞1 ...(46)pred8×8_L [x,y]=(p'[x+(y>>1),-1]+p'[x+(y>>1)+1,-1]+1)>>1 .. .(46)

pred8×8_L[x，y]＝(p′[x+(y＞＞1)，-1]+2^＊p′[x+(y＞＞1)+1，-1]+p′[x+(y＞＞1)+2，-1]+2)＞＞2 ...(47)pred8×8_L [x, y]=(p'[x+(y>>1),-1]+2^* p'[x+(y>>1)+1,-1]+p'[x+( y>>1)+2, -1]+2)>>2...(47)

模式8是Horizontal_Up_prediction模式(上水平预测模式)，并且，按照如下产生预测值pred8×8_L[x，y]。具体地说，Horizontal_Up_prediction模式仅仅在p[-1，y]，y＝0、......、7是“可用的”时适用。现在，如下述表达式(48)一样定义zHU。Mode 8 is a Horizontal_Up_prediction mode (upper horizontal prediction mode), and a predicted value pred8×8_L [x, y] is generated as follows. Specifically, the Horizontal_Up_prediction mode is only applicable when p[-1, y], y=0, . . . , 7 are "available". Now, zHU is defined as in Expression (48) below.

zHU＝x+2^＊y...(48)zHU=x+2^* y...(48)

此时，在zHU的值是0、2、4、6、8、10或12的情况中，如下述表达式(49)一样产生预测像素值，并且，在zHU的值是1、3、5、7、9或11的情况中，如下述表达式(50)一样产生预测像素值。At this time, in the case where the value of zHU is 0, 2, 4, 6, 8, 10, or 12, the predicted pixel value is generated as in the following expression (49), and, when the value of zHU is 1, 3, 5 , 7, 9, or 11, the predicted pixel value is generated as in Expression (50) below.

pred8×8_L[x，y]＝(p′[-1，y+(x＞＞1)]+p′[-1，y+(x＞＞1)+1]+1)＞＞1...(49)pred8×8_L [x, y]=(p'[-1, y+(x>>1)]+p'[-1, y+(x>>1)+1]+1)>>1.. .(49)

pred8×8_L[x，y]＝(p′[-1，y+(x＞＞1)] ...(50)pred8×8_L [x, y]=(p'[-1, y+(x>>1)] ... (50)

此外，在zHU的值是13的情况中，如下述表达式(51)一样产生预测像素值，并且，在除此之外的情况中，即，在zHU的值大于13的情况中，如下述表达式(52)一样产生预测像素值。Furthermore, in the case where the value of zHU is 13, the predicted pixel value is generated as in the following expression (51), and, in other cases, that is, in the case where the value of zHU is greater than 13, as in the following Expression (52) yields predicted pixel values as well.

pred8×8_L[x，y]＝(p′[-1，6]+3^＊p′[-1，7]+2)＞＞2 ...(51)pred8×8_L [x, y]=(p'[-1,6]+3^* p'[-1,7]+2)>>2 ... (51)

pred8×8_L[x，y]＝p′[-1，7]...(52)pred8×8_L [x, y] = p'[-1, 7]...(52)

接下来，将描述16×16像素帧内预测模式。图10和图11是示出亮度信号的四种16×16像素帧内预测模式(Intra_16×16_pred_mode)的示图。Next, the 16×16 pixel intra prediction mode will be described. 10 and 11 are diagrams showing four 16×16 pixel intra prediction modes (Intra_16×16_pred_mode) of a luma signal.

将参照图12描述四种帧内预测模式。对于图12中的例子，示出要进行帧内处理的当前宏块A，并且，P(x，y)；x，y＝-1，0、......、15表示与当前宏块A邻近的像素的像素值。Four intra prediction modes will be described with reference to FIG. 12 . For the example in Fig. 12, the current macroblock A to be processed in the frame is shown, and, P(x, y); x, y=-1, 0, ..., 15 represent the The pixel values of pixels adjacent to block A.

模式0是垂直预测模式，并且仅仅在P(x，-1)；x，y＝-1，0、......、15是“可用的”时适用。在这种情况下，如下述表达式(53)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 0 is a vertical prediction mode and is only applicable when P(x, -1); x, y=-1, 0, . . . , 15 are "available". In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (53).

Pred(x，y)＝P(x，-1)；x，y＝0、......、15 ...(53)Pred(x,y)=P(x,-1); x,y=0,...,15...(53)

模式1是水平预测模式，并且仅仅在P(-1，y)；x，y＝-1，0、......、15是“可用的”时适用。在这种情况下，如下述表达式(54)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 1 is a horizontal prediction mode and is only applicable when P(-1, y); x, y=-1, 0, . . . , 15 are "available". In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (54).

Pred(x，y)＝P(-1，y)；x，y＝0、......、15 ...(54)Pred(x, y)=P(-1, y); x, y=0,...,15...(54)

模式2是DC预测模式，并且，在P(x，-1)和P(-1，y)；x，y＝-1，0、......、15全部是“可用的”的情况中，如下述表达式(55)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 2 is the DC prediction mode, and, at P(x,-1) and P(-1,y); x, y=-1, 0,...,15 are all "available" In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (55).

[数学式4][mathematical formula 4]

$\mathrm{<span><span>Pred</span>Pred</span>}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>)</span>)</span><span><span>=</span>=</span><span><span>[</span>[</span>\underset{{\mathrm{<span><span>x</span>x</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>15</span>15</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span>{\mathrm{<span><span>x</span>x</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span><span><span>+</span>+</span>\underset{{\mathrm{<span><span>y</span>the\; y</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>15</span>15</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span>{\mathrm{<span><span>y</span>the\; y</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>)</span>)</span><span><span>+</span>+</span>\mathrm{<span><span>16</span>16</span>}<span><span>]</span>]</span><span><span>></span>></span><span><span>></span>></span>\mathrm{<span><span>5</span>5</span>}$

其中x，y＝0，…，15 …(55)where x,y=0,...,15...(55)

此外，在P(x，-1)；x，y＝-1，0、......、15是“不可用的”的情况中，如下述表达式(56)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Furthermore, in the case where P(x, -1); x, y=-1, 0, . The predicted pixel value Pred(x, y) of each pixel of A.

[数学式5][mathematical formula 5]

$\mathrm{Pred}(x,y)=[\underset{y^{\&prime;}=0}{\overset{15}{\mathrm{\&Sigma;}}}P(-1,y^{\&prime;})+8]>>4$其中x，y＝0，…，15 …(56)$\mathrm{Pred}(x,\mathrm{the\; y})=[\underset{{\mathrm{the\; y}}^{\&prime;}=0}{\overset{15}{\mathrm{\&Sigma;}}}P(-1,{\mathrm{the\; y}}^{\&prime;})+8]>>4$ where x,y=0,...,15...(56)

此外，在P(-1，y)；x，y＝-1，0、......、15是“不可用的”的情况中，如下述表达式(57)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Furthermore, in the case where P(-1, y); x, y=-1, 0, . The predicted pixel value Pred(x, y) of each pixel of A.

[数学式6][mathematical formula 6]

$\mathrm{Pred}(x,y)=[\underset{y^{\&prime;}=0}{\overset{15}{\mathrm{\&Sigma;}}}P(x^{\&prime;},-1)+8]>>4$其中x，y＝0，…，15 …(57)$\mathrm{Pred}(x,\mathrm{the\; y})=[\underset{{\mathrm{the\; y}}^{\&prime;}=0}{\overset{15}{\mathrm{\&Sigma;}}}P(x^{\&prime;},-1)+8]>>4$ where x,y=0,...,15...(57)

在P(x，-1)和P(-1，y)；x，y＝-1，0、......、15全部是“不可用的”的情况中，128用作预测像素值。In case P(x,-1) and P(-1,y); x,y=-1, 0,...,15 are all "not available", 128 are used as prediction pixels value.

模式3是平面预测模式，并且仅仅在P(x，-1)和P(-1，y)；x，y＝-1，0、......、15全部是“可用的”时适用。在这种情况下，如下述表达式(58)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 3 is a planar prediction mode, and only when P(x,-1) and P(-1,y); x,y=-1,0,...,15 are all "available" Be applicable. In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (58).

[数学式7][mathematical formula 7]

Pred(x，y)＝Clip1((a+b·(x-7)+c·(y-7)+16)＞＞5)Pred(x,y)=Clip1((a+b·(x-7)+c·(y-7)+16)>>5)

a＝16·(P(-1，15)+P(15，-1))a=16·(P(-1, 15)+P(15,-1))

b＝(5·H+32)＞＞6b=(5·H+32)>>6

c＝(5·V+32)＞＞6c＝(5·V+32)＞＞6

$\mathrm{<span><span>H</span>h</span>}<span><span>=</span>=</span>\underset{\mathrm{<span><span>x</span>x</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>8</span>8</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\mathrm{<span><span>x</span>x</span>}<span><span>\&CenterDot;</span>\&CenterDot;</span><span><span>(</span>(</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span>\mathrm{<span><span>7</span>7</span>}<span><span>+</span>+</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span><span><span>-</span>-</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span>\mathrm{<span><span>7</span>7</span>}<span><span>-</span>-</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span><span><span>)</span>)</span>$

$\mathrm{<span><span>V</span>V</span>}<span><span>=</span>=</span>\underset{\mathrm{<span><span>y</span>the\; y</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>8</span>8</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\mathrm{<span><span>y</span>the\; y</span>}<span><span>\&CenterDot;</span>\&Center\; Dot;</span><span><span>(</span>(</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span><span><span>-</span>-</span>\mathrm{<span><span>1,7</span>1,7</span>}<span><span>+</span>+</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>)</span>)</span><span><span>-</span>-</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span><span><span>-</span>-</span>\mathrm{<span><span>1,7</span>1,7</span>}<span><span>-</span>-</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>)</span>)</span><span><span>)</span>)</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>58</span>58</span>}<span><span>)</span>)</span>$

首先，将描述关于颜色差信号的帧内预测模式。图13是示出颜色差信号的四种帧内预测模式(Intra_chroma_pred_mode)的示图。可以与亮度信号的帧内预测模式独立地设置颜色差信号的帧内预测模式。关于颜色差信号的帧内预测模式与上述的亮度信号的16×16像素帧内预测模式一致。First, an intra prediction mode with respect to a color difference signal will be described. FIG. 13 is a diagram showing four intra prediction modes (Intra_chroma_pred_mode) of a color difference signal. The intra prediction mode of the color difference signal can be set independently from the intra prediction mode of the luma signal. The intra prediction mode for the color difference signal is the same as the above-mentioned 16×16 pixel intra prediction mode for the luminance signal.

然而，亮度信号的16×16像素帧内预测模式取16×16像素块作为目标，但是，另一方面，关于颜色差信号的帧内预测模式取8×8像素块作为目标。此外，如上述的图10和图13所示，二者之间的模式编号不对应。However, the 16×16-pixel intra prediction mode for luminance signals targets 16×16 pixel blocks, but on the other hand, the intra prediction mode for color difference signals targets 8×8 pixel blocks. In addition, as shown in FIG. 10 and FIG. 13 described above, the pattern numbers do not correspond between the two.

现在，假设我们遵照上文中参照图12描述的亮度信号的16×16像素帧内预测模式中的当前块A的像素值和邻近像素值的定义。例如，假设说，与要进行帧内处理的当前宏块A(在颜色差信号的情况中，8×8像素)邻近的像素的像素值被取为P(x，y)；x，y＝-1，0、......、7。Now, suppose we follow the definition of the pixel value and neighboring pixel values of the current block A in the 16×16 pixel intra prediction mode of the luma signal described above with reference to FIG. 12 . For example, suppose that the pixel value of a pixel adjacent to the current macroblock A (in the case of a color difference signal, 8×8 pixels) to be intra-processed is taken as P(x, y); x, y= -1, 0, ..., 7.

模式0是DC预测模式，并且，在P(x，-1)和P(-1，y)；x，y＝-1，0、......、7全部是“可用的”的情况中，如下述表达式(59)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 0 is the DC prediction mode, and, at P(x,-1) and P(-1,y); x, y=-1, 0,...,7 are all "available" In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (59).

[数学式8][mathematical formula 8]

$\mathrm{<span><span>Pred</span>Pred</span>}<span><span>(</span>(</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>)</span>)</span><span><span>=</span>=</span><span><span>(</span>(</span><span><span>(</span>(</span>\underset{\mathrm{<span><span>n</span>no</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>7</span>7</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}<span><span>(</span>(</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>+</span>+</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span>\mathrm{<span><span>n</span>no</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span><span><span>)</span>)</span><span><span>)</span>)</span><span><span>+</span>+</span>\mathrm{<span><span>8</span>8</span>}<span><span>)</span>)</span><span><span>></span>></span><span><span>></span>></span>\mathrm{<span><span>4</span>4</span>}$

其中x，y＝0，…，7 …(59)where x,y=0,...,7...(59)

此外，在P(-1，y)；x，y＝-1，0、......、7是“不可用的”的情况中，如下述表达式(60)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Furthermore, in the case where P(-1, y); x, y=-1, 0, . The predicted pixel value Pred(x, y) of each pixel of A.

[数学式9][mathematical formula 9]

$\mathrm{Pred}(x,y)=[\left(\underset{n=0}{\overset{7}{\mathrm{\&Sigma;}}}P(n,-1)\right)+4]>>3$其中x，y＝0，…，7 …(60)$\mathrm{Pred}(x,\mathrm{the\; y})=[\left(\underset{\mathrm{no}=0}{\overset{7}{\mathrm{\&Sigma;}}}P(\mathrm{no},-1)\right)+4]>>3$ where x,y=0,...,7...(60)

此外，在P(x，-1)；x，y＝-1，0、......、7是“不可用的”的情况中，如下述表达式(61)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Furthermore, in the case where P(x, -1); x, y=-1, 0, . . . , 7 are "unavailable", the current macroblock is generated as in Expression (61) below The predicted pixel value Pred(x, y) of each pixel of A.

[数学式10][mathematical formula 10]

$\mathrm{Pred}(x,y)=[\left(\underset{n=0}{\overset{7}{\mathrm{\&Sigma;}}}P(-1,n)\right)+4]>>3$其中x，y＝0，…，7 …(61)$\mathrm{Pred}(x,\mathrm{the\; y})=[\left(\underset{\mathrm{no}=0}{\overset{7}{\mathrm{\&Sigma;}}}P(-1,\mathrm{no})\right)+4]>>3$ where x,y=0,...,7...(61)

模式1是水平预测模式，并且仅仅在P(-1，y)；x，y＝-1，0、......、7是“可用的”时适用。在这种情况下，如下述表达式(62)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 1 is a horizontal prediction mode and only applies when P(-1, y); x, y=-1, 0, . . . , 7 are "available". In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (62).

Pred(x，y)＝P(-1，y)；x，y＝0、......、7 ...(62)Pred(x, y)=P(-1, y); x, y=0,...,7...(62)

模式2是垂直预测模式，并且仅仅在P(x，-1)；x，y＝-1，0、......、7是“可用的”时适用。在这种情况下，如下述表达式(63)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 2 is a vertical prediction mode, and is only applicable when P(x, -1); x, y=-1, 0, . . . , 7 are "available". In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (63).

Pred(x，y)＝P(x，-1)；x，y＝0、......、7 ...(63)Pred(x, y)=P(x,-1); x, y=0,...,7...(63)

模式3是平面预测模式，并且仅仅在P(x，-1)和P(-1，y)；x，y＝-1，0、......、7是“可用的”时适用。在这种情况下，如下述表达式(64)一样产生当前宏块A的每一个像素的预测像素值Pred(x，y)。Mode 3 is a planar prediction mode and only applies when P(x,-1) and P(-1,y); x,y=-1,0,...,7 are "available" . In this case, the predicted pixel value Pred(x, y) of each pixel of the current macroblock A is generated as in the following expression (64).

[数学式11][mathematical formula 11]

Pred(x，y)＝Clip1(a+b·(x-3)+c·(y-3)+16)＞＞5；x，y＝0，...，7Pred(x, y)=Clip1(a+b·(x-3)+c·(y-3)+16)>>5; x, y=0,...,7

a＝16·(P(-1，7)+P(7，-1))a=16·(P(-1,7)+P(7,-1))

b＝(17·H+16)＞＞5b=(17·H+16)>>5

c＝(17·V+16)＞＞5c＝(17·V+16)＞＞5

$\mathrm{<span><span>H</span>h</span>}<span><span>=</span>=</span>\underset{\mathrm{<span><span>x</span>x</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>4</span>4</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\mathrm{<span><span>x</span>x</span>}<span><span>\&CenterDot;</span>\&CenterDot;</span><span><span>[</span>[</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span>\mathrm{<span><span>3</span>3</span>}<span><span>+</span>+</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span><span><span>-</span>-</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span>\mathrm{<span><span>3</span>3</span>}<span><span>-</span>-</span>\mathrm{<span><span>x</span>x</span>}<span><span>,</span>,</span><span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}<span><span>)</span>)</span><span><span>]</span>]</span>$

$\mathrm{<span><span>V</span>V</span>}<span><span>=</span>=</span>\underset{\mathrm{<span><span>y</span>the\; y</span>}<span><span>=</span>=</span>\mathrm{<span><span>1</span>1</span>}}{\overset{\mathrm{<span><span>4</span>4</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\mathrm{<span><span>y</span>the\; y</span>}<span><span>\&CenterDot;</span>\&Center\; Dot;</span><span><span>[</span>[</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span><span><span>-</span>-</span>\mathrm{<span><span>1,3</span>1,3</span>}<span><span>+</span>+</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>)</span>)</span><span><span>-</span>-</span>\mathrm{<span><span>P</span>P</span>}<span><span>(</span>(</span><span><span>-</span>-</span>\mathrm{<span><span>1,3</span>1,3</span>}<span><span>-</span>-</span>\mathrm{<span><span>y</span>the\; y</span>}<span><span>)</span>)</span><span><span>]</span>]</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>64</span>64</span>}<span><span>)</span>)</span>$

如上所述，亮度信号的帧内预测模式包括4×4像素和8×8像素块单位的九种预测模式以及16×16像素宏块单位的四种预测模式。针对每一个宏块单位，设置这些块单位的模式。颜色差信号的帧内预测模式包括8×8像素宏块单位的四种预测模式。可以与亮度信号的帧内预测模式独立地设置颜色差信号的帧内预测模式。As described above, the intra prediction mode of the luma signal includes nine prediction modes in units of 4×4 pixel and 8×8 pixel blocks and four prediction modes in units of 16×16 pixel macroblocks. For each macroblock unit, the modes of these block units are set. The intra prediction mode of the color difference signal includes four prediction modes in units of 8×8 pixel macroblocks. The intra prediction mode of the color difference signal can be set independently from the intra prediction mode of the luma signal.

此外，关于亮度信号的4×4像素帧内预测模式(帧内4×4预测模式)和8×8像素帧内预测模式(帧内8×8预测模式)，针对4×4像素和8×8像素亮度信号块，设置一种帧内预测模式。关于亮度信号的16×16像素帧内预测模式(帧内16×16预测模式)和颜色差信号的帧内预测模式，针对一个宏块设置一种预测模式。Also, regarding the 4×4 pixel intra prediction mode (intra 4×4 prediction mode) and the 8×8 pixel intra prediction mode (intra 8×8 prediction mode) for luminance signals, for 4×4 pixels and 8× An 8-pixel luminance signal block, setting an intra-frame prediction mode. Regarding the 16×16 pixel intra prediction mode (intra 16×16 prediction mode) of the luminance signal and the intra prediction mode of the color difference signal, one prediction mode is set for one macroblock.

请注意，这些种类的预测模式对应于图5中的用上述的编号0、1、3至8指示的方向。预测模式2是平均值预测。Please note that these kinds of prediction modes correspond to the directions indicated by the above-mentioned numbers 0, 1, 3 to 8 in FIG. 5 . Forecast mode 2 is average forecast.

如上所述，对于根据H.264/AVC格式的帧内预测，仅仅在用表达式(14)至表达式(24)以上述的8×8像素的块增量执行帧内预测之前，用确定的滤波系数对邻近像素的像素值执行滤波处理。相反，采用图像编码装置51，在执行所有的帧内预测模式的帧内预测之前，用根据要预测的块设置的滤波系数对邻近像素的像素值执行滤波处理。As described above, for the intra prediction according to the H.264/AVC format, just before intra prediction is performed in the above-mentioned 8×8 pixel block increments by Expression (14) to Expression (24), with the determination The filter coefficients of perform filtering on the pixel values of neighboring pixels. In contrast, with the image encoding device 51 , before performing intra prediction in all intra prediction modes, filter processing is performed on pixel values of adjacent pixels with filter coefficients set according to a block to be predicted.

[帧内预测单元和邻近像素内插滤波切换单元的配置例子][Configuration Example of Intra Prediction Unit and Adjacent Pixel Interpolation Filtering Switching Unit]

图14是图示图1中示出的邻近像素内插滤波切换单元75和帧内预测单元74的详细配置例子的框图。FIG. 14 is a block diagram illustrating a detailed configuration example of the adjacent pixel interpolation filtering switching unit 75 and the intra prediction unit 74 shown in FIG. 1 .

在图14中的例子的情况中，帧内预测单元74由邻近图像设置单元81、预测图像产生单元82和最佳预测模式确定单元83构成。In the case of the example in FIG. 14 , the intra prediction unit 74 is composed of an adjacent image setting unit 81 , a predicted image generation unit 82 , and an optimum prediction mode determination unit 83 .

邻近像素内插滤波切换单元75由预测模式缓冲器91、量化参数缓冲器92和低通滤波设置单元93构成。请注意，低通滤波设置单元93具有内置的滤波系数存储器94。The adjacent pixel interpolation filter switching unit 75 is composed of a prediction mode buffer 91 , a quantization parameter buffer 92 , and a low-pass filter setting unit 93 . Note that the low-pass filter setting unit 93 has a built-in filter coefficient memory 94 .

从帧存储器72向邻近图像设置单元81供应用于帧内预测的当前块的邻近像素值。虽然在图14中省略了开关73的图示，但是，实际上，经由开关73从帧存储器72向邻近图像设置单元81执行供应。请注意，在帧内预测的情况中，未经过去块滤波器71的去块滤波的像素值用作邻近像素值。The adjacent pixel values of the current block used for intra prediction are supplied from the frame memory 72 to the adjacent image setting unit 81 . Although illustration of the switch 73 is omitted in FIG. 14 , actually, supply is performed from the frame memory 72 to the adjacent image setting unit 81 via the switch 73 . Note that in the case of intra prediction, pixel values that have not been deblock-filtered by the deblock filter 71 are used as adjacent pixel values.

邻近图像设置单元81使用由低通滤波设置单元93设置的滤波系数来对来自帧存储器72的当前块的邻近像素值执行滤波处理，并且将经过滤波处理的邻近像素值供应给预测图像产生单元82。The adjacent image setting unit 81 performs filter processing on the adjacent pixel values of the current block from the frame memory 72 using the filter coefficient set by the low-pass filter setting unit 93 , and supplies the filtered adjacent pixel values to the predicted image generation unit 82 .

预测图像产生单元82将其模式是当前正在处理的帧内预测模式的信息供应给预测模式缓冲器91。预测图像产生单元82使用来自邻近图像设置单元81的经过滤波处理的邻近像素值以供应给预测模式缓冲器91的帧内预测模式对当前块执行帧内预测，并且产生预测图像。将产生的预测图像与帧内预测模式信息一起供应给最佳预测模式确定单元83。The predicted image generation unit 82 supplies information whose mode is the intra prediction mode currently being processed to the prediction mode buffer 91 . The predicted image generating unit 82 performs intra prediction on the current block in the intra prediction mode supplied to the prediction mode buffer 91 using the filter-processed adjacent pixel values from the adjacent image setting unit 81 , and generates a predicted image. The generated predicted image is supplied to the optimum prediction mode determination unit 83 together with the intra prediction mode information.

向最佳预测模式确定单元83供应从画面重新排列缓冲器62读出的要进行帧内预测的图像、由预测图像产生单元82产生的预测图像及其帧内预测模式信息。The image to be subjected to intra prediction read out from the screen rearranging buffer 62 , the predicted image generated by the predicted image generation unit 82 , and intra prediction mode information thereof are supplied to the optimum prediction mode determination unit 83 .

最佳预测模式确定单元83使用供应的信息来计算关于已经产生预测图像的帧内预测模式的成本函数值，并且将产生计算出的成本函数值中的最小值的帧内预测模式决定为最佳帧内预测模式。最佳预测模式确定单元83将最佳帧内预测模式的预测图像和相应的成本函数值输出到预测图像选择单元77。The optimum prediction mode determination unit 83 calculates a cost function value regarding the intra prediction mode that has generated the predicted image using the supplied information, and decides the intra prediction mode that produces the smallest value among the calculated cost function values as the optimum Intra prediction mode. The optimum prediction mode determination unit 83 outputs the prediction image of the optimum intra prediction mode and the corresponding cost function value to the prediction image selection unit 77 .

此外，在预测图像选择单元77选择了在最佳帧内预测模式中产生的预测图像的情况中，最佳预测模式确定单元83将指示最佳帧内预测模式的信息供应给无损耗编码单元66。Furthermore, in a case where the predicted image selection unit 77 has selected a predicted image generated in the optimum intra prediction mode, the optimum prediction mode determination unit 83 supplies information indicating the optimum intra prediction mode to the lossless encoding unit 66 .

预测模式缓冲器91存储来自预测图像产生单元82的帧内预测模式信息。量化参数缓冲器92存储来自速率控制单元78的量化参数。The prediction mode buffer 91 stores intra prediction mode information from the prediction image generation unit 82 . The quantization parameter buffer 92 stores quantization parameters from the rate control unit 78 .

低通滤波设置单元93从预测模式缓冲器91读出当前块的帧内预测模式信息，并且，从量化参数缓冲器92读出与当前块对应的量化参数。低通滤波设置单元93从存储于内置滤波系数存储器94中的滤波系数设置与该信息对应的滤波系数，并且，将设置的滤波系数供应给邻近图像设置单元81。The low-pass filter setting unit 93 reads out the intra prediction mode information of the current block from the prediction mode buffer 91 , and reads out the quantization parameter corresponding to the current block from the quantization parameter buffer 92 . The low-pass filter setting unit 93 sets filter coefficients corresponding to this information from the filter coefficients stored in the built-in filter coefficient memory 94 , and supplies the set filter coefficients to the adjacent image setting unit 81 .

滤波系数存储器94存储与量化参数对应的滤波系数和通过在稍后描述的图28中的学习装置251处使用训练图像进行学习而获得的帧内预测模式。例如，针对每一片(slice)，计算和存储滤波系数，如下所述。The filter coefficient memory 94 stores filter coefficients corresponding to quantization parameters and intra prediction modes obtained by learning using training images at the learning means 251 in FIG. 28 described later. For example, for each slice, filter coefficients are calculated and stored as described below.

[最佳滤波系数的计算的描述][Description of Calculation of Optimal Filter Coefficients]

接下来，将参照图15描述用于对邻近像素的该滤波处理的最佳滤波系数的计算方法。请注意，在图15的例子中，示出对4×4像素的当前块执行垂直预测(垂直预测)的例子，但是，以下描述是可适用于任何帧内预测模式的情况。Next, a calculation method of an optimum filter coefficient used for this filter process on adjacent pixels will be described with reference to FIG. 15 . Note that in the example of FIG. 15 , an example of performing vertical prediction (vertical prediction) on a current block of 4×4 pixels is shown, however, the following description is applicable to any intra prediction mode.

对于上文中用表达式(14)至表达式(24)描述的8×8像素的块增量的帧内预测，{1，2，1}的3-抽头滤波系数被定义为用于邻近像素的低通滤波器，但是我们将{c₀，c₁，c₂}视为3-抽头的一般形式。此外，对于本发明，还引入第四参数c₃作为偏移值。For intra prediction in block increments of 8×8 pixels described above with Expressions (14) to (24), 3-tap filter coefficients of {1, 2, 1} are defined as , but we consider {c₀ , c₁ , c₂ } as a general form of 3-tap. In addition, for the present invention, a fourth parameter c₃ is also introduced as an offset value.

请注意，虽然在下面的描述中该3-抽头滤波器被描述为可针对每一个片段设置，但是，该3-抽头滤波器不限于此，并且，例如，可以针对整个序列或针对每一个GOP设置。Please note that although the 3-tap filter is described as being settable for each slice in the following description, the 3-tap filter is not limited thereto, and, for example, may be set for the entire sequence or for each GOP set up.

在图15中的例子中，a_km(0≤k，m≤3)是在当前块中包含的像素的像素值，并且，bm(-1≤m≤4)是用于垂直预测的邻近像素的像素值。In the example in Fig. 15, a_km (0≤k, m≤3) is the pixel value of a pixel contained in the current block, and bm (-1≤m≤4) is a neighboring pixel for vertical prediction pixel value.

首先，通过对邻近像素值b_m执行的3-抽头滤波处理来产生在下述表达式(65)中示出的b′m(0≤m≤3)。First, b′m (0≦m≦3) shown in the following expression (65) is generated by 3-tap filter processing performed on the adjacent pixel value b_m .

[数学式12][mathematical formula 12]

b′_m＝c₀*b_m-1+c₁*b_m+c₂*b_m+1+c₃(0≤m≤3) …(65)b′_m ＝c₀ *b_m-1 +c₁ *b_m +c₂ *b_m+1 +c₃ (0≤m≤3) …(65)

也就是说，在执行滤波处理并且使用滤波系数的情况中，我们将说，也使用相应的偏移值，如表达式(65)所示，甚至以下无需特别地提及。换句话说，滤波系数和偏移值是用于滤波处理的系数。以相同的方式，在滤波系数以编码的方式被发送到解码侧的情况中，我们将说，相应的偏移值也以编码的方式被发送。That is, in the case of performing filter processing and using a filter coefficient, we will say that a corresponding offset value is also used as shown in Expression (65), and even need not be particularly mentioned below. In other words, the filter coefficients and offset values are coefficients used for filter processing. In the same way, in the case where the filter coefficients are sent to the decoding side in encoded form, we will say that the corresponding offset values are also sent in encoded form.

现在，如果我们说在帧内预测模式是n时的预测像素值是p_ij(b′m，n)；0≤i，j≤3，则关于帧内预测像素值，下述表达式(66)成立，因为通过上文中参照图2至14描述的线性表达式来产生预测像素。Now, if we say that the predicted pixel value when the intra prediction mode is n is p_ij (b'm, n); 0≤i, j≤3, then regarding the intra predicted pixel value, the following expression (66 ) holds because predicted pixels are generated by the linear expressions described above with reference to FIGS. 2 to 14 .

[数学式13][mathematical formula 13]

p_ij(b′_m，n)p_ij (b′_m , n)

＝p_ij(c₀*b_m-1+c₁*b_m+c₂*b_m+1+c₃，n)＝p_ij (c₀ *b_m-1 +c₁ *b_m +c₂ *b_m+1 +c₃ ,n)

＝c₀*p_ij(b_m-1，n)+c₁*p_ij(b_m，n)+c₂*p_ij(b_m+1，n)+c₃ …(66)=c₀ *p_ij (b_m-1 ,n)+c₁ *p_ij (b_m ,n)+c₂ *p_ij (b_m+1 ,n)+c₃ ...(66)

此时，关于以a_ij作为原始图像像素值的当前块Ω的预测平方误差如下述表达式(67)所示。At this time, the prediction square error with respect to the current block Ω with a_ij as the original image pixel value is expressed in the following expression (67).

[数学式14][mathematical formula 14]

$\mathrm{<span><span>Err</span>Err</span>}<span><span>(</span>(</span>\mathrm{<span><span>\&Omega;</span>\&Omega;</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>3</span>3</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\underset{\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>3</span>3</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}{<span><span>(</span>(</span>{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>-</span>-</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}}^{<span><span>\&prime;</span>\&prime;</span>}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>)</span>)</span>}^{\mathrm{<span><span>2</span>2</span>}}$

$<span><span>=</span>=</span>\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>3</span>3</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\underset{\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>3</span>3</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}{<span><span>(</span>(</span>{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>km</span>km</span>}}<span><span>-</span>-</span><span><span>\{</span>\{</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>0</span>0</span>}}<span><span>\&CenterDot;</span>\&Center\; Dot;</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>+</span>+</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>\&CenterDot;</span>\&Center\; Dot;</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>+</span>+</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>\&CenterDot;</span>\&CenterDot;</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>+</span>+</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>3</span>3</span>}}<span><span>\}</span>\}</span><span><span>)</span>)</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>67</span>67</span>}<span><span>)</span>)</span>$

现在，如果我们用Φ表示当前片段中的用帧内预测模式n编码的帧内块的集合，则关于属于Φ的块的预测平方误差的和用下述表达式(68)表示。Now, if we denote by Φ the set of intra blocks encoded with intra prediction mode n in the current slice, the sum of the prediction square errors with respect to the blocks belonging to Φ is represented by the following expression (68).

[数学式15][mathematical formula 15]

$\mathrm{<span><span>Err</span>Err</span>}<span><span>(</span>(</span>\mathrm{<span><span>\&Omega;</span>\&Omega;</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>\&Phi;</span>\&Phi;</span>}<span><span>)</span>)</span><span><span>=</span>=</span>\underset{\mathrm{<span><span>\&Omega;</span>\&Omega;</span>}<span><span>\&Element;</span>\&Element;</span>\mathrm{<span><span>\&Phi;</span>\&Phi;</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}\underset{\mathrm{<span><span>i</span>i</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>3</span>3</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}\underset{\mathrm{<span><span>j</span>j</span>}<span><span>=</span>=</span>\mathrm{<span><span>0</span>0</span>}}{\overset{\mathrm{<span><span>3</span>3</span>}}{\mathrm{<span><span>\&Sigma;</span>\&Sigma;</span>}}}<span><span>(</span>(</span>{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>km</span>km</span>}}<span><span>-</span>-</span><span><span>\{</span>\{</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>0</span>0</span>}}<span><span>\&CenterDot;</span>\&CenterDot;</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>+</span>+</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>1</span>1</span>}}<span><span>\&CenterDot;</span>\&CenterDot;</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span>$

$<span><span>+</span>+</span>{{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>2</span>2</span>}}<span><span>\&CenterDot;</span>\&CenterDot;</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>m</span>m</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>,</span>,</span>\mathrm{<span><span>n</span>no</span>}<span><span>)</span>)</span><span><span>+</span>+</span>{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>3</span>3</span>}}<span><span>\}</span>\}</span><span><span>)</span>)</span>}^{\mathrm{<span><span>2</span>2</span>}}<span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>68</span>68</span>}<span><span>)</span>)</span>$

在上述表达式(68)中，我们将Err(Ω∈Φ)视为c₀、c₁、c₂、c₃的函数，即，Err(Ω∈Φ；c₀，c₁，c₂，c₃)，所以使Err(Ω∈Φ；c₀，c₁，c₂，c₃)最小化的c₀、c₁、c₂、c₃将是当前片段中的最佳滤波系数值。也就是说，在下述表达式(69)成立的情况中足以获得c₀、c₁、c₂、c₃。In the above expression (68), we regard Err(Ω∈Φ) as a function of c₀ , c₁ , c₂ , c₃ , that is, Err(Ω∈Φ; c₀ , c₁ , c₂ , c₃ ), so c₀ , c₁ , c₂ , c₃ that minimize Err(Ω∈Φ; c₀ , c₁ , c₂ , c₃ ) will be the best filter coefficient values in the current segment. That is, it is sufficient to obtain c₀ , c₁ , c₂ , c₃ in the case where the following expression (69) holds.

[数学式16][mathematical formula 16]

也就是说，从表达式(69)获得下述表达式(70)中所示的联立方程。That is, simultaneous equations shown in Expression (70) below are obtained from Expression (69).

[数学式17][mathematical formula 17]

使用矩阵对该表达式(70)进行公式变形，得到表达式(71)。The expression (70) is transformed by using a matrix to obtain the expression (71).

[数学式18][mathematical formula 18]

$\left[\begin{array}{cccc}\mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>\\ \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>\\ \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>\\ \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>& \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}\mathrm{<span><span>1</span>1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>0</span>0</span>}}\\ {\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>1</span>1</span>}}\\ {\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>2</span>2</span>}}\\ {\mathrm{<span><span>c</span>c</span>}}_{\mathrm{<span><span>3</span>3</span>}}\end{array}\right]$

$<span><span>=</span>=</span>\left[\begin{array}{c}\mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>-</span>-</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>\\ \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}}<span><span>)</span>)</span>\\ \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}{\mathrm{<span><span>p</span>p</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}<span><span>(</span>(</span>{\mathrm{<span><span>b</span>b</span>}}_{\mathrm{<span><span>k</span>k</span>}<span><span>+</span>+</span>\mathrm{<span><span>1</span>1</span>}}<span><span>)</span>)</span>\\ \mathrm{<span><span>\&Sigma;\&Sigma;\&Sigma;</span>\&Sigma;\&Sigma;\&Sigma;</span>}{\mathrm{<span><span>a</span>a</span>}}_{\mathrm{<span><span>ij</span>ij</span>}}\end{array}\right]<span><span>.</span>.</span><span><span>.</span>.</span><span><span>.</span>.</span><span><span>(</span>(</span>\mathrm{<span><span>71</span>71</span>}<span><span>)</span>)</span>$

求解该表达式(71)，能够针对当前片段获得最佳滤波系数和偏移值{c₀，c₁，c₂，c₃}。By solving the expression (71), the optimal filter coefficients and offset values {c₀ , c₁ , c₂ , c₃ } can be obtained for the current segment.

请注意，通过求解表达式(70)中的联立方程，获得最佳滤波系数和偏移值{c₀，c₁，c₂，c₃}作为浮点值，但是，例如，在图1中的图像编码装置51和图22中的相应的图像解码装置151的情况中，这些被舍入为8比特系数。Note that by solving the simultaneous equations in Expression (70), the optimal filter coefficients and offset values {c₀ , c₁ , c₂ , c₃ } are obtained as floating-point values, but, for example, in Fig. 1 In the case of the image encoding device 51 in and the corresponding image decoding device 151 in FIG. 22, these are rounded to 8-bit coefficients.

也就是说，即使滤波系数是浮点，滤波系数存储器94也根据例如处理器的寄存器长度保持这些作为n比特(其中，n是整数)值。That is, even if the filter coefficients are floating points, the filter coefficient memory 94 holds these as n-bit (where n is an integer) values according to, for example, the processor's register length.

通过对其它帧内预测方法也应用与上述方法相同的方法，也可以针对其它帧内预测方法获得最佳滤波系数。此外，通过相同的方法，不仅对于帧内4×4预测模式，而且对于帧内8×8预测模式、帧内16×16预测模式和颜色差信号的帧内预测模式，都可以获得最佳滤波系数。Optimal filter coefficients can also be obtained for other intra prediction methods by applying the same method as the above method to other intra prediction methods as well. Furthermore, by the same method, optimal filtering can be obtained not only for intra 4×4 prediction mode, but also for intra 8×8 prediction mode, intra 16×16 prediction mode and intra prediction mode of color difference signal coefficient.

虽然在以上描述中针对每一个帧内预测模式获得了一个滤波系数，但是，该滤波系数不限于此，并且，可以进行这样的安排：其中，对于所有的帧内预测模式，仅仅获得一个滤波系数。特别地，对于上文中参照图2至图14描述的帧内预测模式，针对垂直(垂直，vertical)和水平(水平，horizontal)模式原样地使用预测像素值，但是，针对其它模式执行某种平均处理或加权平均处理以产生预测像素，所以其特性不同。因此，执行垂直水平模式和其它模式的两种类别分类，并且，计算每一种类别的滤波系数，可以实现编码效率的进一步提高。此外，例如，对于亮度信号，可以存在用于帧内4×4预测模式的一个滤波系数、用于帧内8×8预测模式的一个滤波系数、以及用于帧内16×16预测模式的一个滤波系数。例如，对于颜色差信号，可以针对Cb/Cr分别获得滤波系数。Although one filter coefficient is obtained for each intra prediction mode in the above description, the filter coefficient is not limited thereto, and an arrangement may be made in which only one filter coefficient is obtained for all intra prediction modes . In particular, for the intra prediction modes described above with reference to FIGS. 2 to 14 , the predicted pixel values are used as-is for the vertical and horizontal modes, but some averaging processing or weighted average processing to produce predicted pixels, so their characteristics are different. Therefore, performing two-category classification of the vertical-horizontal mode and other modes, and calculating filter coefficients for each category, further improvement in coding efficiency can be achieved. In addition, for the luminance signal, for example, there may be one filter coefficient for the intra 4×4 prediction mode, one filter coefficient for the intra 8×8 prediction mode, and one filter coefficient for the intra 16×16 prediction mode. filter coefficient. For example, for a color difference signal, filter coefficients can be obtained separately for Cb/Cr.

此外，在以上描述中，对于用于低通滤波处理的滤波系数，使用了{c₀，c₁，c₂}的三抽头，但是，这不限于3抽头，也可以使用任何数目的抽头的滤波器。也就是说，获得了抽头数目的滤波系数+偏移值。然而，随着抽头数目增加，要求解的联立方程的次数也增加。Also, in the above description, for the filter coefficients used for the low-pass filter processing, three taps of {c₀ , c₁ , c₂ } are used, however, this is not limited to 3 taps, and any number of taps may be used. filter. That is, the filter coefficient+offset value of the number of taps is obtained. However, as the number of taps increases, the number of simultaneous equations to be solved also increases.

此外，可以进行这样的排列：其中，准备和应用根据图像帧不同的滤波系数，例如CIF(通用中间格式，Common Intermediate Format)/QCIF(四分之一CIF，Quarter CIF)，SD(标准清晰度)，HD(高清晰度)等。In addition, an arrangement may be made in which different filter coefficients are prepared and applied according to image frames, such as CIF (Common Intermediate Format, Common Intermediate Format)/QCIF (Quarter CIF, Quarter CIF), SD (Standard Definition ), HD (High Definition), etc.

此外，对于上述方法，通过使帧内预测残差(预测平方误差)最小化来计算滤波系数。但是，滤波系数计算方法不限于此，并且，在需要将滤波系数发送到解码侧的情况中，也可以执行包含用于发送滤波系数的比特的优化。Also, with the above method, the filter coefficient is calculated by minimizing the intra prediction residual (prediction square error). However, the filter coefficient calculation method is not limited thereto, and in the case where the filter coefficient needs to be transmitted to the decoding side, optimization including bits for transmitting the filter coefficient may also be performed.

此外，对于上述的滤波系数，我们假设系数的对称性，如下述表达式(72)所示。Furthermore, for the filter coefficients described above, we assume the symmetry of the coefficients as shown in the following expression (72).

C0＝C2...(72)C0=C2...(72)

也就是说，计算滤波系数，以关于对应于0相位的中心系数具有对称性，如{c₀，c₁，c₀}一样。因此，如上述表达式(70)所示的三个联立方程可以被简化为2个。结果，可以减少计算量。That is, filter coefficients are calculated to have symmetry about the center coefficient corresponding to 0 phase, like {c₀ , c₁ , c₀ }. Therefore, the three simultaneous equations shown in the above expression (70) can be simplified to two. As a result, the amount of calculation can be reduced.

通过设置适合于输入图像的滤波系数并以自适应的方式对邻近像素执行低通滤波处理，使用上述方法，能够使用适合于该图像的预测图像、量化参数和预测模式来执行编码，从而可以提高编码效率。By setting filter coefficients suitable for an input image and performing low-pass filter processing on adjacent pixels in an adaptive manner, using the above method, encoding can be performed using a predicted image, quantization parameter, and prediction mode suitable for the image, thereby improving Coding efficiency.

关于上述的最佳滤波系数计算，可以想到两种方法。一种方法是离线处理，即，其中，在执行编码处理之前，预先使用用于训练的图像信号来计算滤波系数，从而使所有的图像信号优化。稍后将参照图28描述作为该离线处理的学习处理，并且，通过该学习处理计算出的滤波系数和偏移值被存储于图14中的滤波系数存储器94中。Regarding the calculation of the optimum filter coefficient described above, two methods are conceivable. One method is offline processing, that is, in which, before performing encoding processing, filter coefficients are calculated in advance using image signals for training so that all image signals are optimized. Learning processing as this offline processing will be described later with reference to FIG. 28 , and filter coefficients and offset values calculated by this learning processing are stored in filter coefficient memory 94 in FIG. 14 .

第二种方法是在线处理，即，针对每一个片段连续地计算最佳滤波系数。在这种情况下，将计算出的滤波系数和偏移值发送到解码侧。稍后将参照图20描述执行作为第二种方法的离线处理的情况的例子。The second method is online processing, ie, the optimal filter coefficients are continuously calculated for each segment. In this case, the calculated filter coefficients and offset values are sent to the decoding side. An example of a case where offline processing is performed as the second method will be described later with reference to FIG. 20 .

[图像编码装置的编码处理的描述][Description of Coding Process of Image Coding Device]

接下来，将参照图16的流程图描述图1中的图像编码装置51的编码处理。Next, encoding processing by the image encoding device 51 in FIG. 1 will be described with reference to the flowchart of FIG. 16 .

在步骤S11中，A/D转换单元61对输入图像执行模拟数字转换。在步骤S12中，画面重新排列缓冲器62存储从A/D转换单元61供应的图像，并且执行从用于显示图片的序列到用于编码的序列的重新排列。In step S11, the A/D conversion unit 61 performs analog-to-digital conversion on the input image. In step S12 , the screen rearrangement buffer 62 stores the images supplied from the A/D conversion unit 61 , and performs rearrangement from a sequence for displaying pictures to a sequence for encoding.

在步骤S13中，计算单元63计算在步骤S12中重新排列的图像和预测图像之间的差。在执行帧间预测的情况中，经由预测图像选择单元77将预测图像从运动预测/补偿单元76供应到计算单元63，并且，在执行帧内预测的情况中，经由预测图像选择单元77将预测图像从帧内预测单元74供应到计算单元63。In step S13, the calculation unit 63 calculates the difference between the image rearranged in step S12 and the predicted image. In the case of performing inter prediction, the predicted image is supplied from the motion prediction/compensation unit 76 to the computing unit 63 via the predicted image selection unit 77, and, in the case of performing intra prediction, the predicted image is supplied via the predicted image selection unit 77. Images are supplied to the calculation unit 63 from the intra prediction unit 74 .

与原始图像数据相比，差分数据的数据量较小。因此，与在没有变化的情况下对原始图像进行编码的情况相比，可以压缩数据量。Compared with the original image data, the data amount of the difference data is small. Therefore, the amount of data can be compressed compared to the case of encoding the original image without change.

在步骤S14中，正交变换单元64对从计算单元63供应的差分信息进行正交变换。具体地说，执行诸如离散余弦变换、Karhunen-Loéve变换等的正交变换，并且，输出变换系数。在步骤S15中，量化单元65对变换系数进行量化。在该量化时，控制速率，从而将描述稍后描述的步骤S25中的处理。In step S14 , the orthogonal transform unit 64 performs orthogonal transform on the difference information supplied from the calculation unit 63 . Specifically, orthogonal transform such as discrete cosine transform, Karhunen-Loéve transform, etc. is performed, and transform coefficients are output. In step S15, the quantization unit 65 quantizes the transform coefficients. At the time of this quantization, the rate is controlled, so that the processing in step S25 described later will be described.

对这样量化的差分信息进行如下的局部解码。具体地说，在步骤S16中，逆量化单元68使用与量化单元65的特性对应的特性来对由量化单元65量化的变换系数进行逆量化。在步骤S17中，逆正交变换单元69使用与正交变换单元64的特性对应的特性来对经过逆量化单元68的逆量化的变换系数进行逆正交变换。The following local decoding is performed on the difference information quantized in this way. Specifically, in step S16 , the inverse quantization unit 68 dequantizes the transform coefficient quantized by the quantization unit 65 using the characteristic corresponding to the characteristic of the quantization unit 65 . In step S17 , the inverse orthogonal transform unit 69 performs inverse orthogonal transform on the transform coefficients subjected to the inverse quantization by the inverse quantization unit 68 using characteristics corresponding to the characteristics of the orthogonal transform unit 64 .

在步骤S18中，计算单元70将经由预测图像选择单元77输入的预测图像与局部解码的差分信息相加，并且产生局部解码的图像(该图像对应于到计算单元63的输入)。在步骤S19中，去块滤波器71对从计算单元70输出的图像进行滤波。因此，去除了块失真。在步骤S20中，帧存储器72存储经过滤波的图像。请注意，未经过去块滤波器71的滤波处理的图像也从计算单元70被供应到帧存储器72，以便存储。In step S18, the calculation unit 70 adds the predicted image input via the predicted image selection unit 77 to the locally decoded difference information, and generates a locally decoded image (which corresponds to the input to the calculation unit 63). In step S19 , the deblocking filter 71 filters the image output from the calculation unit 70 . Therefore, block distortion is removed. In step S20, the frame memory 72 stores the filtered image. Note that an image that has not been filtered by the past block filter 71 is also supplied from the calculation unit 70 to the frame memory 72 to be stored.

在步骤S21中，帧内预测单元74和运动预测/补偿单元76均执行图像预测处理。具体地说，在步骤S21中，帧内预测单元74以帧内预测模式执行帧内预测处理。运动预测/补偿单元76以帧间预测模式执行运动预测和补偿处理。In step S21 , both the intra prediction unit 74 and the motion prediction/compensation unit 76 perform image prediction processing. Specifically, in step S21, the intra prediction unit 74 performs intra prediction processing in the intra prediction mode. The motion prediction/compensation unit 76 performs motion prediction and compensation processing in the inter prediction mode.

稍后将参照图17描述步骤S21中的预测处理的细节，但是，根据本处理，执行所有的候选预测模式中的预测处理，并且，计算所有的候选预测模式中的成本函数值。然后，基于计算出的成本函数值来选择最佳帧内预测模式，并且，将通过最佳帧内预测模式中的帧内预测产生的预测图像及其成本函数值供应给预测图像选择单元77。Details of the prediction processing in step S21 will be described later with reference to FIG. 17 , but according to the present processing, prediction processing in all candidate prediction modes is performed, and cost function values in all candidate prediction modes are calculated. Then, an optimal intra prediction mode is selected based on the calculated cost function value, and a predicted image generated by intra prediction in the optimal intra prediction mode and its cost function value are supplied to the predicted image selection unit 77 .

请注意，此时，在帧内预测处理之前，帧内预测单元74使用由邻近像素内插滤波切换单元75设置的滤波系数来对用于当前块的帧内预测的邻近像素执行滤波处理。然后，在帧内预测单元74处使用经过滤波处理的邻近像素来执行帧内预测，并且，产生预测图像。Note that at this time, before the intra prediction process, the intra prediction unit 74 uses the filter coefficient set by the adjacent pixel interpolation filter switching unit 75 to perform filter processing on adjacent pixels used for intra prediction of the current block. Then, intra prediction is performed at the intra prediction unit 74 using the filter-processed adjacent pixels, and a predicted image is generated.

在步骤S22中，预测图像选择单元77基于从帧内预测单元74和运动预测/补偿单元76输出的成本函数值将最佳帧内预测模式和最佳帧间预测模式之一确定为最佳预测模式。预测图像选择单元77然后选择确定的最佳预测模式中的预测图像，并且将其供应给计算单元63和70。如上所述，该预测图像用于步骤S13和S18中的计算。In step S22, the predicted image selection unit 77 determines one of the optimal intra prediction mode and the optimal inter prediction mode as the optimal prediction based on the cost function values output from the intra prediction unit 74 and the motion prediction/compensation unit 76 model. The predicted image selection unit 77 then selects the predicted image in the determined optimal prediction mode, and supplies it to the calculation units 63 and 70 . As described above, this predicted image is used for the calculations in steps S13 and S18.

请注意，该预测图像的选择信息被供应给帧内预测单元74或运动预测/补偿单元76。在选择了最佳帧内预测模式中的预测图像的情况中，帧内预测单元74将指示最佳帧内预测模式的信息(即，帧内预测模式信息)供应给无损耗编码单元66。Note that selection information of this predicted image is supplied to the intra prediction unit 74 or the motion prediction/compensation unit 76 . In a case where a predicted image in the optimum intra prediction mode is selected, the intra prediction unit 74 supplies information indicating the optimum intra prediction mode (ie, intra prediction mode information) to the lossless encoding unit 66 .

在选择了最佳帧间预测模式中的预测图像的情况中，运动预测/补偿单元76将指示最佳帧间预测模式的信息输出给无损耗编码单元66，并且，根据需要，将与最佳帧间预测模式对应的信息输出给无损耗编码单元66。与最佳帧间预测模式对应的信息的例子包括运动向量信息、标志信息和参考帧信息。也就是说，当选择了与作为最佳帧间预测模式的帧间预测模式对应的预测图像时，运动预测/补偿单元76将帧间预测模式信息、运动向量信息和参考帧信息输出到无损耗编码单元66。In the case where the predicted image in the optimum inter prediction mode is selected, the motion prediction/compensation unit 76 outputs information indicating the optimum inter prediction mode to the lossless encoding unit 66, and, if necessary, combines the best The information corresponding to the inter prediction mode is output to the lossless encoding unit 66 . Examples of information corresponding to the optimal inter prediction mode include motion vector information, flag information, and reference frame information. That is, when a predicted image corresponding to the inter prediction mode that is the optimum inter prediction mode is selected, the motion prediction/compensation unit 76 outputs the inter prediction mode information, motion vector information, and reference frame information to the lossless Encoding unit 66.

在步骤S23中，无损耗编码单元66对从量化单元65输出的量化变换系数进行编码。具体地说，对差分图像进行诸如可变长度编码、算术编码等的无损耗编码和压缩。此时，在步骤S22中输入到无损耗编码单元66的来自帧内预测单元74的最佳帧内预测模式信息或者与来自运动预测/补偿单元76的最佳帧间预测模式对应的信息、以及来自速率控制单元78的量化参数等也被编码，并且与头信息相加。In step S23 , the lossless encoding unit 66 encodes the quantized transform coefficient output from the quantization unit 65 . Specifically, lossless coding and compression such as variable length coding, arithmetic coding, etc. are performed on the differential image. At this time, the optimal intra prediction mode information from the intra prediction unit 74 or the information corresponding to the optimal inter prediction mode from the motion prediction/compensation unit 76 input to the lossless encoding unit 66 in step S22, and Quantization parameters and the like from the rate control unit 78 are also encoded, and added to header information.

在步骤S24中，存储缓冲器67存储差分图像作为压缩图像。存储于存储缓冲器67中的压缩图像在适当的时候被读出，并且经由传送路径被传送到解码侧。In step S24, the storage buffer 67 stores the differential image as a compressed image. The compressed image stored in the storage buffer 67 is read out at an appropriate time, and transmitted to the decoding side via a transmission path.

在步骤S25中，速率控制单元78基于存储于存储缓冲器67中的压缩图像通过量化参数控制量化单元65的量化操作的速率，以不会导致上溢或下溢。In step S25, the rate control unit 78 controls the rate of the quantization operation of the quantization unit 65 by quantization parameters based on the compressed image stored in the memory buffer 67 so as not to cause overflow or underflow.

将用于在量化单元65处的速率控制的量化参数供应给无损耗编码单元66，在上述的步骤S23中对该量化参数进行无损耗编码处理，并且将其插入到压缩图像的头部。此外，该量化参数被供应给邻近像素内插滤波切换单元75，并且用于设置用于要对邻近像素执行的滤波处理的滤波系数，在帧内预测之前执行该滤波处理。The quantization parameter used for rate control at the quantization unit 65 is supplied to the lossless encoding unit 66, which is subjected to lossless encoding processing in the above-described step S23, and inserted into the header of the compressed image. Furthermore, this quantization parameter is supplied to the adjacent pixel interpolation filter switching unit 75, and is used to set a filter coefficient for filter processing to be performed on the adjacent pixels, which is performed prior to intra prediction.

[预测处理的描述][Description of prediction processing]

接下来，将参照图17中的流程图描述图16中的步骤S21中的预测处理。Next, the prediction processing in step S21 in FIG. 16 will be described with reference to the flowchart in FIG. 17 .

在从画面重新排列缓冲器62供应的要处理的图像是要进行帧内处理的块中的图像的情况中，要参考的解码图像从帧存储器72被读出，并且，经由开关73被供应给帧内预测单元74。In the case where the image to be processed supplied from the screen rearrangement buffer 62 is an image in a block to be subjected to intra-frame processing, the decoded image to be referred to is read out from the frame memory 72 and, via the switch 73, supplied to Intra prediction unit 74 .

在步骤S31中，帧内预测单元74使用供应的图像以所有的候选帧内预测模式对要处理的块的像素执行帧内预测。请注意，未经过去块滤波器71的去块滤波的像素用作要参考的解码像素。In step S31 , the intra prediction unit 74 performs intra prediction on the pixels of the block to be processed in all candidate intra prediction modes using the supplied image. Note that pixels that have not been deblock-filtered by the deblocking filter 71 are used as decoded pixels to be referred to.

将参照图18描述步骤S31中的帧内预测处理的细节，但是，根据该处理，设置最佳滤波系数，并且，使用设置的滤波系数对邻近像素执行滤波处理。然后，使用已经执行了滤波处理的邻近像素来执行帧内预测，从而产生预测图像。Details of the intra prediction processing in step S31 will be described with reference to FIG. 18 , but according to this processing, optimum filter coefficients are set, and filter processing is performed on adjacent pixels using the set filter coefficients. Then, intra prediction is performed using neighboring pixels on which filter processing has been performed, thereby generating a predicted image.

在所有的候选帧内预测模式上执行上述的处理，针对所有的候选帧内预测模式计算成本函数值，并且，基于计算出的成本函数值来决定最佳帧内预测模式。由此产生的预测图像和最佳帧内预测模式的成本函数值被供应给预测图像选择单元77。The above-described processing is performed on all candidate intra prediction modes, cost function values are calculated for all candidate intra prediction modes, and an optimum intra prediction mode is decided based on the calculated cost function values. The resulting predicted image and the cost function value of the optimum intra prediction mode are supplied to the predicted image selection unit 77 .

在从画面重新排列缓冲器62供应的要处理的图像是要进行帧间处理的图像的情况中，要参考的图像从帧存储器72被读出，并且，经由开关73被供应给运动预测/补偿单元76。在步骤S32中，基于这些图像，运动预测/补偿单元76执行帧间运动预测处理。也就是说，运动预测/补偿单元76参考从帧存储器72供应的图像来执行所有的候选帧间预测模式中的运动预测处理。In the case where the image to be processed supplied from the screen rearrangement buffer 62 is an image to be subjected to inter-frame processing, the image to be referred to is read out from the frame memory 72, and is supplied to motion prediction/compensation via the switch 73 Unit 76. In step S32, based on these images, the motion prediction/compensation unit 76 performs inter motion prediction processing. That is to say, the motion prediction/compensation unit 76 performs motion prediction processing in all candidate inter prediction modes with reference to images supplied from the frame memory 72 .

稍后将参照图19描述步骤S32中的帧间运动预测处理的细节，并且，根据本处理，执行所有的候选帧间预测模式中的运动预测处理，但是，针对所有的候选帧间预测模式计算成本函数值。The details of the inter motion prediction processing in step S32 will be described later with reference to FIG. 19, and according to this processing, the motion prediction processing in all candidate inter prediction modes is performed, however, the calculation for all candidate inter prediction modes Cost function value.

在步骤S33中，运动预测/补偿单元76比较在步骤S32中计算出的关于帧间预测模式的成本函数值，并且将提供最小值的预测模式确定为最佳帧间预测模式。运动预测/补偿单元76然后将在最佳帧间预测模式中产生的预测图像及其成本函数值供应给预测图像选择单元77。In step S33 , the motion prediction/compensation unit 76 compares the cost function values for the inter prediction modes calculated in step S32 , and determines the prediction mode providing the smallest value as the optimal inter prediction mode. The motion prediction/compensation unit 76 then supplies the predicted image generated in the optimum inter prediction mode and its cost function value to the predicted image selection unit 77 .

[帧内预测处理的描述][Description of intra prediction processing]

接下来，将参照图17中的流程图描述图17中的步骤S31中的帧内预测处理。请注意，对于图18中的例子，将描述关于亮度信号的情况，作为一个例子。Next, the intra prediction processing in step S31 in FIG. 17 will be described with reference to the flowchart in FIG. 17 . Note that, for the example in FIG. 18 , the case about a luminance signal will be described as an example.

在上述的图16中的步骤S25中，速率控制单元78供应用于当前块的量化参数。在步骤S41中，量化参数缓冲器92从速率控制单元78获得用于当前块的量化参数并存储该量化参数。In step S25 in FIG. 16 described above, the rate control unit 78 supplies quantization parameters for the current block. In step S41, the quantization parameter buffer 92 obtains the quantization parameter for the current block from the rate control unit 78 and stores the quantization parameter.

在步骤S42中，预测图像产生单元82从4×4像素、8×8像素和16×16像素的帧内预测模式当中选择一种帧内预测模式。将选择的帧内预测模式信息存储在预测模式缓冲器91中。In step S42 , the predicted image generating unit 82 selects an intra prediction mode from among intra prediction modes of 4×4 pixels, 8×8 pixels, and 16×16 pixels. The selected intra prediction mode information is stored in the prediction mode buffer 91 .

低通滤波设置单元93从预测模式缓冲器91读出帧内预测模式信息，并且，从量化参数缓冲器92读出量化参数值。在步骤S43中，低通滤波设置单元93然后从存储于滤波系数存储器94中的针对每一个片段计算出的滤波系数当中设置对应于该帧内预测模式和量化参数的滤波系数。将设置的滤波系数供应给邻近图像设置单元81。The low-pass filter setting unit 93 reads out the intra prediction mode information from the prediction mode buffer 91 , and reads out the quantization parameter value from the quantization parameter buffer 92 . In step S43 , the low-pass filter setting unit 93 then sets a filter coefficient corresponding to the intra prediction mode and quantization parameter from among the filter coefficients calculated for each slice stored in the filter coefficient memory 94 . The set filter coefficients are supplied to the adjacent image setting unit 81 .

在步骤S44中，邻近图像设置单元81使用设置的滤波系数来对当前块的邻近像素值执行滤波处理，并且，将经过滤波处理的邻近像素值供应给预测图像产生单元82。In step S44 , the adjacent image setting unit 81 performs filter processing on the adjacent pixel values of the current block using the set filter coefficients, and supplies the filtered adjacent pixel values to the predicted image generation unit 82 .

在步骤S43中，预测图像产生单元82使用经过滤波处理的邻近像素值以在步骤S42中选择的帧内预测模式对当前块执行帧内预测，并且产生预测图像。In step S43 , the predicted image generating unit 82 performs intra prediction on the current block in the intra prediction mode selected in step S42 using the filter-processed neighboring pixel values, and generates a predicted image.

向最佳预测模式确定单元83供应已经从画面重新排列缓冲器62读出的要进行帧内预测的图像、由预测图像产生单元82产生的预测图像及其帧内预测模式信息。The image to be subjected to intra prediction that has been read out from the screen rearrangement buffer 62 , the predicted image generated by the predicted image generation unit 82 , and intra prediction mode information thereof are supplied to the optimum prediction mode determination unit 83 .

在步骤S46中，最佳预测模式确定单元83使用供应的信息来计算已经产生了预测图像的帧内预测模式的成本函数值。这里，基于高复杂度模式或低复杂度模式的技术之一来执行成本函数值的计算。用作为H.264/AVC格式的参考软件的JM(联合模型)来确定这些模式。In step S46 , the optimal prediction mode determination unit 83 uses the supplied information to calculate the cost function value of the intra prediction mode for which the predicted image has been generated. Here, the calculation of the cost function value is performed based on one of techniques of the high complexity mode or the low complexity mode. These modes are determined with JM (Joint Model), which is reference software for the H.264/AVC format.

具体地说，在高复杂度模式中，暂时地，对所有的候选预测模式执行到编码处理，作为步骤S45中的处理。针对各预测模式计算用下述表达式(73)表示的成本函数值，并且，选择提供其最小值的预测模式作为最佳预测模式。Specifically, in the high-complexity mode, temporarily, up to encoding processing is performed on all candidate prediction modes as the processing in step S45. A cost function value represented by the following expression (73) is calculated for each prediction mode, and the prediction mode providing the minimum value thereof is selected as the optimum prediction mode.

Cost(Mode)＝D+λ·R...(73)Cost(Mode)＝D+λ·R...(73)

D表示原始图像和解码图像之间的差(失真)，R表示包含正交变换系数的产生的代码量，并且，λ表示要被提供作为量化参数QP的函数的拉格朗日乘子。D represents a difference (distortion) between an original image and a decoded image, R represents a generated code amount including orthogonal transform coefficients, and λ represents a Lagrangian multiplier to be provided as a function of the quantization parameter QP.

另一方面，在低复杂度模式中，产生预测图像，并且，针对所有的候选预测模式至多计算运动向量信息、预测模式信息、标志信息等的头比特，作为步骤S45中的处理。针对各预测模式计算用下述表达式(74)表示的成本函数值，并且，选择提供其最小值的预测模式作为最佳预测模式。On the other hand, in the low complexity mode, a predicted image is generated, and at most header bits of motion vector information, prediction mode information, flag information, etc. are calculated for all candidate prediction modes as processing in step S45. A cost function value represented by the following expression (74) is calculated for each prediction mode, and the prediction mode providing the minimum value thereof is selected as the optimum prediction mode.

Cost(Mode)＝D+QPtoQuant(QP)+Header_Bit...(74)Cost(Mode)＝D+QPtoQuant(QP)+Header_Bit...(74)

D表示原始图像和解码图像之间的差(失真)，Header_Bit表示关于预测模式的头比特，并且，QptoQuant是被提供作为量化参数QP的函数的函数。D represents a difference (distortion) between an original image and a decoded image, Header_Bit represents a header bit with respect to a prediction mode, and QptoQuant is a function provided as a function of the quantization parameter QP.

在低复杂度模式中，仅仅针对所有的预测模式产生预测图像，并且，无需执行编码处理和解码处理，从而可以减少计算量。In the low complexity mode, only predicted images are generated for all prediction modes, and encoding processing and decoding processing do not need to be performed, so that the amount of calculation can be reduced.

在步骤S47中，最佳预测模式确定单元83确定针对所有的帧内预测模式是否结束了处理。也就是说，在步骤S47中，确定是否针对4×4像素、8×8像素和16×16像素的所有的帧内预测模式执行了步骤S42至S46的处理。In step S47 , the optimum prediction mode determination unit 83 determines whether the processing has ended for all intra prediction modes. That is, in step S47, it is determined whether the processes of steps S42 to S46 are performed for all intra prediction modes of 4×4 pixels, 8×8 pixels, and 16×16 pixels.

当在步骤S47中确定尚未针对所有的帧内预测模式结束处理的情况中，处理返回到步骤S42，并且，重复后续处理。In a case where it is determined in step S47 that the processing has not ended for all intra prediction modes, the processing returns to step S42, and the subsequent processing is repeated.

当在步骤S47中确定针对所有的帧内预测模式结束了处理的情况中，处理前进到步骤S48。在步骤S48中，最佳预测模式确定单元83将其计算出的成本函数值是最小值的帧内预测模式确定为最佳帧内预测模式。When it is determined in step S47 that the processing has ended for all intra prediction modes, the processing proceeds to step S48. In step S48 , the optimum prediction mode determination unit 83 determines the intra prediction mode whose calculated cost function value is the minimum value as the optimum intra prediction mode.

最佳帧内预测模式的预测图像及其对应成本函数值被供应给预测图像选择单元77。The predicted image of the optimum intra prediction mode and its corresponding cost function value are supplied to the predicted image selection unit 77 .

在预测图像选择单元77选择了在最佳帧内预测模式中产生的预测图像的情况中，最佳预测模式确定单元83将指示最佳帧内预测模式的信息供应给无损耗编码单元66。然后在无损耗编码单元66处对此信息进行编码，并且，将该信息与压缩图像的头信息相加(在上述的图16中的步骤S23)。In a case where the predicted image selection unit 77 has selected a predicted image generated in the optimum intra prediction mode, the optimum prediction mode determination unit 83 supplies information indicating the optimum intra prediction mode to the lossless encoding unit 66 . This information is then encoded at the lossless encoding unit 66, and added to the header information of the compressed image (step S23 in FIG. 16 described above).

请注意，存储在滤波系数存储器94中的通过学习处理计算出的滤波系数也类似地被存储在稍后描述的图22中的图像解码装置151中，从而不必将设置的滤波系数与压缩图像的头信息相加并发送。Note that the filter coefficients calculated by the learning process stored in the filter coefficient memory 94 are also similarly stored in the image decoding device 151 in FIG. The header information is added and sent.

因此，在H.264/AVC的情况中，存在51个量化参数，存在4×4像素和8×8像素的九种帧内预测模式，并且，当考虑这些组合时，需要51×9＝459个庞大的滤波系数。不必将关于这样的庞大的滤波系数的信息发送到解码侧，从而可以在不增加系数信息的开销的情况下实现处理。Therefore, in the case of H.264/AVC, there are 51 quantization parameters, there are nine intra prediction modes of 4×4 pixels and 8×8 pixels, and, when considering these combinations, 51×9=459 a large filter coefficient. It is not necessary to send information on such a huge filter coefficient to the decoding side, so that processing can be realized without increasing the overhead of coefficient information.

[帧间运动预测处理的描述][Description of inter motion prediction processing]

接下来，将参照图19中的流程图描述图17中的步骤S32中的帧间运动预测处理。Next, the inter motion prediction process in step S32 in FIG. 17 will be described with reference to the flowchart in FIG. 19 .

在步骤S61中，运动预测/补偿单元76针对由16×16像素至4×4像素构成的八种帧间预测模式中的每一种确定运动向量和参考图像。也就是说，针对要以每一种帧间预测模式处理的块均确定运动向量和参考图像。In step S61 , the motion prediction/compensation unit 76 determines a motion vector and a reference image for each of eight inter prediction modes composed of 16×16 pixels to 4×4 pixels. That is, a motion vector and a reference image are determined for a block to be processed in each inter prediction mode.

在步骤S62中，基于在步骤S61中针对由16×16像素至4×4像素构成的八种帧间预测模式中的每一种确定的运动向量，运动预测/补偿单元76对参考图像进行运动预测和补偿处理。根据该运动预测和补偿处理，产生每一种帧间预测模式中的预测图像。In step S62, based on the motion vector determined in step S61 for each of the eight inter prediction modes consisting of 16×16 pixels to 4×4 pixels, the motion prediction/compensation unit 76 performs motion on the reference image Prediction and Compensation Processing. According to this motion prediction and compensation processing, a predicted image in each inter prediction mode is generated.

在步骤S63中，运动预测/补偿单元76产生关于针对由16×16像素至4×4像素构成的八种帧间预测模式中的每一种确定的运动向量的运动向量信息，以与压缩图像相加。In step S63, the motion prediction/compensation unit 76 generates motion vector information on the motion vector determined for each of the eight inter prediction modes composed of 16×16 pixels to 4×4 pixels, to be compared with the compressed image add up.

在接下来的步骤S64中计算成本函数值时也使用产生的运动向量信息，并且，在预测图像选择单元77最终选择了相应的预测图像的情况中，将产生的运动向量信息与预测模式信息和参考帧信息一起输出给无损耗编码单元66。The generated motion vector information is also used when calculating the cost function value in the next step S64, and, in the case where the predicted image selection unit 77 finally selects the corresponding predicted image, combines the generated motion vector information with the predicted mode information and The reference frame information is output to the lossless encoding unit 66 together.

在步骤S64中，运动预测/补偿单元76针对由16×16像素至4×4像素构成的八种帧间预测模式中的每一种计算上述的表达式(73)或表达式(74)中示出的成本函数值。当在上述的图17中的步骤S34中确定最佳帧间预测模式时，使用这里计算出的成本函数值。In step S64, the motion prediction/compensation unit 76 calculates the above-mentioned expression (73) or expression (74) for each of the eight kinds of inter prediction modes composed of 16×16 pixels to 4×4 pixels. Cost function values shown. The cost function value calculated here is used when the optimal inter prediction mode is determined in step S34 in FIG. 17 described above.

接下来，作为用于计算最佳滤波系数的第二种方法，将参照图20描述下述情况的例子：其中，执行在线处理，即，针对每一个片段连续地计算最佳滤波系数。Next, as a second method for calculating the optimum filter coefficient, an example of a case where in-line processing is performed, that is, the optimum filter coefficient is continuously calculated for each segment, will be described with reference to FIG. 20 .

现在，在这种情况下，需要将在编码侧针对每一个片段计算出的滤波系数发送到解码侧，并且，发送已经被分解为多种情形的滤波系数导致编码效率的劣化。因此，针对片段只发送一个滤波系数，或者，针对每一块尺寸的每一种预测模式只发送一个滤波系数，或者，针对诸如水平预测、垂直预测等的预测模式类型只发送一个滤波系数。Now, in this case, it is necessary to transmit filter coefficients calculated for each slice on the encoding side to the decoding side, and transmitting filter coefficients that have been decomposed into a plurality of cases leads to deterioration of encoding efficiency. Therefore, only one filter coefficient is sent for a slice, or only one filter coefficient is sent for each prediction mode per block size, or only one filter coefficient is sent for a prediction mode type such as horizontal prediction, vertical prediction, etc.

此外，在上述的离线处理的情况中，对使用帧内预测模式和量化参数作为用于计算滤波系数的参数的例子进行描述。另一方面，在在线处理的情况中，用于计算滤波系数的大量的参数增加了处理量，从而，将用图20描述关于对于参数只使用帧内预测模式的例子的例子。虽然将省略描述，但是，当然，可以只使用量化参数，或者，可以使用两种参数。Also, in the case of the above-described offline processing, description will be given of an example using the intra prediction mode and the quantization parameter as parameters for calculating the filter coefficient. On the other hand, in the case of online processing, a large number of parameters for calculating filter coefficients increases the amount of processing, and thus, an example about an example using only intra prediction mode for parameters will be described using FIG. 20 . Although description will be omitted, of course, only quantization parameters may be used, or two kinds of parameters may be used.

[帧内预测单元和邻近像素内插滤波切换单元的其它配置例子][Other Configuration Examples of Intra Prediction Unit and Neighboring Pixel Interpolation Filtering Switching Unit]

图20是图示在下述情况中的帧内预测单元74和邻近像素内插滤波切换单元75的另一配置例子的框图：针对每一个片段执行在线处理，以连续地计算最佳滤波系数。20 is a block diagram illustrating another configuration example of the intra prediction unit 74 and the adjacent pixel interpolation filter switching unit 75 in a case where online processing is performed for each slice to continuously calculate optimal filter coefficients.

在图20中的例子的情况中，在帧内预测单元74和邻近像素内插滤波切换单元75之间引入开关101，并且，与图14所示的情况不同，通过接通和断开开关101，帧内预测单元74执行帧内预测两次。也就是说，对于帧内预测单元74，在开关101断开的状态中执行在H.264/AVC中定义的帧内预测，并且，计算适合于帧内预测的滤波系数。在开关101接通的状态中，用计算出的滤波系数当中的由邻近像素内插滤波切换单元75设置的滤波系数执行帧内预测。In the case of the example in FIG. 20, a switch 101 is introduced between the intra prediction unit 74 and the adjacent pixel interpolation filter switching unit 75, and, unlike the case shown in FIG. 14, by turning on and off the switch 101 , the intra prediction unit 74 performs intra prediction twice. That is, with the intra prediction unit 74 , intra prediction defined in H.264/AVC is performed in a state where the switch 101 is off, and a filter coefficient suitable for intra prediction is calculated. In a state where the switch 101 is turned on, intra prediction is performed with the filter coefficient set by the adjacent pixel interpolation filter switching unit 75 among the calculated filter coefficients.

图20中的帧内预测单元74由邻近图像设置单元111、预测图像产生单元112和最佳预测模式确定单元113构成。The intra prediction unit 74 in FIG. 20 is composed of an adjacent image setting unit 111 , a predicted image generation unit 112 , and an optimum prediction mode determination unit 113 .

邻近像素内插滤波切换单元75由预测模式缓冲器121、最佳滤波计算单元122和低通滤波设置单元123构成。The adjacent pixel interpolation filter switching unit 75 is composed of a prediction mode buffer 121 , an optimum filter calculation unit 122 , and a low-pass filter setting unit 123 .

从帧存储器72向邻近像素设置单元111供应用于帧内预测的当前片段的所有当前块的邻近像素值。在图20的情况中，也省略了开关73的图示。请注意，在帧内预测的情况中，未经过去块滤波器71的去块滤波的像素值用作邻近像素值。The adjacent pixel values of all current blocks of the current segment used for intra prediction are supplied from the frame memory 72 to the adjacent pixel setting unit 111 . Also in the case of FIG. 20 , illustration of the switch 73 is omitted. Note that in the case of intra prediction, pixel values that have not been deblock-filtered by the deblock filter 71 are used as adjacent pixel values.

在开关101处于断开状态的情况中，邻近像素设置单元111使用仅仅用于在H.264/AVC中定义的帧内预测模式的滤波系数来对来自帧存储器72的当前块的邻近像素值进行滤波处理，并且将其供应给预测图像产生单元112。也就是说，仅仅在上文的用表达式(14)至表达式(24)描述的8×8预测模式的情况中，经过滤波处理的邻近像素值被供应给预测图像产生单元112。在所有的其它情况中，来自帧存储器72的当前块的邻近像素值原样地被供应给预测图像产生单元112。In the case where the switch 101 is in the OFF state, the neighboring pixel setting unit 111 performs the neighboring pixel values of the current block from the frame memory 72 using only filter coefficients for the intra prediction mode defined in H.264/AVC. filter processing, and supply it to the predicted image generation unit 112 . That is to say, only in the case of the 8×8 prediction mode described with Expression (14) to Expression (24) above, the adjacent pixel values subjected to filter processing are supplied to the predicted image generating unit 112 . In all other cases, the neighboring pixel values of the current block from the frame memory 72 are supplied to the predicted image generation unit 112 as they are.

在开关101处于接通(on)状态的情况中，从低通滤波设置单元123向邻近像素设置单元111供应滤波系数。因此，邻近像素设置单元111使用由低通滤波设置单元123设置的滤波系数来对来自帧存储器72的当前块的邻近像素值进行滤波处理，并且将经过滤波处理的邻近像素值供应给预测图像产生单元112。In a case where the switch 101 is in an on state, the filter coefficient is supplied from the low-pass filter setting unit 123 to the adjacent pixel setting unit 111 . Accordingly, the adjacent pixel setting unit 111 performs filter processing on the adjacent pixel values of the current block from the frame memory 72 using the filter coefficient set by the low-pass filter setting unit 123, and supplies the filtered adjacent pixel values to the predicted image generation Unit 112.

预测图像产生单元112使用来自邻近像素设置单元111的邻近像素值来以所有的帧内预测模式对当前块执行帧内预测，并且产生预测图像。将产生的预测图像与帧内预测模式信息一起供应给最佳预测模式确定单元113。The predicted image generating unit 112 uses the adjacent pixel values from the adjacent pixel setting unit 111 to perform intra prediction on the current block in all intra prediction modes, and generates a predicted image. The generated predicted image is supplied to the optimum prediction mode determination unit 113 together with the intra prediction mode information.

向最佳预测模式确定单元113供应已经从画面重新排列缓冲器62读出的用于帧内预测的图像、由预测图像产生单元112产生的预测图像及其帧内预测模式信息。The image for intra prediction that has been read out from the screen rearrangement buffer 62 , the predicted image generated by the predicted image generation unit 112 , and intra prediction mode information thereof are supplied to the optimum prediction mode determination unit 113 .

最佳预测模式确定单元113使用供应的信息来计算已经产生预测图像的帧内预测模式的成本函数值，并且将产生计算出的成本函数值中的最小值的帧内预测模式决定为最佳帧内预测模式。The optimum prediction mode determination unit 113 calculates the cost function value of the intra prediction mode that has produced the predicted image using the supplied information, and decides the intra prediction mode that produces the smallest value among the calculated cost function values as the optimum frame Intra-prediction mode.

在开关101处于断开(off)状态的情况中，最佳预测模式确定单元113将最佳帧内预测模式的信息供应给预测模式缓冲器121。在开关101处于接通状态的情况中，最佳预测模式确定单元113将最佳帧内预测模式的预测图像和相应的成本函数值供应给预测图像选择单元77。In a case where the switch 101 is in an off state, the optimum prediction mode determination unit 113 supplies information of the optimum intra prediction mode to the prediction mode buffer 121 . In a case where the switch 101 is in the ON state, the optimum prediction mode determination unit 113 supplies the prediction image of the optimum intra prediction mode and the corresponding cost function value to the prediction image selection unit 77 .

此外，在预测图像选择单元77选择了在最佳帧内预测模式中产生的预测图像的情况中，最佳预测模式确定单元113将指示最佳帧内预测模式的信息供应给无损耗编码单元66。Furthermore, in a case where the predicted image selection unit 77 has selected a predicted image generated in the optimum intra prediction mode, the optimum prediction mode determination unit 113 supplies information indicating the optimum intra prediction mode to the lossless encoding unit 66 .

预测模式缓冲器121存储来自最佳预测模式确定单元113的帧内预测模式信息。The prediction mode buffer 121 stores intra prediction mode information from the optimum prediction mode determination unit 113 .

向最佳滤波计算单元122供应已经从画面重新排列缓冲器62读出的帧内预测的图像和来自帧存储器72的当前块的邻近像素值。最佳滤波计算单元122从预测模式缓冲器121读出当前片段中包含的每一块的帧内预测模式。最佳滤波计算单元122然后使用该信息来计算当前片段的帧内预测模式的最佳滤波系数，如上文中参照图15所述，并且，最佳滤波计算单元122将计算出的滤波系数供应给低通滤波设置单元123。The intra-predicted image that has been read out from the screen rearrangement buffer 62 and the adjacent pixel values of the current block from the frame memory 72 are supplied to the optimal filter calculation unit 122 . The optimal filter calculation unit 122 reads out the intra prediction mode of each block contained in the current slice from the prediction mode buffer 121 . The optimum filter calculation unit 122 then uses this information to calculate the optimum filter coefficient for the intra prediction mode of the current slice, as described above with reference to FIG. 15 , and supplies the calculated filter coefficient to the low pass filter setting unit 123.

低通滤波设置单元123从已经计算出的当前片段的滤波系数当中设置用于当前块的滤波系数，接通开关101的端子，并且将设置的滤波系数供应给邻近像素设置单元111。此外，低通滤波设置单元123将用于当前片段的滤波系数供应给无损耗编码单元66。The low-pass filter setting unit 123 sets filter coefficients for the current block from among the filter coefficients of the current slice that have been calculated, turns on the terminal of the switch 101 , and supplies the set filter coefficients to the adjacent pixel setting unit 111 . Also, the low-pass filter setting unit 123 supplies the filter coefficients for the current slice to the lossless encoding unit 66 .

[帧内预测处理的其它描述][Other description of intra prediction processing]

接下来，将参照图21中的流程图描述图20中的邻近像素内插滤波切换单元75和帧内预测单元74执行的帧内预测处理。请注意，该帧内预测处理是图17中的步骤S31的帧内预测处理的另一例子。Next, intra prediction processing performed by the adjacent pixel interpolation filter switching unit 75 and the intra prediction unit 74 in FIG. 20 will be described with reference to the flowchart in FIG. 21 . Note that this intra prediction processing is another example of the intra prediction processing of step S31 in FIG. 17 .

首先，开关101处于断开状态。从帧存储器72向邻近像素设置单元111供应要进行帧内预测的当前片段的所有当前块的邻近像素值。邻近像素设置单元111使用仅仅用于在H.264/AVC中定义的8×8像素帧内预测模式的滤波系数来对来自帧存储器72的当前块的邻近像素值执行滤波处理，并且将其供应给预测图像产生单元112。也就是说，在其它的帧内预测模式的情况中，来自帧存储器72的当前块的邻近像素值原样地被供应给预测图像产生单元112。First, the switch 101 is in an off state. The adjacent pixel values of all the current blocks of the current segment to be intra-predicted are supplied from the frame memory 72 to the adjacent pixel setting unit 111 . The adjacent pixel setting unit 111 performs filter processing on the adjacent pixel values of the current block from the frame memory 72 using only filter coefficients for the 8×8-pixel intra prediction mode defined in H.264/AVC, and supplies to the predictive image generating unit 112. That is, in the case of other intra prediction modes, the adjacent pixel values of the current block from the frame memory 72 are supplied to the predicted image generation unit 112 as they are.

在步骤S101中，预测图像产生单元112对当前片段中包含的所有块执行帧内预测处理。也就是说，预测图像产生单元112使用来自邻近像素设置单元111的当前块的邻近像素值来以每一种帧内预测模式执行帧内预测，并且产生预测图像。In step S101 , the predicted image generating unit 112 performs intra prediction processing on all blocks included in the current slice. That is, the predicted image generating unit 112 performs intra prediction in each intra prediction mode using the adjacent pixel values of the current block from the adjacent pixel setting unit 111 , and generates a predicted image.

向最佳预测模式确定单元113供应已经从画面重新排列缓冲器62读出的要进行帧内预测的图像、由预测图像产生单元112产生的预测图像及其帧内预测模式信息。The image to be subjected to intra prediction that has been read out from the screen rearrangement buffer 62 , the predicted image generated by the predicted image generation unit 112 , and intra prediction mode information thereof are supplied to the optimum prediction mode determination unit 113 .

在步骤S102中，最佳预测模式确定单元113使用供应的信息来计算关于产生了预测图像的所有的帧内预测模式的上述的表达式(73)或表达式(74)中的成本函数值。In step S102 , the optimum prediction mode determination unit 113 uses the supplied information to calculate the cost function values in the above-described Expression (73) or Expression (74) with respect to all the intra prediction modes that generated the predicted image.

在步骤S103中，最佳预测模式确定单元113将表达式(73)或表达式(74)中的成本函数为最小的帧内预测模式确定为最佳帧内预测模式，并且将确定的帧内预测模式的信息供应给预测模式缓冲器121。In step S103, the optimal prediction mode determining unit 113 determines the intra prediction mode whose cost function in expression (73) or expression (74) is the smallest as the optimal intra prediction mode, and sets the determined intra prediction mode Information of the prediction mode is supplied to the prediction mode buffer 121 .

向最佳滤波计算单元122供应已经从画面重新排列缓冲器62读出的要进行帧内预测的图像和来自帧存储器72的当前块的邻近像素值。最佳滤波计算单元122从预测模式缓冲器121读出当前片段中包含的每一块的帧内预测模式。The image to be intra-predicted, which has been read out from the screen rearrangement buffer 62 , and the adjacent pixel values of the current block from the frame memory 72 are supplied to the optimum filter calculation unit 122 . The optimal filter calculation unit 122 reads out the intra prediction mode of each block contained in the current slice from the prediction mode buffer 121 .

在步骤S104中，最佳滤波计算单元122使用该信息来计算使整个当前片段的残差最小的滤波系数，作为当前片段的每一种帧内预测模式的最佳滤波系数。将上文中参照图15描述的滤波系数供应给低通滤波设置单元123。In step S104, the optimal filter calculation unit 122 uses the information to calculate the filter coefficient that minimizes the residual error of the entire current segment as the optimal filter coefficient for each intra prediction mode of the current segment. The filter coefficients described above with reference to FIG. 15 are supplied to the low-pass filter setting unit 123 .

低通滤波设置单元123从计算出的当前片段的滤波系数当中设置对应于当前块的滤波系数，接通开关101的端子，并且将设置的滤波系数供应给邻近像素设置单元111。The low-pass filter setting unit 123 sets a filter coefficient corresponding to the current block from among the calculated filter coefficients of the current segment, turns on the terminal of the switch 101 , and supplies the set filter coefficient to the adjacent pixel setting unit 111 .

在步骤S105中，邻近像素设置单元111使用由低通滤波设置单元123设置的滤波系数来对来自帧存储器72的当前块的邻近像素值执行滤波处理。In step S105 , the adjacent pixel setting unit 111 performs filter processing on the adjacent pixel values of the current block from the frame memory 72 using the filter coefficient set by the low-pass filter setting unit 123 .

将经过滤波处理的邻近像素值供应给预测图像产生单元112。在步骤S106中，预测图像产生单元112再次使用经过滤波处理的邻近像素值来对当前片段中包含的所有块执行帧内预测，从而产生预测图像。将产生的预测图像与帧内预测模式信息一起供应给最佳预测模式确定单元113。The filter-processed adjacent pixel values are supplied to the predicted image generation unit 112 . In step S106 , the predicted image generating unit 112 again uses the filtered adjacent pixel values to perform intra prediction on all blocks included in the current slice, thereby generating a predicted image. The generated predicted image is supplied to the optimum prediction mode determination unit 113 together with the intra prediction mode information.

在开关101处于接通状态的情况中，最佳预测模式确定单元113将最佳帧内预测模式的预测图像和相应的成本函数值供应给预测图像选择单元77。In a case where the switch 101 is in the ON state, the optimum prediction mode determination unit 113 supplies the prediction image of the optimum intra prediction mode and the corresponding cost function value to the prediction image selection unit 77 .

在上述的图16的步骤S22中，预测图像选择单元77将最佳帧内预测模式和最佳帧间预测模式之一确定为最佳预测模式，并且供应预测图像的选择信息。In step S22 of FIG. 16 described above, the predicted image selection unit 77 determines one of the optimal intra prediction mode and the optimal inter prediction mode as the optimal prediction mode, and supplies selection information of the predicted image.

在步骤S107中，最佳预测模式确定单元113根据预测图像的选择信息确定是否选择了最佳帧内预测模式的预测图像。当在步骤S107中确定选择了最佳帧内预测模式的预测图像的情况中，处理前进到步骤S108。In step S107 , the optimal prediction mode determining unit 113 determines whether the predicted image of the optimal intra prediction mode is selected according to the selection information of the predicted image. When it is determined in step S107 that the predicted image of the optimal intra prediction mode is selected, the process proceeds to step S108.

在步骤S108中，最佳预测模式确定单元113将帧内预测模式信息供应给无损耗编码单元66。请注意，在针对当前片段尚未供应滤波系数的情况中，来自最佳滤波计算单元122的滤波系数也被供应给无损耗编码单元66。In step S108 , the optimum prediction mode determination unit 113 supplies the intra prediction mode information to the lossless encoding unit 66 . Note that the filter coefficients from the optimal filter calculation unit 122 are also supplied to the lossless encoding unit 66 in the case where filter coefficients have not been supplied for the current segment.

当在步骤S107中确定尚未选择最佳帧内预测模式的预测图像的情况中，帧内预测处理结束。In the case where it is determined in step S107 that the predicted image of the optimum intra prediction mode has not been selected, the intra prediction process ends.

请注意，可以通过重复上述的步骤S104至S106的处理来获得进一步优化的滤波系数。Please note that further optimized filter coefficients can be obtained by repeating the above steps S104 to S106.

编码的压缩图像经由预定的传送路径来传送，并且由图像解码装置解码。The encoded compressed image is transmitted via a predetermined transmission path, and decoded by an image decoding device.

[图像解码装置的配置例子][Configuration Example of Image Decoding Device]

图22表示用作应用了本发明的图像处理装置的图像解码装置的实施例的配置。FIG. 22 shows the configuration of an embodiment of an image decoding device serving as an image processing device to which the present invention is applied.

图像解码装置151由存储缓冲器161、无损耗解码单元162、逆量化单元163、逆正交变换单元164、计算单元165、去块滤波器166、画面重新排列缓冲器167、D/A转换单元168、帧存储器169、开关170、帧内预测单元171、邻近像素内插滤波切换单元172、运动预测/补偿单元173和开关174构成。The image decoding device 151 is composed of a storage buffer 161, a lossless decoding unit 162, an inverse quantization unit 163, an inverse orthogonal transformation unit 164, a calculation unit 165, a deblocking filter 166, a screen rearrangement buffer 167, and a D/A conversion unit. 168 , a frame memory 169 , a switch 170 , an intra prediction unit 171 , an adjacent pixel interpolation filtering switching unit 172 , a motion prediction/compensation unit 173 and a switch 174 .

存储缓冲器161存储传送的压缩图像。无损耗解码单元162使用与图1中的无损耗编码单元66的编码格式对应的格式来对从存储缓冲器161供应且由该无损耗编码单元66编码的信息进行解码。逆量化单元163使用与图1中的量化单元65的量化格式对应的格式对由无损耗解码单元162解码的图像进行逆量化。逆正交变换单元164使用与图1中的正交变换单元64的正交变换格式对应的格式来对逆量化单元163的输出进行逆正交变换。The storage buffer 161 stores the transmitted compressed image. The lossless decoding unit 162 decodes information supplied from the storage buffer 161 and encoded by the lossless encoding unit 66 using a format corresponding to the encoding format of the lossless encoding unit 66 in FIG. 1 . The inverse quantization unit 163 dequantizes the image decoded by the lossless decoding unit 162 using a format corresponding to the quantization format of the quantization unit 65 in FIG. 1 . The inverse orthogonal transform unit 164 performs inverse orthogonal transform on the output of the inverse quantization unit 163 using a format corresponding to the orthogonal transform format of the orthogonal transform unit 64 in FIG. 1 .

通过计算单元165将经过逆正交变换的输出与从开关174供应的预测图像相加来对该输出进行解码。去块滤波器166去除解码图像的块失真，然后，将其供应给帧存储器169以便存储，并且还将其输出给画面重新排列缓冲器167。The output subjected to the inverse orthogonal transformation is decoded by the calculation unit 165 adding the predicted image supplied from the switch 174 . The deblocking filter 166 removes block distortion of the decoded image, then supplies it to the frame memory 169 for storage, and also outputs it to the screen rearrangement buffer 167 .

画面重新排列缓冲器167对图像执行重新排列。具体地说，将由图1中的画面重新排列缓冲器62对编码序列重新排列的帧的序列重新排列为原始的显示序列。D/A转换单元168对从画面重新排列缓冲器167供应的图像进行数值模拟转换，并且将其输出给未示出的显示器以便显示。The screen rearrangement buffer 167 performs rearrangement of images. Specifically, the sequence of frames rearranged by the frame rearrangement buffer 62 in FIG. 1 to the coded sequence is rearranged into the original display sequence. The D/A conversion unit 168 performs numerical-analog conversion on the image supplied from the screen rearrangement buffer 167, and outputs it to an unillustrated display for display.

开关170从帧存储器169读出要进行帧间处理的图像和要参考的图像，将其输出给运动预测/补偿单元173，还从帧存储器169读出要用于帧内预测的图像并将其供应给帧内预测单元171。The switch 170 reads out an image to be inter-frame processed and an image to be referred to from the frame memory 169, and outputs it to the motion prediction/compensation unit 173, and also reads out an image to be used for intra-frame prediction from the frame memory 169 and converts it to It is supplied to the intra prediction unit 171.

将通过对头信息进行解码而获得的指示帧内预测模式的信息从无损耗解码单元162供应给帧内预测单元171。帧内预测单元171基于该信息通过使用由邻近像素内插滤波切换单元172设置的滤波系数对邻近像素值执行滤波处理和帧内预测来产生预测图像，并且将产生的预测图像输出到开关174。Information indicating the intra prediction mode obtained by decoding the header information is supplied from the lossless decoding unit 162 to the intra prediction unit 171 . The intra prediction unit 171 generates a predicted image based on this information by performing filter processing and intra prediction on adjacent pixel values using the filter coefficient set by the adjacent pixel interpolation filter switching unit 172 , and outputs the generated predicted image to the switch 174 .

从无损耗解码单元162向邻近像素内插滤波切换单元172供应通过根据在图像编码装置51处的编码对头信息进行解码而获得的指示帧内预测模式的信息和量化参数的信息中的至少一个。以与图1中的邻近像素内插滤波切换单元75相同的方式，邻近像素内插滤波切换单元172存储通过在稍后描述的图28中的学习装置251处学习而获得的与量化参数和帧内预测模式中的至少一个对应的滤波系数。At least one of information indicating an intra prediction mode and information of a quantization parameter obtained by decoding header information according to encoding at the image encoding device 51 is supplied from the lossless decoding unit 162 to the adjacent pixel interpolation filtering switching unit 172 . In the same manner as the adjacent pixel interpolation filter switching unit 75 in FIG. 1, the adjacent pixel interpolation filter switching unit 172 stores the quantization parameter and frame information obtained by learning at the learning device 251 in FIG. 28 described later. Filter coefficients corresponding to at least one of the intra prediction modes.

邻近像素内插滤波切换单元172设置与来自无损耗解码单元162的量化参数和帧内预测模式中的至少一个对应的滤波系数。每一个片段邻近像素内插滤波切换单元172将设置的滤波系数供应给帧内预测单元74。The adjacent pixel interpolation filter switching unit 172 sets a filter coefficient corresponding to at least one of the quantization parameter and the intra prediction mode from the lossless decoding unit 162 . The per-slice adjacent pixel interpolation filter switching unit 172 supplies the set filter coefficients to the intra prediction unit 74 .

请注意，对于邻近像素内插滤波切换单元172，存储预先离线学习的滤波系数。但是，请注意，在用图1中的邻近像素内插滤波切换单元75在线计算滤波系数的情况中，例如，针对每一个片段向其传送这些滤波系数。在这种情况下，邻近像素内插滤波切换单元172使用由无损耗解码单元162解码的滤波系数。Note that for the adjacent pixel interpolation filter switching unit 172, filter coefficients learned offline in advance are stored. Note, however, that in the case of calculating filter coefficients online with the adjacent pixel interpolation filter switching unit 75 in FIG. 1 , for example, these filter coefficients are transmitted thereto for each slice. In this case, the adjacent pixel interpolation filter switching unit 172 uses the filter coefficient decoded by the lossless decoding unit 162 .

将通过对头信息进行解码而获得的信息(预测模式信息、运动向量信息和参考帧信息)从无损耗解码单元162供应给运动预测/补偿单元173。在供应了指示帧间预测模式的信息的情况中，运动预测/补偿单元173基于运动向量信息和参考帧信息对图像进行运动预测和补偿处理，以产生预测图像。运动预测/补偿单元173将在帧间预测模式中产生的预测图像输出到开关174。Information (prediction mode information, motion vector information, and reference frame information) obtained by decoding header information is supplied from the lossless decoding unit 162 to the motion prediction/compensation unit 173 . In a case where information indicating an inter prediction mode is supplied, the motion prediction/compensation unit 173 performs motion prediction and compensation processing on an image based on motion vector information and reference frame information to generate a predicted image. The motion prediction/compensation unit 173 outputs the predicted image generated in the inter prediction mode to the switch 174 .

开关174选择由运动预测/补偿单元173或帧内预测单元171产生的预测图像并将其供应给计算单元165。The switch 174 selects the predicted image generated by the motion prediction/compensation unit 173 or the intra prediction unit 171 and supplies it to the calculation unit 165 .

请注意，采用图1中的图像编码装置51，为了基于成本函数的预测模式确定，针对所有的帧内预测模式执行帧内预测处理。另一方面，采用图像解码装置151，仅仅基于编码的向其发送的帧内预测模式的信息，执行帧内预测处理。Note that with the image encoding device 51 in FIG. 1 , for prediction mode determination based on a cost function, intra prediction processing is performed for all intra prediction modes. On the other hand, with the image decoding device 151 , intra prediction processing is performed based only on the encoded information of the intra prediction mode sent thereto.

[帧内预测单元和邻近像素内插滤波切换单元的配置例子][Configuration Example of Intra Prediction Unit and Adjacent Pixel Interpolation Filtering Switching Unit]

图23是图示邻近像素内插滤波切换单元和帧内预测单元的详细配置例子的框图。请注意，图23中的功能块对应于在采用图14中所示的图像编码装置51的离线处理的情况中的功能块。Fig. 23 is a block diagram illustrating a detailed configuration example of a neighboring pixel interpolation filtering switching unit and an intra prediction unit. Note that the functional blocks in FIG. 23 correspond to functional blocks in the case of offline processing employing the image encoding device 51 shown in FIG. 14 .

在图23的例子的情况中，帧内预测单元71由预测图像产生单元181和邻近像素设置单元182构成。邻近像素内插滤波切换单元172由预测模式缓冲器191、量化参数缓冲器192和低通滤波设置单元193构成。低通滤波设置单元193具有内置的滤波系数存储器194。In the case of the example of FIG. 23 , the intra prediction unit 71 is composed of a predicted image generation unit 181 and an adjacent pixel setting unit 182 . The adjacent pixel interpolation filter switching unit 172 is composed of a prediction mode buffer 191 , a quantization parameter buffer 192 , and a low-pass filter setting unit 193 . The low-pass filter setting unit 193 has a built-in filter coefficient memory 194 .

向预测图像产生单元181供应来自无损耗解码单元162的帧内预测模式信息和来自邻近像素设置单元182的经过滤波处理的邻近像素值。预测图像产生单元181使用向其供应的邻近像素值来以来自无损耗解码单元162的帧内预测模式执行帧内预测，产生预测图像，并且将产生的预测图像供应给开关174。The intra prediction mode information from the lossless decoding unit 162 and the filter-processed adjacent pixel values from the adjacent pixel setting unit 182 are supplied to the predicted image generation unit 181 . The predicted image generation unit 181 performs intra prediction in the intra prediction mode from the lossless decoding unit 162 using the neighboring pixel values supplied thereto, generates a predicted image, and supplies the generated predicted image to the switch 174 .

从帧存储器169向邻近像素设置单元182供应要进行帧内预测的当前块的邻近像素值。在图23的情况中，省略了开关170的图示，但是，实际上，将邻近像素值经由开关170从帧存储器169供应给邻近像素设置单元182。The adjacent pixel values of the current block to be intra-predicted are supplied from the frame memory 169 to the adjacent pixel setting unit 182 . In the case of FIG. 23 , illustration of the switch 170 is omitted, but, actually, adjacent pixel values are supplied from the frame memory 169 to the adjacent pixel setting unit 182 via the switch 170 .

邻近像素设置单元182使用由低通滤波设置单元193设置的滤波系数来对来自帧存储器169的当前块的邻近像素值执行滤波处理，并且将经过了滤波处理的邻近像素值供应给预测图像产生单元181。The adjacent pixel setting unit 182 performs filter processing on the adjacent pixel values of the current block from the frame memory 169 using the filter coefficient set by the low-pass filter setting unit 193, and supplies the filtered adjacent pixel values to the predicted image generation unit 181.

预测模式缓冲器191存储来自无损耗解码单元162的帧内预测模式信息。量化参数缓冲器192存储来自无损耗解码单元162的量化参数。The prediction mode buffer 191 stores intra prediction mode information from the lossless decoding unit 162 . The quantization parameter buffer 192 stores quantization parameters from the lossless decoding unit 162 .

低通滤波设置单元193从预测模式缓冲器191读出当前块的帧内预测模式信息，并且，从量化参数缓冲器192读出与当前块对应的量化参数。低通滤波设置单元193从存储于内置滤波系数存储器194中的滤波系数设置与该信息对应的滤波系数，并且，将设置的滤波系数供应给邻近像素设置单元182。The low-pass filter setting unit 193 reads out the intra prediction mode information of the current block from the prediction mode buffer 191 , and reads out the quantization parameter corresponding to the current block from the quantization parameter buffer 192 . The low-pass filter setting unit 193 sets a filter coefficient corresponding to this information from the filter coefficient stored in the built-in filter coefficient memory 194 , and supplies the set filter coefficient to the adjacent pixel setting unit 182 .

以与图14中的滤波系数存储器94相同的方式，滤波系数存储器194存储与通过在稍后描述的图28中的学习装置处学习而获得的帧内预测模式和量化参数对应的滤波系数。In the same manner as filter coefficient memory 94 in FIG. 14 , filter coefficient memory 194 stores filter coefficients corresponding to intra prediction modes and quantization parameters obtained by learning at a learning device in FIG. 28 described later.

例如，针对每一个片段，计算和存储滤波系数，如上文中参照图15所述。请注意，对于滤波系数存储器194，也根据处理器的寄存器长度将滤波系数保持为n比特值(其中，n是整数)。For example, for each segment, filter coefficients are calculated and stored as described above with reference to FIG. 15 . Note that for the filter coefficient memory 194, the filter coefficients are also maintained as n-bit values (where n is an integer) according to the processor's register length.

[图像解码装置的解码处理的描述][Description of Decoding Processing of Image Decoding Device]

接下来，将参照图24的流程图描述图像解码装置151执行的解码处理。Next, decoding processing performed by the image decoding device 151 will be described with reference to the flowchart of FIG. 24 .

在步骤S131中，存储缓冲器161存储传送的图像。在步骤S132中，无损耗解码单元162对从存储缓冲器161供应的压缩图像进行解码。具体地说，对由图1中的无损耗编码单元66编码的I图片、P图片和B图片进行解码。In step S131, the storage buffer 161 stores the transferred image. In step S132 , the lossless decoding unit 162 decodes the compressed image supplied from the storage buffer 161 . Specifically, I pictures, P pictures, and B pictures encoded by the lossless encoding unit 66 in FIG. 1 are decoded.

此时，也对运动向量信息、参考帧信息、预测模式信息(指示帧内预测模式或帧间预测模式的信息)、量化参数信息、标志信息等进行解码。At this time, motion vector information, reference frame information, prediction mode information (information indicating intra prediction mode or inter prediction mode), quantization parameter information, flag information, and the like are also decoded.

具体地说，在预测模式信息是帧内预测模式信息的情况中，将预测模式信息供应给帧内预测单元171和邻近像素内插滤波切换单元172。此外，在已经对量化参数信息进行解码的情况中，也将其供应给邻近像素内插滤波切换单元172。在预测模式信息是帧间预测模式信息的情况中，将对应于预测模式信息的参考帧信息和运动向量信息供应给运动预测/补偿单元173。Specifically, in the case where the prediction mode information is intra prediction mode information, the prediction mode information is supplied to the intra prediction unit 171 and the adjacent pixel interpolation filtering switching unit 172 . Furthermore, in the case where the quantization parameter information has already been decoded, it is also supplied to the adjacent pixel interpolation filter switching unit 172 . In a case where the prediction mode information is inter prediction mode information, reference frame information and motion vector information corresponding to the prediction mode information are supplied to the motion prediction/compensation unit 173 .

在步骤S133中，逆量化单元163使用与图1中的量化单元65的特性对应的特性来对由无损耗解码单元162解码的变换系数进行逆量化。在步骤S134中，逆正交变换单元164使用与图1中的正交变换单元64的特性对应的特性来对由逆量化单元163逆量化的变换系数进行逆正交变换。这意味着，已经对与图1中的正交变换单元64的输入(计算单元63的输出)对应的差分信息进行解码。In step S133 , the inverse quantization unit 163 dequantizes the transform coefficient decoded by the lossless decoding unit 162 using characteristics corresponding to those of the quantization unit 65 in FIG. 1 . In step S134 , the inverse orthogonal transform unit 164 inversely orthogonally transforms the transform coefficients inversely quantized by the inverse quantization unit 163 using characteristics corresponding to those of the orthogonal transform unit 64 in FIG. 1 . This means that difference information corresponding to the input of the orthogonal transform unit 64 (the output of the calculation unit 63 ) in FIG. 1 has been decoded.

在步骤S135中，计算单元165将经由开关174输入且在稍后描述的步骤S141中的处理中选择的预测图像与该差分信息相加。因此，对原始图像进行解码。在步骤S136中，去块滤波器166对从计算单元165输出的图像进行滤波。因此，去除了块失真。在步骤S137中，帧存储器169存储经过滤波的图像。In step S135 , the calculation unit 165 adds the predicted image input via the switch 174 and selected in the processing in step S141 described later to the difference information. Therefore, the original image is decoded. In step S136 , the deblocking filter 166 filters the image output from the calculation unit 165 . Therefore, block distortion is removed. In step S137, the frame memory 169 stores the filtered image.

在步骤S138中，帧内预测单元171和运动预测/补偿单元173响应于从无损耗解码单元162供应的预测模式信息来执行相应的图像预测处理。In step S138 , the intra prediction unit 171 and the motion prediction/compensation unit 173 perform corresponding image prediction processing in response to the prediction mode information supplied from the lossless decoding unit 162 .

具体地说，在已经从无损耗解码单元162供应帧内预测模式信息的情况中，帧内预测单元171以帧内预测模式执行帧内预测处理。此时，帧内预测单元171使用由邻近像素内插滤波切换单元172设置的滤波系数来对邻近像素执行滤波处理和帧内预测处理。Specifically, in a case where intra prediction mode information has been supplied from the lossless decoding unit 162 , the intra prediction unit 171 performs intra prediction processing in the intra prediction mode. At this time, the intra prediction unit 171 performs filter processing and intra prediction processing on adjacent pixels using the filter coefficient set by the adjacent pixel interpolation filter switching unit 172 .

稍后将参照图25描述步骤S138中的预测处理的细节，但是，根据该处理，将由帧内预测单元171产生的预测图像或者由运动预测/补偿单元173产生的预测图像供应给开关174。Details of the prediction processing in step S138 will be described later with reference to FIG. 25 , but according to this processing, the predicted image generated by the intra prediction unit 171 or the predicted image generated by the motion prediction/compensation unit 173 is supplied to the switch 174 .

在步骤S139中，开关174选择预测图像。具体地说，供应由帧内预测单元171产生的预测图像或者由运动预测/补偿单元173产生的预测图像。因此，供应的预测图像被选择，被供应给计算单元165，并且，在步骤S134中，如上所述，将该预测图像与逆正交变换单元164的输出相加。In step S139, the switch 174 selects a predicted image. Specifically, a predicted image generated by the intra prediction unit 171 or a predicted image generated by the motion prediction/compensation unit 173 is supplied. Accordingly, the supplied predicted image is selected, supplied to the calculation unit 165 , and, in step S134 , the predicted image is added to the output of the inverse orthogonal transform unit 164 as described above.

在步骤S140中，画面重新排列缓冲器167执行重新排列。具体地说，将由图像编码装置51的画面重新排列缓冲器62为编码而重新排列的帧的序列重新排列为原始的显示序列。In step S140, the screen rearrangement buffer 167 performs rearrangement. Specifically, the sequence of frames rearranged for encoding by the screen rearrangement buffer 62 of the image encoding device 51 is rearranged into an original display sequence.

在步骤S141中，D/A转换单元168对来自画面重新排列缓冲器167的图像进行数字模拟转换。将该图像输出给未示出的显示器，并且，显示该图像。In step S141 , the D/A conversion unit 168 performs digital-to-analog conversion on the image from the screen rearranging buffer 167 . This image is output to a display not shown, and the image is displayed.

[预测处理的描述][Description of prediction processing]

接下来，将参照图25中的流程图描述图24中的步骤S138中的预测处理。Next, the prediction processing in step S138 in FIG. 24 will be described with reference to the flowchart in FIG. 25 .

在步骤S171中，预测图像产生单元181确定是否对当前块进行了帧内编码。在将帧内预测模式信息从无损耗解码单元162供应给预测图像产生单元181时，在步骤S171中，预测图像产生单元181确定对当前块进行了帧内编码，并且，本处理前进到步骤S172。In step S171, the predicted image generating unit 181 determines whether the current block is intra-coded. When the intra prediction mode information is supplied from the lossless decoding unit 162 to the predicted image generation unit 181, in step S171, the predicted image generation unit 181 determines that the current block is intra-coded, and the process proceeds to step S172 .

在步骤S172中，预测图像产生单元181从无损耗解码单元162接收和获得帧内预测模式信息。此时，帧内预测模式信息也被供应给预测模式缓冲器191并被存储。In step S172 , the predicted image generation unit 181 receives and obtains intra prediction mode information from the lossless decoding unit 162 . At this time, intra prediction mode information is also supplied to the prediction mode buffer 191 and stored.

此外，在将来自无损耗解码单元162的量化参数信息供应给量化参数缓冲器192时，在步骤S173中，量化参数缓冲器192获得并存储量化参数。Furthermore, when the quantization parameter information from the lossless decoding unit 162 is supplied to the quantization parameter buffer 192, in step S173, the quantization parameter buffer 192 obtains and stores the quantization parameter.

低通滤波设置单元193从预测模式缓冲器191读出当前块的帧内预测模式信息，并且，从量化参数缓冲器192读出关于当前块的量化参数。在步骤S174中，低通滤波设置单元193从存储于内置的滤波系数存储器194中的每一个片段的滤波系数当中设置对应于该信息的邻近像素的滤波系数。将设置的滤波系数供应给邻近像素设置单元182。The low-pass filter setting unit 193 reads out the intra prediction mode information of the current block from the prediction mode buffer 191 , and reads out the quantization parameter on the current block from the quantization parameter buffer 192 . In step S174 , the low-pass filter setting unit 193 sets the filter coefficients of adjacent pixels corresponding to the information from among the filter coefficients of each segment stored in the built-in filter coefficient memory 194 . The set filter coefficients are supplied to the adjacent pixel setting unit 182 .

在步骤S175中，邻近像素设置单元182使用由低通滤波设置单元193设置的滤波系数来对来自帧存储器169的当前块的邻近像素值执行滤波处理，并且将经过了滤波处理的邻近像素值供应给预测图像产生单元181。In step S175, the adjacent pixel setting unit 182 performs filter processing on the adjacent pixel values of the current block from the frame memory 169 using the filter coefficient set by the low-pass filter setting unit 193, and supplies the filtered adjacent pixel values to to the predicted image generating unit 181.

预测图像产生单元181使用从邻近像素设置单元182供应的邻近像素值来以在步骤S172中获得的帧内预测模式执行帧内预测，并且产生预测图像。将产生的预测图像供应给开关174。The predicted image generating unit 181 uses the adjacent pixel values supplied from the adjacent pixel setting unit 182 to perform intra prediction in the intra prediction mode obtained in step S172 , and generates a predicted image. The generated predicted image is supplied to the switch 174 .

另一方面，当在步骤S171中确定尚未执行帧内编码的情况中，本处理前进到步骤S177。On the other hand, in a case where it is determined in step S171 that intra encoding has not been performed, the process proceeds to step S177.

当要处理的图像是要进行帧内处理的图像的情况中，将帧间预测模式信息、参考帧信息和运动向量信息从无损耗解码单元162供应给运动预测/补偿单元173。在步骤S177中，运动预测/补偿单元173从无损耗解码单元162获得帧间预测模式信息、参考帧信息、运动向量信息等。When the image to be processed is an image to be subjected to intra processing, inter prediction mode information, reference frame information, and motion vector information are supplied from the lossless decoding unit 162 to the motion prediction/compensation unit 173 . In step S177 , the motion prediction/compensation unit 173 obtains inter prediction mode information, reference frame information, motion vector information, and the like from the lossless decoding unit 162 .

在步骤S178中，运动预测/补偿单元173然后执行帧间运动预测。具体地说，在要处理的图像是要进行帧间预测处理的图像的情况中，从帧存储器169读出必要的图像，并且，将必要的图像经由开关170供应给运动预测/补偿单元173。在步骤S177中，运动预测/补偿单元173基于在步骤S176中获得的运动向量以帧间预测模式执行运动预测，以产生预测图像。将产生的预测图像输出给开关174。In step S178, the motion prediction/compensation unit 173 then performs inter motion prediction. Specifically, in the case where the image to be processed is an image to be subjected to inter prediction processing, necessary images are read out from the frame memory 169 and supplied to the motion prediction/compensation unit 173 via the switch 170 . In step S177, the motion prediction/compensation unit 173 performs motion prediction in the inter prediction mode based on the motion vector obtained in step S176 to generate a prediction image. The generated predicted image is output to the switch 174 .

[帧内预测单元和邻近像素内插滤波切换单元的其它配置例子][Other Configuration Examples of Intra Prediction Unit and Neighboring Pixel Interpolation Filtering Switching Unit]

图26是图示邻近像素内插滤波切换单元和帧内预测单元的详细配置例子的框图。请注意，图26中的功能块对应于在采用图20中所示的图像编码装置51的在线处理的情况中的功能块。Fig. 26 is a block diagram illustrating a detailed configuration example of a neighboring pixel interpolation filtering switching unit and an intra prediction unit. Note that the functional blocks in FIG. 26 correspond to functional blocks in the case of online processing employing the image encoding device 51 shown in FIG. 20 .

在图26的例子的情况中，帧内预测单元71由图23中的邻近像素设置单元182和预测图像产生单元181构成。邻近像素内插滤波切换单元172由图23中的预测模式缓冲器191、内插滤波缓冲器201和低通滤波设置单元202构成。请注意，在图26中的例子中，对应于图23中的情况的部分用相应的附图标记表示，并且，基本上执行相同的处理，从而将省略其描述。In the case of the example of FIG. 26 , the intra prediction unit 71 is composed of the adjacent pixel setting unit 182 and the predicted image generation unit 181 in FIG. 23 . The adjacent pixel interpolation filter switching unit 172 is composed of the prediction mode buffer 191 , the interpolation filter buffer 201 , and the low-pass filter setting unit 202 in FIG. 23 . Note that in the example in FIG. 26 , parts corresponding to the case in FIG. 23 are denoted by corresponding reference numerals, and basically the same processing is performed, so that description thereof will be omitted.

在图26的情况中，针对当前片段计算的滤波系数被编码并从图像编码装置51发送。因此，无损耗解码单元162将其与其它信息一同解码，并供应给邻近像素内插滤波切换单元172的内插滤波缓冲器201。In the case of FIG. 26 , the filter coefficient calculated for the current slice is encoded and transmitted from the image encoding device 51 . Therefore, the lossless decoding unit 162 decodes it together with other information, and supplies it to the interpolation filter buffer 201 of the adjacent pixel interpolation filter switching unit 172 .

内插滤波缓冲器201从无损耗解码单元162获得用于当前片段的滤波系数并存储它。The interpolation filter buffer 201 obtains the filter coefficient for the current segment from the lossless decoding unit 162 and stores it.

低通滤波设置单元202从预测模式缓冲器191读出当前块的帧内预测模式信息。低通滤波设置单元202从存储在内插滤波缓冲器201中的当前片段的滤波系数读出对应于已经被读出的帧内预测模式的滤波系数，并将其设置为用于当前块的滤波系数。将设置的滤波系数供应给邻近像素设置单元182。The low-pass filter setting unit 202 reads out the intra prediction mode information of the current block from the prediction mode buffer 191 . The low-pass filter setting unit 202 reads out the filter coefficient corresponding to the already read intra prediction mode from the filter coefficient of the current segment stored in the interpolation filter buffer 201, and sets it as the filter coefficient for the current block. coefficient. The set filter coefficients are supplied to the adjacent pixel setting unit 182 .

[预测处理的其它描述][Additional description of prediction processing]

接下来，将参照图27中的流程图描述帧内预测单元171和邻近像素内插滤波切换单元172的情况中的预测处理。请注意，该帧内预测处理是图24中的步骤S138中的预测处理的另一例子。此外，在图27中步骤S181、S182和S185到S188的处理基本上执行与图25中步骤S171、S172和S175到S178相同的处理，因此省略对其的详细描述。Next, prediction processing in the case of the intra prediction unit 171 and the adjacent pixel interpolation filtering switching unit 172 will be described with reference to the flowchart in FIG. 27 . Note that this intra prediction processing is another example of the prediction processing in step S138 in FIG. 24 . Furthermore, the processing of steps S181, S182, and S185 to S188 in FIG. 27 basically performs the same processing as steps S171, S172, and S175 to S178 in FIG. 25, and thus detailed description thereof is omitted.

在步骤S181中，预测图像产生单元181确定是否对当前块进行了帧内编码。在将帧内预测模式信息从无损耗解码单元162供应给预测图像产生单元181时，在步骤S181中，预测图像产生单元181确定对当前块进行了帧内编码，并且，本处理前进到步骤S182。In step S181, the predicted image generating unit 181 determines whether the current block is intra-coded. When the intra prediction mode information is supplied from the lossless decoding unit 162 to the predicted image generating unit 181, in step S181, the predicted image generating unit 181 determines that the current block is intra-encoded, and the process proceeds to step S182 .

在步骤S182中，预测图像产生单元181从无损耗解码单元162接收和获得帧内预测模式信息。此时，该帧内预测模式信息也被供应给预测模式缓冲器191并被存储。In step S182 , the predicted image generation unit 181 receives and obtains intra prediction mode information from the lossless decoding unit 162 . At this time, this intra prediction mode information is also supplied to the prediction mode buffer 191 and stored.

此外，在用于当前片段的滤波系数的信息被从无损耗解码单元162供应给内插滤波缓冲器201时，内插滤波缓冲器201在步骤S183获得用于当前片段的滤波系数，并存储。请注意，为每个片段供应滤波系数。Furthermore, when the information of the filter coefficient for the current slice is supplied from the lossless decoding unit 162 to the interpolation filter buffer 201, the interpolation filter buffer 201 obtains the filter coefficient for the current slice in step S183, and stores it. Note that filter coefficients are supplied for each segment.

低通滤波设置单元202从预测模式缓冲器191读出用于当前块的帧内预测模式信息。在步骤S184中，除了存储在内插滤波缓冲器201中的当前片段的滤波系数，低通滤波设置单元202还对应于当前块的帧内预测模式设置用于邻近像素的滤波系数。将设置的滤波系数供应给邻近像素设置单元182。The low-pass filter setting unit 202 reads out intra prediction mode information for the current block from the prediction mode buffer 191 . In step S184, in addition to the filter coefficients of the current slice stored in the interpolation filter buffer 201, the low-pass filter setting unit 202 sets filter coefficients for adjacent pixels corresponding to the intra prediction mode of the current block. The set filter coefficients are supplied to the adjacent pixel setting unit 182 .

在步骤S185中，邻近像素设置单元182使用由低通滤波设置单元202设置的滤波系数来对来自帧存储器169的当前块的邻近像素值执行滤波处理，并且将经过了滤波处理的邻近像素值供应给预测图像产生单元181。In step S185, the adjacent pixel setting unit 182 performs filter processing on the adjacent pixel values of the current block from the frame memory 169 using the filter coefficient set by the low-pass filter setting unit 202, and supplies the filtered adjacent pixel values to to the predicted image generating unit 181.

在步骤S186中，预测图像产生单元181使用从邻近像素设置单元182供应的邻近像素值来使用在步骤S172中获得的帧内预测模式执行帧内预测，并且产生预测图像。将产生的预测图像供应给开关174。In step S186 , the predicted image generation unit 181 performs intra prediction using the intra prediction mode obtained in step S172 using the adjacent pixel values supplied from the adjacent pixel setting unit 182 , and generates a predicted image. The generated predicted image is supplied to the switch 174 .

另一方面，当在步骤S181中确定其不是帧内编码的情况中，本处理前进到步骤S187。On the other hand, in a case where it is determined in step S181 that it is not intra-frame encoding, the process proceeds to step S187.

在步骤S187中，运动预测/补偿单元173从无损耗解码单元162获得帧间预测模式信息、参考帧信息、运动向量信息等。In step S187 , the motion prediction/compensation unit 173 obtains inter prediction mode information, reference frame information, motion vector information, and the like from the lossless decoding unit 162 .

在步骤S188中，运动预测/补偿单元173执行帧间运动预测。由于这一处理，产生的预测图像被输出到开关174。In step S188, the motion prediction/compensation unit 173 performs inter motion prediction. Due to this processing, the predicted image generated is output to the switch 174 .

因此，先于帧内处理，通过图1中的图像编码装置51和图22中的图像解码装置151，使用为图像自适应地设置的滤波系数对用于帧内预测的邻近像素执行滤波处理。例如，滤波系数根据帧内预测模式或量化参数设置。Therefore, prior to intra processing, filter processing is performed on adjacent pixels for intra prediction by the image encoding device 51 in FIG. 1 and the image decoding device 151 in FIG. 22 using filter coefficients adaptively set for the image. For example, filter coefficients are set according to intra prediction modes or quantization parameters.

因此，可以执行对应于图像和比特率的噪声去除。结果，可以提高预测效率。Therefore, noise removal corresponding to the image and the bit rate can be performed. As a result, prediction efficiency can be improved.

图28表示应用了本发明的学习装置的一个实施例的配置。在图28中的例子中，学习装置251使用训练图像信号执行滤波系数的学习处理。Fig. 28 shows the configuration of an embodiment of a learning device to which the present invention is applied. In the example in FIG. 28 , the learning means 251 performs learning processing of filter coefficients using a training image signal.

请注意，训练图像信号是用于获得滤波系数的测试图像，并且可以使用，例如，可以在www.vqeg.org获得的用于图像压缩编码的标准化的标准序列。或者，还可以使用对应于每一个应用的输入图像。例如，在输入是照相机信号的情况中，可以使用使用CCD或CMOS传感器成像的基带信号执行学习。Note that the training image signal is a test image for obtaining filter coefficients, and can use, for example, a standardized standard sequence for image compression encoding available at www.vqeg.org. Alternatively, an input image corresponding to each application may also be used. For example, where the input is a camera signal, learning can be performed using baseband signals imaged using a CCD or CMOS sensor.

图28中的学习装置251与图1中的图像编码装置51的共同之处在于具有A/D转换单元61、画面重新排列缓冲器62、计算单元63、正交变换单元64、量化单元65、无损耗编码单元66、存储缓冲器67、逆量化单元68、逆正交变换单元69、计算单元70、去块滤波器71、帧存储器72、开关73、帧内预测单元74、运动预测/补偿单元76、预测图像选择单元77和速率控制单元78。The learning device 251 in FIG. 28 is common to the image coding device 51 in FIG. Lossless encoding unit 66, storage buffer 67, inverse quantization unit 68, inverse orthogonal transformation unit 69, calculation unit 70, deblocking filter 71, frame memory 72, switch 73, intra prediction unit 74, motion prediction/compensation unit 76 , predicted image selection unit 77 and rate control unit 78 .

此外，学习装置251与图1中的图像编码装置51的不同之处在于：对于使用的信号，使用训练图像信号，并且，包括邻近像素内插滤波计算单元261，而不是邻近像素内插滤波切换单元75。In addition, the learning means 251 is different from the image coding means 51 in FIG. 1 in that, for the signal used, a training image signal is used, and a neighboring pixel interpolation filter calculation unit 261 is included instead of a neighboring pixel interpolation filter switching Unit 75.

具体地说，对于学习装置251，仅仅使用包含在I图片中的块来执行学习。或者，对于学习装置251，仅仅使用包含在B图片和P图片中的帧内宏块内的块来执行学习。前者比后者需要更少的用于学习的计算量。此外，在前者的情况中，对于包含在I图片中的块获得的系数可以仅仅适用于在I图片中包含的块，或者可以适用于在B图片和P图片中包含的帧内宏块。Specifically, with the learning means 251, learning is performed using only blocks contained in I pictures. Alternatively, with the learning means 251, learning is performed using only blocks within intra macroblocks contained in B pictures and P pictures. The former requires less computation for learning than the latter. Also, in the former case, coefficients obtained for blocks included in I pictures may be applied only to blocks included in I pictures, or may be applied to intra macroblocks included in B pictures and P pictures.

即，对于学习装置251，执行仅仅通过采用帧内预测单元74的帧内预测的学习。因此，我们将说，运动预测/补偿单元76实际上不起作用。That is, with the learning means 251 , learning by only intra prediction using the intra prediction unit 74 is performed. Therefore, we will say that the motion prediction/compensation unit 76 does not actually function.

此外，图29中的邻近像素内插滤波计算单元261与图20中的邻近像素内插滤波切换单元75的共同之处在于，具有预测模式缓冲器121、最佳滤波计算单元122和低通滤波设置单元123。In addition, the adjacent pixel interpolation filter calculation unit 261 in FIG. 29 is in common with the adjacent pixel interpolation filter switching unit 75 in FIG. Set unit 123.

另一方面，图29中的邻近像素内插滤波计算单元261与图20中的邻近像素内插滤波切换单元75的不同之处在于，添加了滤波系数存储单元271，并且，将来自速率控制单元78的量化参数供应给最佳滤波计算单元122。On the other hand, the difference between the neighboring pixel interpolation filtering calculation unit 261 in FIG. 29 and the neighboring pixel interpolation filtering switching unit 75 in FIG. The quantization parameters of 78 are supplied to the optimal filter calculation unit 122 .

具体地说，在图29中的例子中，以与图20中的例子的情况相同的方式，在帧内预测单元74和邻近像素内插滤波切换单元75之间设置开关101，并且，帧内预测单元74根据开关101的打开/关闭执行帧内预测两次。Specifically, in the example in FIG. 29, in the same manner as in the case of the example in FIG. The prediction unit 74 performs intra prediction twice according to ON/OFF of the switch 101 .

也就是说，对于帧内预测单元74，在开关101的关闭状态中，执行在H.264/AVC中定义的帧内预测，并且，针对每一个片段计算对于帧内预测模式最佳的滤波系数和量化参数。将针对每一个片段计算出的滤波系数存储在滤波系数存储单元271中。然后，在开关101的打开状态中，用已经计算出的每一个片段的滤波系数中的由邻近像素内插滤波切换单元75设置的滤波条件执行帧内预测。That is, with the intra prediction unit 74, in the off state of the switch 101, intra prediction defined in H.264/AVC is performed, and a filter coefficient optimal for the intra prediction mode is calculated for each slice and quantization parameters. The filter coefficients calculated for each segment are stored in the filter coefficient storage unit 271 . Then, in the open state of the switch 101 , intra prediction is performed using the filter conditions set by the adjacent pixel interpolation filter switching unit 75 among the filter coefficients of each slice that have been calculated.

经由存储介质或网络等，将在该滤波系数存储单元271中存储的滤波系数存储在图1中的图像编码装置51的滤波系数存储器94(图14)和图22中的图像解码装置151的滤波系数存储器194(图23)。The filter coefficients stored in this filter coefficient storage unit 271 are stored in the filter coefficient memory 94 (FIG. 14) of the image coding device 51 in FIG. 1 and the filter coefficient of the image decoding device 151 in FIG. Coefficient memory 194 (FIG. 23).

[学习处理中的帧内预测处理的描述][Description of intra prediction processing in learning processing]

接下来，将参照图30中的流程图描述作为学习处理的一个过程的图28中的学习装置251执行的帧内预测处理。请注意，对于学习处理，除了步骤S21中的预测处理被步骤S30中的帧内预测处理替代以外，学习装置251基本上执行与图17中的编码处理相同的处理。Next, intra prediction processing performed by the learning device 251 in FIG. 28 as one procedure of the learning processing will be described with reference to the flowchart in FIG. 30 . Note that, for the learning processing, the learning device 251 basically performs the same processing as the encoding processing in FIG. 17 except that the prediction processing in step S21 is replaced by the intra prediction processing in step S30.

此外，图30中的步骤S201至S203和S206至S209基本上执行与步骤S101至S103和S105至S108相同的处理，从而，将省略其赘述。也就是说，在图30中的步骤S204中，最佳滤波计算单元122计算使整个片段的残差最小的当前片段的每一种帧内预测模式的滤波系数和对应的量化参数，作为最佳滤波系数。将计算出的滤波系数供应给滤波系数存储单元271。In addition, steps S201 to S203 and S206 to S209 in FIG. 30 basically perform the same processing as steps S101 to S103 and S105 to S108, and thus, redundant description thereof will be omitted. That is to say, in step S204 in FIG. 30 , the optimal filter calculation unit 122 calculates the filter coefficients and corresponding quantization parameters of each intra prediction mode of the current segment that minimizes the residual error of the entire segment, as the optimal filter coefficient. The calculated filter coefficients are supplied to the filter coefficient storage unit 271 .

在步骤S205中，滤波系数存储单元271存储从最佳滤波计算单元122供应的滤波系数。In step S205 , the filter coefficient storage unit 271 stores the filter coefficient supplied from the optimum filter calculation unit 122 .

低通滤波设置单元123从存储在滤波系数存储单元271中的当前片段的滤波系数中设置对应于当前块的滤波系数，接通开关101的端子，并且将设置的滤波系数供应给邻近像素设置单元111。The low-pass filter setting unit 123 sets the filter coefficient corresponding to the current block from the filter coefficients of the current segment stored in the filter coefficient storage unit 271, turns on the terminal of the switch 101, and supplies the set filter coefficient to the adjacent pixel setting unit 111.

因此，在步骤S206中，使用设置的滤波系数来对来自帧存储器72的当前块的邻近像素值执行滤波处理。Therefore, in step S206, filter processing is performed on neighboring pixel values of the current block from the frame memory 72 using the set filter coefficients.

当然，以与图21中的例子相同的方式，重复上述的步骤S204至S207的处理能够获得进一步优化的滤波系数。Of course, in the same manner as the example in FIG. 21 , further optimized filter coefficients can be obtained by repeating the above-mentioned processing of steps S204 to S207 .

如上所述，对于学习装置251，使用训练图像信号执行与实际上使用的编码处理相同的处理，并且，将由此计算出的滤波系数存储到滤波系数存储单元271中。因此，可以获得最佳滤波系数。As described above, with the learning means 251 , the same processing as the encoding processing actually used is performed using the training image signal, and the filter coefficient thus calculated is stored in the filter coefficient storage unit 271 . Therefore, optimum filter coefficients can be obtained.

经由存储介质或网络等，将在该滤波系数存储单元271中存储的滤波系数存储在图1中的图像编码装置51的滤波系数存储器94(图14)和图22中的图像解码装置151的滤波系数存储器194(图23)。The filter coefficients stored in this filter coefficient storage unit 271 are stored in the filter coefficient memory 94 (FIG. 14) of the image coding device 51 in FIG. 1 and the filter coefficient of the image decoding device 151 in FIG. Coefficient memory 194 (FIG. 23).

此外，采用图像编码装置251，如上所述，对于在I图片中包含的块(或在B图片和P图片中包含的帧内宏块)获得的系数可以仅仅适用于I图片中包含的块。或者，该系数不仅可以适用于I图片中包含的块，还可以适用于B图片和P图片中包含的帧内宏块。Furthermore, with the image encoding device 251, as described above, coefficients obtained for blocks contained in I pictures (or intra macroblocks contained in B pictures and P pictures) can be applied only to blocks contained in I pictures. Alternatively, the coefficient can be applied not only to blocks included in I pictures, but also to intra macroblocks included in B pictures and P pictures.

或者，可以用图1中的图像编码装置51和图22中的图像解码装置151实现高编码效率。Alternatively, high encoding efficiency can be achieved with the image encoding device 51 in FIG. 1 and the image decoding device 151 in FIG. 22 .

请注意，采用图1中的图像编码装置51的上述的正交变换单元64和逆正交变换单元69、以及图22中的图像解码装置151的逆正交变换单元164，执行在H.264/AVC中定义的正交变换/逆正交变换。或者，可以进行这样的布置：其中，图1中的图像编码装置51的正交变换单元64和逆正交变换单元69、以及图22中的图像解码装置151的逆正交变换单元164，执行在非专利文献1中提出的正交变换/逆正交变换。Note that, using the above-mentioned orthogonal transform unit 64 and inverse orthogonal transform unit 69 of the image encoding device 51 in FIG. 1 and the inverse orthogonal transform unit 164 of the image decoding device 151 in FIG. Orthogonal transform/inverse orthogonal transform defined in /AVC. Alternatively, an arrangement may be made in which the orthogonal transform unit 64 and the inverse orthogonal transform unit 69 of the image encoding device 51 in FIG. 1 and the inverse orthogonal transform unit 164 of the image decoding device 151 in FIG. 22 perform Orthogonal transform/inverse orthogonal transform proposed in Non-Patent Document 1.

因此，可以进一步提高在非专利文献1中提出的格式的编码效率。Therefore, the encoding efficiency of the format proposed in Non-Patent Document 1 can be further improved.

请注意，虽然在以上的描述中已经描述了执行帧内预测的例子，但是，本发明也可以适用于非专利文献2中提出的二次预测中的帧内预测。Note that although an example of performing intra prediction has been described in the above description, the present invention can also be applied to intra prediction in secondary prediction proposed in Non-Patent Document 2.

<2.第二实施例><2. Second Embodiment>

[图像编码装置的其它配置例子][Other Configuration Examples of Image Coding Device]

图31示出用作应用了本发明的图像处理装置的图像编码装置的另一实施例的配置。Fig. 31 shows the configuration of another embodiment of an image encoding device serving as an image processing device to which the present invention is applied.

图像编码装置351与图1中的图像编码装置51的共同之处在于具有A/D转换单元61、画面重新排列缓冲器62、计算单元63、正交变换单元64、量化单元65、无损耗编码单元66、存储缓冲器67、逆量化单元68、逆正交变换单元69、计算单元70、去块滤波器71、帧存储器72、开关73、帧内预测单元74、运动预测/补偿单元76、预测图像选择单元77和速率控制单元78。The image coding device 351 has in common with the image coding device 51 in FIG. unit 66, memory buffer 67, inverse quantization unit 68, inverse orthogonal transform unit 69, calculation unit 70, deblocking filter 71, frame memory 72, switch 73, intra prediction unit 74, motion prediction/compensation unit 76, Predicted image selection unit 77 and rate control unit 78 .

此外，图像编码装置351与图1中的图像编码装置51的不同之处在于，省略了邻近像素内插滤波切换单元75，并且，添加了二次预测单元361和邻近像素内插滤波切换单元362。In addition, the image encoding device 351 differs from the image encoding device 51 in FIG. 1 in that the adjacent pixel interpolation filtering switching unit 75 is omitted, and the secondary prediction unit 361 and the adjacent pixel interpolation filtering switching unit 362 are added. .

也就是说，对于图31中的例子，帧内预测单元74执行H.264/AVC帧内预测。That is, with the example in FIG. 31 , the intra prediction unit 74 performs H.264/AVC intra prediction.

另一方面，运动预测/补偿单元76基于用于帧间处理的图像和参考图像检测所有的候选帧间预测模式的运动向量，基于运动向量对参考图像进行补偿处理，并且产生预测图像。On the other hand, the motion prediction/compensation unit 76 detects motion vectors of all candidate inter prediction modes based on the image for inter processing and the reference image, performs compensation processing on the reference image based on the motion vector, and generates a prediction image.

运动预测/补偿单元76向二次预测单元361供应检测到的运动向量信息、用于帧间处理的图像的信息(地址等)和一次残差，该一次残差是用于帧间预测的图像和产生的预测图像之间的差。The motion prediction/compensation unit 76 supplies the secondary prediction unit 361 with detected motion vector information, information (address, etc.) of an image used for inter-frame processing, and a primary residual which is an image used for inter-frame prediction and the resulting predicted image.

运动预测/补偿单元76通过比较来自二次预测单元361的二次残差来确定二次预测中的最佳帧内预测模式。此外，运动预测/补偿单元76通过比较二次残差和一次残差来确定对二次残差进行编码还是对一次残差进行编码。请注意，对所有的候选帧间预测模式执行该处理。The motion prediction/compensation unit 76 determines the optimal intra prediction mode in the secondary prediction by comparing the secondary residuals from the secondary prediction unit 361 . Furthermore, the motion prediction/compensation unit 76 determines whether to encode the secondary residual or the primary residual by comparing the secondary residual with the primary residual. Note that this process is performed for all candidate inter prediction modes.

运动预测/补偿单元76针对所有的候选帧间预测模式计算成本函数值。此外，使用一次残差和二次残差中的针对每一种帧间预测模式确定的残差来计算成本函数值。运动预测/补偿单元76将产生计算出的成本函数值中的最小值的预测模式确定为最佳帧间预测模式。Motion prediction/compensation unit 76 computes cost function values for all candidate inter prediction modes. Also, the cost function value is calculated using the residual determined for each inter prediction mode among the primary residual and the secondary residual. The motion prediction/compensation unit 76 determines the prediction mode that produces the smallest value among the calculated cost function values as the optimal inter prediction mode.

运动预测/补偿单元76将在最佳帧间预测模式中产生的预测图像(或者用于帧间预测的图像和二次残差之间的差)及其成本函数值供应给预测图像选择单元77。在预测图像选择单元77选择了在最佳帧间预测模式中产生的预测图像的情况中，运动预测/补偿单元76将指示最佳帧间预测模式的信息输出给无损耗编码单元66。此时，运动向量信息、参考帧信息、指示要执行二次预测的二次预测标志、二次预测中的帧内预测模式的信息等也被输出到无损耗编码单元66。The motion prediction/compensation unit 76 supplies the predicted image generated in the optimum inter prediction mode (or the difference between the image used for inter prediction and the quadratic residual) and its cost function value to the predicted image selection unit 77 . In a case where the predicted image selection unit 77 has selected a predicted image generated in the optimum inter prediction mode, the motion prediction/compensation unit 76 outputs information indicating the optimum inter prediction mode to the lossless encoding unit 66 . At this time, motion vector information, reference frame information, a secondary prediction flag indicating that secondary prediction is to be performed, information of an intra prediction mode in secondary prediction, and the like are also output to the lossless encoding unit 66 .

无损耗编码单元66也对来自运动预测/补偿单元76的信息进行诸如可变长度编码、算术编码等的无损耗编码处理，并且将其插入压缩图像的头部中。The lossless encoding unit 66 also performs lossless encoding processing such as variable length encoding, arithmetic encoding, etc. on the information from the motion prediction/compensation unit 76, and inserts it into the header of the compressed image.

基于来自运动预测/补偿单元76的运动向量信息和要进行帧间处理的图像的信息，二次预测单元361从帧存储器72读出与要进行帧间处理的当前块邻近的当前邻近像素。此外，二次预测单元361从帧存储器72读出与通过运动向量信息同当前块相关联的与参考块邻近的参考邻近像素。Based on the motion vector information from the motion prediction/compensation unit 76 and the information of the image to be inter-processed, the secondary prediction unit 361 reads out from the frame memory 72 current adjacent pixels adjacent to the current block to be inter-processed. Furthermore, the secondary prediction unit 361 reads out, from the frame memory 72 , reference adjacent pixels adjacent to the reference block associated with the current block by the motion vector information.

二次预测单元361执行二次预测处理。二次预测处理是这样的处理：其中，在一次残差和当前邻近像素与参考邻近像素间的差之间执行帧内预测，从而产生二次差(二次残差)的信息。The secondary prediction unit 361 performs secondary prediction processing. The secondary prediction process is a process in which intra prediction is performed between the primary residual and the difference between the current adjacent pixel and the reference adjacent pixel, thereby generating information of the secondary difference (secondary residual).

现在，将参照图32描述二次预测处理。Now, the secondary prediction process will be described with reference to FIG. 32 .

对于图32中的例子，示出了当前帧和参考帧，其中，在当前帧中示出了当前块A。For the example in Fig. 32, a current frame and a reference frame are shown, where current block A is shown in the current frame.

在针对当前块A的参考帧和当前帧中获得运动向量mv(mv_x，mv_y)的情况中，计算当前块A和通过运动向量mv与当前块A相关联的块之间的差分信息(残差)。In the case where the motion vector mv(mv_x, mv_y) is obtained in the reference frame for the current block A and the current frame, differential information (residual error) between the current block A and the block associated with the current block A by the motion vector mv is calculated ).

对于二次预测系统，不仅计算与当前块A有关的差分信息，而且还计算邻近当前块A的邻近像素组R和通过运动向量mv与邻近像素组R相关联的邻近像素组R1之间的差分信息。For the quadratic prediction system, not only the difference information related to the current block A is calculated, but also the difference between the neighboring pixel group R adjacent to the current block A and the neighboring pixel group R1 associated with the neighboring pixel group R through the motion vector mv information.

也就是说，从当前块A的左上坐标(x，y)获得邻近像素组R的坐标。此外，从通过运动向量mv与当前块A相关联的块的左上坐标(x+mv_x，y+mv_y)获得邻近像素组R1的坐标。从这些坐标值计算邻近像素组的差分信息。That is, the coordinates of the adjacent pixel group R are obtained from the upper left coordinates (x, y) of the current block A. Furthermore, the coordinates of the adjacent pixel group R1 are obtained from the upper left coordinates (x+mv_x, y+mv_y) of the block associated with the current block A by the motion vector mv. Difference information of adjacent pixel groups is calculated from these coordinate values.

对于二次预测系统，在这样计算出的关于当前块的差分信息和关于参考像素的差分信息之间执行根据H.264/AVC格式的帧内预测，从而产生二次差分信息。对产生的二次差分信息进行正交变换和量化，与压缩图像一起被编码，并且被发送到解码侧。For the quadratic prediction system, intra prediction according to the H.264/AVC format is performed between thus calculated differential information on a current block and differential information on a reference pixel, thereby generating secondary differential information. The generated secondary difference information is subjected to orthogonal transformation and quantization, encoded together with the compressed image, and sent to the decoding side.

在该二次预测之前，二次预测单元361使用由邻近像素内插滤波切换单元362设置的滤波系数来对用于帧内预测的当前邻近像素和参考邻近像素之间的差执行滤波处理。二次预测单元361然后使用经过滤波处理的当前邻近像素和参考邻近像素之间的滤波的差来执行二次预测处理，并且将二次差分信息(二次残差)输出到运动预测/补偿单元76。Prior to this secondary prediction, the secondary prediction unit 361 uses the filter coefficient set by the neighboring pixel interpolation filter switching unit 362 to perform filter processing on the difference between the current neighboring pixel used for intra prediction and the reference neighboring pixel. The secondary prediction unit 361 then performs secondary prediction processing using the filtered difference between the filtered current neighboring pixel and the reference neighboring pixel, and outputs secondary difference information (secondary residual) to the motion prediction/compensation unit 76.

也就是说，二次预测单元361被配置为包括图14中所示的帧内预测单元74等。That is, the secondary prediction unit 361 is configured to include the intra prediction unit 74 shown in FIG. 14 and the like.

邻近像素内插滤波切换单元362基本上以与图1中的邻近像素内插滤波切换单元75相同的方式配置，并且执行相同的处理。也就是说，邻近像素内插滤波切换单元362根据来自二次预测单元361的帧内预测模式信息和来自速率控制单元78的量化参数来设置滤波系数，并且将设置的滤波系数供应给二次预测单元361。The adjacent pixel interpolation filter switching unit 362 is basically configured in the same manner as the adjacent pixel interpolation filter switch unit 75 in FIG. 1 , and performs the same processing. That is, the adjacent pixel interpolation filter switching unit 362 sets the filter coefficients according to the intra prediction mode information from the secondary prediction unit 361 and the quantization parameter from the rate control unit 78, and supplies the set filter coefficients to the secondary prediction Unit 361.

请注意，图31中的图像编码装置351的编码处理与由图1中的图像编码装置51执行的图16中的编码处理的不同之处仅仅在于下述的帧内处理和运动预测处理，并且，其它的处理基本上相同，因此，将省略其描述。Note that the encoding process of the image encoding device 351 in FIG. 31 differs from the encoding process in FIG. 16 performed by the image encoding device 51 in FIG. 1 only in intra processing and motion prediction processing described below, and , other processing is basically the same, and therefore, description thereof will be omitted.

也就是说，采用图31中的图像编码装置351，作为帧内处理，执行根据H.264/AVC格式的帧内预测。此外，作为运动预测处理，在运动预测处理时使用由邻近像素内插滤波切换单元362设置的滤波系数，从而产生二次差分信息。从一次差分信息和二次差分信息中选择更好的二次差分信息，并且，通过比较成本函数值来确定最佳帧内预测模式。That is, with the image encoding device 351 in FIG. 31 , as intra processing, intra prediction according to the H.264/AVC format is performed. Furthermore, as the motion prediction processing, the filter coefficient set by the adjacent pixel interpolation filter switching unit 362 is used at the time of the motion prediction processing, thereby generating secondary difference information. Select better secondary difference information from primary difference information and secondary difference information, and determine the best intra prediction mode by comparing cost function values.

将参照图33描述图像解码装置，该图像解码装置接收由该图像编码装置351编码的压缩图像并解码它。An image decoding device that receives the compressed image encoded by the image encoding device 351 and decodes it will be described with reference to FIG. 33 .

[图像解码装置的其它例子][Other Examples of Image Decoding Device]

图33表示用作应用了本发明的图像处理装置的图像解码装置的另一实施例的配置。Fig. 33 shows the configuration of another embodiment of an image decoding device serving as an image processing device to which the present invention is applied.

图像解码装置401与图22中的图像解码装置151的共同之处在于包含存储缓冲器161、无损耗解码单元162、逆量化单元163、逆正交变换单元164、计算单元165、去块滤波器166、画面重新排列缓冲器167、D/A转换单元168、帧存储器169、开关170、帧内预测单元171、运动预测/补偿单元173和开关174。The image decoding device 401 is common to the image decoding device 151 in FIG. 22 in that it includes a storage buffer 161, a lossless decoding unit 162, an inverse quantization unit 163, an inverse orthogonal transformation unit 164, a calculation unit 165, and a deblocking filter. 166 . Screen rearrangement buffer 167 , D/A conversion unit 168 , frame memory 169 , switch 170 , intra prediction unit 171 , motion prediction/compensation unit 173 and switch 174 .

此外，图像解码装置401与图22中的图像解码装置151的不同之处在于，省略了邻近像素内插滤波切换单元172，并且，添加了二次预测单元411和邻近像素内插滤波切换单元412。Furthermore, the image decoding device 401 differs from the image decoding device 151 in FIG. 22 in that the adjacent pixel interpolation filtering switching unit 172 is omitted, and the secondary prediction unit 411 and the adjacent pixel interpolation filtering switching unit 412 are added. .

也就是说，将通过对头信息进行解码而获得的指示帧内预测模式的信息从无损耗解码单元162供应给帧内预测单元171。基于该信息，帧内预测单元171产生预测图像并将产生的预测图像输出到开关174。That is, information indicating the intra prediction mode obtained by decoding the header information is supplied from the lossless decoding unit 162 to the intra prediction unit 171 . Based on this information, the intra prediction unit 171 generates a predicted image and outputs the generated predicted image to the switch 174 .

将通过对头信息进行解码而获得的信息中的预测模式信息、运动向量信息和参考帧信息等从无损耗解码单元162供应给运动预测/补偿单元173。此外，在对当前块应用了二次预测处理的情况中，将指示要执行该二次预测的二次预测标志和二次预测的帧内模式信息也从无损耗解码单元162供应给运动预测/补偿单元173。The prediction mode information, motion vector information, reference frame information, and the like among the information obtained by decoding the header information are supplied from the lossless decoding unit 162 to the motion prediction/compensation unit 173 . Furthermore, in the case where the secondary prediction process is applied to the current block, the secondary prediction flag indicating that the secondary prediction is to be performed and the intra mode information of the secondary prediction are also supplied from the lossless decoding unit 162 to the motion prediction/ Compensation unit 173.

在确定应用二次预测处理的情况中，运动预测/补偿单元173控制二次预测单元411，使得以二次预测的帧内预测模式信息指示的帧内预测模式执行二次预测。In a case where it is determined that the secondary prediction process is applied, the motion prediction/compensation unit 173 controls the secondary prediction unit 411 so that the secondary prediction is performed in the intra prediction mode indicated by the intra prediction mode information of the secondary prediction.

运动预测/补偿单元173基于运动向量信息和参考帧信息对该图像进行运动预测和补偿处理，并且产生预测图像。也就是说，在参考块内，使用与当前块相关联的参考块的像素值，产生当前块的预测图像。运动预测/补偿单元173然后将来自二次预测单元411的预测差分值和产生的预测图像相加，并将这些输出到开关174。The motion prediction/compensation unit 173 performs motion prediction and compensation processing on the image based on motion vector information and reference frame information, and generates a predicted image. That is, within the reference block, a predicted image of the current block is generated using pixel values of the reference block associated with the current block. The motion prediction/compensation unit 173 then adds the prediction difference value from the secondary prediction unit 411 and the generated prediction image, and outputs these to the switch 174 .

二次预测单元411使用从帧存储器169读出的当前邻近像素和参考邻近像素之间的差来执行二次预测。在该二次预测之前，二次预测单元411使用由邻近像素内插滤波切换单元412设置的滤波系数来对当前邻近像素和参考邻近像素之间的差执行滤波处理。二次预测单元411然后使用经过滤波处理的当前邻近像素和参考邻近像素之间的差来执行二次预测处理，并且将获得的二次差分信息(二次残差)输出到运动预测/补偿单元173。The secondary prediction unit 411 performs secondary prediction using the difference between the current neighboring pixel and the reference neighboring pixel read out from the frame memory 169 . Prior to this secondary prediction, the secondary prediction unit 411 performs filter processing on the difference between the current neighboring pixel and the reference neighboring pixel using the filter coefficient set by the neighboring pixel interpolation filter switching unit 412 . The secondary prediction unit 411 then performs secondary prediction processing using the difference between the filter-processed current neighboring pixel and the reference neighboring pixel, and outputs the obtained secondary difference information (secondary residual) to the motion prediction/compensation unit 173.

也就是说，二次预测单元411被配置为包括图26中所示的帧内预测单元171等。That is, the secondary prediction unit 411 is configured including the intra prediction unit 171 shown in FIG. 26 and the like.

邻近像素内插滤波切换单元412基本上以与邻近像素内插滤波切换单元172相同的方式配置。也就是说，邻近像素内插滤波切换单元412设置与来自无损耗解码单元162的量化参数和帧内预测模式中的至少一个对应的滤波系数。邻近像素内插滤波切换单元412将设置的滤波系数供应给二次预测单元411。The adjacent pixel interpolation filter switching unit 412 is basically configured in the same manner as the adjacent pixel interpolation filter switch unit 172 . That is, the adjacent pixel interpolation filter switching unit 412 sets a filter coefficient corresponding to at least one of the quantization parameter and the intra prediction mode from the lossless decoding unit 162 . The adjacent pixel interpolation filter switching unit 412 supplies the set filter coefficient to the secondary prediction unit 411 .

请注意，图33中的图像解码装置401的解码处理与由图22中的图像解码装置151执行的图24中的解码处理的不同之处仅仅在于下述的帧内处理和运动预测处理，并且，其它的处理是基本上相同的处理，因此，将省略其描述。Note that the decoding process of the image decoding device 401 in FIG. 33 differs from the decoding process in FIG. 24 performed by the image decoding device 151 in FIG. 22 only in intra processing and motion prediction processing described below, and , the other processing is basically the same processing, and therefore, description thereof will be omitted.

也就是说，采用图33中的图像解码装置401，作为帧内处理，执行根据H.264/AVC格式的帧内预测。此外，作为运动预测处理，使用由邻近像素内插滤波切换单元412设置的滤波系数在运动预测处理时执行二次预测(帧内预测)，并且产生二次差分信息。That is, with the image decoding device 401 in FIG. 33 , as intra processing, intra prediction according to the H.264/AVC format is performed. Also, as motion prediction processing, secondary prediction (intra prediction) is performed at the time of motion prediction processing using filter coefficients set by the adjacent pixel interpolation filter switching unit 412 , and secondary difference information is generated.

本发明还可以适用于如上所述的二次预测处理中的帧内预测。The present invention can also be applied to intra prediction in the secondary prediction process as described above.

请注意，在以上的描述中，描述了这样的例子：在帧内预测之前，使用对图像适应地设置的滤波系数，对用于帧内预测的邻近像素执行滤波处理。Note that, in the above description, an example has been described in which, prior to intra prediction, filter processing is performed on adjacent pixels for intra prediction using filter coefficients set adaptively to an image.

现在，用于帧内预测系统的邻近像素中包含的噪声根据诸如图像的内容、量化参数等的编码条件不同。因此，存在通过执行例如在H.264/AVC格式中执行的滤波处理来提高编码效率的块和不是这样的块。Now, noise contained in adjacent pixels used in an intra prediction system differs depending on encoding conditions such as the content of an image, quantization parameters, and the like. Therefore, there are blocks that improve coding efficiency by performing filtering processing such as that performed in the H.264/AVC format and blocks that do not.

不管怎样，在用H.264/AVC格式对宏块执行基于8×8块的帧内处理时，对所有的块执行了空白滤波处理(blanket filtering processing)，从而会出现编码效率因而降低的块。However, when performing intra-frame processing based on 8×8 blocks for macroblocks in the H.264/AVC format, blank filtering processing (blanket filtering processing) is performed on all blocks, resulting in blocks with reduced coding efficiency .

因此，接下来将描述对用于帧内预测的邻近像素执行滤波处理的打开/关闭的情况的例子。Therefore, an example of the case of performing ON/OFF of the filter processing for adjacent pixels for intra prediction will be described next.

<3.第三实施例><3. Third Embodiment>

[图像编码装置的其它配置例子][Other Configuration Examples of Image Coding Device]

图34示出用作应用了本发明的图像处理装置的图像编码装置的另一实施例的配置。Fig. 34 shows the configuration of another embodiment of an image encoding device serving as an image processing device to which the present invention is applied.

图像编码装置451与图1中的图像编码装置51的共同之处在于具有A/D转换单元61、画面重新排列缓冲器62、计算单元63、正交变换单元64、量化单元65、无损耗编码单元66、存储缓冲器67、逆量化单元68、逆正交变换单元69、计算单元70、去块滤波器71、帧存储器72、开关73、帧内预测单元74、运动预测/补偿单元76、预测图像选择单元77和速率控制单元78。The image coding device 451 has in common with the image coding device 51 in FIG. unit 66, memory buffer 67, inverse quantization unit 68, inverse orthogonal transform unit 69, calculation unit 70, deblocking filter 71, frame memory 72, switch 73, intra prediction unit 74, motion prediction/compensation unit 76, Predicted image selection unit 77 and rate control unit 78 .

此外，图像编码装置451与图1中的图像编码装置51的不同之处在于，用邻近像素内插滤波控制单元461替代了邻近像素内插滤波切换单元75。Furthermore, the image coding device 451 differs from the image coding device 51 in FIG. 1 in that the neighboring pixel interpolation filtering control unit 461 replaces the neighboring pixel interpolation filtering switching unit 75 .

也就是说，邻近像素内插滤波控制单元461执行对空白滤波处理的打开或关闭的控制，在用H.264/AVC格式对宏块执行基于8×8块的帧内处理时，对所有块的邻近像素执行了该空白滤波处理。请注意，虽然仅仅用以H.264/AVC格式的基于8×8块的帧内处理执行了滤波处理，但是，也用邻近像素内插滤波控制单元461对帧内4×4和帧内16×16执行该滤波处理。That is to say, the neighboring pixel interpolation filter control unit 461 performs the control of turning on or off the blanking filter processing, and when performing intra-frame processing based on 8×8 blocks for macroblocks in the H.264/AVC format, all blocks Neighboring pixels of are subjected to this blank filtering process. Note that although filtering processing is performed only with 8×8 block-based intra processing in the H.264/AVC format, intra 4×4 and intra 16 ×16 performs this filtering process.

将来自邻近像素内插滤波控制单元461的打开/关闭控制信号供应给帧内预测单元74。The on/off control signal from the adjacent pixel interpolation filter control unit 461 is supplied to the intra prediction unit 74 .

帧内预测单元74基于已经从画面重新排列缓冲器62读出的要进行帧内预测的图像和从帧存储器72供应的参考图像执行所有的候选帧内预测模式的帧内预测处理。此时，在帧内预测单元74处，在帧内预测之前，根据来自邻近像素内插滤波控制单元461的控制信号接通或断开滤波处理来执行帧内预测，并且，利用作为结果的具有较小的计算出的成本函数值的帧内预测模式。The intra prediction unit 74 performs intra prediction processing for all candidate intra prediction modes based on the image to be intra predicted that has been read out from the screen rearrangement buffer 62 and the reference image supplied from the frame memory 72 . At this time, at the intra prediction unit 74, prior to the intra prediction, the intra prediction is performed by turning on or off the filter processing according to the control signal from the adjacent pixel interpolation filter control unit 461, and using the resulting Intra prediction mode with smaller computed cost function value.

此外，帧内预测单元74产生指示滤波处理的打开或关闭的标志。在预测图像选择单元77选择以最佳帧内预测模式产生的预测图像的情况中，将该标志信息与指示最佳帧内预测模式的信息一起供应给无损耗编码单元66。Furthermore, the intra prediction unit 74 generates a flag indicating ON or OFF of the filtering process. In the case where the predicted image selection unit 77 selects a predicted image generated in the optimal intra prediction mode, this flag information is supplied to the lossless encoding unit 66 together with information indicating the optimal intra prediction mode.

[帧内预测单元的配置例子][Configuration example of intra prediction unit]

图35是图示图34中的帧内预测单元74的详细的配置例子的框图。在图35中的例子的情况中，帧内预测单元74由预测图像产生单元471、成本函数值产生单元472和模式和打开/关闭标志产生单元473构成。FIG. 35 is a block diagram illustrating a detailed configuration example of the intra prediction unit 74 in FIG. 34 . In the case of the example in FIG. 35 , the intra prediction unit 74 is composed of a predicted image generation unit 471 , a cost function value generation unit 472 , and a mode and on/off flag generation unit 473 .

从帧存储器72向预测图像产生单元471供应帧内预测的当前块的邻近像素值。在图35中的情况中，从图中省略了开关73，但是，实际上，将邻近像素值经由开关73从帧存储器72供应给预测图像产生单元471。请注意，在帧内预测的情况中，未经过去块滤波器71的去块滤波的像素值用作邻近像素值。The adjacent pixel values of the intra-predicted current block are supplied from the frame memory 72 to the predicted image generation unit 471 . In the case in FIG. 35 , the switch 73 is omitted from the figure, but, actually, adjacent pixel values are supplied from the frame memory 72 to the predicted image generation unit 471 via the switch 73 . Note that in the case of intra prediction, pixel values that have not been deblock-filtered by the deblock filter 71 are used as adjacent pixel values.

基于来自邻近像素内插滤波控制单元461的控制信号，预测图像产生单元471以所有的候选帧内预测模式执行帧内预测，对邻近像素值执行滤波处理，或者不执行滤波处理，从而产生预测图像。如稍后参照图36至图38所述，以块为增量或以宏块为增量执行由控制信号指示的邻近像素内插滤波控制单元461的打开/关闭控制。Based on the control signal from the adjacent pixel interpolation filtering control unit 461, the predicted image generating unit 471 performs intra prediction in all candidate intra prediction modes, performs filtering processing on adjacent pixel values, or performs no filtering processing, thereby generating a predicted image . As described later with reference to FIGS. 36 to 38 , on/off control of the adjacent pixel interpolation filter control unit 461 indicated by the control signal is performed in block increments or in macroblock increments.

预测图像产生单元471将产生的预测图像像素值及其帧内预测模式信息供应给成本函数值产生单元472。The predicted image generation unit 471 supplies the generated predicted image pixel values and their intra prediction mode information to the cost function value generation unit 472 .

从画面重新排列缓冲器62向成本函数值产生单元472供应原始图像像素值。对于滤波处理是打开和关闭的情况，成本函数值产生单元472使用原始图像像素值和预测图像像素值来计算每一种帧内预测模式的成本函数值。成本函数值产生单元472将计算出的成本函数值、预测图像像素值和帧内预测模式信息供应给模式和打开/关闭标志产生单元473。The original image pixel values are supplied from the screen rearrangement buffer 62 to the cost function value generating unit 472 . The cost function value generation unit 472 calculates a cost function value for each intra prediction mode using the original image pixel values and the predicted image pixel values for the cases where the filter processing is on and off. The cost function value generation unit 472 supplies the calculated cost function value, predicted image pixel value, and intra prediction mode information to the mode and on/off flag generation unit 473 .

模式和打开/关闭标志产生单元473使用来自成本函数值产生单元472的成本函数值来确定最佳帧内预测模式及其设置滤波处理的打开和关闭，并且产生指示滤波处理是打开或关闭的打开/关闭标志信息。The mode and on/off flag generation unit 473 uses the cost function value from the cost function value generation unit 472 to determine the optimum intra prediction mode and its setting on and off of the filtering process, and generates an on indicating that the filtering process is on or off. / Turn off flag information.

模式和打开/关闭标志产生单元473将最佳帧内预测模式的预测图像像素值供应给预测图像选择单元77。在预测图像选择单元77选择最佳帧内预测模式的预测图像的情况中，模式和打开/关闭标志产生单元473将指示最佳帧内预测模式的信息和相应的打开/关闭标志信息供应给无损耗编码单元66。The mode and on/off flag generation unit 473 supplies the predicted image pixel values of the optimal intra prediction mode to the predicted image selection unit 77 . In the case where the predicted image selection unit 77 selects the predicted image of the optimum intra prediction mode, the mode and on/off flag generation unit 473 supplies information indicating the optimum intra prediction mode and the corresponding on/off flag information to none. Loss coding unit 66 .

请注意，除了接下来描述的帧内预测以外，由图像编码装置451执行的处理基本上与图1中的图像编码装置51的处理相同，从而，将省略其赘述。Note that the processing performed by the image encoding device 451 is basically the same as that of the image encoding device 51 in FIG. 1 except for intra prediction described next, and thus, redundant description thereof will be omitted.

接下来，将参照图36的流程图描述在以块为增量执行打开或关闭控制的情况中的由图34中的帧内预测单元74执行的帧内预测处理。请注意，该处理是图17中的步骤S31中的帧内预测处理的另一例子，并且，在图36的例子中，将描述帧内4×4的例子。此外，在下文中，滤波处理打开/关闭也可以被简称为滤波器打开/关闭。Next, the intra prediction process performed by the intra prediction unit 74 in FIG. 34 in the case where opening or closing control is performed in block increments will be described with reference to the flowchart of FIG. 36 . Note that this processing is another example of the intra prediction processing in step S31 in FIG. 17 , and, in the example of FIG. 36 , an example of intra 4×4 will be described. Also, hereinafter, filter processing on/off may also be simply referred to as filter on/off.

在步骤S401中，针对图3或图4中示出的九种帧内预测模式中的每一种，成本函数值产生单元472产生当前块的成本函数值。In step S401 , for each of the nine intra prediction modes shown in FIG. 3 or FIG. 4 , the cost function value generating unit 472 generates a cost function value of the current block.

也就是说，从帧存储器72向预测图像产生单元471供应帧内预测的当前块的邻近像素值。预测图像产生单元471对图3或图4中示出的九种帧内预测模式中的每一种执行帧内预测，并且，产生当前块的预测图像。That is, the adjacent pixel values of the intra-predicted current block are supplied from the frame memory 72 to the predicted image generation unit 471 . The predicted image generating unit 471 performs intra prediction on each of the nine intra prediction modes shown in FIG. 3 or 4 , and generates a predicted image of the current block.

此时，从邻近像素内插滤波控制单元461供应用于对邻近像素不执行滤波处理的控制信号，并且，预测图像产生单元471对邻近像素执行没有执行滤波处理的帧内预测。请注意，这里，可以进行这样的布置：其中，供应实现要对邻近像素执行滤波处理的控制信号。然而，请注意，针对九种模式不进行不同的控制，例如，对垂直执行而对水平不执行；确切地说，对于九种模式，进行关于执行或不执行的相同控制。此外，请注意，这里，不对所有的模式执行滤波处理允许更少的计算量。At this time, a control signal for not performing filter processing on adjacent pixels is supplied from the adjacent pixel interpolation filter control unit 461 , and the predicted image generation unit 471 performs intra prediction on adjacent pixels without performing filter processing. Note that here, an arrangement may be made in which a control signal realizing that filtering processing is to be performed on adjacent pixels is supplied. Note, however, that no different controls are made for the nine modes, eg, execution for vertical and no execution for horizontal; rather, the same control as to execution or non-execution is made for the nine modes. Also, note that here, not performing filtering processing on all patterns allows for less computation.

预测图像产生单元471将产生的预测图像像素值及帧内预测模式信息供应给成本函数值产生单元472。在滤波是关闭的情况中，针对每一种帧内预测模式，成本函数值产生单元471使用来自画面重新排列缓冲器62的原始图像像素值和预测图像像素值来计算上述的表达式(73)或表达式(74)中示出的成本函数值。成本函数值产生单元472将计算出的成本函数值、预测图像像素值和帧内预测模式信息供应给模式和打开/关闭标志产生单元473。The predicted image generating unit 471 supplies the generated predicted image pixel values and intra prediction mode information to the cost function value generating unit 472 . In the case where filtering is off, for each intra prediction mode, the cost function value generation unit 471 calculates the above-mentioned expression (73) using the original image pixel value and the predicted image pixel value from the screen rearrangement buffer 62 or the cost function value shown in expression (74). The cost function value generation unit 472 supplies the calculated cost function value, predicted image pixel value, and intra prediction mode information to the mode and on/off flag generation unit 473 .

在步骤S402中，模式和打开/关闭标志产生单元473使用来自成本函数值产生单元472的成本函数值来选择用于当前块的最佳帧内预测模式。将选择的帧内预测模式信息供应给邻近像素内插滤波控制单元461。In step S402 , the mode and on/off flag generation unit 473 uses the cost function value from the cost function value generation unit 472 to select an optimal intra prediction mode for the current block. The selected intra prediction mode information is supplied to the adjacent pixel interpolation filtering control unit 461 .

在步骤S403中，邻近像素内插滤波控制单元461使得成本函数值产生单元472在滤波打开和关闭的情况中产生选择的帧内预测模式的成本函数值。请注意，在步骤S401中，产生滤波是关闭的成本函数值，因此，实际上，在步骤S403中，产生滤波打开的成本函数值。In step S403, the adjacent pixel interpolation filtering control unit 461 causes the cost function value generation unit 472 to generate the cost function value of the selected intra prediction mode in the case of filtering ON and OFF. Please note that in step S401, a cost function value with filtering turned off is generated, so actually, in step S403, a cost function value with filtering turned on is generated.

也就是说，邻近像素内插滤波控制单元461将滤波打开控制信号和选择的帧内预测模式信息供应给预测图像产生单元471。预测图像产生单元471对用于选择的帧内预测模式的邻近像素执行滤波处理，以选择的帧内预测模式执行帧内预测，并且产生当前块的预测图像。That is, the adjacent pixel interpolation filter control unit 461 supplies the filter on control signal and the selected intra prediction mode information to the predicted image generation unit 471 . The predicted image generating unit 471 performs filtering processing on neighboring pixels for the selected intra prediction mode, performs intra prediction in the selected intra prediction mode, and generates a predicted image of the current block.

预测图像产生单元471将产生的预测图像像素值和选择的帧内预测模式信息供应给成本函数值产生单元472。在滤波是打开的情况中，针对选择的帧内预测模式，成本函数值产生单元471使用来自画面重新排列缓冲器62的原始图像像素值和预测图像像素值来计算上述的表达式(73)或表达式(74)中示出的成本函数值。成本函数值产生单元472将计算出的成本函数值、预测图像像素值供应给模式和打开/关闭标志产生单元473。The predicted image generation unit 471 supplies the generated predicted image pixel values and the selected intra prediction mode information to the cost function value generation unit 472 . In the case where filtering is on, for the selected intra prediction mode, the cost function value generation unit 471 calculates the above-mentioned expression (73) or The cost function value shown in expression (74). The cost function value generating unit 472 supplies the calculated cost function value, predicted image pixel value to the mode and on/off flag generating unit 473 .

在步骤S404中，模式和打开/关闭标志产生单元473通过比较选择的帧内预测模式的滤波是打开和关闭的成本函数值来确定当前块的滤波打开/关闭。也就是说，在滤波是打开的成本函数值较小的情况中，针对当前块确定滤波打开，并且，在滤波是关闭的成本函数值较小的情况中，针对当前块确定滤波关闭。模式和打开/关闭标志产生单元473然后将确定的预测图像值供应给预测图像选择单元77。In step S404 , the mode and on/off flag generating unit 473 determines filtering on/off of the current block by comparing the cost function values of whether filtering of the selected intra prediction mode is on and off. That is, in a case where the cost function value for which filtering is on is small, filtering is determined to be on for the current block, and in a case where the cost function value for which filtering is off is small, filtering is determined to be off for the current block. The mode and on/off flag generation unit 473 then supplies the determined predicted image value to the predicted image selection unit 77 .

在步骤S405中，模式和打开/关闭标志产生单元473产生指示当前块的在步骤S404中决定的打开或关闭的打开/关闭标志。例如，在滤波打开的情况中，打开/关闭滤波值是1。在滤波关闭的情况中，打开/关闭滤波值是0。In step S405, the mode and open/close flag generating unit 473 generates an open/close flag indicating the open or close decided in step S404 of the current block. For example, in the case of filtering ON, the filter ON/OFF value is 1. In the case of filtering off, the filter on/off value is 0.

当在上述的图16中的步骤S22中选择了帧内预测模式中的预测图像的情况中，将产生的打开/关闭标志信息与指示最佳帧内预测模式的信息一起供应给无损耗编码单元66。供应的信息在图16中的步骤S23中被编码，与压缩图像的头相加，被发送到解码侧。When a predicted image in the intra prediction mode is selected in step S22 in FIG. 16 described above, the generated ON/OFF flag information is supplied to the lossless coding unit together with information indicating the optimal intra prediction mode 66. The supplied information is encoded in step S23 in FIG. 16, added to the header of the compressed image, and sent to the decoding side.

接下来，将参照图37的流程图描述在以块为增量执行打开或关闭控制的情况中的由图34中的帧内预测单元74执行的帧内预测处理的另一例子。在图37的例子的情况中，也将描述帧内4×4的例子。Next, another example of the intra prediction process performed by the intra prediction unit 74 in FIG. 34 in the case where opening or closing control is performed in block increments will be described with reference to the flowchart of FIG. 37 . In the case of the example of FIG. 37 , an example of intra 4×4 will also be described.

在步骤S421中，针对每一种帧内预测模式，成本函数值产生单元472根据滤波是打开和关闭来产生当前块的成本函数值。In step S421, for each intra prediction mode, the cost function value generation unit 472 generates the cost function value of the current block according to whether filtering is on or off.

也就是说，从帧存储器72向预测图像产生单元471供应帧内预测的当前块的邻近像素值。预测图像产生单元471对图3或图4中示出的九种帧内预测模式中的每一种执行帧内预测，并且，产生当前块的预测图像。That is, the adjacent pixel values of the intra-predicted current block are supplied from the frame memory 72 to the predicted image generation unit 471 . The predicted image generating unit 471 performs intra prediction on each of the nine intra prediction modes shown in FIG. 3 or 4 , and generates a predicted image of the current block.

此时，首先，从邻近像素内插滤波控制单元461供应用于对邻近像素不执行滤波处理的控制信号，并且，预测图像产生单元471对邻近像素以没有执行滤波处理的每一种帧内预测模式执行帧内预测，并产生预测图像。此外，从邻近像素内插滤波控制单元461供应实现对邻近像素执行滤波处理的控制信号，并且，预测图像产生单元471对邻近像素以执行了滤波处理的每一种帧内预测模式执行帧内预测，并产生预测图像。At this time, first, a control signal for not performing filter processing on the adjacent pixels is supplied from the adjacent pixel interpolation filter control unit 461, and the predicted image generating unit 471 performs each intra prediction in which the filter processing is not performed on the adjacent pixels. mode performs intra prediction and produces a predicted image. Further, a control signal for realizing filter processing for adjacent pixels is supplied from the adjacent pixel interpolation filtering control unit 461, and the predicted image generation unit 471 performs intra prediction for each of the intra prediction modes in which filter processing is performed for adjacent pixels. , and generate a predicted image.

预测图像产生单元471将在滤波是打开和关闭的情况中的每一种帧内预测模式的信息和相应的预测图像像素值供应给成本函数值产生单元472。在滤波是关闭和打开的每一种情况中，针对每一种帧内预测模式，成本函数值产生单元471使用来自画面重新排列缓冲器62的原始图像像素值和预测图像像素值来计算上述的表达式(73)或表达式(74)中示出的成本函数值。在滤波是关闭和打开的每一种情况中，成本函数值产生单元472将计算出的成本函数值、预测图像像素值和帧内预测模式信息供应给模式和打开/关闭标志产生单元473。The predicted image generation unit 471 supplies the information of each intra prediction mode in the cases where filtering is on and off and the corresponding predicted image pixel values to the cost function value generation unit 472 . In each case where filtering is off and on, for each intra prediction mode, the cost function value generation unit 471 uses the original image pixel values and predicted image pixel values from the screen rearrangement buffer 62 to calculate the above-mentioned The cost function value shown in Expression (73) or Expression (74). The cost function value generation unit 472 supplies the calculated cost function value, predicted image pixel value, and intra prediction mode information to the mode and on/off flag generation unit 473 in each case of filtering being off and on.

在步骤S422中，模式和打开/关闭标志产生单元473使用来自成本函数值产生单元472的成本函数值来确定在每一种帧内预测模式的情况中对于当前块而言滤波应该是打开还是关闭。In step S422, the mode and on/off flag generation unit 473 uses the cost function value from the cost function value generation unit 472 to determine whether filtering should be on or off for the current block in the case of each intra prediction mode .

在步骤S423中，模式和打开/关闭标志产生单元473从关于确定滤波是打开或关闭的帧内预测模式中选择当前块的最佳帧内预测模式。In step S423 , the mode and on/off flag generation unit 473 selects the optimal intra prediction mode of the current block from the intra prediction modes regarding determining whether filtering is on or off.

在步骤S424中，模式和打开/关闭标志产生单元473产生指示针对选择的帧内预测模式的滤波器的状态(打开或关闭)的打开/关闭标志信息。当在上述的图16中的步骤S22中选择了帧内预测模式中的预测图像的情况中，将产生的打开/关闭标志信息与指示最佳帧内预测模式的信息一起供应给无损耗编码单元66。供应的信息在图16中的步骤S23中被编码，与压缩图像的头相加，被发送到解码侧。In step S424, the mode and on/off flag generation unit 473 generates on/off flag information indicating the state (on or off) of the filter for the selected intra prediction mode. When a predicted image in the intra prediction mode is selected in step S22 in FIG. 16 described above, the generated ON/OFF flag information is supplied to the lossless coding unit together with information indicating the optimal intra prediction mode 66. The supplied information is encoded in step S23 in FIG. 16, added to the header of the compressed image, and sent to the decoding side.

接下来，将参照图38的流程图描述在以宏块为增量执行打开或关闭控制的情况中的由图34中的帧内预测单元74执行的帧内预测处理。Next, the intra prediction process performed by the intra prediction unit 74 in FIG. 34 in the case where opening or closing control is performed in increments of macroblocks will be described with reference to the flowchart of FIG. 38 .

请注意，该处理是图17中的步骤S31中的帧内预测处理的另一例子，并且，在图38的例子中，也将描述帧内4×4的例子。Note that this processing is another example of the intra prediction processing in step S31 in FIG. 17 , and, in the example of FIG. 38 , an example of intra 4×4 will also be described.

在步骤S451中，邻近像素内插滤波控制单元461将关于整个宏块的滤波固定为关闭或打开。在这种情况下，邻近像素内插滤波控制单元461将滤波固定为关闭，并且，将滤波关闭的控制信号供应给预测图像产生单元471。滤波的固定可以是打开或关闭，但是，可以用较少的计算量来实现到关闭的固定。In step S451, the adjacent pixel interpolation filter control unit 461 fixes the filtering with respect to the entire macroblock to be off or on. In this case, the adjacent pixel interpolation filter control unit 461 fixes filtering to be off, and supplies a control signal that the filtering is off to the predicted image generation unit 471 . The fixation of filtering can be either on or off, however, fixation to off can be achieved with less computational effort.

在步骤S452中，帧内预测单元74确定每一块的帧内预测模式。也就是说，从帧存储器72向预测图像产生单元471供应帧内预测的当前块的邻近像素值。预测图像产生单元471对图3或图4中示出的九种帧内预测模式中的每一种执行帧内预测，并且，产生当前块的预测图像。In step S452, the intra prediction unit 74 determines an intra prediction mode for each block. That is, the adjacent pixel values of the intra-predicted current block are supplied from the frame memory 72 to the predicted image generation unit 471 . The predicted image generating unit 471 performs intra prediction on each of the nine intra prediction modes shown in FIG. 3 or 4 , and generates a predicted image of the current block.

此时，首先，从邻近像素内插滤波控制单元461供应实现对邻近像素不执行滤波处理的控制信号，并且，预测图像产生单元471对邻近像素执行每一种帧内预测模式上的帧内预测而没有执行滤波处理，并产生预测图像。预测图像产生单元471将产生的预测图像像素值及其帧内预测模式信息供应给成本函数值产生单元472。At this time, first, a control signal for realizing not performing filter processing on adjacent pixels is supplied from the adjacent pixel interpolation filter control unit 461, and the predicted image generation unit 471 performs intra prediction on each intra prediction mode on the adjacent pixels No filter processing is performed, and a predicted image is generated. The predicted image generation unit 471 supplies the generated predicted image pixel values and their intra prediction mode information to the cost function value generation unit 472 .

在滤波是关闭的每一种情况中，针对每一种帧内预测模式，成本函数值产生单元471使用来自画面重新排列缓冲器62的原始图像像素值和预测图像像素值来计算上述的表达式(73)或表达式(74)中示出的成本函数值。在滤波是关闭的每一种情况中，成本函数值产生单元472将计算出的成本函数值、预测图像像素值和帧内预测模式信息供应给模式和打开/关闭标志产生单元473。In each case where filtering is off, for each intra prediction mode, the cost function value generating unit 471 uses the original image pixel values and predicted image pixel values from the screen rearrangement buffer 62 to calculate the above-mentioned expression (73) or the cost function value shown in Expression (74). In each case where filtering is off, the cost function value generation unit 472 supplies the calculated cost function value, predicted image pixel value, and intra prediction mode information to the mode and on/off flag generation unit 473 .

模式和打开/关闭标志产生单元473使用来自成本函数值产生单元472的成本函数值来确定每一块的最佳帧内预测模式。将解码的帧内预测模式信息供应给邻近像素内插滤波控制单元461。The mode and on/off flag generation unit 473 uses the cost function value from the cost function value generation unit 472 to determine the optimum intra prediction mode for each block. The decoded intra prediction mode information is supplied to the adjacent pixel interpolation filtering control unit 461 .

在步骤S453中，邻近像素内插滤波控制单元461使得成本函数值产生单元472针对整个宏块产生滤波打开和关闭的成本函数值。请注意，在步骤S452中产生了在滤波是关闭的情况中针对宏块内的每一块(即，整个宏块)的最佳帧内预测模式的成本函数值。因此，实际上，在步骤S453中，产生在滤波是打开的情况中的整个宏块的成本函数值。In step S453 , the adjacent pixel interpolation filtering control unit 461 causes the cost function value generation unit 472 to generate cost function values for filtering ON and OFF for the entire macroblock. Note that the cost function value for the best intra prediction mode for each block within the macroblock (ie, the entire macroblock) in the case of filtering is off is generated in step S452. Therefore, actually, in step S453, the cost function value of the entire macroblock in the case that filtering is ON is generated.

也就是说，邻近像素内插滤波控制单元461将滤波打开控制信号和针对每一块确定的帧内预测模式的信息供应给预测图像产生单元471。预测图像产生单元471对用于确定的帧内预测模式的邻近像素值执行滤波处理，以确定的帧内预测模式执行帧内预测，并且产生当前块的预测图像。That is, the adjacent pixel interpolation filter control unit 461 supplies the filter-on control signal and information of the intra prediction mode determined for each block to the predicted image generation unit 471 . The predicted image generating unit 471 performs filtering processing on adjacent pixel values for the determined intra prediction mode, performs intra prediction in the determined intra prediction mode, and generates a predicted image of the current block.

预测图像产生单元471将产生的预测图像像素值及确定的帧内预测模式信息供应给成本函数值产生单元472。在滤波是打开的每一种情况中，针对确定的帧内预测模式，成本函数值产生单元471使用来自画面重新排列缓冲器62的原始图像像素值和预测图像像素值来计算上述的表达式(73)或表达式(74)中示出的成本函数值。在滤波是关闭和打开的每一种情况中，成本函数值产生单元472将计算出的成本函数值、预测图像像素值和帧内预测模式信息供应给模式和打开/关闭标志产生单元473。The predicted image generating unit 471 supplies the generated predicted image pixel values and the determined intra prediction mode information to the cost function value generating unit 472 . In each case where filtering is on, for a determined intra prediction mode, the cost function value generating unit 471 calculates the above-mentioned expression ( 73) or the cost function value shown in expression (74). The cost function value generation unit 472 supplies the calculated cost function value, predicted image pixel value, and intra prediction mode information to the mode and on/off flag generation unit 473 in each case of filtering being off and on.

在步骤S454中，模式和打开/关闭标志产生单元473比较来自成本函数值产生单元472的在滤波是打开和关闭的情况中的宏块内的所有块的成本函数值，并且确定对整个宏块应用滤波打开/关闭中的哪一个。In step S454, the mode and on/off flag generation unit 473 compares the cost function values of all blocks within the macroblock in the case that filtering is on and off from the cost function value generation unit 472, and determines the value for the entire macroblock Which of filtering on/off is applied.

在步骤S455中，对于整个宏块，模式和打开/关闭标志产生单元473产生指示在步骤S454中决定的打开或关闭的打开/关闭标志。对于每一个宏块，将产生的打开/关闭标志信息供应给无损耗编码单元66。供应的信息在图16中的步骤S23中被编码，与压缩图像的头相加，被发送到解码侧。In step S455, the mode and on/off flag generating unit 473 generates an on/off flag indicating on or off decided in step S454 for the entire macroblock. The generated on/off flag information is supplied to the lossless encoding unit 66 for each macroblock. The supplied information is encoded in step S23 in FIG. 16, added to the header of the compressed image, and sent to the decoding side.

如上所述，滤波打开/关闭(打开或者关闭)的控制可以以块为增量执行，或者，可以以宏块为增量执行。请注意，虽然可以通过以块为增量控制打开/关闭来提高帧内预测处理的预测精度，但是用于传送每一块的标志信息所需的信息量增加。相反，在以宏块为增量进行控制的情况中，预测精度的提高低于以块为增量执行的预测精度的提高，但是，针对每一个宏块的一个标志信息是足够的，从而，可以减少标志信息量的增加。As described above, control of filter ON/OFF (ON or OFF) may be performed in block increments, or may be performed in macroblock increments. Note that although the prediction accuracy of intra prediction processing can be improved by controlling ON/OFF in block increments, the amount of information required for transmitting flag information for each block increases. Conversely, in the case of controlling in increments of macroblocks, the improvement of prediction accuracy is lower than that performed in increments of blocks, but one piece of flag information for each macroblock is sufficient, and thus, An increase in the amount of flag information can be reduced.

虽然在以上描述中描述了亮度信号的例子，但是这也可以用于关于颜色差信号的帧内预测。此外，要控制的滤波处理的滤波系数不限于H.264/AVC格式中的三抽头{1，2，1}//4，并且，这可以适用于用图1中的图像编码装置51设置的任何抽头长度的任何系数。Although an example of a luma signal was described in the above description, this can also be used for intra prediction with respect to a color difference signal. In addition, the filter coefficients of the filter processing to be controlled are not limited to the three-tap {1, 2, 1}//4 in the H.264/AVC format, and this can be applied to the Any coefficient for any tap length.

也就是说，在滤波打开的情况中，还可以执行用由图1中的图像编码装置51设置的滤波系数的滤波处理。That is to say, in the case of filtering ON, filtering processing with the filter coefficient set by the image encoding device 51 in FIG. 1 can also be performed.

将参照图39描述图像解码装置，该图像解码装置接收由该图像编码装置451编码的压缩图像并解码它。An image decoding device that receives the compressed image encoded by the image encoding device 451 and decodes it will be described with reference to FIG. 39 .

[图像解码装置的其它例子][Other Examples of Image Decoding Device]

图39图示用作应用了本发明的图像处理装置的图像解码装置的另一实施例的配置。Fig. 39 illustrates the configuration of another embodiment of an image decoding device serving as an image processing device to which the present invention is applied.

图像解码装置501与图22中的图像解码装置151的共同之处在于包含存储缓冲器161、无损耗解码单元162、逆量化单元163、逆正交变换单元164、计算单元165、去块滤波器166、画面重新排列缓冲器167、D/A转换单元168、帧存储器169、开关170、帧内预测单元171、运动预测/补偿单元173和开关174。The image decoding device 501 is common to the image decoding device 151 in FIG. 22 in that it includes a storage buffer 161, a lossless decoding unit 162, an inverse quantization unit 163, an inverse orthogonal transformation unit 164, a calculation unit 165, and a deblocking filter. 166 . Screen rearrangement buffer 167 , D/A conversion unit 168 , frame memory 169 , switch 170 , intra prediction unit 171 , motion prediction/compensation unit 173 and switch 174 .

此外，图像解码装置501与图22中的图像解码装置151的不同之处在于，用邻近像素内插滤波控制单元511替代了邻近像素内插滤波切换单元172。Furthermore, the image decoding device 501 differs from the image decoding device 151 in FIG. 22 in that the adjacent pixel interpolation filter control unit 511 replaces the adjacent pixel interpolation filter switching unit 172 .

也就是说，将通过对头信息进行解码而获得的指示帧内预测模式的信息从无损耗解码单元162供应给帧内预测单元171。基于该信息，帧内预测单元171产生预测图像并将产生的预测图像输出到开关174。此时。在帧内预测之前，帧内预测单元171根据来自邻近像素内插滤波控制单元511的控制信号对邻近像素值执行(或者不执行)滤波处理。That is, information indicating the intra prediction mode obtained by decoding the header information is supplied from the lossless decoding unit 162 to the intra prediction unit 171 . Based on this information, the intra prediction unit 171 generates a predicted image and outputs the generated predicted image to the switch 174 . at this time. Before intra prediction, the intra prediction unit 171 performs (or does not perform) filter processing on adjacent pixel values according to a control signal from the adjacent pixel interpolation filter control unit 511 .

根据图像编码装置451的编码，从无损耗解码单元162向邻近像素内插滤波控制单元511供应每一个宏块或每一块的打开/关闭标志信息。On/off flag information for each macroblock or each block is supplied from the lossless decoding unit 162 to the adjacent pixel interpolation filter control unit 511 according to encoding by the image encoding device 451 .

邻近像素内插滤波控制单元511根据供应的打开/关闭标志信息向帧内预测单元171供应实现执行或不执行滤波处理的控制信号。The adjacent pixel interpolation filtering control unit 511 supplies the intra prediction unit 171 with a control signal enabling execution or non-execution of filter processing according to the supplied ON/OFF flag information.

请注意，采用图34中的图像编码装置451，对滤波是打开和关闭的两种情况进行测试，并且，在通过成本函数值选择了产生更高的编码效率的一种情况之后，执行帧内预测处理。另一方面，采用图像解码装置501，基于编码的发送的打开/关闭标志信息控制滤波打开或关闭，并且，执行帧内预测处理。Please note that with the image encoding device 451 in FIG. 34 , two cases where filtering is on and off are tested, and after a case that results in higher coding efficiency is selected by the value of the cost function, intra predictive processing. On the other hand, with the image decoding device 501, filter ON or OFF is controlled based on encoded transmitted ON/OFF flag information, and intra prediction processing is performed.

[帧内预测单元和邻近像素内插滤波控制单元的配置例子][Configuration Example of Intra Prediction Unit and Neighboring Pixel Interpolation Filtering Control Unit]

图40是图示邻近像素内插滤波控制单元和帧内预测单元的详细配置例子的框图。Fig. 40 is a block diagram illustrating a detailed configuration example of a neighboring pixel interpolation filtering control unit and an intra prediction unit.

在图40的例子的情况中，帧内预测单元171由预测模式缓冲器521和预测图像产生单元522构成。邻近像素内插滤波控制单元511由标志缓冲器531和控制信号产生单元532构成。In the case of the example of FIG. 40 , the intra prediction unit 171 is composed of a prediction mode buffer 521 and a predicted image generation unit 522 . The adjacent pixel interpolation filter control unit 511 is composed of a flag buffer 531 and a control signal generation unit 532 .

向预测模式缓冲器521供应来自无损耗解码单元162的帧内预测模式信息。从帧存储器169向预测图像产生单元522供应帧内预测的当前块的邻近像素值。在图40中的情况中，也从图中省略了开关170，但是，实际上，将邻近像素值经由开关170从帧存储器169供应给预测图像产生单元522。The intra prediction mode information from the lossless decoding unit 162 is supplied to the prediction mode buffer 521 . The adjacent pixel values of the intra-predicted current block are supplied from the frame memory 169 to the predicted image generating unit 522 . In the case in FIG. 40 , the switch 170 is also omitted from the figure, but, actually, adjacent pixel values are supplied from the frame memory 169 to the predicted image generation unit 522 via the switch 170 .

预测图像产生单元522从预测模式缓冲器521读出当前块的帧内预测模式信息，以读出的帧内预测模式对当前块执行帧内预测，并且产生预测图像。在该帧内预测之前，预测图像产生单元522根据来自控制信号产生单元532的控制信号对来自帧存储器169的预测图像像素值执行滤波处理。The predicted image generating unit 522 reads out the intra prediction mode information of the current block from the prediction mode buffer 521 , performs intra prediction on the current block in the read intra prediction mode, and generates a predicted image. Prior to this intra prediction, the predicted image generating unit 522 performs filter processing on predicted image pixel values from the frame memory 169 in accordance with a control signal from the control signal generating unit 532 .

针对每一个宏块或每一块，从无损耗解码单元162向标志缓冲器531供应打开/关闭标志信息。控制信号产生单元532从标志缓冲器531读出相应的打开/关闭标志，产生指示对于每一块执行滤波处理还是不执行滤波处理的控制信号，并且将产生的控制信号供应给预测图像产生单元522。On/off flag information is supplied from the lossless decoding unit 162 to the flag buffer 531 for each macroblock or each block. The control signal generation unit 532 reads out the corresponding on/off flags from the flag buffer 531 , generates a control signal indicating whether to perform filter processing or not to perform filter processing for each block, and supplies the generated control signal to the predicted image generation unit 522 .

请注意，除了接下来描述的预测处理以外，由图像解码装置501执行的处理基本上与图22中的图像解码装置151的处理相同，从而，将省略其赘述。Note that the processing performed by the image decoding device 501 is basically the same as that of the image decoding device 151 in FIG. 22 except for the prediction processing described next, and thus, redundant description thereof will be omitted.

[预测处理的描述][Description of prediction processing]

接下来，将参照图41的流程图描述图39中的图像解码装置501的预测处理。请注意，该帧内预测处理是图24中的步骤S138中的预测处理的另一例子。Next, prediction processing by the image decoding device 501 in FIG. 39 will be described with reference to the flowchart of FIG. 41 . Note that this intra prediction processing is another example of the prediction processing in step S138 in FIG. 24 .

在步骤S501中，预测图像产生单元522确定是否对当前块进行了帧内编码。将帧内预测模式信息从无损耗解码单元162供应给预测模式缓冲器521，从而由预测图像产生单元522读取。因此，在步骤S501中，预测图像产生单元522确定对当前块进行帧内编码，并且，本处理前进到步骤S502。In step S501, the predicted image generating unit 522 determines whether the current block is intra-coded. The intra prediction mode information is supplied from the lossless decoding unit 162 to the prediction mode buffer 521 to be read by the predicted image generation unit 522 . Therefore, in step S501, the predicted image generating unit 522 determines to intra-encode the current block, and the present process proceeds to step S502.

在步骤S502中，预测图像产生单元522获得预测模式缓冲器521的帧内预测模式信息。In step S502 , the predicted image generating unit 522 obtains the intra prediction mode information of the prediction mode buffer 521 .

此外，在将来自无损耗解码单元162的打开/关闭标志信息供应给标志缓冲器531时，标志缓冲器531在步骤S503中获得预测图像像素值标志并存储它。Furthermore, when the on/off flag information from the lossless decoding unit 162 is supplied to the flag buffer 531, the flag buffer 531 obtains a predicted image pixel value flag and stores it in step S503.

控制信号产生单元532从标志缓冲器531读出对应于当前标志的打开/关闭标志，并且，在步骤S504中确定打开/关闭标志是否为1。当在步骤S504中确定打开/关闭标志是1，即，滤波处理是打开的情况中，控制信号产生单元532将控制信号供应给预测图像产生单元522，以使得执行滤波处理。The control signal generation unit 532 reads out the on/off flag corresponding to the current flag from the flag buffer 531, and determines whether the on/off flag is 1 in step S504. When it is determined in step S504 that the on/off flag is 1, that is, the filter processing is on, the control signal generation unit 532 supplies a control signal to the predicted image generation unit 522 so that the filter processing is performed.

根据控制信号，在步骤S505中，预测图像产生单元522使用滤波系数对邻近像素进行滤波处理。在步骤S506中，预测图像产生单元522使用经过了滤波处理的邻近像素值来执行帧内预测，并且产生预测图像。According to the control signal, in step S505 , the predicted image generation unit 522 performs filtering processing on the adjacent pixels using the filter coefficients. In step S506 , the predicted image generation unit 522 performs intra prediction using the filter-processed neighboring pixel values, and generates a predicted image.

另一方面，当在步骤S504中确定打开/关闭标志不是1，即，滤波处理是关闭的情况中，跳过步骤S505的滤波处理，并且，本处理前进到步骤S506。On the other hand, when it is determined in step S504 that the ON/OFF flag is not 1, that is, the filter processing is off, the filter processing of step S505 is skipped, and the process proceeds to step S506.

在步骤S506中，预测图像产生单元522使用来自帧存储器169的预测图像像素值来执行帧内预测，并且产生预测图像。In step S506 , the predicted image generation unit 522 performs intra prediction using the predicted image pixel values from the frame memory 169 , and generates a predicted image.

将在步骤S506中产生的预测图像供应给开关174。The predicted image generated in step S506 is supplied to the switch 174 .

另一方面，当在步骤S501中确定不执行帧内编码的情况中，本处理前进到步骤S507。On the other hand, in a case where it is determined in step S501 that intra encoding is not performed, the process proceeds to step S507.

在步骤S507中，运动预测/补偿单元173执行帧间运动预测。也就是说，在要处理的图像是用于帧间预测处理的图像的情况中，从帧存储器169读出必要的图像，并且，将必要的图像经由开关170供应给运动预测/补偿单元173。在步骤S508中，运动预测/补偿单元173基于在步骤S507中获得的运动向量以帧间预测模式执行运动预测，并且产生预测图像。将产生的预测图像输出给开关174。In step S507, the motion prediction/compensation unit 173 performs inter motion prediction. That is, in the case where the image to be processed is an image for inter prediction processing, necessary images are read out from the frame memory 169 and supplied to the motion prediction/compensation unit 173 via the switch 170 . In step S508, the motion prediction/compensation unit 173 performs motion prediction in the inter prediction mode based on the motion vector obtained in step S507, and generates a predicted image. The generated predicted image is output to the switch 174 .

如上所述，采用图像编码装置451和图像解码装置501，控制对于用于帧内预测的邻近像素的滤波处理的打开和关闭，并且，针对编码效率劣化的块不执行滤波处理。因此，可以提高编码效率。As described above, with the image encoding device 451 and the image decoding device 501, ON and OFF of the filter processing for adjacent pixels for intra prediction is controlled, and the filter processing is not performed for a block whose encoding efficiency deteriorates. Therefore, coding efficiency can be improved.

请注意，虽然在以上的描述中描述了执行帧内预测的例子，但是对滤波处理打开和关闭的控制可以适用于上文中参照图32描述的二次预测中的帧内预测。Note that although an example of performing intra prediction has been described in the above description, control of turning on and off filtering processing can be applied to intra prediction in the secondary prediction described above with reference to FIG. 32 .

<4.第四实施例><4. Fourth Embodiment>

[图像编码装置的其它配置例子][Other Configuration Examples of Image Coding Device]

图42示出用作应用了本发明的图像处理装置的图像编码装置的另一实施例的配置。Fig. 42 shows the configuration of another embodiment of an image encoding device serving as an image processing device to which the present invention is applied.

图像编码装置551与图34中的图像编码装置451的共同之处在于具有A/D转换单元61、画面重新排列缓冲器62、计算单元63、正交变换单元64、量化单元65、无损耗编码单元66、存储缓冲器67、逆量化单元68、逆正交变换单元69、计算单元70、去块滤波器71、帧存储器72、开关73、帧内预测单元74、运动预测/补偿单元76、预测图像选择单元77和速率控制单元78。The image encoding device 551 has in common with the image encoding device 451 in FIG. unit 66, memory buffer 67, inverse quantization unit 68, inverse orthogonal transform unit 69, calculation unit 70, deblocking filter 71, frame memory 72, switch 73, intra prediction unit 74, motion prediction/compensation unit 76, Predicted image selection unit 77 and rate control unit 78 .

此外，图像编码装置551与图34中的图像编码装置451的不同之处在于，省略了邻近像素内插滤波控制单元461，并且，添加了图31中的二次预测单元361和邻近像素内插滤波控制单元561。Furthermore, the image encoding device 551 differs from the image encoding device 451 in FIG. 34 in that the adjacent pixel interpolation filter control unit 461 is omitted, and the secondary prediction unit 361 and the adjacent pixel interpolation control unit 461 in FIG. 31 are added. filter control unit 561 .

也就是说，对于图42中的例子，帧内预测单元74根据H.264/AVC执行帧内预测。That is, with the example in FIG. 42 , the intra prediction unit 74 performs intra prediction according to H.264/AVC.

另一方面，运动预测/补偿单元76基于用于帧间处理的图像和参考图像检测所有的候选帧间预测模式的运动向量，基于运动向量对参考图像进行补偿处理，并且产生预测图像。On the other hand, the motion prediction/compensation unit 76 detects motion vectors of all candidate inter prediction modes based on the image for inter processing and the reference image, performs compensation processing on the reference image based on the motion vector, and generates a prediction image.

运动预测/补偿单元76向二次预测单元361供应检测到的运动向量信息、用于帧间处理的图像的信息(地址等)和一次残差，该一次残差是用于帧间预测的图像和产生的预测图像之间的差。The motion prediction/compensation unit 76 supplies the secondary prediction unit 361 with detected motion vector information, information (address, etc.) of an image used for inter-frame processing, and a primary residual which is an image used for inter-frame prediction and the resulting predicted image.

运动预测/补偿单元76通过比较来自二次预测单元361的二次残差来确定二次预测中的最佳帧内预测模式。此外，运动预测/补偿单元76通过比较二次残差和一次残差来确定对二次残差进行编码还是对一次残差进行编码。请注意，对所有的候选帧间预测模式执行该处理。The motion prediction/compensation unit 76 determines the optimal intra prediction mode in the secondary prediction by comparing the secondary residuals from the secondary prediction unit 361 . Furthermore, the motion prediction/compensation unit 76 determines whether to encode the secondary residual or the primary residual by comparing the secondary residual with the primary residual. Note that this process is performed for all candidate inter prediction modes.

运动预测/补偿单元76针对所有的候选帧间预测模式计算成本函数值。此时，使用一次残差和二次残差中的针对每一种帧间预测模式确定的残差来计算成本函数值。运动预测/补偿单元76将产生计算出的成本函数值中的最小值的预测模式确定为最佳帧间预测模式。Motion prediction/compensation unit 76 computes cost function values for all candidate inter prediction modes. At this time, the cost function value is calculated using the residual determined for each inter prediction mode among the primary residual and the secondary residual. The motion prediction/compensation unit 76 determines the prediction mode that produces the smallest value among the calculated cost function values as the optimal inter prediction mode.

运动预测/补偿单元76将在最佳帧间预测模式中产生的预测图像(或者用于帧间预测的图像和二次残差之间的差)及其成本函数值供应给预测图像选择单元77。在预测图像选择单元77选择了在最佳帧间预测模式中产生的预测图像的情况中，运动预测/补偿单元76将指示最佳帧间预测模式的信息输出给无损耗编码单元66。The motion prediction/compensation unit 76 supplies the predicted image generated in the optimum inter prediction mode (or the difference between the image used for inter prediction and the quadratic residual) and its cost function value to the predicted image selection unit 77 . In a case where the predicted image selection unit 77 has selected a predicted image generated in the optimum inter prediction mode, the motion prediction/compensation unit 76 outputs information indicating the optimum inter prediction mode to the lossless encoding unit 66 .

此时，运动向量信息、参考帧信息、指示要执行二次预测的二次预测标志、二次预测中的帧内预测模式的信息等也被输出到无损耗编码单元66。无损耗编码单元66也对来自运动预测/补偿单元76的信息进行诸如可变长度编码、算术编码等的无损耗编码处理，并且将其插入压缩图像的头部中。At this time, motion vector information, reference frame information, a secondary prediction flag indicating that secondary prediction is to be performed, information of an intra prediction mode in secondary prediction, and the like are also output to the lossless encoding unit 66 . The lossless encoding unit 66 also performs lossless encoding processing such as variable length encoding, arithmetic encoding, etc. on the information from the motion prediction/compensation unit 76, and inserts it into the header of the compressed image.

基于来自运动预测/补偿单元76的运动向量信息和要进行帧间处理的图像的信息，二次预测单元361从帧存储器72读出与要进行帧间处理的当前块邻近的当前邻近像素。此外，二次预测单元361从帧存储器72读出与通过运动向量信息同当前块相关联的参考块邻近的参考邻近像素。Based on the motion vector information from the motion prediction/compensation unit 76 and the information of the image to be inter-processed, the secondary prediction unit 361 reads out from the frame memory 72 current adjacent pixels adjacent to the current block to be inter-processed. Furthermore, the secondary prediction unit 361 reads out reference adjacent pixels adjacent to the reference block associated with the current block by the motion vector information from the frame memory 72 .

二次预测单元361执行上文参照图32描述的二次预测处理。二次预测处理是这样的处理：其中，在一次残差和当前邻近像素与参考邻近像素间的差之间执行帧内预测，从而产生二次差(二次残差)的信息。The secondary prediction unit 361 performs the secondary prediction process described above with reference to FIG. 32 . The secondary prediction process is a process in which intra prediction is performed between the primary residual and the difference between the current adjacent pixel and the reference adjacent pixel, thereby generating information of the secondary difference (secondary residual).

然而，请注意，在该二次预测之前，图42中的二次预测单元361根据来自邻近像素内插滤波控制单元561的控制信号对用于帧间预测的参考邻近像素和当前邻近像素之间的差执行(或者不执行)滤波处理。二次预测单元361然后使用当前邻近像素和参考邻近像素之间的滤波的(或未滤波的)差来执行二次预测处理，并且将获得的二次差分信息(二次残差)输出到运动预测/补偿单元76。此时，二次预测单元361也将指示是否执行滤波处理的打开/关闭标志信息输出到运动预测/补偿单元76。However, please note that before the secondary prediction, the secondary prediction unit 361 in FIG. Perform (or not perform) filtering processing on the difference. The secondary prediction unit 361 then performs secondary prediction processing using the filtered (or unfiltered) difference between the current adjacent pixel and the reference adjacent pixel, and outputs the obtained secondary difference information (secondary residual) to the motion Prediction/compensation unit 76 . At this time, the secondary prediction unit 361 also outputs on/off flag information indicating whether to perform filter processing to the motion prediction/compensation unit 76 .

也就是说，二次预测单元361包括图35中所示的帧内预测单元74。That is, the secondary prediction unit 361 includes the intra prediction unit 74 shown in FIG. 35 .

邻近像素内插滤波控制单元561基本上以与邻近像素内插滤波控制单元461相同的方式配置，并且执行相同的处理。也就是说，邻近像素内插滤波控制单元561将实现是否以块为增量或以宏块为增量执行滤波处理的控制的控制信号供应给二次预测单元361。The adjacent pixel interpolation filter control unit 561 is basically configured in the same manner as the adjacent pixel interpolation filter control unit 461 , and performs the same processing. That is, the adjacent pixel interpolation filtering control unit 561 supplies the secondary prediction unit 361 with a control signal realizing control of whether to perform filter processing in increments of blocks or in increments of macroblocks.

请注意，除了下述的帧内处理和运动预测处理以外，由图42中的图像编码装置551执行的处理基本上与图34中的图像编码装置451的处理(即，图16中的编码处理)相同，从而，将省略其描述。Note that the processing performed by the image coding device 551 in FIG. 42 is basically the same as that of the image coding device 451 in FIG. 34 (that is, the coding processing in FIG. ) are the same, and thus, description thereof will be omitted.

也就是说，采用图42中的图像编码装置551，作为帧内处理，执行根据H.264/AVC格式的帧内预测。此外，作为运动预测处理，在运动预测处理时，根据来自邻近像素内插滤波控制单元561的控制信号，控制滤波处理，从而产生滤波的(或未滤波的)二次差分信息。选择一次差分信息和二次差分信息中的具有较好的编码效率的差分信息，并且比较成本函数值，从而确定最佳帧间预测模式。That is, with the image encoding device 551 in FIG. 42 , as intra processing, intra prediction according to the H.264/AVC format is performed. Also, as motion prediction processing, at the time of motion prediction processing, filter processing is controlled based on a control signal from the adjacent pixel interpolation filter control unit 561, thereby generating filtered (or unfiltered) secondary difference information. The differential information with better encoding efficiency among the primary differential information and the secondary differential information is selected, and the cost function value is compared, so as to determine the best inter-frame prediction mode.

将参照图43描述图像解码装置，该图像解码装置接收由该图像编码装置551编码的压缩图像并解码它。An image decoding device that receives the compressed image encoded by the image encoding device 551 and decodes it will be described with reference to FIG. 43 .

[图像解码装置的其它配置例子][Other Configuration Examples of Image Decoding Device]

图43图示用作应用了本发明的图像处理装置的图像解码装置的另一实施例的配置。Fig. 43 illustrates the configuration of another embodiment of an image decoding device serving as an image processing device to which the present invention is applied.

图像解码装置601与图39中的图像解码装置501的共同之处在于包含存储缓冲器161、无损耗解码单元162、逆量化单元163、逆正交变换单元164、计算单元165、去块滤波器166、画面重新排列缓冲器167、D/A转换单元168、帧存储器169、开关170、帧内预测单元171、运动预测/补偿单元173和开关174The image decoding device 601 is common to the image decoding device 501 in FIG. 39 in that it includes a storage buffer 161, a lossless decoding unit 162, an inverse quantization unit 163, an inverse orthogonal transformation unit 164, a calculation unit 165, and a deblocking filter. 166, picture rearrangement buffer 167, D/A conversion unit 168, frame memory 169, switch 170, intra prediction unit 171, motion prediction/compensation unit 173 and switch 174

此外，图像解码装置601与图39中的图像解码装置501的不同之处在于，省略了邻近像素内插滤波控制单元511，并且，添加了图33中的二次预测单元411和邻近像素内插滤波控制单元611。Furthermore, the image decoding device 601 differs from the image decoding device 501 in FIG. 39 in that the adjacent pixel interpolation filter control unit 511 is omitted, and the secondary prediction unit 411 and the adjacent pixel interpolation control unit 511 in FIG. 33 are added. filter control unit 611 .

也就是说，将通过对头信息进行解码而获得的指示帧内预测模式的信息从无损耗解码单元162供应给帧内预测单元171。基于该信息，帧内预测单元171产生预测图像并将产生的预测图像输出到开关174。That is, information indicating the intra prediction mode obtained by decoding the header information is supplied from the lossless decoding unit 162 to the intra prediction unit 171 . Based on this information, the intra prediction unit 171 generates a predicted image and outputs the generated predicted image to the switch 174 .

将通过对头信息进行解码而获得的信息中的预测模式信息、运动向量信息和参考帧信息等从无损耗解码单元162供应给运动预测/补偿单元173。此外，在对当前块应用了二次预测处理的情况中，将二次预测标志和二次预测的帧内模式信息从无损耗解码单元162供应给运动预测/补偿单元173。The prediction mode information, motion vector information, reference frame information, and the like among the information obtained by decoding the header information are supplied from the lossless decoding unit 162 to the motion prediction/compensation unit 173 . Also, in the case where the secondary prediction process is applied to the current block, the secondary prediction flag and the secondary predicted intra mode information are supplied from the lossless decoding unit 162 to the motion prediction/compensation unit 173 .

在确定应用二次预测处理的情况中，运动预测/补偿单元173控制二次预测单元411，从而以二次预测的帧内预测模式信息指示的帧内预测模式执行二次预测。In a case where it is determined to apply the secondary prediction process, the motion prediction/compensation unit 173 controls the secondary prediction unit 411 so that secondary prediction is performed in the intra prediction mode indicated by the intra prediction mode information of the secondary prediction.

运动预测/补偿单元173基于运动向量信息和参考帧信息对该图像执行运动预测和补偿处理，并且产生预测图像。也就是说，在参考块内，使用与当前块相关联的参考块的像素值，产生当前块的预测图像。运动预测/补偿单元173然后将来自二次预测单元411的预测差分值和产生的预测图像相加，并将这些输出到开关174。The motion prediction/compensation unit 173 performs motion prediction and compensation processing on the image based on motion vector information and reference frame information, and generates a predicted image. That is, within the reference block, a predicted image of the current block is generated using pixel values of the reference block associated with the current block. The motion prediction/compensation unit 173 then adds the prediction difference value from the secondary prediction unit 411 and the generated prediction image, and outputs these to the switch 174 .

二次预测单元411使用从帧存储器169读出的当前邻近像素和参考邻近像素之间的差来执行二次预测。然而，请注意，在从邻近像素内插滤波控制单元611接收了实现执行滤波处理的控制的控制信号的情况中，二次预测单元411在该二次预测之前对当前邻近像素和参考邻近像素的差执行滤波处理。二次预测单元411然后使用经过滤波处理的当前邻近像素和参考邻近像素之间的差来执行二次预测处理，并且将获得的二次差分信息(二次残差)输出到运动预测/补偿单元173。The secondary prediction unit 411 performs secondary prediction using the difference between the current neighboring pixel and the reference neighboring pixel read out from the frame memory 169 . Note, however, that in the case where a control signal for realizing control to perform filtering processing is received from the neighboring pixel interpolation filtering control unit 611, the secondary prediction unit 411 compares the values of the current neighboring pixel and the reference neighboring pixel before the secondary prediction. Poor filter processing is performed. The secondary prediction unit 411 then performs secondary prediction processing using the difference between the filter-processed current neighboring pixel and the reference neighboring pixel, and outputs the obtained secondary difference information (secondary residual) to the motion prediction/compensation unit 173.

请注意，在从邻近像素内插滤波控制单元611接收了实现不执行滤波处理的控制的控制信号的情况中，二次预测单元411不执行滤波处理，并且使用当前邻近像素和参考邻近像素的差执行二次处理。Note that, in the case where a control signal realizing control not to perform filter processing is received from the neighboring pixel interpolation filtering control unit 611, the secondary prediction unit 411 does not perform the filtering process, and uses the difference between the current neighboring pixel and the reference neighboring pixel Perform secondary processing.

也就是说，二次预测单元411被配置为包括图40中所示的帧内预测单元171。That is, the secondary prediction unit 411 is configured to include the intra prediction unit 171 shown in FIG. 40 .

邻近像素内插滤波控制单元611基本上以与图39中的邻近像素内插滤波控制单元511相同的方式配置，并且基本上执行相同的处理。也就是说，从无损耗解码单元162向邻近像素内插滤波控制单元611供应通过对头信息解码而获得的信息中的打开/关闭标志信息。邻近像素内插滤波控制单元611根据打开/关闭标志信息供应控制信号，以使得二次预测单元411对邻近像素执行或不执行滤波处理。The adjacent pixel interpolation filter control unit 611 is basically configured in the same manner as the adjacent pixel interpolation filter control unit 511 in FIG. 39 , and basically performs the same processing. That is, the ON/OFF flag information in the information obtained by decoding the header information is supplied from the lossless decoding unit 162 to the adjacent pixel interpolation filter control unit 611 . The adjacent pixel interpolation filter control unit 611 supplies a control signal in accordance with the on/off flag information so that the secondary prediction unit 411 performs or does not perform filter processing on the adjacent pixels.

请注意，除了下述的帧内处理和运动预测处理以外，由图43中的图像解码装置601执行的处理基本上与图39中的图像解码装置501的处理(即，图24中的解码处理)相同，从而，将省略其描述。Note that the processing performed by the image decoding device 601 in FIG. 43 is basically the same as the processing of the image decoding device 501 in FIG. 39 (that is, the decoding processing in FIG. ) are the same, and thus, description thereof will be omitted.

也就是说，采用图43中的图像解码装置601，作为帧内处理，执行根据H.264/AVC格式的帧内预测。此外，作为运动预测处理，在运动预测处理时，根据来自邻近像素内插滤波控制单元611的控制信号控制滤波处理，执行二次预测(帧内预测)，并且产生二次差分信息。That is, with the image decoding device 601 in FIG. 43 , as intra processing, intra prediction according to the H.264/AVC format is performed. Also, as motion prediction processing, at the time of motion prediction processing, filter processing is controlled according to a control signal from the adjacent pixel interpolation filter control unit 611, secondary prediction (intra prediction) is performed, and secondary difference information is generated.

滤波处理的打开和关闭控制也可以适用于如上所述的具有二次预测处理的帧内预测。On and off control of filter processing can also be applied to intra prediction with secondary prediction processing as described above.

请注意，虽然在以上描述中描述了宏块的尺寸是16×16像素的情况，但是，本发明也可以适用于非专利文献3中描述的扩展的宏块尺寸。Note that although the above description describes the case where the size of the macroblock is 16×16 pixels, the present invention can also be applied to the extended macroblock size described in Non-Patent Document 3.

图44是图示扩展宏块尺寸的例子的示图。对于非专利文献3，宏块尺寸被扩展直到32×32像素。Fig. 44 is a diagram illustrating an example of extending a macroblock size. With Non-Patent Document 3, the macroblock size is extended up to 32×32 pixels.

在图44中的上层从左侧起依次地示出由被分割为32×32像素、32×16像素、16×32像素和16×16像素的块(分区)的32×32像素构成的宏块。在图44中的中层从左侧起依次地示出由被分割为16×16像素、16×8像素、8×16像素和8×8像素的块(分区)的16×16像素构成的块。在图44中的下层从左侧起依次地示出由被分割为8×8像素、8×4像素、4×8像素和4×4像素的块(分区)的8×8像素构成的块。The upper layer in FIG. 44 shows a macro composed of 32×32 pixels divided into blocks (partitions) of 32×32 pixels, 32×16 pixels, 16×32 pixels, and 16×16 pixels in order from the left. piece. The middle layer in FIG. 44 shows blocks composed of 16×16 pixels divided into blocks (partitions) of 16×16 pixels, 16×8 pixels, 8×16 pixels, and 8×8 pixels in sequence from the left . The lower layer in FIG. 44 shows blocks composed of 8×8 pixels divided into blocks (partitions) of 8×8 pixels, 8×4 pixels, 4×8 pixels, and 4×4 pixels sequentially from the left. .

换句话说，可以用在图44中的上层示出的32×32像素、32×16像素、16×32像素和16×16像素的块来处理32×32像素的宏块。In other words, a macroblock of 32×32 pixels can be processed with blocks of 32×32 pixels, 32×16 pixels, 16×32 pixels, and 16×16 pixels shown in the upper layer in FIG. 44 .

此外，以与H.264/AVC格式相同的方式，可以用在中层示出的16×16像素、16×8像素、8×16像素和8×8像素的块来处理在上层的右边示出的16×16像素的块。Also, in the same manner as the H.264/AVC format, blocks of 16×16 pixels, 16×8 pixels, 8×16 pixels, and 8×8 pixels shown in the middle layer can be processed with blocks shown on the right side of the upper layer blocks of 16x16 pixels.

此外，以与H.264/AVC格式相同的方式，可以用在下层示出的8×8像素、8×4像素、4×8像素和4×4像素的块来处理在中层的右边示出的8×8像素的块。Also, in the same manner as the H.264/AVC format, blocks of 8×8 pixels, 8×4 pixels, 4×8 pixels, and 4×4 pixels shown on the lower layer can be processed. blocks of 8x8 pixels.

采用扩展的宏块尺寸，通过利用这样的层状结构，关于16×16像素块或更小，在保持与H.264/AVC格式兼容的同时，将较大的块定义为其超集。With the extended macroblock size, by utilizing such a layered structure, a larger block is defined as a superset thereof while maintaining compatibility with the H.264/AVC format with respect to a 16×16 pixel block or smaller.

根据本发明的滤波系数设置、计算和滤波处理打开/关闭控制也可以适用于如上所述地扩展的提出的宏块尺寸。Filter coefficient setting, calculation, and filter processing on/off control according to the present invention can also be applied to proposed macroblock sizes extended as described above.

迄今为止在H.264/AVC格式用作编码格式的情况下进行了描述，但是本发明并不限于此，并且，可以利用用于使用邻近像素执行预测(例如，帧内预测或二次预测)的另一种编码格式/解码格式。The description has been made so far in the case where the H.264/AVC format is used as the encoding format, but the present invention is not limited thereto, and a method for performing prediction using adjacent pixels (for example, intra prediction or secondary prediction) can be used Another encoding format/decoding format.

请注意，例如，与MPEG、H.26X等一样，本发明可以适用于在经由诸如卫星广播、有线电视、互联网、蜂窝电话等的网络媒体接收通过诸如离散余弦变换等的正交变换和运动补偿压缩的图像信息(比特流)时使用的图像编码装置和图像解码装置。此外，本发明可以适用于在处理诸如光盘、磁盘和闪速存储器的存储介质上的图像信息时使用的图像编码装置和图像解码装置。此外，本发明可以适用于在这样的图像编码装置和图像解码装置中包含的运动预测补偿装置。Note that, for example, like MPEG, H.26X, etc., the present invention can be applied to reception via a network medium such as satellite broadcasting, cable television, the Internet, cellular phones, etc. An image encoding device and an image decoding device used when compressing image information (bit stream). Furthermore, the present invention can be applied to an image encoding device and an image decoding device used when processing image information on a storage medium such as an optical disk, a magnetic disk, and a flash memory. Furthermore, the present invention can be applied to a motion prediction compensation device included in such an image encoding device and an image decoding device.

上述的一系列处理可以通过硬件来执行，或者可以通过软件来执行。在通过软件执行该一系列处理的情况中，构成该软件的程序被安装在计算机中。这里，计算机的例子包括置于专用硬件中的计算机和通用个人计算机，由此，可以通过在其中安装各种程序来执行各种功能。The series of processing described above can be executed by hardware, or can be executed by software. In the case of executing the series of processes by software, a program constituting the software is installed in a computer. Here, examples of the computer include a computer built in dedicated hardware and a general-purpose personal computer, whereby various functions can be performed by installing various programs therein.

图45是图示使用程序执行上述的一系列处理的计算机的硬件的配置例子的框图。FIG. 45 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing using a program.

对于计算机，CPU(中央处理单元)801、ROM(只读存储器)802和RAM(随机存取存储器)803通过总线804相互连接。For a computer, a CPU (Central Processing Unit) 801 , a ROM (Read Only Memory) 802 , and a RAM (Random Access Memory) 803 are connected to each other via a bus 804 .

此外，输入/输出接口805连接到总线804。输入单元86、输出单元807、存储单元808、通信单元809和驱动器810与输入/输出接口805连接。Furthermore, an input/output interface 805 is connected to the bus 804 . An input unit 86 , an output unit 807 , a storage unit 808 , a communication unit 809 , and a driver 810 are connected to the input/output interface 805 .

输入单元806由键盘、鼠标、麦克风等构成。输出单元807由显示器、扬声器等构成。存储单元807由硬盘、非易失性存储器等构成。通信单元809由网络接口等构成。驱动器810驱动诸如磁盘、光盘、磁光盘、半导体存储器等的可移动介质811。The input unit 806 is constituted by a keyboard, a mouse, a microphone, and the like. The output unit 807 is constituted by a display, a speaker, and the like. The storage unit 807 is constituted by a hard disk, a nonvolatile memory, and the like. The communication unit 809 is constituted by a network interface and the like. The drive 810 drives a removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like.

对于这样配置的计算机，例如，CPU 801将存储于存储单元808中的程序经由输入/输出接口805和总线804加载到RAM 803，执行该程序，从而执行上述的一系列处理。With such a configured computer, for example, the CPU 801 loads a program stored in the storage unit 808 to the RAM 803 via the input/output interface 805 and the bus 804 , executes the program, thereby performing the series of processes described above.

计算机(CPU 801)执行的程序可以通过被记录在用作封装介质等的可移动介质811中来被提供。此外，程序可以经由诸如局域网、互联网或数字卫星广播的有线或无线传输媒体来被提供。A program executed by a computer (CPU 801 ) can be provided by being recorded in a removable medium 811 serving as a package medium or the like. Also, the program can be provided via wired or wireless transmission media such as a local area network, the Internet, or digital satellite broadcasting.

对于计算机，通过将可移动介质808安装到驱动器810上，可以将程序经由输入/输出接口805安装到存储单元808中。此外，该程序可以经由有线或无线传输媒体在通信单元809处被接收，并且被安装在存储单元808中。For the computer, by mounting the removable medium 808 on the drive 810 , the program can be installed into the storage unit 808 via the input/output interface 805 . Also, the program can be received at the communication unit 809 via a wired or wireless transmission medium, and installed in the storage unit 808 .

另外，可以将程序预先安装在ROM 802或存储单元808中。In addition, the program may be preinstalled in the ROM 802 or the storage unit 808 .

请注意，计算机执行的程序可以是其中沿着本说明书中描述的顺序按照时间序列执行处理的程序，或者可以是其中其中并行地或在诸如执行呼叫的必要定时处执行处理的程序。Note that the computer-executed program may be a program in which processing is performed in time series along the order described in this specification, or may be a program in which processing is performed in parallel or at necessary timing such as executing a call.

本发明的实施例并不限于上述的实施例，并且，在不脱离本发明的实质的情况中可以进行各种修改。Embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

例如，上述的图像编码装置51、351、451和551以及图像解码装置151、401、501和601可以适用于可任选的电子装置。在下文中，将描述其例子。For example, the image encoding devices 51, 351, 451, and 551 and the image decoding devices 151, 401, 501, and 601 described above may be applied to optional electronic devices. Hereinafter, examples thereof will be described.

图46是图示使用应用了本发明的图像解码装置的电视接收器的主要配置例子的框图。Fig. 46 is a block diagram illustrating a main configuration example of a television receiver using an image decoding device to which the present invention is applied.

图46中所示的电视接收器1300包括地面调谐器1313、视频解码器1315、视频信号处理电路1318、图形产生电路1319、面板驱动电路1320和显示面板1321。A television receiver 1300 shown in FIG. 46 includes a terrestrial tuner 1313 , a video decoder 1315 , a video signal processing circuit 1318 , a graphics generating circuit 1319 , a panel driving circuit 1320 and a display panel 1321 .

地面调谐器1313经由天线接收地面模拟广播的广播波信号，对该广播波信号进行解调，获得视频信号，并且将这些视频信号供应给视频解码器1315。The terrestrial tuner 1313 receives broadcast wave signals of terrestrial analog broadcasts via an antenna, demodulates the broadcast wave signals, obtains video signals, and supplies the video signals to the video decoder 1315 .

视频解码器1315对从地面调谐器1313供应的视频信号进行解码处理，并且将获得的数字成分信号供应给视频信号处理电路1318。The video decoder 1315 performs decoding processing on the video signal supplied from the terrestrial tuner 1313 , and supplies the obtained digital component signal to the video signal processing circuit 1318 .

视频信号处理电路1318对从视频解码器1315供应的视频数据进行诸如噪声去除等的预定处理，并且，将获得的视频数据供应给图形产生电路1319。The video signal processing circuit 1318 performs predetermined processing such as noise removal on the video data supplied from the video decoder 1315 , and supplies the obtained video data to the graphic generation circuit 1319 .

图形产生电路1319产生要显示在显示面板1321上的节目的视频数据，或者由于基于要经由网络供应的应用的处理而导致的图像数据，并且，将产生的视频数据或图像数据供应给面板驱动电路1320。此外，图形产生电路1319也执行诸如将通过为用户产生视频数据(图形)而获得的视频数据在适当的时候供应给面板驱动电路1320的处理，所述视频数据显示用于项目等的选择的画面，并将其重叠在节目的视频数据上。The graphic generating circuit 1319 generates video data of a program to be displayed on the display panel 1321, or image data due to processing based on an application to be supplied via the network, and supplies the generated video data or image data to the panel driving circuit 1320. In addition, the graphic generating circuit 1319 also performs processing such as supplying to the panel driving circuit 1320 video data obtained by generating video data (graphics) for the user displaying a screen for selection of an item or the like at an appropriate time. , and overlay it on the show's video data.

面板驱动电路1320基于从图形产生电路1319供应的数据来驱动显示面板1321，以在显示面板1321上显示节目的视频或者上述的各种画面。The panel driving circuit 1320 drives the display panel 1321 based on the data supplied from the graphics generating circuit 1319 to display video of a program or the above-mentioned various screens on the display panel 1321 .

显示面板1321由LCD(液晶显示器)等构成，并且根据面板驱动电路1320的控制显示节目的视频等。The display panel 1321 is constituted by an LCD (Liquid Crystal Display) or the like, and displays video of a program or the like according to control of the panel drive circuit 1320 .

此外，电视接收器1300还包括音频A/D(模拟/数字)转换单元1314、音频信号处理电路1322、回声消除/音频合成电路1323和音频放大器电路1324和扬声器1325。In addition, the television receiver 1300 includes an audio A/D (Analog/Digital) conversion unit 1314 , an audio signal processing circuit 1322 , an echo cancellation/audio synthesis circuit 1323 and an audio amplifier circuit 1324 and a speaker 1325 .

地面调谐器1313对接收到的广播波信号进行解调，从而不仅获得视频信号还获得音频信号。地面调谐器1313将获得的音频信号供应给音频A/D转换电路1314。The terrestrial tuner 1313 demodulates the received broadcast wave signal to obtain not only a video signal but also an audio signal. The terrestrial tuner 1313 supplies the obtained audio signal to the audio A/D conversion circuit 1314 .

该音频A/D转换电路1314对从地面调谐器1313供应的音频信号进行A/D转换处理，并且，将获得的数字音频信号供应给音频信号处理电路1322。The audio A/D conversion circuit 1314 performs A/D conversion processing on the audio signal supplied from the terrestrial tuner 1313 , and supplies the obtained digital audio signal to the audio signal processing circuit 1322 .

音频信号处理电路1322对从音频A/D转换电路1314供应的音频数据进行诸如噪声去除等的预定处理，并且，将获得的音频数据供应给回声消除/音频合成电路1323。The audio signal processing circuit 1322 performs predetermined processing such as noise removal on the audio data supplied from the audio A/D conversion circuit 1314 , and supplies the obtained audio data to the echo cancellation/audio synthesis circuit 1323 .

回声消除/音频合成电路1323将从音频信号处理电路1322供应的音频数据供应给音频放大器电路1324。The echo cancellation/audio synthesis circuit 1323 supplies the audio data supplied from the audio signal processing circuit 1322 to the audio amplifier circuit 1324 .

音频放大器电路1324对从回声消除/音频合成电路1323供应的音频数据进行D/A转换处理，进行放大器处理以调节到预定的音量，然后，将从扬声器1325输出音频。The audio amplifier circuit 1324 performs D/A conversion processing on the audio data supplied from the echo canceling/audio synthesizing circuit 1323 , performs amplifier processing to adjust to a predetermined volume, and then, will output audio from the speaker 1325 .

此外，电视接收器1300还包括数字调谐器1316和MPEG解码器1317。In addition, the television receiver 1300 also includes a digital tuner 1316 and an MPEG decoder 1317 .

数字调谐器1316经由天线接收数字广播(地面数字广播、BS(广播卫星)/CS(通信卫星)数字广播)的广播波信号，对其进行解调以获得MPEG-TS(运动图片专家组-传输流)，并且将其供应给MPEG解码器1317。The digital tuner 1316 receives a broadcast wave signal of digital broadcasting (terrestrial digital broadcasting, BS (Broadcasting Satellite)/CS (Communication Satellite) digital broadcasting) via an antenna, demodulates it to obtain MPEG-TS (Moving Picture Experts Group-Transmission stream), and supply it to the MPEG decoder 1317.

MPEG解码器1317对给予从数字调谐器1316供应的MPEG-TS的加扰进行解扰，并且提取包含用作重放目标(观看目标)的节目的数据的流。MPEG解码器1317对构成提取的流的音频分组进行解码，将获得的音频数据供应给视频信号处理电路1322，并且，还对构成流的视频分组进行解码，并将获得的视频数据供应给视频信号处理电路1318。此外，MPEG解码器1317将从MPEG-TS提取的EPG(电子节目指南)数据经由未示出的路径供应给CPU 1332。The MPEG decoder 1317 descrambles the scramble given to the MPEG-TS supplied from the digital tuner 1316, and extracts a stream containing data of a program serving as a playback target (viewing target). The MPEG decoder 1317 decodes the audio packets constituting the extracted stream, supplies the obtained audio data to the video signal processing circuit 1322, and also decodes the video packets constituting the stream, and supplies the obtained video data to the video signal processing circuitry 1318 . Also, the MPEG decoder 1317 supplies EPG (Electronic Program Guide) data extracted from the MPEG-TS to the CPU 1332 via a path not shown.

电视接收器1300使用上述的图像解码装置151、401、501或601，作为用于以这样的方式对视频分组进行解码的MPEG解码器1317。因此，以与图像编码装置151和401相同的方式，MPEG解码器1317根据量化参数和预测模式切换滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像编码装置501和601相同的方式，MPEG解码器1317基于打开/关闭标志控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。The television receiver 1300 uses the above-described image decoding device 151, 401, 501, or 601 as the MPEG decoder 1317 for decoding video packets in this manner. Therefore, in the same manner as the image encoding devices 151 and 401, the MPEG decoder 1317 switches filter coefficients according to quantization parameters and prediction modes, and, before intra prediction, performs filter processing on adjacent pixels. Alternatively, in the same manner as the image encoding devices 501 and 601 , the MPEG decoder 1317 controls whether or not to perform filter processing on neighboring pixels before intra prediction based on an on/off flag. Therefore, coding efficiency can be improved.

以与从视频解码器1315供应的视频数据的情况相同的方式，在视频信号处理电路1318处对从MPEG解码器1317供应的视频数据进行预定的处理。然后，在适当的时候，在图形产生电路1319处，将经过预定处理的视频数据重叠在产生的视频数据等上，经由面板驱动电路1320将该视频数据供应给显示面板1321，并且，在显示面板1321上显示其图像。The video data supplied from the MPEG decoder 1317 is subjected to predetermined processing at the video signal processing circuit 1318 in the same manner as in the case of the video data supplied from the video decoder 1315 . Then, at an appropriate time, at the graphics generating circuit 1319, the video data subjected to predetermined processing is superimposed on the generated video data, etc., the video data is supplied to the display panel 1321 via the panel driving circuit 1320, and, on the display panel 1321 to display its image.

以与从音频A/D转换电路1314供应的音频数据的情况相同的方式，在音频信号处理电路1322处对从MPEG解码器1317供应的音频数据进行预定的处理。经过预定处理的音频数据然后经由回声消除/音频合成电路1323被供应给音频放大器电路1324，并且经过D/A转换处理和放大器处理。作为其结果，将以预定音量调节的音频从扬声器1325输出。The audio data supplied from the MPEG decoder 1317 is subjected to predetermined processing at the audio signal processing circuit 1322 in the same manner as in the case of the audio data supplied from the audio A/D conversion circuit 1314 . The audio data subjected to predetermined processing is then supplied to the audio amplifier circuit 1324 via the echo canceling/audio synthesizing circuit 1323, and undergoes D/A conversion processing and amplifier processing. As a result thereof, audio adjusted at a predetermined volume is output from the speaker 1325 .

此外，电视接收器1300还包含麦克风1326和A/D转换电路1327。In addition, the television receiver 1300 also includes a microphone 1326 and an A/D conversion circuit 1327 .

A/D转换电路1327接收由设置给电视接收器1300的麦克风1326收集的用户的音频信号，用于音频转换。A/D转换电路1327对接收到的音频信号进行A/D转换处理，并且，将获得的数字音频数据供应给回声消除/音频合成电路1323。The A/D conversion circuit 1327 receives the user's audio signal collected by the microphone 1326 provided to the television receiver 1300 for audio conversion. The A/D conversion circuit 1327 performs A/D conversion processing on the received audio signal, and supplies the obtained digital audio data to the echo cancellation/audio synthesis circuit 1323 .

在从A/D转换电路1327供应了电视接收器1300的用户(用户A)的音频数据的情况中，回声消除/音频合成电路1323在以用户A的音频数据作为目标的情况下执行回声消除。在回声消除之后，回声消除/音频合成电路1323将通过合成用户A的音频数据和其它音频数据等而获得的音频数据经由音频放大器电路1324从扬声器1325输出。In the case where the audio data of the user (user A) of the television receiver 1300 is supplied from the A/D conversion circuit 1327, the echo cancellation/audio synthesis circuit 1323 performs echo cancellation with the audio data of the user A as the target. After echo cancellation, the echo cancellation/audio synthesis circuit 1323 outputs audio data obtained by synthesizing user A's audio data and other audio data, etc., from the speaker 1325 via the audio amplifier circuit 1324 .

此外，电视接收器1300还包括音频编解码器1328、内部总线1329、SDRAM(同步动态随机存取存储器)1330、闪速存储器1331、CPU 1332、USB(通用串行总线)I/F 1333和网络I/F 1334。In addition, the television receiver 1300 includes an audio codec 1328, an internal bus 1329, an SDRAM (Synchronous Dynamic Random Access Memory) 1330, a flash memory 1331, a CPU 1332, a USB (Universal Serial Bus) I/F 1333, and a network I/F 1334.

A/D转换电路1327接收由设置给电视接收器1300的麦克风1326收集的用户的音频信号，用于音频转换。A/D转换电路1327对接收到的音频信号进行A/D转换处理，并且，将获得的数字音频数据供应给音频编解码器1328。The A/D conversion circuit 1327 receives the user's audio signal collected by the microphone 1326 provided to the television receiver 1300 for audio conversion. The A/D conversion circuit 1327 performs A/D conversion processing on the received audio signal, and supplies the obtained digital audio data to the audio codec 1328 .

音频编解码器1328将从A/D转换电路1327供应的音频数据转换为预定格式的数据以便经由网络传送，并且，将其经由内部总线1329供应给网络I/F 1334。The audio codec 1328 converts the audio data supplied from the A/D conversion circuit 1327 into data of a predetermined format for transmission via the network, and supplies it to the network I/F 1334 via the internal bus 1329 .

网络I/F 1334经由安装在网络端子1335上的线缆与网络连接。例如，网络I/F1334将从音频编解码器1328供应的音频数据传送到与其网络连接的另一装置。此外，例如，网络I/F 1334经由网络端子1335接收从经由网络与其连接的另一装置传送的音频数据，并且将其经由内部总线1329供应给音频编解码器1328。The network I/F 1334 is connected to the network via a cable attached to the network terminal 1335 . For example, the network I/F 1334 transfers the audio data supplied from the audio codec 1328 to another device connected to its network. Also, for example, the network I/F 1334 receives audio data transmitted from another device connected thereto via the network via the network terminal 1335 , and supplies it to the audio codec 1328 via the internal bus 1329 .

音频编解码器1328将从网络I/F 1334供应的音频数据转换为预定格式的数据，并且将其供应给回声消除/音频合成电路1323。The audio codec 1328 converts the audio data supplied from the network I/F 1334 into data of a predetermined format, and supplies it to the echo cancellation/audio synthesis circuit 1323 .

回声消除/音频合成电路1323在以从音频编解码器1328供应的音频数据取为目标的情况中执行回声消除，并且，经由音频放大器电路1324从扬声器1325输出通过合成该音频数据和其它音频数据等获得的音频的数据。The echo cancellation/audio synthesis circuit 1323 performs echo cancellation in the case of targeting audio data supplied from the audio codec 1328, and outputs the audio data by synthesizing the audio data and other audio data, etc., from the speaker 1325 via the audio amplifier circuit 1324. Get the audio data.

SDRAM 1330存储CPU 1332执行处理所需的各种数据。The SDRAM 1330 stores various data necessary for the CPU 1332 to perform processing.

闪速存储器1331存储要由CPU 1332执行的程序。通过在诸如激活电视接收器1300等的预定定时处由CPU 1332读出在闪速存储器1331中存储的程序。经由数字广播获得的EPG数据、经由网络从预定服务器获得的数据等也被存储在闪速存储器1331中。The flash memory 1331 stores programs to be executed by the CPU 1332 . The program stored in the flash memory 1331 is read out by the CPU 1332 at a predetermined timing such as activation of the television receiver 1300 or the like. EPG data obtained via digital broadcasting, data obtained from a predetermined server via a network, and the like are also stored in the flash memory 1331 .

例如，包含通过CPU 1331的控制经由网络从预定服务器获得的内容数据的MPEG-TS被存储在闪速存储器1331中。例如，通过CPU1332的控制，经由内部总线1329，闪速存储器1331将其MPEG-TS供应给MPEG解码器1317。For example, MPEG-TS containing content data obtained from a predetermined server via a network by the control of the CPU 1331 is stored in the flash memory 1331 . For example, the flash memory 1331 supplies its MPEG-TS to the MPEG decoder 1317 via the internal bus 1329 under the control of the CPU 1332 .

MPEG解码器1317以与从数字调谐器1316供应的MPEG-TS的情况相同的方式来处理其MPEG-TS。以这样的方式，电视接收器1300经由网络接收由视频、音频等构成的内容数据，使用MPEG解码器1317进行解码，从而可以显示其视频，并且，可以输出其音频。The MPEG decoder 1317 processes the MPEG-TS thereof in the same manner as the case of the MPEG-TS supplied from the digital tuner 1316 . In this way, the television receiver 1300 receives content data composed of video, audio, etc. via the network, decodes it using the MPEG decoder 1317, so that its video can be displayed, and its audio can be output.

此外，电视接收器1300还包含用于接收从遥控器1351发射的红外信号的光接收单元1337。In addition, the television receiver 1300 also includes a light receiving unit 1337 for receiving an infrared signal transmitted from the remote controller 1351 .

光接收单元1337从遥控器1351接收红外线，并且，将通过解调获得的表示用户的操作的内容的控制代码输出给CPU 1332。The light receiving unit 1337 receives infrared rays from the remote controller 1351 , and outputs, to the CPU 1332 , a control code indicating the content of the user's operation obtained through demodulation.

CPU 1332根据从光接收单元1337供应的控制代码等执行在闪速存储器1331中存储的程序，以控制电视接收器1300的整个操作。CPU1332和电视接收器1300的各个单元经由未示出的路径连接。The CPU 1332 executes programs stored in the flash memory 1331 in accordance with control codes and the like supplied from the light receiving unit 1337 to control the entire operation of the television receiver 1300 . The CPU 1332 and respective units of the television receiver 1300 are connected via paths not shown.

USB I/F 1333对经由安装在USB端子1336上的USB线缆连接的电视接收器1300的外部装置执行数据的发送/接收。网络I/F 1334经由安装在网络端子1335上的线缆与网络连接，还对与网络连接的各种装置执行除音频数据以外的数据的发送/接收。The USB I/F 1333 performs transmission/reception of data to an external device of the television receiver 1300 connected via a USB cable mounted on the USB terminal 1336 . The network I/F 1334 is connected to the network via a cable mounted on the network terminal 1335, and also performs transmission/reception of data other than audio data to various devices connected to the network.

电视接收器1300使用图像解码装置151、401、501或601作为MPEG解码器1317，从而可以提高编码效率。作为其结果，电视接收器1300可以以更高的速度从经由天线接收到的广播波信号或者经由网络获得的内容数据中获得具有更高精度的解码图像并显示它。The television receiver 1300 uses the image decoding device 151, 401, 501, or 601 as the MPEG decoder 1317, so that encoding efficiency can be improved. As a result thereof, the television receiver 1300 can obtain a decoded image with higher precision from a broadcast wave signal received via an antenna or content data obtained via a network at a higher speed and display it.

图47是图示使用应用了本发明的图像编码装置和图像解码装置的蜂窝电话的主要配置例子的框图。Fig. 47 is a block diagram illustrating a main configuration example of a cellular phone using an image encoding device and an image decoding device to which the present invention is applied.

图47中所示的蜂窝电话1400包括被配置为整体地控制各个单元的主控制单元1450、电源电路单元1451、操作输入控制单元1452、图像编码器1453、照相机I/F单元1454、LCD控制单元1455、图像解码器1456、多路复用/分离单元1457、记录/重放单元1462、调制/解调电路单元1458和音频编解码器1459。这些单元经由总线1460相互连接。A cellular phone 1400 shown in FIG. 47 includes a main control unit 1450 configured to integrally control the respective units, a power supply circuit unit 1451, an operation input control unit 1452, an image encoder 1453, a camera I/F unit 1454, an LCD control unit 1455 , image decoder 1456 , multiplexing/separating unit 1457 , recording/playback unit 1462 , modulation/demodulation circuit unit 1458 and audio codec 1459 . These units are connected to each other via a bus 1460 .

此外，蜂窝电话1400包括操作键1419、CCD(电荷耦合器件)照相机1416、液晶显示器1418、存储单元1423、发送/接收电路单元1463、天线1414、麦克风(MIC)1421和扬声器1417。Further, the cellular phone 1400 includes operation keys 1419 , a CCD (Charge Coupled Device) camera 1416 , a liquid crystal display 1418 , a storage unit 1423 , a transmission/reception circuit unit 1463 , an antenna 1414 , a microphone (MIC) 1421 and a speaker 1417 .

在呼叫结束并通过用户的操作接通电源键时，电源电路单元1451通过从电池组向各个单元供电来在操作状态中激活蜂窝电话1400。When the call ends and the power key is turned on by the user's operation, the power supply circuit unit 1451 activates the cellular phone 1400 in the operation state by supplying power from the battery pack to the respective units.

基于由CPU、ROM、RAM等构成的主控制单元1450的控制，在诸如语音呼叫模式、数据通信模式等的各种模式中，蜂窝电话1400执行各种操作，例如，音频信号的发送/接收、电子邮件和图像数据的发送/接收、图像拍摄、数据记录等。In various modes such as voice call mode, data communication mode, etc., the cellular phone 1400 performs various operations such as transmission/reception of audio signals, E-mail and image data transmission/reception, image capture, data logging, etc.

例如，在语音呼叫模式中，蜂窝电话1400通过音频编解码器1459将由麦克风(话筒)1421收集的音频信号转换为数字音频数据，在调制/解调电路单元1458处对其进行光谱扩展处理，并且，在发送/接收电路单元1463处对其进行数字/模拟转换处理和频率转换处理。蜂窝电话1400将通过其转换处理获得的用于发送的信号经由天线1414发送到未示出的基站。将发送到基站的用于发送的信号(音频信号)经由公用电话网络供应给通信伙伴的蜂窝电话。For example, in the voice call mode, the cellular phone 1400 converts the audio signal collected by the microphone (microphone) 1421 into digital audio data through the audio codec 1459, performs spectral expansion processing on it at the modulation/demodulation circuit unit 1458, and , which is subjected to digital/analog conversion processing and frequency conversion processing at the transmission/reception circuit unit 1463. Cellular phone 1400 transmits the signal for transmission obtained through its conversion process to an unillustrated base station via antenna 1414 . The signal for transmission (audio signal) transmitted to the base station is supplied to the cellular phone of the communication partner via the public telephone network.

此外，例如，在语音呼叫模式中，蜂窝电话1400在发送/接收电路单元1463处对在天线1414处接收的接收信号进行放大，进一步对其进行频率转换处理和模拟/数字转换处理，在调制/解调电路单元1458处对其进行光谱逆扩展处理，并且通过音频编解码器将其转换为模拟音频信号。蜂窝电话1400从扬声器1417输出其转换的和获得的模拟音频信号。In addition, for example, in the voice call mode, the cellular phone 1400 amplifies the reception signal received at the antenna 1414 at the transmission/reception circuit unit 1463, further performs frequency conversion processing and analog/digital conversion processing on it, The demodulation circuit unit 1458 performs spectrum inverse expansion processing on it, and converts it into an analog audio signal through an audio codec. The cellular phone 1400 outputs its converted and obtained analog audio signal from the speaker 1417 .

此外，例如，当在数据通信模式中发送电子邮件的情况中，蜂窝电话1400在操作输入控制单元1452处接受通过操作键1419的操作而输入的电子邮件的文本数据。蜂窝电话1400在主控制单元1450处处理其文本数据，并且经由LCD控制单元1455将其作为图像显示在液晶显示器1418上。Also, for example, in the case of sending e-mail in the data communication mode, the cellular phone 1400 accepts text data of e-mail input by operation of the operation key 1419 at the operation input control unit 1452 . The cellular phone 1400 processes its text data at the main control unit 1450 and displays it as an image on the liquid crystal display 1418 via the LCD control unit 1455 .

此外，蜂窝电话1400基于由操作输入控制单元1452接受的文本数据、用户的指示等在主控制单元1450处产生电子邮件数据。蜂窝电话1400在调制/解调电路单元1458处对其电子邮件数据进行光谱扩展处理，并且在发送/接收电路单元1463处对其进行数字/模拟转换处理和频率转换处理。蜂窝电话1400将通过其转换处理获得的用于发送的信号经由天线1414发送到未示出的基站。将发送到基站的用于发送的信号(电子邮件)经由网络、邮件服务器等供应给预定的目的地。Furthermore, the cellular phone 1400 generates e-mail data at the main control unit 1450 based on text data accepted by the operation input control unit 1452, a user's instruction, and the like. The cellular phone 1400 subjects its e-mail data to spectrum spreading processing at the modulation/demodulation circuit unit 1458 , and to digital/analog conversion processing and frequency conversion processing at the transmission/reception circuit unit 1463 . Cellular phone 1400 transmits the signal for transmission obtained through its conversion process to an unillustrated base station via antenna 1414 . A signal for transmission (e-mail) transmitted to the base station is supplied to a predetermined destination via a network, a mail server, or the like.

此外，例如，当在数据通信模式中接收电子邮件的情况中，蜂窝电话1400用发送/接收电路单元1463接收经由天线1414从基站发送的信号，对该信号进行放大，并且，对其进一步进行频率转换处理和模拟/数字转换处理。蜂窝电话1400在调制/解调电路单元1458处对其接收信号进行光谱逆扩展处理，以恢复原始电子邮件数据。蜂窝电话1400经由LCD控制单元1455在液晶显示器1418上显示恢复的电子邮件数据。Also, for example, in the case of receiving e-mail in the data communication mode, the cellular phone 1400 receives a signal transmitted from the base station via the antenna 1414 with the transmission/reception circuit unit 1463, amplifies the signal, and further performs frequency analysis on it. conversion processing and analog/digital conversion processing. The cellular phone 1400 performs spectrum despreading processing on its received signal at the modulation/demodulation circuit unit 1458 to restore the original email data. The cellular phone 1400 displays the recovered email data on the liquid crystal display 1418 via the LCD control unit 1455 .

请注意，蜂窝电话1400可以经由记录/重放单元1462在存储单元1423中记录(存储)接收到的电子邮件数据。Note that the cellular phone 1400 can record (store) received e-mail data in the storage unit 1423 via the recording/playback unit 1462 .

该存储单元1423是可任选的可重写的记录介质。存储单元1423可以是诸如RAM、内置闪速存储器等的半导体存储器，可以是硬盘，或者可以是诸如磁盘、磁光盘、光盘、USB存储器、存储器卡等的可移动介质。不用说，存储单元1423可以是除了这些以外的存储单元。The storage unit 1423 is an optional rewritable recording medium. The storage unit 1423 may be a semiconductor memory such as a RAM, a built-in flash memory, etc., may be a hard disk, or may be a removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, a memory card, or the like. It goes without saying that the storage unit 1423 may be a storage unit other than these.

此外，例如，当在数据通信模式中发送图像数据的情况中，蜂窝电话1400通过在CCD照相机1416处成像来产生图像数据。CCD照相机1416包括用作诸如透镜、光圈等的光学装置且用作光电转换装置的对被摄体进行成像的CCD，将接收到的光的强度转换为电信号，并且产生被摄体的图像的图像数据。经由照相机I/F单元1451，使用诸如MPEG2、MPEG4等的预定的编码格式，在图像编码器1453处对其图像数据进行压缩编码，因此，将其图像数据转换为编码的图像数据。Also, for example, the cellular phone 1400 generates image data by imaging at the CCD camera 1416 when image data is transmitted in the data communication mode. The CCD camera 1416 includes a CCD that images a subject as an optical device such as a lens, a diaphragm, etc. and serves as a photoelectric conversion device, converts the intensity of received light into an electrical signal, and generates an image of the subject. image data. The image data thereof is compression-encoded at the image encoder 1453 via the camera I/F unit 1451 using a predetermined encoding format such as MPEG2, MPEG4, etc., and thus converted into encoded image data.

蜂窝电话1400利用上述的图像编码装置61、351、451和551作为用于执行这种处理的图像编码器1453。因此，以与图像编码装置51和351相同的方式，图像编码器1453根据量化参数和预测模式设置滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像编码装置451和551相同的方式，图像编码器1453控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。The cellular phone 1400 utilizes the above-described image encoding devices 61, 351, 451, and 551 as the image encoder 1453 for performing such processing. Therefore, in the same manner as the image encoding devices 51 and 351, the image encoder 1453 sets filter coefficients according to quantization parameters and prediction modes, and, before intra prediction, performs filter processing on adjacent pixels. Alternatively, in the same manner as the image encoding devices 451 and 551 , the image encoder 1453 controls whether to perform filtering processing on adjacent pixels before intra prediction. Therefore, coding efficiency can be improved.

请注意，此时，同时，在用CCD照相机1416拍摄的同时，蜂窝电话1400在音频编解码器1459处对在麦克风(话筒)1421处收集的音频进行模拟数字转换，并且进一步对其进行编码。Please note that at this time, while shooting with the CCD camera 1416, the cellular phone 1400 performs analog-to-digital conversion at the audio codec 1459 on the audio collected at the microphone (microphone) 1421 and further encodes it.

蜂窝电话1400使用预定的方法在多路复用/分离单元1457处对从图像编码器1453供应的编码图像数据和从音频编解码器1459供应的数字音频数据进行多路复用。蜂窝电话1400在调制/解调电路单元1458处对作为其结果获得的多路复用数据进行光谱扩展处理，并且在发送/接收电路单元1463处对其进行数字/模拟转换处理和频率转换处理。蜂窝电话1400将通过其转换处理获得的用于发送的信号经由天线1414发送到未示出的基站。将发送到基站的用于发送的信号(图像信号)经由网络等供应给通信伙伴。The cellular phone 1400 multiplexes the encoded image data supplied from the image encoder 1453 and the digital audio data supplied from the audio codec 1459 at the multiplexing/demultiplexing unit 1457 using a predetermined method. The cellular phone 1400 subjects the multiplexed data obtained as a result thereof to spectrum extension processing at the modulation/demodulation circuit unit 1458 , and to digital/analog conversion processing and frequency conversion processing at the transmission/reception circuit unit 1463 . Cellular phone 1400 transmits the signal for transmission obtained through its conversion process to an unillustrated base station via antenna 1414 . The signal for transmission (image signal) transmitted to the base station is supplied to a communication partner via a network or the like.

请注意，在不发送图像数据的情况中，蜂窝电话1400也可以经由LCD控制单元1455而不是图像编码器1453在液晶显示器1418上显示在CCD照相机1416处产生的图像数据。Note that the cellular phone 1400 can also display the image data generated at the CCD camera 1416 on the liquid crystal display 1418 via the LCD control unit 1455 instead of the image encoder 1453 in the case of not transmitting image data.

此外，例如，当在数据通信模式中接收与简单网站等链接的运动图像文件的数据的情况中，蜂窝电话1400经由天线1414在发送/接收电路单元1463处接收从基站发送的信号，对该信号进行放大，并且，对其进一步进行频率转换处理和模拟/数字转换处理。Also, for example, in the case of receiving data of a moving picture file linked to a simple website or the like in the data communication mode, the cellular phone 1400 receives a signal transmitted from the base station at the transmission/reception circuit unit 1463 via the antenna 1414, and the signal Amplification is performed, and it is further subjected to frequency conversion processing and analog/digital conversion processing.

蜂窝电话1400在调制/解调电路单元1458处对接收到的信号进行光谱逆扩展处理，以恢复原始的多路复用数据。蜂窝电话1400在多路复用/分离单元1457处将其多路复用数据分离为编码的图像数据和音频数据。The cellular phone 1400 performs spectrum despreading processing on the received signal at the modulation/demodulation circuit unit 1458 to restore the original multiplexed data. The cellular phone 1400 separates its multiplexed data into encoded image data and audio data at the multiplexing/separating unit 1457 .

蜂窝电话1400使用与诸如MPEG2、MPEG4等的预定的编码格式对应的解码格式在图像解码器1456处对编码的图像数据进行解码，从而产生重放的运动图像数据，并且，蜂窝电话1400经由LCD控制单元1455在液晶显示器1418上显示该运动图像数据。因此，例如，在液晶显示器1418上显示与简单网站链接的运动图像文件中包含的运动图像数据。The cellular phone 1400 decodes the encoded image data at the image decoder 1456 using a decoding format corresponding to a predetermined encoding format such as MPEG2, MPEG4, etc., thereby generating reproduced moving image data, and the cellular phone 1400 is controlled via the LCD. Unit 1455 displays the moving image data on liquid crystal display 1418 . Thus, for example, moving image data contained in a moving image file linked to a simple website is displayed on the liquid crystal display 1418 .

蜂窝电话1400利用上述的图像解码装置151、401、501或601作为用于执行这种处理的图像解码器1456。因此，以与图像解码装置151和401相同的方式，图像解码器1456根据量化参数和预测模式切换滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像解码装置501和601相同的方式，图像解码器1456基于打开/关闭标志控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。The cellular phone 1400 utilizes the image decoding device 151, 401, 501, or 601 described above as the image decoder 1456 for performing such processing. Therefore, in the same manner as the image decoding devices 151 and 401 , the image decoder 1456 switches filter coefficients according to quantization parameters and prediction modes, and, before intra prediction, performs filter processing on adjacent pixels. Alternatively, in the same manner as the image decoding devices 501 and 601 , the image decoder 1456 controls whether or not to perform filter processing on neighboring pixels before intra prediction based on an on/off flag. Therefore, coding efficiency can be improved.

此时，同时，蜂窝电话1400在音频编解码器1459处将数字音频数据转换为模拟音频信号，并且从扬声器1417处输出它。因此，例如，播放与简单网站链接的运动图像文件中包含的音频数据。At this time, at the same time, the cellular phone 1400 converts the digital audio data into an analog audio signal at the audio codec 1459 and outputs it from the speaker 1417 . Therefore, for example, audio data contained in a moving picture file linked to a simple website is played.

请注意，以与电子邮件的情况相同的方式，蜂窝电话1400可以经由记录/重放单元1462将与简单网站等链接的接收到的数据记录(存储)在存储单元1423中。Note that the cellular phone 1400 can record (store) received data linked to a simple website or the like in the storage unit 1423 via the recording/playback unit 1462 in the same manner as in the case of e-mail.

此外，蜂窝电话1400在主控制单元1450处对由CCD照相机1416获得的成像的二维代码进行分析，从而可以获得在二维代码中记录的信息。In addition, the cellular phone 1400 analyzes the imaged two-dimensional code obtained by the CCD camera 1416 at the main control unit 1450 so that information recorded in the two-dimensional code can be obtained.

此外，蜂窝电话1400可以使用红外线在红外通信单元1481处与外部装置通信。Also, the cellular phone 1400 can communicate with an external device at the infrared communication unit 1481 using infrared rays.

例如，蜂窝电话1400利用图像编码装置51、351、451或551作为图像编码器1453，从而，可以提高通过对在CCD照相机1416处产生的图像数据进行编码而产生的编码数据的编码效率。结果，蜂窝电话1400可以向另一个装置提供具有优异的编码效率的编码数据(图像数据)。For example, the cellular phone 1400 utilizes the image encoding device 51, 351, 451, or 551 as the image encoder 1453, whereby encoding efficiency of encoded data generated by encoding image data generated at the CCD camera 1416 can be improved. As a result, the cellular phone 1400 can provide encoded data (image data) with excellent encoding efficiency to another device.

此外，蜂窝电话1400利用图像解码装置151、401、501或601作为图像解码器1456，从而可以产生具有高精度的预测图像。例如，作为其结果，蜂窝电话1400可以从与简单网站链接的运动图像文件获得具有更高精度的解码图像，并且显示它。Furthermore, the cellular phone 1400 utilizes the image decoding device 151, 401, 501, or 601 as the image decoder 1456, so that a predicted image with high precision can be generated. For example, as a result thereof, the cellular phone 1400 can obtain a decoded image with higher precision from a moving image file linked to a simple website, and display it.

请注意，迄今为止已经进行了这样的描述：其中，蜂窝电话1400利用CCD照相机1416，但是，蜂窝电话1400可以利用使用CMOS(互补金属氧化物半导体)的图像传感器(CMOS图像传感器)代替该CCD照相机1416。在这种情况下，以与利用CCD照相机1416的情况相同的方式，蜂窝电话1400也可以对被摄体进行成像并产生被摄体的图像的图像数据。Note that the description has been made so far in which the cellular phone 1400 utilizes the CCD camera 1416, however, the cellular phone 1400 may utilize an image sensor (CMOS image sensor) using CMOS (Complementary Metal Oxide Semiconductor) instead of the CCD camera 1416. In this case, also in the same manner as the case of using the CCD camera 1416, the cellular phone 1400 can image a subject and generate image data of an image of the subject.

此外，迄今为止已经针对蜂窝电话1400进行了描述，但是，以与蜂窝电话1400的情况相同的方式，图像编码装置51、351、451和55，以及图像解码装置151、401、501和601可以适用于任何类型的装置，只要它是具有与蜂窝电话1400的成像功能和图像功能相同的成像功能和图像功能的装置，例如，PDA(个人数字助理)、智能电话、UMPC(超移动个人计算机)、网络书(net book)、笔记本型个人计算机等。In addition, the description has been made so far regarding the cellular phone 1400, but in the same manner as in the case of the cellular phone 1400, the image encoding devices 51, 351, 451, and 55, and the image decoding devices 151, 401, 501, and 601 can be applied Any type of device as long as it is a device having the same imaging function and image function as those of the cellular phone 1400, for example, PDA (Personal Digital Assistant), smart phone, UMPC (Ultra Mobile Personal Computer), Internet book (net book), notebook personal computer, etc.

图48是图示使用应用了本发明的图像编码装置和图像解码装置的硬盘记录器的主要配置例子的框图。Fig. 48 is a block diagram illustrating a main configuration example of a hard disk recorder using an image encoding device and an image decoding device to which the present invention is applied.

图48中所示的硬盘记录器(HDD记录器)1500是这样的装置，该装置在内置的硬盘中存储由调谐器接收且从卫星或地面天线等发送的广播波信号(电视信号)中包含的广播节目的音频数据和视频数据，并且，根据用户的指示在某一定时处向用户提供存储的数据。A hard disk recorder (HDD recorder) 1500 shown in FIG. 48 is a device that stores in a built-in hard disk a broadcast wave signal (television signal) received by a tuner and transmitted from a satellite or terrestrial antenna, etc. audio data and video data of the broadcast program, and the stored data is provided to the user at a certain timing according to the user's instruction.

例如，硬盘记录器1500可以从广播波信号提取音频数据和视频数据，在适当的时候对这些音频数据和视频数据进行解码，并且将其存储在内置的硬盘中。此外，例如，硬盘记录器1500也可以经由网络从另一装置获得音频数据和视频数据，在适当的时候对这些音频数据和视频数据进行解码，并且将其存储在内置的硬盘中。For example, the hard disk recorder 1500 can extract audio data and video data from a broadcast wave signal, decode them when appropriate, and store them in a built-in hard disk. In addition, for example, the hard disk recorder 1500 can also obtain audio data and video data from another device via a network, decode them when appropriate, and store them in a built-in hard disk.

此外，例如，硬盘记录器1500对在内置的硬盘中记录的音频数据和视频数据进行解码，将其供应给监视器1460，并且，在监视器1560的屏幕上显示其图像。此外，硬盘记录器1500可以从监视器1560的扬声器输出其音频。Also, for example, the hard disk recorder 1500 decodes audio data and video data recorded in a built-in hard disk, supplies it to the monitor 1460 , and displays images thereof on the screen of the monitor 1560 . In addition, the hard disk recorder 1500 can output its audio from the speaker of the monitor 1560 .

例如，硬盘记录器1500对从经由调谐器获得的广播波信号提取的音频数据和视频数据或者经由网络从另一装置获得的音频数据和视频数据进行解码，将其供应给监视器1560，并且，在监视器1560的屏幕上显示其图像。此外，硬盘记录器1500可以从监视器1560的扬声器输出其音频。For example, the hard disk recorder 1500 decodes audio data and video data extracted from a broadcast wave signal obtained via a tuner or audio data and video data obtained from another device via a network, supplies it to the monitor 1560, and, Its image is displayed on the screen of the monitor 1560 . In addition, the hard disk recorder 1500 can output its audio from the speaker of the monitor 1560 .

不用说，可以执行除这些操作以外的操作。Needless to say, operations other than these operations can be performed.

如图48所示，硬盘记录器1500包括接收单元1521、解调单元1522、分用器1523、音频解码器1524、视频解码器1525和记录器控制单元1526。硬盘记录器1500还包括EPG数据存储器1527、程序存储器1528、工作存储器1529、显示转换器1530、OSD(屏幕上显示)控制单元1531、显示控制单元1523、记录/重放单元1533、D/A转换器1534和通信单元1535。As shown in FIG. 48 , the hard disk recorder 1500 includes a receiving unit 1521 , a demodulating unit 1522 , a demultiplexer 1523 , an audio decoder 1524 , a video decoder 1525 and a recorder control unit 1526 . The hard disk recorder 1500 also includes an EPG data memory 1527, a program memory 1528, a work memory 1529, a display converter 1530, an OSD (On Screen Display) control unit 1531, a display control unit 1523, a recording/playback unit 1533, a D/A conversion device 1534 and communication unit 1535.

此外，显示转换器1530包括视频编码器1541。记录/重放单元1533包括编码器1551和解码器1552。Furthermore, the display converter 1530 includes a video encoder 1541 . The recording/playback unit 1533 includes an encoder 1551 and a decoder 1552 .

接收单元1521从遥控器(未示出)接收红外信号，将其转换为电信号，并且输出给记录器控制单元1526。记录器控制单元1526由例如微处理器等构成，并且，根据存储在程序存储器1528中的程序执行各种处理。此时，记录器控制单元1526根据需要使用各种存储器1529。The receiving unit 1521 receives an infrared signal from a remote controller (not shown), converts it into an electric signal, and outputs it to the recorder control unit 1526 . The recorder control unit 1526 is constituted by, for example, a microprocessor or the like, and executes various kinds of processing according to programs stored in the program memory 1528 . At this time, the recorder control unit 1526 uses various memories 1529 as necessary.

与网络连接的通信单元1535经由网络与另一装置执行图像处理。例如，通信单元1535由记录器控制单元1526控制，以与调谐器(未示出)通信并且，主要向调谐器输出信道选择控制信号。The communication unit 1535 connected to the network performs image processing with another device via the network. For example, the communication unit 1535 is controlled by the recorder control unit 1526 to communicate with a tuner (not shown) and mainly output a channel selection control signal to the tuner.

解调单元1522对从调谐器供应的信号进行解调，并且输出给分用器1523。分用器1523将从解调单元1522供应的数据分离为音频数据、视频数据和EPG数据，并且分别输出给音频解码器1524、视频解码器1525和记录器控制单元1526。The demodulation unit 1522 demodulates the signal supplied from the tuner, and outputs to the demultiplexer 1523 . The demultiplexer 1523 separates the data supplied from the demodulation unit 1522 into audio data, video data, and EPG data, and outputs to the audio decoder 1524, video decoder 1525, and recorder control unit 1526, respectively.

音频解码器1524例如使用MPEG格式对输入的音频数据进行解码，并且输出给记录/重放单元1533。视频解码器1525例如使用MPEG格式对输入的视频数据进行解码，并且输出给显示转换器1530。记录器控制单元1526将输入的EPG数据供应给EPG数据存储器1527以便存储。The audio decoder 1524 decodes the input audio data using, for example, MPEG format, and outputs to the recording/playback unit 1533 . The video decoder 1525 decodes input video data using, for example, MPEG format, and outputs to the display converter 1530 . The recorder control unit 1526 supplies the input EPG data to the EPG data storage 1527 for storage.

显示转换器1530使用视频编码器1541将从视频解码器1525或记录器控制单元1526供应的视频数据编码为例如符合NTSC(国家电视标准委员会)格式的视频数据，并且输出给记录/重放单元1533。此外，显示转换器1530将从视频解码器1525或记录器控制单元1526供应的视频数据的画面的尺寸转换为对应于监视器1560的尺寸的尺寸。显示转换器1530使用视频编码器1541将转换了画面尺寸的视频数据进一步转换为符合NTSC格式的视频数据，并且转换为模拟信号，并且输出给显示控制单元1532。The display converter 1530 encodes the video data supplied from the video decoder 1525 or the recorder control unit 1526 into, for example, video data conforming to the NTSC (National Television Standards Committee) format using the video encoder 1541, and outputs to the recording/playback unit 1533 . Also, the display converter 1530 converts the size of the screen of video data supplied from the video decoder 1525 or the recorder control unit 1526 into a size corresponding to the size of the monitor 1560 . The display converter 1530 further converts the video data whose screen size has been converted into video data conforming to the NTSC format using the video encoder 1541 , and converts into an analog signal, and outputs to the display control unit 1532 .

在记录器控制单元1526的控制下，显示控制单元1523将从OSD(屏幕上显示)控制单元1531输出的OSD信号重叠在从显示转换器1530输入的视频信号上，并且输出给监视器1560的显示器以便显示。Under the control of the recorder control unit 1526, the display control unit 1523 superimposes the OSD signal output from the OSD (On Screen Display) control unit 1531 on the video signal input from the display converter 1530, and outputs to the display of the monitor 1560 for display.

此外，使用D/A转换器1534将从音频解码器1524输出的音频数据转换为模拟信号，并且将其供应给监视器1560。监视器1560从内置的扬声器输出该音频信号。Also, the audio data output from the audio decoder 1524 is converted into an analog signal using the D/A converter 1534 and supplied to the monitor 1560 . The monitor 1560 outputs the audio signal from a built-in speaker.

记录/重放单元1533包括硬盘，作为记录有视频数据、视频数据等的记录介质。The recording/playback unit 1533 includes a hard disk as a recording medium recorded with video data, video data, and the like.

例如，记录/重放单元1533使用MPEG格式通过编码器1551对从音频解码器1524供应的音频数据进行编码。此外，记录/重放单元1533使用MPEG格式通过编码器1551对从显示转换器1530的视频编码器1541供应的视频数据进行编码。记录/重放单元1533使用多路复用器合成其音频数据的编码数据和其视频数据的编码数据。记录/重放单元1533通过信道编码放大合成的数据，并且经由记录头将其数据写入在硬盘中。For example, the recording/playback unit 1533 encodes the audio data supplied from the audio decoder 1524 by the encoder 1551 using the MPEG format. Furthermore, the recording/playback unit 1533 encodes the video data supplied from the video encoder 1541 of the display converter 1530 by the encoder 1551 using the MPEG format. The recording/playback unit 1533 synthesizes encoded data of its audio data and encoded data of its video data using a multiplexer. The recording/playback unit 1533 amplifies the synthesized data by channel encoding, and writes its data in the hard disk via the recording head.

记录/重放单元1533经由重放头播放在硬盘中记录的数据，放大该数据，并且，使用分用器将其分离为音频数据和视频数据。记录/重放单元1533使用MPEG格式通过解码器1552对音频数据和视频数据进行解码。记录/重放单元1533对解码的音频数据进行数字模拟转换，并且输出给监视器1560的扬声器。The recording/playback unit 1533 plays data recorded in the hard disk via a playback head, amplifies the data, and separates it into audio data and video data using a demultiplexer. The recording/playback unit 1533 decodes audio data and video data by the decoder 1552 using the MPEG format. The recording/playback unit 1533 digital-analog-converts the decoded audio data, and outputs to a speaker of the monitor 1560 .

此外，记录/重放单元1533对解码的视频数据进行数字模拟转换，并且输出给监视器1560的显示器。Also, the recording/playback unit 1533 performs digital-to-analog conversion on the decoded video data, and outputs to the display of the monitor 1560 .

记录器控制单元1526基于由经由接收单元1521接收的来自遥控器的红外信号指示的用户的指示从EPG数据存储器1527读出最新的EPG数据，并且供应给OSD控制单元1531。OSD控制单元1531产生对应于输入的EPG数据的图像数据，并且输出给显示控制单元1532。显示控制单元1532将从OSD控制单元1531输入的视频数据输出给监视器1560的显示器以便显示。因此，在监视器1560的显示器上显示EPG(电子节目指南)。The recorder control unit 1526 reads out the latest EPG data from the EPG data storage 1527 based on the user's instruction indicated by the infrared signal received via the receiving unit 1521 from the remote controller, and supplies to the OSD control unit 1531 . The OSD control unit 1531 generates image data corresponding to the input EPG data, and outputs to the display control unit 1532 . The display control unit 1532 outputs the video data input from the OSD control unit 1531 to the display of the monitor 1560 for display. Accordingly, an EPG (Electronic Program Guide) is displayed on the display of the monitor 1560 .

此外，硬盘记录器1500可以经由诸如互联网等的网络获得从另一装置供应的各种数据，例如，视频数据、音频数据、EPG数据等。In addition, the hard disk recorder 1500 can obtain various data supplied from another device, for example, video data, audio data, EPG data, etc., via a network such as the Internet.

通信单元1535由记录器控制单元1526控制，以获得经由网络从另一装置发送的诸如视频数据、音频数据、EPG数据等的编码数据，并且将其供应给记录器控制单元1526。例如，记录器控制单元1526将获得的视频数据和音频数据的编码数据供应给记录/重放单元1533，并且将其存储在硬盘中。此时，记录器控制单元1526和记录/重放单元153可以根据需要执行诸如重新编码等的处理。The communication unit 1535 is controlled by the recorder control unit 1526 to obtain encoded data such as video data, audio data, EPG data, etc. transmitted from another device via a network, and supply it to the recorder control unit 1526 . For example, the recorder control unit 1526 supplies the obtained encoded data of video data and audio data to the recording/playback unit 1533, and stores it in the hard disk. At this time, the recorder control unit 1526 and the recording/playback unit 153 can perform processing such as re-encoding as necessary.

此外，记录器控制单元1526对获得的视频数据和音频数据的编码数据进行解码，并且将获得的视频数据供应给显示转换器1530。以与从视频解码器1525供应的视频数据相同的方式，显示转换器1530对从记录器控制单元1526供应的视频数据进行处理，经由显示控制单元1532供应给监视器1560，以便显示其图像。Furthermore, the recorder control unit 1526 decodes the obtained encoded data of video data and audio data, and supplies the obtained video data to the display converter 1530 . In the same manner as the video data supplied from the video decoder 1525, the display converter 1530 processes the video data supplied from the recorder control unit 1526, supplies to the monitor 1560 via the display control unit 1532 to display images thereof.

或者，可以进行这样的布置：其中，根据该图像显示，记录器控制单元1526将解码的音频数据经由D/A转换器1534供应给监视器1560，并且从扬声器输出其音频。Alternatively, an arrangement may be made in which, according to the image display, the recorder control unit 1526 supplies decoded audio data to the monitor 1560 via the D/A converter 1534 and outputs its audio from a speaker.

此外，记录器控制单元1526对获得的EPG数据的编码数据进行解码，并且，将解码的EPG数据供应给EPG数据存储器1527。Furthermore, the recorder control unit 1526 decodes the obtained encoded data of the EPG data, and supplies the decoded EPG data to the EPG data memory 1527 .

这样配置的硬盘记录器1500利用图像解码装置151、401、501或601作为视频解码器1525、解码器1552和容纳在记录器控制单元1526中的解码器。因此，以与图像解码装置151和401相同的方式，视频解码器1525、解码器1552和容纳在记录器控制单元1526中的解码器根据量化参数和预测模式切换滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像解码装置501和601相同的方式，视频解码器1525、解码器1552和容纳在记录器控制单元1526中的解码器基于打开/关闭标志控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。The hard disk recorder 1500 thus configured utilizes the image decoding device 151 , 401 , 501 , or 601 as the video decoder 1525 , the decoder 1552 , and the decoder housed in the recorder control unit 1526 . Therefore, in the same manner as the image decoding devices 151 and 401, the video decoder 1525, the decoder 1552, and the decoder housed in the recorder control unit 1526 switch filter coefficients according to quantization parameters and prediction modes, and, in intra prediction Before, filter processing is performed on neighboring pixels. Alternatively, in the same manner as the image decoding devices 501 and 601, the video decoder 1525, the decoder 1552, and the decoder housed in the recorder control unit 1526 control based on an ON/OFF flag whether to perform filter processing. Therefore, coding efficiency can be improved.

因此，硬盘记录器1500可以产生具有高精度的预测图像。作为其结果，硬盘记录器1500可以例如从经由调谐器接收的视频数据的编码数据、从记录/重放单元1533的硬盘读出的视频数据的编码数据或者经由网络获得的视频数据的编码数据获得具有更高精度的解码图像，并且在监视器1560上显示。Therefore, the hard disk recorder 1500 can generate predicted images with high precision. As a result, the hard disk recorder 1500 can obtain, for example, from encoded data of video data received via a tuner, encoded data of video data read out from the hard disk of the recording/playback unit 1533, or encoded data of video data obtained via a network. The decoded image has higher precision and is displayed on the monitor 1560.

此外，硬盘记录器1500利用图像编码装置51、351、451或551作为编码器1551。因此，以与图像编码装置51和351相同的方式，编码器1551根据量化参数和预测模式设置滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像编码装置451和551相同的方式，编码器1551控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。Furthermore, the hard disk recorder 1500 utilizes the image encoding device 51 , 351 , 451 , or 551 as the encoder 1551 . Therefore, in the same manner as the image encoding devices 51 and 351, the encoder 1551 sets filter coefficients according to quantization parameters and prediction modes, and, before intra prediction, performs filter processing on adjacent pixels. Alternatively, in the same manner as the image encoding devices 451 and 551 , the encoder 1551 controls whether to perform filtering processing on adjacent pixels before intra prediction. Therefore, coding efficiency can be improved.

因此，例如，硬盘记录器1500可以提高要记录在硬盘中的编码数据的编码效率。作为其结果，硬盘记录器1500可以以更有效的方式使用硬盘的存储区域。Therefore, for example, the hard disk recorder 1500 can improve encoding efficiency of encoded data to be recorded in a hard disk. As a result thereof, the hard disk recorder 1500 can use the storage area of the hard disk in a more efficient manner.

请注意，迄今为止已经对用于在硬盘中记录视频数据和音频数据的硬盘记录器1500进行了描述，但是，不用说，可以利用任何类型的记录介质。例如，甚至采用应用了诸如闪速存储器、光盘、视频带等的除硬盘以外的记录介质的记录器，以与上述的硬盘记录器1500的情况相同的方式，图像编码装置51、351、451和551以及图像解码装置151、401、501和601也可以适用于此。Note that the hard disk recorder 1500 for recording video data and audio data in a hard disk has been described so far, but it goes without saying that any type of recording medium can be utilized. For example, even with a recorder to which a recording medium other than a hard disk such as a flash memory, an optical disk, a video tape, etc. is applied, in the same manner as in the case of the hard disk recorder 1500 described above, the image encoding devices 51, 351, 451 and 551 and image decoding devices 151, 401, 501, and 601 are also applicable here.

图49是图示使用应用了本发明的图像编码装置和图像解码装置的照相机的主要配置例子的框图。Fig. 49 is a block diagram illustrating a main configuration example of a camera using an image encoding device and an image decoding device to which the present invention is applied.

图49中所示的照相机1600对被摄体进行成像，在LCD 1616上显示被摄体的图像，并且将其作为图像数据记录在记录介质1633中。A camera 1600 shown in FIG. 49 images a subject, displays the image of the subject on an LCD 1616 , and records it as image data in a recording medium 1633 .

透镜块1611将光(即，被摄体的视频)输入到CCD/CMOS 1612。CCD/CMOS 1612是利用CCD或CMOS的图像传感器，将接收到的光的强度转换为电信号，并且供应给照相机信号处理单元1613。The lens block 1611 inputs light (ie, the video of the subject) to the CCD/CMOS 1612 . The CCD/CMOS 1612 is an image sensor using a CCD or CMOS, converts the intensity of received light into an electrical signal, and supplies it to the camera signal processing unit 1613 .

照相机信号处理单元1613将从CCD/CMOS 1612供应的电信号转换为Y、Cr和Cb的颜色差信号，并且供应给图像信号处理单元1614。在控制器1621的控制下，图像信号处理单元1614对从照相机信号处理单元1613供应的图像信号进行预定的图像处理，或者，使用例如MPEG格式通过编码器1641对其图像信号进行编码。图像信号处理单元1614将通过对图像信号进行编码而产生的编码数据供应给解码器1615。此外，图像信号处理单元1614获得在屏幕上显示器(OSD)1620处产生的用于显示的数据，并且将其供应给解码器1615。The camera signal processing unit 1613 converts the electric signal supplied from the CCD/CMOS 1612 into color difference signals of Y, Cr, and Cb, and supplies to the image signal processing unit 1614 . Under the control of the controller 1621 , the image signal processing unit 1614 performs predetermined image processing on the image signal supplied from the camera signal processing unit 1613 , or encodes the image signal thereof by the encoder 1641 using, for example, MPEG format. The image signal processing unit 1614 supplies encoded data generated by encoding the image signal to the decoder 1615 . Furthermore, the image signal processing unit 1614 obtains data for display generated at an on-screen display (OSD) 1620 and supplies it to the decoder 1615 .

对于上述的处理，照相机信号处理单元1613根据需要适当地利用经由总线1617连接的DRAM(动态随机存取存储器)1617来将图像数据、从其图像数据编码的编码数据等保持在其DRAM 1618中。For the above-described processing, the camera signal processing unit 1613 appropriately utilizes a DRAM (Dynamic Random Access Memory) 1617 connected via a bus 1617 as necessary to hold image data, encoded data encoded from its image data, and the like in its DRAM 1618 .

解码器1615对从图像信号处理单元1614供应的编码数据进行解码，并且，将获得的图像数据(解码图像数据)供应给LCD 1616。此外，解码器1615将从图像信号处理单元1614供应的用于显示的数据供应给LCD 1616。LCD 1616在适当的时候合成从解码器1615供应的用于显示的数据的图像和解码图像数据的图像，并且显示其合成图像。The decoder 1615 decodes the encoded data supplied from the image signal processing unit 1614 , and supplies the obtained image data (decoded image data) to the LCD 1616 . Also, the decoder 1615 supplies data for display supplied from the image signal processing unit 1614 to the LCD 1616 . The LCD 1616 synthesizes the image of the data for display supplied from the decoder 1615 and the image of the decoded image data at an appropriate time, and displays the synthesized image thereof.

在控制器1621的控制下，屏幕上显示器1620将由符号、字符或图形构成的诸如菜单画面或图标等的用于显示的数据经由总线1617输出到图像信号处理单元1614。Under the control of the controller 1621 , the on-screen display 1620 outputs data for display, such as menu screens or icons, constituted by symbols, characters, or graphics, to the image signal processing unit 1614 via the bus 1617 .

基于指示由用户使用操作单元1622命令的内容的信号，控制单元1621执行各种处理，并且，经由总线1617还控制图像信号处理单元1614、DRAM 1618、外部接口1619、屏幕上显示器1620、介质驱动器1623等。用于控制器1621执行各种处理所需的程序、数据等存储在闪速ROM 1624中。The control unit 1621 executes various processing based on a signal indicating the contents ordered by the user using the operation unit 1622, and also controls the image signal processing unit 1614, DRAM 1618, external interface 1619, on-screen display 1620, media drive 1623 via the bus 1617 Wait. Programs, data, and the like necessary for the controller 1621 to execute various processes are stored in the flash ROM 1624 .

例如，控制器1621可以代替图像信号处理单元1614和解码器1615对存储于DRAM1618中的图像数据进行编码，或者对存储于DRAM 1618中的编码数据进行解码。此时，控制器1621可以使用与图像信号处理单元1614和解码器1615的编码和解码格式相同的格式执行编码和解码处理，或者，可以使用图像信号处理单元1614和解码器1615都不能处理的格式来执行编码和解码处理。For example, the controller 1621 may encode image data stored in the DRAM 1618 or decode encoded data stored in the DRAM 1618 instead of the image signal processing unit 1614 and the decoder 1615 . At this time, the controller 1621 may perform encoding and decoding processing using the same format as those of the image signal processing unit 1614 and the decoder 1615, or may use a format that neither the image signal processing unit 1614 nor the decoder 1615 can process to perform encoding and decoding processing.

此外，例如，在从操作单元1622指示了开始图像打印的情况中，控制器1621从DRAM1618读出图像数据，并且经由总线1617将其供应给与外部接口1619连接的打印机1634以便打印。Also, for example, in the case where start of image printing is instructed from the operation unit 1622 , the controller 1621 reads out image data from the DRAM 1618 and supplies it to the printer 1634 connected to the external interface 1619 via the bus 1617 for printing.

此外，例如，在从操作单元1622指示了图像记录的情况中，控制器1621从DRAM1618读出编码数据，并且经由总线1617将其供应给安装在介质驱动器1623上的记录介质1633以便存储。Also, for example, in the case where image recording is instructed from the operation unit 1622, the controller 1621 reads out encoded data from the DRAM 1618, and supplies it to the recording medium 1633 mounted on the media drive 1623 via the bus 1617 for storage.

记录介质1633是可任选的可读写可移动介质，例如，磁盘、磁光盘、光盘、半导体存储器等。不用说，关于可移动介质的类型，记录介质1633也是可任选的，因此，可以是带装置，或者可以是盘，或者可以是存储器卡。不用说，记录介质1633可以是非接触式IC卡等。The recording medium 1633 is an optional readable and writable removable medium, for example, a magnetic disk, a magneto-optical disk, an optical disk, a semiconductor memory, and the like. It goes without saying that the recording medium 1633 is also optional as to the type of removable medium, and therefore, may be a tape device, or may be a disc, or may be a memory card. Needless to say, the recording medium 1633 may be a non-contact IC card or the like.

或者，介质驱动器1623和记录介质1633可以被配置为集成在非可搬性记录介质中，所述非可搬性记录介质例如是内置的硬盘驱动器、SSD(固态驱动器)等。Alternatively, the media driver 1623 and the recording medium 1633 may be configured to be integrated in a non-removable recording medium such as a built-in hard disk drive, SSD (Solid State Drive), or the like.

外部接口1619由例如USB输入/输出端子等构成，并且在执行图像打印的情况中与打印机1634连接。此外，驱动器1631根据需要与外部接口1619连接，诸如磁盘、光盘或磁光盘的可移动介质1632在适当的时候被安装在其上，并且，从其读出的计算机程序根据需要被安装在闪速ROM 1624中。The external interface 1619 is constituted by, for example, a USB input/output terminal or the like, and is connected to a printer 1634 in the case of performing image printing. Also, a drive 1631 is connected to the external interface 1619 as needed, and a removable medium 1632 such as a magnetic disk, an optical disk, or a magneto-optical disk is mounted thereon when appropriate, and a computer program read therefrom is installed in a flash memory as needed. ROM 1624.

此外，外部接口1619包括要与诸如LAN、互联网等的预定网络连接的网络接口。例如，根据来自操作单元122的指示，控制器1621可以从DRAM 1618读出编码数据，并且将其从外部接口1619供应给经由网络连接的另一装置。此外，控制器1621可以经由外部接口1619获得经由网络从另一装置供应的编码数据或图像数据，并且将其保持在DRAM 1618中，或者，将其供应给图像信号处理单元1614。Furthermore, the external interface 1619 includes a network interface to be connected with a predetermined network such as LAN, the Internet, or the like. For example, according to an instruction from the operation unit 122, the controller 1621 can read out encoded data from the DRAM 1618, and supply it from the external interface 1619 to another device connected via a network. Furthermore, the controller 1621 can obtain encoded data or image data supplied from another device via a network via the external interface 1619 and hold it in the DRAM 1618 or supply it to the image signal processing unit 1614 .

这样配置的照相机1600利用图像解码装置151、401、501或601作为解码器1615。The camera 1600 thus configured utilizes the image decoding device 151 , 401 , 501 , or 601 as the decoder 1615 .

因此，以与图像解码装置151和401相同的方式，解码器1615根据量化参数和预测模式切换滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像解码装置501和601相同的方式，解码器1615基于打开/关闭标志控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。Therefore, in the same manner as the image decoding devices 151 and 401, the decoder 1615 switches filter coefficients according to quantization parameters and prediction modes, and performs filter processing on adjacent pixels before intra prediction. Alternatively, in the same manner as the image decoding devices 501 and 601 , the decoder 1615 controls whether to perform filter processing on adjacent pixels before intra prediction based on an on/off flag. Therefore, coding efficiency can be improved.

因此，照相机1600可以产生具有高精度的预测图像。作为其结果，例如，照相机1600可以从在CCD/CMOS 1612处产生的图像数据、从DRAM 1618或记录介质1633读出的视频数据的编码数据、或者经由网络获得的视频数据的编码数据获得具有更高精度的解码图像，并且在LCD 1616上显示。Accordingly, the camera 1600 can generate a predicted image with high precision. As a result, for example, the camera 1600 can obtain an image having a higher quality from image data generated at the CCD/CMOS 1612, coded data of video data read out from the DRAM 1618 or the recording medium 1633, or coded data of video data obtained via a network. High-precision decoded images are displayed on LCD 1616.

此外，照相机1600利用图像编码装置51、351、451或551作为编码器1641。Furthermore, the camera 1600 utilizes the image encoding device 51 , 351 , 451 , or 551 as the encoder 1641 .

因此，以与图像编码装置51和351相同的方式，编码器1641根据量化参数和预测模式设置滤波系数，并且，在帧内预测之前，对邻近像素执行滤波处理。或者，以与图像编码装置451和551相同的方式，编码器1641控制在帧内预测之前是否对邻近像素执行滤波处理。因此，可以提高编码效率。Therefore, in the same manner as the image encoding devices 51 and 351, the encoder 1641 sets filter coefficients according to quantization parameters and prediction modes, and, before intra prediction, performs filter processing on adjacent pixels. Alternatively, in the same manner as the image encoding devices 451 and 551 , the encoder 1641 controls whether to perform filtering processing on neighboring pixels before intra prediction. Therefore, coding efficiency can be improved.

因此，例如，照相机1600可以提高要记录在硬盘中的编码数据的编码效率。作为其结果，照相机1600可以以更有效的方式使用DRAM 1618或记录介质1633的存储区域。Therefore, for example, the camera 1600 can improve encoding efficiency of encoded data to be recorded in a hard disk. As a result thereof, the camera 1600 can use the storage area of the DRAM 1618 or the recording medium 1633 in a more efficient manner.

请注意，图像解码装置151、401、501和601的解码方法可以适用于控制器1621执行的解码处理。类似地，图像编码装置51、351、451和551的编码方法可以适用于控制器1621执行的编码处理。Note that the decoding methods of the image decoding devices 151 , 401 , 501 , and 601 can be applied to the decoding process performed by the controller 1621 . Similarly, the encoding methods of the image encoding devices 51 , 351 , 451 , and 551 can be applied to the encoding process performed by the controller 1621 .

此外，照相机1600成像的图像数据可以是运动图像，或者可以是静止图像。In addition, image data imaged by the camera 1600 may be a moving image, or may be a still image.

不用说，图像编码装置51、351、451和551以及图像解码装置151、401、501和601可以适用于除上述装置以外的装置或系统。It goes without saying that the image encoding devices 51, 351, 451, and 551 and the image decoding devices 151, 401, 501, and 601 may be applied to devices or systems other than the above-described devices.

附图标记列表List of reference signs

51 图像编码装置51 Image coding device

66 无损耗编码单元66 lossless coding units

74 帧内预测单元74 intra prediction units

75 邻近像素内插滤波切换单元75 adjacent pixel interpolation filter switching unit

81 邻近像素设置单元81 adjacent pixel setting unit

82 预测图像产生单元82 predictive image generation unit

83 最佳预测模式确定单元83 best prediction mode determination unit

91 预测模式缓冲器91 Predictive Mode Buffer

92 量化参数缓冲器92 Quantization parameter buffer

93 低通滤波设置单元93 Low-pass filter setting unit

94 帧系数存储器94 frame coefficient memory

111 邻近像素设置单元111 adjacent pixel setting unit

112 预测图像产生单元112 predictive image generation unit

113 最佳预测模式确定单元113 optimal prediction mode determination unit

121 预测模式缓冲器121 Prediction Mode Buffer

122 最佳滤波计算单元122 optimal filtering calculation unit

151 图像解码装置151 Image decoding device

162 无损耗解码单元162 lossless decoding units

171 帧内预测单元171 intra prediction units

172 邻近像素内插滤波切换单元172 adjacent pixel interpolation filter switching unit

181 预测图像产生单元181 predictive image generation unit

182 邻近像素设置单元182 adjacent pixel set units

191 预测模式缓冲器191 Predictive Mode Buffer

192 量化参数缓冲器192 Quantization parameter buffer

193 低通滤波设置单元193 Low-pass filter setting unit

194 滤波系数存储器194 filter coefficient memory

202 低通滤波设置单元202 Low-pass filter setting unit

251 学习装置251 learning devices

261 邻近内插滤波计算单元261 adjacent interpolation filter calculation unit

271 滤波系数存储单元271 filter coefficient storage unit

351 图像编码装置351 Image coding device

361 二次预测单元361 quadratic prediction unit

362 邻近像素内插滤波切换单元362 adjacent pixel interpolation filter switching unit

401 图像解码装置401 Image decoding device

411 二次预测单元411 Quadratic Prediction Unit

412 邻近像素内插滤波切换单元412 Neighboring pixel interpolation filter switching unit

451 图像编码装置451 Image coding device

461 邻近像素内插滤波控制单元461 adjacent pixel interpolation filter control unit

501 图像解码装置501 Image decoding device

511 邻近像素内插滤波控制单元511 adjacent pixel interpolation filter control unit

551 图像编码装置551 Image coding device

561 邻近像素内插滤波控制单元561 adjacent pixel interpolation filter control unit

601 图像解码装置601 Image decoding device

611 邻近像素内插滤波控制单元611 Adjacent pixel interpolation filter control unit

Claims (4)

Translated fromChinese

1.一种图像处理装置，包括：1. An image processing device, comprising:滤波部件，被配置为根据方向和控制信息，使用滤波系数对当前块的邻近像素进行滤波，所述方向对应于帧内预测模式，所述控制信息控制是否对所述当前块的邻近像素进行滤波处理；A filtering component configured to filter neighboring pixels of the current block using filter coefficients according to a direction corresponding to an intra prediction mode and control information, the control information controlling whether to filter neighboring pixels of the current block deal with;帧内预测部件，被配置为：在所述控制信息指示对所述邻近像素进行滤波处理的条件下，使用经过了滤波处理的邻近像素值，对所述当前块的像素执行帧内预测，并且产生预测图像，而在所述控制信息指示不对所述邻近像素进行滤波处理的条件下，使用邻近像素值，对所述当前块的像素执行帧内预测，并且产生预测图像；以及an intra-frame prediction component configured to: perform intra-frame prediction on pixels of the current block using filtered adjacent pixel values under the condition that the control information indicates that the adjacent pixels are filtered, and generating a predicted image, performing intra prediction on pixels of the current block using adjacent pixel values under the condition that the control information indicates not to perform filtering processing on the adjacent pixels, and generating a predicted image; and解码部件，被配置为使用由所述帧内预测部件所生成的预测图像，来对比特流进行解码。decoding means configured to decode the bitstream using the predicted image generated by the intra prediction means.2.根据权利要求1所述的图像处理装置，其中2. The image processing apparatus according to claim 1, wherein所述图像以具有层级结构的单位而被编码，the images are coded in units having a hierarchical structure,所述解码部件以具有层级结构的单位对所述图像进行解码。The decoding section decodes the image in units having a hierarchical structure.3.根据权利要求2所述的图像处理装置，其中3. The image processing apparatus according to claim 2, wherein所述帧内预测部件将块大小设为4×4像素、8×8像素或16×16像素，来进行帧内预测。The intra prediction section sets the block size to 4×4 pixels, 8×8 pixels, or 16×16 pixels to perform intra prediction.4.一种图像处理方法，由图像处理装置进行如下步骤：4. An image processing method, the following steps are carried out by an image processing device:根据方向和控制信息，使用滤波系数对当前块的邻近像素进行滤波，所述方向对应于帧内预测模式，所述控制信息控制是否对所述当前块的邻近像素进行滤波处理；Filtering adjacent pixels of the current block by using filter coefficients according to the direction and control information, the direction corresponds to an intra prediction mode, and the control information controls whether to filter the adjacent pixels of the current block;在所述控制信息指示对所述邻近像素进行滤波处理的条件下，使用经过了滤波处理的邻近像素值，对所述当前块的像素执行帧内预测，并且产生预测图像，而在所述控制信息指示不对所述邻近像素进行滤波处理的条件下，使用邻近像素值，对所述当前块的像素执行帧内预测，并且产生预测图像；以及Under the condition that the control information indicates to perform filtering processing on the adjacent pixels, intra prediction is performed on the pixels of the current block using the filtered adjacent pixel values, and a predicted image is generated, and in the control The information indicates that under the condition that the filtering process is not performed on the adjacent pixels, using the adjacent pixel values, performing intra prediction on the pixels of the current block, and generating a predicted image; and使用执行帧内预测所生成的预测图像，来对比特流进行解码。The bitstream is decoded using a predicted image generated by performing intra prediction.