1. Introduction

2. Motivation for This Work

3. Proposed Method

3.2. Spatial Correlation-Based Motion-Vector-Prediction Algorithm

The performance of motion estimation highly depends on the MVP [13,14,15,16]. If the MVP is close to the calculated MV, the MVD between the MVP and the calculated MV is small, and the MVP is more accurate. However, in the H.265/HEVC standard, a total of seven spatial and temporal MVs are added to the candidate list to predict the MVP. There are two disadvantages to the current AMVP mechanism in H.265/HEVC [17]. For one thing, the number of reference MVs is limited. For another, a fixed selecting pattern is not adaptive to selecting reference MVs; therefore, it is not generating an accurate MVP by using the current AMVP mechanism. In order to further improve encoding efficiency, more reference MVs can be added to the candidate list. Owing to spatial and temporal motion consistency, the MVs surrounding the current PU are useful for determining the MVP. However, too many MVP candidates may cause a large number of calculations, so it is necessary to reduce the calculation redundancy of the MVP decision.

Based on the above-mentioned views, the reference MVs of the current PU can be used to search for an accurate MVP. In neighborhood subsetM, the MVs of the neighboring CUs are shown inFigure 4, where$M{V}_L$,$M{V}_{TR}$ and$M{V}_{TL}$ indicate the MV candidates in the left, top right, and top left of the current PU, respectively. In H.265/HEVC, the simplified rate-distortion optimization (RDO) method is performed to estimate the motion vector [12]. In the RDO process, the rate-distortion cost (RD-cost) function ($J_{cost}=D_{distortion}+R_{bits}\times \lambda$) is minimized by the encoder, where$\lambda$ is the Lagrange multiplier,$D_{distortion}$ represents the distortion between the original block and reference block, and$R_{bits}$ represents the number of coding bits. The MVD between the MVP and the calculated MV is also signalled in the$R_{bits}$.

For the different texture of video content, the reliability of candidate MVs can be evaluated by the MVs of spatial neighborhood subsetM: {$M{V}_L$,$M{V}_{TR}$,$M{V}_{TL}$}. When these three MVs are equal, the MVs of adjacent CUs tend to the same direction. In this case, the reliability of candidate MVs is the highest, and a simple MV can be selected as the final MVP. That is, reference MVs satisfy as

$$\begin{array}{c}\hfill M{V}_{TR}=M{V}_{TL}=M{V}_L\end{array}$$

(3)

Then,$M{V}_L$ is selected as the optimized MVP.

Furthermore, when the MV absolute difference of$M{V}_{TL}$ and$M{V}_{TR}$ is more than the MV absolute difference of$M{V}_{TL}$ and$M{V}_L$, motion consistency in the top of PU is lower than motion consistency to the left of PU. In this case, the reliability of the MVs in the left of PU is higher than the reliability of the MVs in the top of PU. Otherwise, the reliability of the MVs in the top of PU is higher than the reliability of the MVs in the left of PU. Thus, when reference MVs satisfy as

$$\begin{array}{c}\hfill |M{V}_{TR}-M{V}_{TL}|>|M{V}_{TL}-M{V}_L|\end{array}$$

(4)

the MVP position tends to the left of PU. Then,$M{V}_{TL}$ and$M{V}_L$ are selected as MVP candidates. Otherwise, the MVP position tends to the top of PU, and$M{V}_{TR}$ and$M{V}_{TL}$ are selected as MVP candidates.

When the MVs of subsetM are not available, the reliability of candidate MVs is the lowest in the spatial domain, and it is hard to obtain an accurate MVP by using the fixed AMVP mechanism. In this case, the MVP position may tend to the left of PU, and it is also possible to tend to the top of PU. Therefore, all available MVs of spatial neighborhood setG need to be checked. In order to obtain a more accurate MVP, all surrounding MVs ofG can be added to the candidate MVs, and the cost of these MVs is checked to obtain an optimized MVP by comparing one with a method. When all components of G are not available, MV (0, 0) is added to the candidate list, which is the same as that in H.265/HEVC.

It is noted that the encoder codes$mvp\_lx\_flag$ for indicating the number of coded bits for the MVP candidates. Different from the H.265/HEVC standard, in this work, the indicating method for the codec is that$mvp\_lx\_flag$ is designed as a variable-length code, and the length of$mvp\_lx\_flag$ is expressed asL. The relationship between the coded bits of$mvp\_lx\_flag$ and MVP is shown as inTable 1. When M is available, the length of$mvp\_lx\_flag$ satisfies$L=1\mathrm{bit}$. However, when M is not available, the maximum value ofL with a different PU size is shown inTable 2.

It can be seen fromTable 2 that, when the current PU size is equal to$64\times 64$, the maximum value of$mvp\_lx\_flag$ can be calculated as follows: (1) if the smaller surrounding PU size is$8\times 8$, the number of coded bits that need to index the MVP candidates is$lo{g}_2(64/8+64/8)=4$. Thus, the length of$mvp\_lx\_flag$ satisfiesL = 4 bit. (2) if the smaller surrounding PU size is$16\times 16$, the number of coded bits that need to index the MVP candidates is$lo{g}_2(64/16+64/16)=3$. Thus, the length of$mvp\_lx\_flag$ satisfiesL = 3 bit. (3) if the smaller surrounding PU size is$32\times 32$, the number of coded bits that need to index the MVP candidates is$lo{g}_2(64/32+64/32)=2$. Thus, the length of$mvp\_lx\_flag$ satisfiesL = 2 bit. (4) If both surrounding PUs are$64\times 64$ in size, the number of coded bits that need to index the MVP candidates is$lo{g}_2(64/64+64/64)=1$. Thus, the length of$mvp\_lx\_flag$ satisfiesL = 1 bit. In this case, the maximum-value length of$mvp\_lx\_flag$ satisfiesL = 4 bit. Moreover, in order to clearly specify the length of$mvp\_lx\_flag$,Figure 5 shows the length range of$mvp\_lx\_flag$ with$64\times 64$ PU size. For others PU size ($32\times 64$,$64\times 32$,$48\times 64$,$64\times 48$,$16\times 64$, and$64\times 16$), the maximum value length of$mvp\_lx\_flag$ satisfiesL = 4 bit. Similarly, when the current PU size is equal to$32\times 32$, the maximum value of$mvp\_lx\_flag$ satisfiesL = 3 bit. When the current PU size is equal to$16\times 16$, the maximum value of$mvp\_lx\_flag$ satisfiesL= 2 bit. When the current PU size is equal to$8\times 8$, and the smaller surrounding PU size is$8\times 8$, the number of coded bits that need to index the MVP candidates is$lo{g}_2(8/8+8/8)=1$. In this case, the length of$mvp\_lx\_flag$ satisfiesL = 1 bit.

In H.265/HEVC standards, the distribution of the selected spatial-motion candidates is far greater than the distribution of the temporal-motion candidates [4]. In this work, the more-available spatial candidates are used to decide the MVP, and the temporal-motion candidates have little overall effect on coding efficiency. Thus, the temporal-motion candidates have been removed in the proposed method.

As per the aforementioned approaches, the spatial correlation-based motion-vector-prediction selection algorithm for interprediction is shown in Algorithm 1. Firstly, the MVP candidate list is established by using the proposed spatial-neighborhood motion vector. After that, the rate-distortion-optimal MVP is generated by executing motion estimation in the MVP candidate list, which is the search center to search for the optimal MV. Motion estimation (ME) is the process of determining a motion vector by using a block-matching algorithm [18], which is regarded as a time-consuming process. There are two advantages of this proposed method: (1) MVP accuracy was improved with the proposed method. Thus, the MVD of the current PU becomes smaller for InterMode. The length of$mvp\_lx\_flag$ is variable. (2) Using the proposed method, the possibility that the MV and MVP of the current PU are consistent increased, and the probability that CU selects MergeMode increased. In the case when MVD is equal to zero, the majority of CUs select MergeMode. Therefore, by using the proposed algorithm, the effect of the proposed method (MVD becoming zero) and the effect of MergeMode overlap. The length of$merge\_idx$ from the merge candidate list in MergeMode is fixed, which is the same as the definition in H.265/HEVC standards. As a result, the proposed algorithm can significantly reduce the amount of bits.

Algorithm 1: Spatial correlation-based MVP algorithm.

The main idea of this work is the sacrifice of computational complexity for higher coding efficiency. Thus, more MVs surrounding the current PU are added to the MV candidate list by the proposed method; therefore, most computational cost in this work is to search for an accurate MVP with the RDO process.

3.3. Spatial Correlation-Based CU Depth-Prediction Algorithm

The above spatial correlation-based MVP algorithm can significantly improve coding efficiency, while computation complexity is increased by a lot. There are quite a few related works that can reduce computation complexity [13,14,19]. However, three important issues are carefully considered to design the conditions. Firstly, arithmetic-complexity reduction is the design motivation. Secondly, the robustness of the design condition is higher. Thirdly, owing to high availability, depth information should be used. In this paper, a spatial correlation-based CU depth-prediction algorithm is presented. In order to evaluate depth-level prediction, several experiments were performed on different conditions with different configurations. In the experiments, the accuracy rate when the predicted depth level was equal to the depth level selected by the original H.265/HEVC test model was verified.

Generally, the texture complexity of image content is directly related to the depth of the image. When the depth range of the CU is higher, the texture complexity of the CU tends to be complex. On the contrary, when CU depth range is lower, CU texture complexity tends to be simple. Based on CU depth, CU texture complexity ($TC$) is classified into simple or complex as

$$TC=\left\{\begin{array}{cc}\mathrm{simple},\hfill & \mathrm{if}D\le 1\hfill \\ \mathrm{complex},\hfill & \mathrm{if}D>1\hfill \end{array}\right.$$

(5)

where$TC$ represents the texture complexity of a CU.D is the maximal depth of the$CU$ in the motion-estimation processing, and default valueD is set to 3 in H.265/HEVC reference software.

In this verification, the test conditions have to be carefully designed. It is clear that when the$TC$ of the left neighboring$C{U}_L$, the top-right neighboring$C{U}_{TR}$, and the top-left neighboring$C{U}_{TL}$ are simple, the$TC$ of the current CU tends to be not complex. On the contrary, when the$TC$ of the left neighboring$C{U}_L$, the top-right neighboring$C{U}_{TR}$, and the top-left neighboring$C{U}_{TL}$ are complex, the$TC$ of the current CU tends to be not simple. Thus, based on the above conclusions, two conditions (C1 and C2) are proposed as follows:

$$\left\{\begin{array}{cc}\mathrm{C}1:\hfill & D_{TR}\le 1\&\&D_{TL}\le 1\&\&D_L\le 1\&\&D^{\ast }\le 2\hfill \\ \mathrm{C}2:\hfill & D_{TR}>1\&\&D_{TL}>1\&\&D_L>1\&\&D^{\ast }\le 1\hfill \end{array}\right.$$

where$D_L$,$D_{TR}$,$D_{TL}$, and$D^{\ast }$ are the maximal depth of$C{U}_L$,$C{U}_{TR}$,$C{U}_{TL}$, and the current CU, respectively.

In order to verify the accuracy of the two conditions, accuracy rate$AR$ is defined as

$$\begin{array}{c}\hfill AR=\frac{n_1}{N}\times 100\%\end{array}$$

(6)

while$n_1$ represents the number of correct-matching test cases by using the depth of the neighboring CUs to predict the depth of the current CU, andN represents the total number of test cases. In this work, four typical sequences (PeopleOnStreet, BasketballDrill, BQSquare, Vidyo1) were applied to test with low-delay (LD) and random-access (RA) profiles. From the results ofTable 3, the rates of Condition 1 and Condition 2 are about 99% and 93%, respectively. That is, the depth of$C{U}_L$,$C{U}_{TR}$, and$C{U}_{TL}$ has strong spatial correlation with the depth of the current CU. Thus, it is high availability to predict the depth of the current CU by utilizing the depth of the neighboring CUs.

Hence, the spatial correlation-based CU depth-prediction algorithm for interprediction is shown in Algorithm 2. Firstly, the predicted depth range of the current CU is determined by the depth of the neighboring CUs. Secondly, the RD-cost of the current CU is checked in the predicted depth range. The advantage of this method is simple and easy to achieve. Moreover, the robustness of this method is high.

Algorithm 2: Spatial correlation-based CU depth-prediction algorithm.

4. Experiment Results

5. Conclusions

References

1. Sullivan, G.J.; Ohm, J.R.; Han, J.R.; Wiegand, T. Overview of the high efficiency video coding (HEVC) standard.IEEE Trans. Circuits Syst. Video Technol.2012,22, 1649–1668. [Google Scholar] [CrossRef]
2. Jiang, X.; Song, T.; Shi, W.; Shimamoto, T.; Wang, L. Fast coding unit size decision based on probabilistic graphical model in high efficiency video coding inter prediction.IEICE Trans. Inf. Syst.2016,99, 2836–2839. [Google Scholar] [CrossRef]
3. Laroche, G.; Jung, J.; Pesquet-Popescu, B. RD optimized coding for motion vector predictor selection.IEEE Trans. Circuits Syst. Video Technol.2008,18, 1247–1257. [Google Scholar] [CrossRef]
4. Lin, J.L.; Chen, Y.W.; Huang, Y.W.; Lei, S.M. Motion vector coding in the HEVC standard.IEEE J. Sel. Top. Signal Process.2013,7, 957–968. [Google Scholar] [CrossRef]
5. Peng, W.H.; Chen, C.C. An interframe prediction technique combining template matching prediction and block-motion compensation for high-efficiency video coding.IEEE Trans. Circuits Syst. Video Technol.2013,23, 1432–1446. [Google Scholar] [CrossRef]
6. Kim, I.K.; Lee, S.; Cheon, M.S.; Lee, T.; Park, J. Coding efficiency improvement of HEVC using asymmetric motion partitioning. In Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Seoul, South Korea, 27–29 June 2012; pp. 1–4. [Google Scholar]
7. Chien, W.D.; Liao, K.Y.; Yang, J.F. Enhanced AMVP mechanism based adaptive motion search range decision algorithm for fast HEVC coding. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Paris, France, 27–30 October 2014; pp. 3696–3699. [Google Scholar]
8. Zhang, Y.; Wang, H.; Li, Z. Fast coding unit depth decision algorithm for interframe coding in HEVC. In Proceedings of the Data Compression Conference (DCC), Snowbird, UT, USA, 20–22 March 2013; pp. 53–62. [Google Scholar]
9. Tok, M.; Glantz, A.; Krutz, A.; Sikora, T. Parametric motion vector prediction for hybrid video coding. In Proceedings of the Picture Coding Symposium (PCS), Krakow, Poland, 7–9 May 2012; pp. 381–384. [Google Scholar]
10. Tok, M.; Eiselein, V.; Sikora, T. Motion modeling for motion vector coding in HEVC. In Proceedings of the Picture Coding Symposium (PCS), Cairns, QLD, Australia, 31 May–3 June 2015; pp. 154–158. [Google Scholar]
11. Springer, D.; Simmet, F.; Niederkorn, D.; Kaup, A. Robust Rotational Motion Estimation for efficient HEVC compression of 2D and 3D navigation video sequences. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013; pp. 1379–1383. [Google Scholar]
12. Rosewarne, C. High Efficiency Video Coding (HEVC) Test Model 16 (HM 16). Document JCTVC-V1002, JCT-VC. October 2015. [Google Scholar]
13. Lei, J.; Duan, J.; Wu, F.; Ling, N.; Hou, C. Fast Mode Decision Based on Grayscale Similarity and Inter-View Correlation for Depth Map Coding in 3D-HEVC.IEEE Trans. Circuits Syst. Video Technol.2018,28, 706–718. [Google Scholar] [CrossRef]
14. Chen, M.; Wu, Y.; Yeh, C.; Lin, K.; Lin, S.D. Efficient CU and PU Decision Based on Motion Information for Interprediction of HEVC.IEEE Trans. Ind. Inform.2018,14, 4735–4745. [Google Scholar] [CrossRef]
15. Jiang, X.; Song, T.; Shimamoto, T.; Shi, W.; Wang, L. Spatio-Temporal Prediction Based Algorithm for Parallel Improvement of HEVC.IEICE Trans. Fundam. Electron. Commun. Comput. Sci.2015,98, 2229–2237. [Google Scholar] [CrossRef]
16. Kazui, K.; Shimada, S.; Koyama, J.; Nakagawa, A. Improvement on Simplified Motion Vector Prediction. Document JCTVC-E062, JCT-VC. March 2011. [Google Scholar]
17. Park, J.; Park, S.; Jeon, B. Improvement on Median Motion Vector of AMVP. Document JCTVC-D095, JCT-VC. January 2011. [Google Scholar]
18. Shilpa, M.; Sanjay, T.Motion Estimation Techniques for Digital Video Coding; Springerbriefs in Applied Sciences and Technology; Springer: Delhi, India, 2013. [Google Scholar]
19. Zhou, C.; Zhou, F.; Chen, Y. Spatio-temporal correlation-based fast coding unit depth decision for high efficiency video coding.J. Electron. Imaging2013,22, 043001. [Google Scholar] [CrossRef]
20. Bossen, F. Common Test Conditions and Software Reference Configurations. JCTVC-L1100, JCT-VC. January 2013. [Google Scholar]
21. Bjontegaard, G. Calculation of Average PSNR Differences between RD-Curves. ITU-T SG16 Q.6, VCEG-M33, JVT. April 2001. [Google Scholar]
22. Hanhart, P.; Rerabek, M.; Simone, F.D.; Ebrahimi, T. Subjective quality evaluation of the upcoming HEVC video compression standard.Proc. SPIE2012,8499, 84990V. [Google Scholar]