Network Working Group A. FilippovInternet Draft Huawei TechnologiesIntended status: Informational A. Norkin Netflix J.R. Alvarez Huawei TechnologiesExpires: December 27, 2016 June 27, 2016 <Video Codec Requirements and Evaluation Methodology>draft-ietf-netvc-requirements-02.txtStatus of this Memo This Internet-Draft is submitted in full conformance with the provisions ofBCP 78 andBCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. The list of current Internet-Drafts can be accessed athttp://datatracker.ietf.org/drafts/current/ Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed athttp://www.ietf.org/1id-abstracts.html The list of Internet-Draft Shadow Directories can be accessed athttp://www.ietf.org/shadow.html This Internet-Draft will expire on December 27, 2016.Copyright Notice Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject toBCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with<Filippov> Expires December 27, 2016 [Page 1]
Internet-Draft Video Codec Requirements and Evaluation June 2016 respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of theTrust Legal Provisions and are provided without warranty as described in the Simplified BSD License.Abstract This document provides requirements for a video codec designed mainly for use over the Internet. In addition, an evaluation methodology needed for measuring the parameters (compression efficiency, computational complexity, etc.) to ensure whether the stated requirements are fulfilled or not.Table of Contents1. Introduction...................................................32. Applications...................................................32.1. Internet Video Streaming..................................42.2. Internet Protocol Television (IPTV).......................42.3. Video conferencing........................................72.4. Video sharing.............................................82.5. Screencasting.............................................92.6. Game streaming...........................................102.7. Video monitoring / surveillance..........................113. Requirements..................................................113.1. General requirements.....................................133.2. Basic requirements.......................................133.2.1. Input source formats:...............................133.2.2. Coding delay........................................143.2.3. Complexity..........................................143.2.4. Scalability.........................................143.2.5. Error resilience....................................143.3. Optional requirements....................................153.3.1. Input source formats................................153.3.2. Scalability.........................................153.3.3. Complexity..........................................154. Evaluation methodology........................................154.1. Compression performance evaluation.......................154.2. Reference software.......................................155. Security Considerations.......................................176. Conclusions...................................................177. References....................................................187.1. Normative References.....................................187.2. Informative References...................................188. Acknowledgments...............................................19Appendix A. Abbreviations used in the text of this document......20Appendix B. Used terms...........................................21<Filippov> Expires December 27, 2016 [Page 2]
Internet-Draft Video Codec Requirements and Evaluation June 20161. Introduction In this document, the requirements for a video codec designed mainly for use over the Internet are presented. The requirements encompass a wide range of applications that use data transmission over the Internet including IPTV (broadcasting over IP-based networks), peer- to-peer video conferencing, video sharing, screencasting, and video monitoring/ surveillance. For each application, typical resolutions, frame-rates and picture access modes are presented. Specific requirements related to data transmission over packet-loss networks are considered as well. In this document, when we discuss data protection techniques we only refer to methods designed and implemented to protect data inside the video codec since there are many existing techniques that protect generic data transmitted over packet-loss networks. From the theoretical point of view, both packet-loss and bit-error robustness can be beneficial for video codecs. In practice, packet losses are a more significant problem than bit corruption in IP networks. It is worth noting that there is an evident interdependence between possible amount of delay and the necessity of error robust video streams: o If an amount of delay is not crucial for an application, then reliable transport protocols such as TCP that resends undelivered packets can be used to guarantee correct decoding of transmitted data. o If the amount of delay must be kept low, then either data transmission should be error free (e.g., by using managed networks) or compressed video stream should be error resilient. Thus, error resilience can be useful for delay-critical applications to provide low delay in packet-loss environment.2. Applications In this chapter, an overview of video codec applications that are currently available on the Internet market is presented. It is worth noting that there are different use cases for each application that define a target platform, and hence there are different types of communication channels involved (e.g., wired or wireless channels) that are characterized by different quality of service as well as bandwidth; for instance, wired channels are considerably more error- free than wireless channels and therefore require different QoS approaches. The target platform, the channel bandwidth and the channel quality determine resolutions, frame-rates and quality or bit-rates for video streams to be encoded or decoded. By default,<Filippov> Expires December 27, 2016 [Page 3]
Internet-Draft Video Codec Requirements and Evaluation June 2016 color format YCbCr 4:2:0 is assumed for the application scenarios listed below.2.1. Internet Video Streaming Typical content for this application is movies, TV-series and shows, and animation. Internet video streaming uses a variety of client devices and has to operate under changing network conditions. For this reason, an adaptive streaming model has been widely adopted. Video material is encoded at different quality levels and different resolutions, which are then chosen by a client depending on its capabilities and current network bandwidth. An example combination of resolutions and bitrates is shown in a Table 1 below. +----------------------+-------------------------+-----------------+ | Resolution * | Frame-rate, fps | PAM | +----------------------+-------------------------+-----------------+ +----------------------+-------------------------+-----------------+ | 4K, 3840x2160 | 24/1.001, 24, 25, | RA | +----------------------+ +-----------------+ | 2K (1080p), 1920x1080| 30/1.001, 30, 50, | RA | +----------------------+ +-----------------+ | 1080i, 1920x1080* | 60/1.001, 60, 100, | RA | +----------------------+ +-----------------+ | 720p, 1280x720 | 120/1.001, 120 | RA | +----------------------+ +-----------------+ | 576p (EDTV), 720x576 | The set of frame-rates | RA | +----------------------+ +-----------------+ | 576i (SDTV), 720x576*| presented in this table | RA | +----------------------+ +-----------------+ | 480p (EDTV), 720x480 | is taken from Table 2 | RA | +----------------------+ +-----------------+ | 480i (SDTV), 720x480*| in [1] | RA | +----------------------+ +-----------------+ | 512x384 | | RA | +----------------------+ +-----------------+ | QVGA, 320x240 | | RA | +----------------------+-------------------------+-----------------+ Table 1. Internet Video Streaming: typical values of resolutions, frame-rates, and RAPs NB *: Interlaced content can be handled at the higher system level and not necessarily by using specialized video coding tools. It is included in this table only for the sake of completeness as most video content today is in progressive format.<Filippov> Expires December 27, 2016 [Page 4]
Internet-Draft Video Codec Requirements and Evaluation June 2016 A video encoding pipeline in on-demand Internet video streaming typically operates as follows: o Video is encoded in the cloud by software encoders. o Source video is split into chunks, each of which is encoded separately, in parallel. o Closed-GOP encoding with 2-5 second intra-picture intervals (or more) is used. o Encoding is perceptually optimized. Perceptual quality is important and should be considered during the codec development. Characteristics and requirements of this application scenario are as follows: o High encoder complexity (up to 10x and more) can be tolerated since encoding happens once and in parallel for different segments. o Decoding complexity should be kept at reasonable levels to enable efficient decoder implementation. o Support and efficient encoding of a wide range of content types and formats is required: o High Dynamic Range (HDR), Wide Color Gamut (WCG), high resolution (currently, up to 4K), high frame-rate content are important use cases, the codec should be able to encode such content efficiently. o Coding efficiency improvement at both lower and higher resolutions is important since low resolutions are used when streaming in low bandwidth conditions. o Improvement on both "easy" and "difficult" content in terms of compression efficiency at the same quality level contributes to the overall bitrate/storage savings. o Film grain (and sometimes other types of noise) is often present in the streaming movie-type content and is usually a part of the creative intent. o Significant improvements in compression efficiency between generations of video standards are desirable since this scenario typically assumes long-term support of legacy video codecs.<Filippov> Expires December 27, 2016 [Page 5]
Internet-Draft Video Codec Requirements and Evaluation June 2016 o Random access points are inserted frequently (one per 2-5 seconds) to enable switching between resolutions and fast-forward playback. o Elementary stream should have a model that allows easy parsing and identification of the sample components. o Middle QP values are normally used in streaming, this is also the range where compression efficiency is important for this scenario. . Scalability or other forms of supporting multiple quality representations are beneficial if they do not incur significant bitrate overhead and if mandated in the first version.2.2. Internet Protocol Television (IPTV) This is a service for delivering television content over IP-based networks. IPTV may be classified into two main groups based on the type of delivery, as follows: o unicast (e.g., for video on demand), where delay is not crucial; o multicast/broadcast (e.g., for transmitting news) where zapping, i.e. stream changing, delay is important. In the IPTV scenario, traffic is transmitted over managed (QoS- based) networks. Typical content used in this application is news, movies, cartoons, series, TV shows, etc. One important requirement for both groups is Random access to pictures, i.e. random access period (RAP) should be kept small enough (approximately, 1-5 seconds). Optional requirements are as follows: o Temporal (frame-rate) scalability; o Resolution and quality (SNR) scalability. For this application, typical values of resolutions, frame-rates, and RAPs are presented in Table 2. +----------------------+-------------------------+-----------------+ | Resolution * | Frame-rate, fps | PAM | +----------------------+-------------------------+-----------------+ +----------------------+-------------------------+-----------------+ | 2160p (4K),3840x2160 | 24/1.001, 24, 25, | RA |<Filippov> Expires December 27, 2016 [Page 6]
Internet-Draft Video Codec Requirements and Evaluation June 2016 +----------------------+ +-----------------+ | 1080p, 1920x1080 | 30/1.001, 30, 50, | RA | +----------------------+ +-----------------+ | 1080i, 1920x1080* | 60/1.001, 60, 100, | RA | +----------------------+ +-----------------+ | 720p, 1280x720 | 120/1.001, 120 | RA | +----------------------+ +-----------------+ | 576p (EDTV), 720x576 | The set of frame-rates | RA | +----------------------+ +-----------------+ | 576i (SDTV), 720x576*| presented in this table | RA | +----------------------+ +-----------------+ | 480p (EDTV), 720x480 | is taken from Table 2 | RA | +----------------------+ +-----------------+ | 480i (SDTV), 720x480*| in [1] | RA | +----------------------+-------------------------+-----------------+ Table 2. IPTV: typical values of resolutions, frame-rates, and RAPs NB *: Interlaced content can be handled at the higher system level and not necessarily by using specialized video coding tools. It is included in this table only for the sake of completeness as most video content today is in progressive format.2.3. Video conferencing This is a form of video connection over the Internet. This form allows users to establish connections to two or more people by two- way video and audio transmission for communication in real-time. For this application, both stationary and mobile devices can be used. The main requirements are as follows: o Delay should be kept as low as possible (the preferable and maximum end-to-end delay values should be less than 100 ms [7] and 320 ms [2], respectively); o Temporal (frame-rate) scalability; o Error robustness. Support of resolution and quality (SNR) scalability is highly desirable. For this application, typical values of resolutions, frame-rates, and RAPs are presented in Table 3. +----------------------+-------------------------+----------------+ | Resolution | Frame-rate, fps | PAM | +----------------------+-------------------------+----------------+ +----------------------+-------------------------+----------------+ | 1080p, 1920x1080 | 15, 30 | FIZD |<Filippov> Expires December 27, 2016 [Page 7]
Internet-Draft Video Codec Requirements and Evaluation June 2016 +----------------------+-------------------------+----------------+ | 720p, 1280x720 | 30, 60 | FIZD | +----------------------+-------------------------+----------------+ | 4CIF, 704x576 | 30, 60 | FIZD | +----------------------+-------------------------+----------------+ | 4SIF, 704x480 | 30, 60 | FIZD | +----------------------+-------------------------+----------------+ | VGA, 640x480 | 30, 60 | FIZD | +----------------------+-------------------------+----------------+ | 360p, 640x360 | 30, 60 | FIZD | +----------------------+-------------------------+----------------+ Table 3. Video conferencing: typical values of resolutions, frame- rates, and RAPs2.4. Video sharing This is a service that allows people to upload and share video data (using live streaming or not) and to watch them. It is also known as video hosting. A typical User-generated Content (UGC) scenario for this application is to capture video using mobile cameras such as GoPro or cameras integrated into smartphones (amateur video). The main requirements are as follows: o Random access to pictures for downloaded video data; o Temporal (frame-rate) scalability; o Error robustness. Support of resolution and quality (SNR) scalability is highly desirable. For this application, typical values of resolutions, frame-rates, and RAPs are presented in Table 4. +----------------------+-------------------------+----------------+ | Resolution | Frame-rate, fps | PAM | +----------------------+-------------------------+----------------+ +----------------------+-------------------------+----------------+ | 2160p (4K),3840x2160 | 24, 25, 30, 48, 50, 60 | RA | +----------------------+-------------------------+----------------+ | 1440p (2K),2560x1440 | 24, 25, 30, 48, 50, 60 | RA | +----------------------+-------------------------+----------------+ | 1080p, 1920x1080 | 24, 25, 30, 48, 50, 60 | RA | +----------------------+-------------------------+----------------+ | 720p, 1280x720 | 24, 25, 30, 48, 50, 60 | RA | +----------------------+-------------------------+----------------+ | 480p, 854x480 | 24, 25, 30, 48, 50, 60 | RA |<Filippov> Expires December 27, 2016 [Page 8]
Internet-Draft Video Codec Requirements and Evaluation June 2016 +----------------------+-------------------------+----------------+ | 360p, 640x360 | 24, 25, 30, 48, 50, 60 | RA | +----------------------+-------------------------+----------------+ Table 4. Video sharing: typical values of resolutions, frame-rates [8], and RAPs2.5. Screencasting This is a service that allows users to record and distribute computer desktop screen output. This service requires efficient compression of computer-generated content with high visual quality (up to visually and mathematically lossless) [9]. Currently, this application includes business presentations (powerpoint, word documents, email messages, etc.), animation (cartoons), gaming content, data visualization, i.e. such type of content that is characterized by fast motion, rotation, smooth shade, 3D effect, highly saturated colors with full resolution, clear textures and sharp edges with distinct colors [9]), virtual desktop infrastructure (VDI), screen/desktop sharing and collaboration, supervisory control and data acquisition (SCADA) display, automotive/navigation display, cloud gaming, factory automation display, wireless display, display wall, digital operating room (DiOR), etc. For this application, an important requirement is the support of a wide range of video formats (e.g., RGB) in addition to YCbCr 4:2:0 and YCbCr 4:4:4 [9]. For this application, typical values of resolutions, frame-rates, and RAPs are presented in Table 5. +----------------------+-------------------------+----------------+ | Resolution | Frame-rate, fps | PAM | +----------------------+-------------------------+----------------+ +----------------------+-------------------------+----------------+ | Input color format: RGB 4:4:4 | +----------------------+-------------------------+----------------+ | 5k, 5120x2880 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | 4k, 3840x2160 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | WQXGA, 2560x1600 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | WUXGA, 1920x1200 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | WSXGA+, 1680x1050 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | WXGA, 1280x800 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | XGA, 1024x768 | 15, 30, 60 | AI, RA, FIZD |<Filippov> Expires December 27, 2016 [Page 9]
Internet-Draft Video Codec Requirements and Evaluation June 2016 +----------------------+-------------------------+----------------+ | SVGA, 800x600 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | VGA, 640x480 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | Input color format: YCbCr 4:4:4 | +----------------------+-------------------------+----------------+ | 5k, 5120x2880 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | 4k, 3840x2160 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | 1440p (2K), 2560x1440| 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | 1080p, 1920x1080 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ | 720p, 1280x720 | 15, 30, 60 | AI, RA, FIZD | +----------------------+-------------------------+----------------+ Table 5. Screencasting for RGB and YCbCr 4:4:4 format: typical values of resolutions, frame-rates, and RAPs2.6. Game streaming This is a service that provides game content over the Internet to different local devices such as notebooks, gaming tablets, etc. In this category of applications, server renders 3D games in cloud server, and streams the game to any device with a wired or wireless broadband connection [10]. There are low latency requirements for transmitting user interactions and receiving game data in less than a turn-around delay of 100 ms. This allows anyone to play (or resume) full featured games from anywhere in the Internet [10]. An example of this application is Nvidia Grid [10]. Another category application is broadcast of video games played by people over the Internet in real time or for later viewing [10]. There are many companies such as Twitch, YY in China enable game broadcasting [10]. Games typically contain a lot of sharp edges and large motion [10]. The main requirements are as follows: o Random access to pictures for game broadcasting; o Temporal (frame-rate) scalability; o Error robustness. Support of resolution and quality (SNR) scalability is highly desirable. For this application, typical values of resolutions, frame-rates, and RAPs are similar to ones presented in Table 5.<Filippov> Expires December 27, 2016 [Page 10]
Internet-Draft Video Codec Requirements and Evaluation June 20162.7. Video monitoring / surveillance This is a type of live broadcasting over IP-based networks. Video streams are sent to many receivers at the same time. A new receiver may connect to the stream at an arbitrary moment, so random access period should be kept small enough (approximately, ~1-5 seconds). Data are transmitted publicly in the case of video monitoring and privately in the case of video surveillance, respectively. For IP- cameras that have to capture, process and encode video data, complexity including computational and hardware complexity as well as memory bandwidth should be kept low to allow real-time processing. In addition, support of high dynamic range and a monochrome mode (e.g., for infrared cameras) as well as resolution and quality (SNR) scalability is an essential requirement for video surveillance. For this application, typical values of resolutions, frame-rates, and RAPs are presented in Table 6. +----------------------+-------------------------+-----------------+ | Resolution | Frame-rate, fps | PAM | +----------------------+-------------------------+-----------------+ +----------------------+-------------------------+-----------------+ | 2160p (4K),3840x2160 | 12 | RA, FIZD | +----------------------+-------------------------+-----------------+ | 5Mpixels, 2560x1920 | 12 | RA, FIZD | +----------------------+-------------------------+-----------------+ | 1080p, 1920x1080 | 25 | RA, FIZD | +----------------------+-------------------------+-----------------+ | 1.3Mpixels, 1280x960 | 25, 30 | RA, FIZD | +----------------------+-------------------------+-----------------+ | 720p, 1280x720 | 25, 30 | RA, FIZD | +----------------------+-------------------------+-----------------+ | SVGA, 800x600 | 25, 30 | RA, FIZD | +----------------------+-------------------------+-----------------+ Table 6. Video monitoring / surveillance: typical values of resolutions, frame-rates, and RAPs3. Requirements Taking the requirements discussed above for specific video applications, this chapter proposes requirements for an internet video codec.3.1. General requirements 3.1.1. The most basic requirement is coding efficiency, i.e. compression performance. It should provide higher coding efficiency over state-of-the-art video codecs such as HEVC/H.265 and VP9, at<Filippov> Expires December 27, 2016 [Page 11]
Internet-Draft Video Codec Requirements and Evaluation June 2016 least by 20-25% in accordance with the methodology described inSection 4.1 of this document. For higher resolutions, the coding efficiency is expected to be higher than for lower resolutions. 3.1.2. Good quality specification and well-defined profiles and levels are required to enable device interoperability and facilitate decoder implementations. An example of codec levels to be supported is presented in Table 7. +------------------------------------------------------------------+ | Level | Example picture resolution at highest frame rate | +-------------+----------------------------------------------------+ | | 128x96(12,288*)@30.0 | | 1 | 176x144(25,344*)@15.0 | +-------------+----------------------------------------------------+ | 2 | 352x288(101,376*)@30.0 | +-------------+----------------------------------------------------+ | | 352x288(101,376*)@60.0 | | 3 | 640x360(230,400*)@30.0 | +-------------+----------------------------------------------------+ | | 640x360(230,400*)@60.0 | | 4 | 960x540(518,400*)@30.0 | +-------------+----------------------------------------------------+ | | 720x576(414,720*)@75.0 | | 5 | 960x540(518,400*)@60.0 | | | 1280x720(921,600*)@30.0 | +-------------+----------------------------------------------------+ | | 1,280x720(921,600*)@68.0 | | 6 | 2,048x1,080(2,211,840*)@30.0 | +-------------+----------------------------------------------------+ | | 1,280x720(921,600*)@120.0 | | 7 | 2,048x1,080(2,211,840*)@60.0 | +-------------+----------------------------------------------------+ | | 1,920x1,080(2,073,600*)@120.0 | | 8 | 3,840x2,160(8,294,400*)@30.0 | | | 4,096x2,160(8,847,360*)@30.0 | +-------------+----------------------------------------------------+ | | 1,920x1,080(2,073,600*)@250.0 | | 9 | 4,096x2,160(8,847,360*)@60.0 | +-------------+----------------------------------------------------+ | | 1,920x1,080(2,073,600*)@300.0 | | 10 | 4,096x2,160(8,847,360*)@120.0 | +-------------+----------------------------------------------------+ | | 3,840x2,160(8,294,400*)@120.0 | | 11 | 8,192x4,320(35,389,440*)@30.0 | +-------------+----------------------------------------------------+ | | 3,840x2,160(8,294,400*)@250.0 |<Filippov> Expires December 27, 2016 [Page 12]
Internet-Draft Video Codec Requirements and Evaluation June 2016 | 12 | 8,192x4,320(35,389,440*)@60.0 | +-------------+----------------------------------------------------+ | | 3,840x2,160(8,294,400*)@300.0 | | 13 | 8,192x4,320(35,389,440*)@120.0 | +-------------+----------------------------------------------------+ Table 7. Codec levels NB *: The quantities of pixels are presented for such applications where a picture can have an arbitrary size (e.g., screencasting) 3.1.3. High-level syntax should allow extensibility, so that new features can be supported easily by using metadata (e.g., such as SEI messages, VUI, headers). 3.1.4. Any elementary stream should have a model that allows easy parsing and identification of the sample components (such as ISO/IEC14496-10, Annex B or ISO/IEC 14496-15). 3.1.5. Perceptual quality tools (such as adaptive QP and quantization matrices) should be supported by the codec bit-stream. 3.1.6. The codec specification shall define a buffer model, such as hypothetical reference decoder (HRD). 3.1.7. Specifications providing integration with system and delivery layers should be developed.3.2. Basic requirements 3.2.1. Input source formats: o Bit depth: 8- and 10-bits (up to 12-bits for a high profile) per color component; o Color sampling formats: o YCbCr 4:2:0; o YCbCr 4:4:4, YCbCr 4:2:2 and YCbCr 4:0:0 (preferably in different profile(s)). o For profiles with bit depth of 10 bits per sample or higher, support of high dynamic range and wide color gamut. o Support of arbitrary resolution for such applications where a picture can have an arbitrary size (e.g., in screencasting)<Filippov> Expires December 27, 2016 [Page 13]
Internet-Draft Video Codec Requirements and Evaluation June 2016 3.2.2. Coding delay: o Support of configurations with zero structural delay also referred to as "low-delay" configurations. o Note 1: end-to-end delay should be up to 320 ms [2] but it's preferable value should be less than 100 ms [7] o Support of configurations with non-zero structural delay (such as out-of-order or multi-pass encoding) for applications without low-delay requirements if such configurations provide additional compression efficiency improvements. 3.2.3. Complexity: o Feasible real-time implementation of both an encoder and a decoder supporting a chosen subset of tools for hardware and software implementation on a wide range of state-of-the-art platforms. The real-time encoder tools subset should provide sufficient improvement in compression efficiency and take into account complexity of hardware and software encoder implementations. o High-complexity software encoder implementations used by offline encoding applications can have 10x or more complexity increase compared to state-of-the-art video compression technologies such as HEVC/H.265 and VP9. 3.2.4. Scalability: o Temporal (frame-rate) scalability should be supported. 3.2.5. Error resilience: o Error resilience tools that are complementary to the error protection mechanisms implemented on transport level should be supported. o The codec should support mechanisms that facilitate packetization of a bitstream for common network protocols. o Packetization mechanisms should enable frame-level error recovery by means of retransmission or error concealment.<Filippov> Expires December 27, 2016 [Page 14]
Internet-Draft Video Codec Requirements and Evaluation June 20163.3. Optional requirements 3.3.1. Input source formats o Bit depth: up to 16-bits per color component; o Color sampling formats: RGB 4:4:4. o Auxiliary channel (e.g., alpha channel) support. 3.3.2. Scalability: o Resolution and quality (SNR) scalability that provide low compression efficiency penalty (up to 5% of BD-rate [12] increase per layer with reasonable increase of both computational and hardware complexity) can be supported in the main profile of the codec being developed by the NETVC WG. Otherwise, a separate profile is needed to support these types of scalability. o Computational complexity scalability(i.e. computational complexity is decreasing along with degrading picture quality) is desirable. 3.3.3. Complexity: Tools that enable parallel processing (e.g., slices, tiles, wave front propagation processing) at both encoder and decoder sides are highly desirable for many applications. o High-level multi-core parallelism: encoder and decoder operation, especially entropy encoding and decoding, should allow multiple frames or sub-frame regions (e.g. 1D slices, 2D tiles, or partitions) to be processed concurrently, either independently or with deterministic dependencies that can be efficiently pipelined o Low-level instruction set parallelism: favor algorithms that are SIMD/GPU friendly over inherently serial algorithms4. Evaluation methodology4.1. Compression performance evaluation As shown in Fig.1, compression performance testing is performed in 3 ranges that encompass 12 different bit-rate values: o Low bit-rate range (LBR) is the range that contains the 4 lowest bit-rates of the 12 specified bit-rates;<Filippov> Expires December 27, 2016 [Page 15]
Internet-Draft Video Codec Requirements and Evaluation June 2016 o Medium bit-rate range (MBR) is the range that contains the 4 medium bit-rates of the 12 specified bit-rates; o High bit-rate range (HBR) is the range that contains the 4 highest bit-rates of the 12 specified bit-rates. To avoid any rate control mechanisms that can significantly impact evaluation results, just nominal values of bit-rates should be specified in a separate document on Internet video codec testing. The deviation between nominal and actual values of bit-rates obtained for both reference and tested codecs should be less than the threshold value defined in the above-mention document on Internet video codec testing. This deviation is calculated as follows: D = abs((BRa - BRn)/ BRn)*100%, where BRn is a nominal value of bit-rate, BRa is an actual value of bit-rate obtained for either reference or tested codecs. Bit-rate ------------------------------------------------------------> BR0 BR1 BR2 BR3 BR4 BR5 BR6 BR7 BR8 BR9 BR10 BR11 ^ ^ ^ ^ ^ ^ |------LBR------| |------MBR------| |------HBR------| Figure 1 Bit-rate ranges for the CBR mode To assess the quality of output (decoded) sequences, two indexes, PSNR [3] and MS-SSIM [3,11], should be separately calculated for each color plane. For obtaining an integral estimation, BD-rate [12] should be computed for each range and each quality index. Finally, 18 values should be obtained for a color format, which contains 3 color planes (e.g., for YCbCr or RGB). A list of video sequences that should be used for testing as well as the 6 values of bit-rates are defined in a separate document. Testing processes should use the information on the codec applications presented in this document. As the reference for evaluation, the HEVC/H.265 codec [4,5] must be used. The reference source code of the HEVC/H.265 codec can be found at [6]. The HEVC/H.265 codec must be configured according to [13] and Table 8. +----------------------+-------------------------------------------+ | Intra-period, second | HEVC/H.265 encoding mode according to [13]| +----------------------+-------------------------------------------+ | AI | Intra Main or Intra Main10 | +----------------------+-------------------------------------------+<Filippov> Expires December 27, 2016 [Page 16]
Internet-Draft Video Codec Requirements and Evaluation June 2016 | RA | Random access Main or | | | Random access Main10 | +----------------------+-------------------------------------------+ | JIZD | Low delay Main or | | | Low delay Main10 | +----------------------+-------------------------------------------+ Table 8. Intra-periods for different HEVC/H.265 encoding modes according to [13] In addition to the objective quality measures defined above, subjective evaluation must also be performed before adopting any new tool and a final codec standard. For subjective tests, the MOS-based evaluation procedure must be used as described in section 2.1 of [3]. For perception-oriented tools that primarily impact subjective quality, additional tests may also be individually assigned even for intermediate evaluation, subject to a decision of the NETVC WG.4.2. Reference software Reference software provided to the NETVC WG for candidate codecs should comprise a fully operational encoder supporting necessary rate controls, subjective quality optimization features and some degree of speed optimization and a "real-time" decoder.5. Security Considerations This document itself does not address any security considerations. However, it is worth noting that a codec implementation (for both an encoder and a decoder) should cover the worst case of computational complexity, memory bandwidth, and physical memory size (e.g., for decoded pictures used as references). Otherwise, it can be considered as a security vulnerability and lead to denial-of-service (DoS) in the case of attacks.6. Conclusions In this document, an overview of Internet video codec applications and typical use cases as well as a prioritized list of requirements for an Internet video codec are presented. An evaluation methodology for this codec is also proposed.<Filippov> Expires December 27, 2016 [Page 17]
Internet-Draft Video Codec Requirements and Evaluation June 20167. References7.1. Normative References [1] Recommendation ITU-R BT.2020-2: Parameter values for ultra- high definition television systems for production and international programme exchange, 2015. [2] Recommendation ITU-T G.1091: Quality of Experience requirements for telepresence services, 2014. [3] ISO/IEC PDTR 29170-1: Information technology -- Advanced image coding and evaluation methodologies -- Part 1: Guidelines for codec evaluation. [4] ISO/IEC 23008-2:2015. Information technology -- High efficiency coding and media delivery in heterogeneous environments -- Part 2: High efficiency video coding [5] Recommendation ITU-T H.265: High efficiency video coding, 2013. [6]https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/7.2. Informative References [7] S. Wenger, "The case for scalability support in version 1 of Future Video Coding," contribution COM 16-C 988 R1-E to ITU-T SG16/Q6, September 2015."Recommended upload encoding settings (Advanced)" [8] "Recommended upload encoding settings (Advanced)"https://support.google.com/youtube/answer/1722171?hl=en [9] H. Yu, K. McCann, R. Cohen, and P. Amon, "Requirements for future extensions of HEVC in coding screen content", ISO/IEC JTC1/SC29/WG11 MPEG2013/N14174, San Jose, USA, Jan. 2014 [10] Manindra Parhy, "Game streaming requirement for Future Video Coding," MPEG Contribution m36771, June 2015, Warsaw, Poland. [11] Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multi-scale structural similarity for image quality assessment," Invited Paper, IEEE Asilomar Conference on Signals, Systems and Computers, Nov. 2003, Vol. 2, pp. 1398-1402.<Filippov> Expires December 27, 2016 [Page 18]
Internet-Draft Video Codec Requirements and Evaluation June 2016 [12] G. Bjontegaard, "Calculation of average PSNR differences between RD-curves (VCEG-M33)," in VCEG Meeting (ITU-T SG16 Q.6), Austin, Texas, USA, Apr. 2-4 2001. [13] F. Bossen, "Common test conditions and software reference configurations," JCTVC-L1100, Geneva, Switzerland, Jan. 2013. [14]http://www.digitizationguidelines.gov/term.php?term=compressio nvisuallylossless)8. Acknowledgments9. The authors would like to thank Mr. Jiantong Zhou, Mr. Paul Coverdale, Mr. Vasily Rufitskiy, and Dr. Haitao Yang for many useful discussions on this document and their help while preparing it as well as Mr. Mo Zanaty, Dr. Minhua Zhou, Dr. Ali Begen, Mr. Thomas Daede, Mr. Jonathan Lennox, Dr. Timothy Terriberry, Mr. Peter Thatcher, Dr. Jean-Marc Valin, Mr. Jack Moffitt, Mr. Greg Coppa and Mr. Andrew Krupiczka for their valuable comments on different revisions of this document. This document was prepared using 2-Word-v2.0.template.dot.<Filippov> Expires December 27, 2016 [Page 19]
Internet-Draft Video Codec Requirements and Evaluation June 2016Appendix A. Abbreviations used in the text of this document +--------------+---------------------------------------------------+ | Abbreviation | Meaning | +--------------+---------------------------------------------------+ | AI | All-Intra (each picture is intra-coded) | | BD-Rate | Bjontegaard Delta Rate | | GOP | Group of Picture | | HBR | High Bit-rate Range | | HDR | High Dynamic Range | | HRD | Hypothetical Reference Decoder | | PAM | Picture Access Mode | | RA | Random Access | | RAP | Random Access Period | | IPTV | Internet Protocol Television | | FIZD | just the First picture is Intra-coded, Zero | | | structural Delay | | LBR | Low Bit-rate Range | | MBR | Medium Bit-rate Range | | MOS | Mean Opinion Score | | MS-SSIM | Multi-Scale Structural Similarity quality index | | OTT | Over-The-Top | | PSNR | Peak Signal-to-Noise Ratio | | QoS | Quality of Service | | QP | Quantization Parameter | | SEI | Supplemental Enhancement Information | | UGC | User-Generated Content | | VDI | Virtual Desktop Infrastructure | | VUI | Video Usability Information | | WCG | Wide Color Gamut | +--------------+---------------------------------------------------+<Filippov> Expires December 27, 2016 [Page 20]
Internet-Draft Video Codec Requirements and Evaluation June 2016Appendix B. Used terms +------------------+-----------------------------------------------+ | Term | Meaning | +------------------+-----------------------------------------------+ | High dynamic | is a set of techniques that allow a greater | | range imaging | dynamic range of exposures or values (i.e., | | | a wide range of values between light and dark | | | areas) than normal digital imaging techniques.| | | The intention is to accurately represent the | | | wide range of intensity levels found in such | | | examples as exterior scenes that include | | | light-colored items struck by direct sunlight | | | and areas of deep shadow [14]. | | | | | Random access | is the period of time between two closest | | period | independently decodable frames (pictures). | | | | | Visually | is a form or manner of lossy compression | | lossless | where the data that are lost after the file | | compression | is compressed and decompressed is not | | | detectable to the eye; the compressed data | | | appearing identical to the uncompressed | | | data [14]. | | | | | Wide color gamut | is a certain complete color subset (e.g., | | | considered in ITU-R BT.2020) that supports a | | | wider range of colors (i.e., an extended range| | | of colors that can be generated by a specific | | | input or output device such as a video camera,| | | monitor or printer and can be interpreted by | | | a color model) than conventional color gamuts | | | (e.g., considered in ITU-R BT.601 or BT.709). | +------------------+-----------------------------------------------+<Filippov> Expires December 27, 2016 [Page 21]
Internet-Draft Video Codec Requirements and Evaluation June 2016Authors' Addresses Alexey Filippov Huawei Technologies Email: alexey.filippov@huawei.com Andrey Norkin Netflix Email: anorkin@netflix.com Jose Roberto Alvarez Huawei Technologies Email: jose.roberto.alvarez@huawei.com<Filippov> Expires December 27, 2016 [Page 22]Datatracker
draft-ietf-netvc-requirements-02
This is an older version of an Internet-Draft that was ultimately published asRFC 8761.
| Document | Document type | This is an older version of an Internet-Draft that was ultimately published asRFC 8761. | |
|---|---|---|---|
| Select version | |||
| Compare versions | |||
| Authors | Alexey Filippov,Andrey Norkin,José Roberto Alvarez | ||
| Replaces | draft-filippov-netvc-requirements | ||
| RFC stream |   | ||
| Other formats | |||
| Additional resources | Mailing list discussion |