3.1.Internet Video Streaming

Typical content for this application is movies, TV series and shows, andanimation. Internet video streaming uses a variety of client devices and hasto operate under changing network conditions. For this reason, an adaptivestreaming model has been widely adopted. Video material is encoded atdifferent quality levels and different resolutions, which are then chosen by aclient depending on its capabilities and current network bandwidth. An examplecombination of resolutions and bitrates is shown inTable 1.¶

A video encoding pipeline in on-demand Internet video streaming typically operates as follows:¶

• Video is encoded in the cloud by software encoders.¶
• Source video is split into chunks, each of which is encoded separately, in parallel.¶
• Closed-GOP encoding with intrapicture intervals of 2-5seconds (or longer) is used.¶
• Encoding is perceptually optimized. Perceptual quality is important and should be considered during the codec development.¶

*Note: Interlaced content can be handled at the higher system level and not necessarily by using specialized video coding tools. It is included in this table only for the sake of completeness, as most video content today is in the progressive format.¶

**Note: The set of frame rates presented in this table is taken from Table 2 in[1].¶

The characteristics and requirements of this application scenario are as follows:¶

• High encoder complexity (up to 10x and more) can be tolerated sinceencoding happens once and in parallel for different segments.¶
• Decoding complexity should be kept at reasonable levels to enable efficient decoder implementation.¶

• Support and efficient encoding of a wide range of content types and formats is required:¶

  • High Dynamic Range (HDR), Wide Color Gamut (WCG), high-resolution(currently, up to 4K), and high-frame-rate content are important use cases; thecodec should be able to encode such content efficiently.¶
  • Improvement of coding efficiency at both lower and higher resolutions isimportant since low resolutions are used when streaming in low-bandwidthconditions.¶
  • Improvement on both "easy" and "difficult" content in terms of compression efficiency at the same quality level contributes to the overall bitrate/storage savings.¶
  • Film grain (and sometimes other types of noise) is often present in moviesand similar content; this is usually part of the creative intent.¶
• Significant improvements in compression efficiency between generations ofvideo standards are desirable since this scenario typically assumes long-termsupport of legacy video codecs.¶
• Random access points are inserted frequently (one per 2-5 seconds) to enable switching between resolutions and fast-forward playback.¶
• The elementary stream should have a model that allows easy parsing andidentification of the sample components.¶
• Middle QP values are normally used in streaming; this is also the rangewhere compression efficiency is important for this scenario.¶
• Scalability or other forms of supporting multiple quality representationsare beneficial if they do not incur significant bitrate overhead and ifmandated in the first version.¶

3.2.Internet Protocol Television (IPTV)

This is a service for delivering television content over IP-based networks. IPTV may be classified into two main groups based on the type of delivery, as follows:¶

• unicast (e.g., for video on demand), where delay is not crucial; and¶
• multicast/broadcast (e.g., for transmitting news) wherezapping (i.e., stream changing) delay is important.¶

In the IPTV scenario, traffic is transmitted over managed (QoS-based)networks. Typical content used in this application is news, movies, cartoons,series, TV shows, etc. One important requirement for both groups is that randomaccess to pictures (i.e., the random access period (RAP)) should be kept smallenough (approximately 1-5 seconds). Optional requirements are as follows:¶

• Temporal (frame-rate) scalability; and¶
• Resolution and quality (SNR) scalability.¶

For this application, typical values of resolutions, frame rates, and PAMsare presented inTable 2.¶

*Note: Interlaced content can be handled at the higher system level and not necessarily by using specialized video coding tools. It is included in this table only for the sake of completeness, as most video content today is in a progressive format.¶

**Note: The set of frame rates presented in this table is takenfrom Table 2 in[1].¶

3.3.Video Conferencing

This is a form of video connection over the Internet. This form allowsusers to establish connections to two or more people by two- way video andaudio transmission for communication in real time. For this application, bothstationary and mobile devices can be used. The main requirements are asfollows:¶

• Delay should be kept as low as possible (the preferable and maximumend-to-end delay values should be less than 100 ms[9] and 320 ms[2], respectively);¶
• Temporal (frame-rate) scalability; and¶
• Error robustness.¶

Support of resolution and quality (SNR) scalability is highlydesirable. For this application, typical values of resolutions, frame rates,and PAMs are presented inTable 3.¶

3.4.Video Sharing

This is a service that allows people to upload and share video data (usinglive streaming or not) and watch those videos. It is also known as video hosting. Atypical User-Generated Content (UGC) scenario for this application is tocapture video using mobile cameras such as GoPros or cameras integrated intosmartphones (amateur video). The main requirements are as follows:¶

• Random access to pictures for downloaded video data;¶
• Temporal (frame-rate) scalability; and¶
• Error robustness.¶

Support of resolution and quality (SNR) scalability is highlydesirable. For this application, typical values of resolutions, frame rates,and PAMs are presented inTable 4.¶

Typical values of resolutions and frame rates inTable 4 are taken from[10].¶

3.5.Screencasting

This is a service that allows users to record and distribute video data from a computer screen. This service requires efficient compression ofcomputer-generated content with high visual quality up to visually andmathematically (numerically) lossless[11]. Currently, this applicationincludes business presentations (PowerPoint, Word documents, email messages,etc.), animation (cartoons), gaming content, and data visualization. Thistype of content is characterized by fast motion, rotation, smooth shade,3D effect, highly saturated colors with full resolution, clear textures andsharp edges with distinct colors[11], virtual desktopinfrastructure (VDI),screen/desktop sharing and collaboration, supervisory control and dataacquisition (SCADA) display, automotive/navigation display, cloud gaming, factory automationdisplay, wireless display, display wall,digital operating room (DiOR), etc. For this application, an importantrequirement is the support of low-delay configurations with zero structuraldelay for a wide range of video formats (e.g., RGB) in addition to YCbCr 4:2:0and YCbCr 4:4:4[11]. For this application, typical values of resolutions,frame rates, and PAMs are presented inTable 5.¶

3.6.Game Streaming

This is a service that provides game content over the Internet to differentlocal devices such as notebooks and gaming tablets. In this category ofapplications, the server renders 3D games in a cloud server and streams the game toany device with a wired or wireless broadband connection[12]. There are low-latency requirements for transmitting user interactions andreceiving game data with a turnaround delay of less than 100 ms. This allowsanyone to play (or resume) full-featured games from anywhere on the Internet[12]. An example of this application is Nvidia Grid[12]. Another application scenario of this category is broadcast of video gamesplayed by people over the Internet in real time or for later viewing[12]. There are many companies, such as Twitch and YY in China, that enablegame broadcasting[12]. Games typically contain a lot ofsharp edges and large motion[12]. The main requirements areas follows:¶

• Random access to pictures for game broadcasting;¶
• Temporal (frame-rate) scalability; and¶
• Error robustness.¶

Support of resolution and quality (SNR) scalability is highlydesirable. For this application, typical values of resolutions, frame rates,and PAMs are similar to ones presented inTable 3.¶

3.7.Video Monitoring and Surveillance

This is a type of live broadcasting over IP-based networks. Video streamsare sent to many receivers at the same time. A new receiver may connect to thestream at an arbitrary moment, so the random access period should be keptsmall enough (approximately, 1-5 seconds). Data are transmitted publicly inthe case of video monitoring and privately in the case of videosurveillance. For IP cameras that have to capture, process, and encode videodata, complexity -- including computational and hardware complexity, as wellas memory bandwidth -- should be kept low to allow real-time processing. Inaddition, support of a high dynamic range and a monochrome mode (e.g., forinfrared cameras) as well as resolution and quality (SNR) scalability is anessential requirement for video surveillance.In some use cases, highvideo signal fidelity is required even after lossy compression. Typical valuesof resolutions, frame rates, and PAMs for video monitoring and surveillanceapplications are presented inTable 6.¶