JPEG XS
Filename extension	.jxs
Internet media type	image/jxsc, video/jxsv[1]
Magic number	0xFF10 FF50
Developed by	Joint Photographic Experts Group
Initial release	May 1, 2019; 6 years ago (2019-05-01)
Type of format	Lossy andlosslessimage compressionformat
Standard	ISO/IEC 21122
Website	jpeg.org/jpegxs/

JPEG XS (standardized asISO/IEC 21122) is an image and video codec that offers both visually and mathematically lossless quality. It is a special-purpose codec that is specifically designed to allow for low-complexity andlow-latency implementations.^[2][3] Target applications of the standard include streaming high-quality content forprofessional video over IP (SMPTEST 2022 andST 2110) in broadcast and other applications,virtual reality, drones, autonomous vehicles using cameras, gaming.^[2][4][5] Although there is not an official acronym definition, XS was chosen to highlight theextra small andextra speed characteristics of the codec.

Three main features are key to JPEG XS:

1. Visually transparent compression: XS compressed content is indistinguishable from the original uncompressed content (passing ISO/IEC 29170-2 tests, also called AIC-2) for compression ratios between 2:1 and 10:1.
2. Low latency: The total end-to-end latency, introduced by the JPEG XS compression-decompression cycle, is minimal. Depending on the configuration, XS typically imposes only between 1 and 32 lines of additional end-to-end latency, when compared to the same system using uncompressed video.
3. Lightweight: JPEG XS is designed to have lowcomputational and memory complexity, allowing for efficient low-power and low-resource implementations on various platforms such asCPU,GPU,FPGA andASIC.

Relying on these key features, JPEG XS is suitable for any application where uncompressed content is the norm, yet still allowing for significant savings in the required bandwidth usage, preserving quality and low latency. Among the targeted use cases are video transport over professional video links (likeSDI andprofessional video over IP), real-time video storage, memory buffers, omnidirectional video capture and rendering, and image sensor compression (for example in cameras and in the automotive industry).^[6] JPEG XS favors visually lossless quality in combination with low latency and low complexity, over data reduction through compression. It is not a direct competitor to alternative image codecs likeJPEG 2000 andJPEG XL or video codecs likeAV1,AVC/H.264 andHEVC/H.265 which tend to focus on compression efficiency.

Other important features are:

1. Exact bitrate allocation: JPEG XS allows an accurately targeted bitrate to perfectly match the available bandwidth (also referred to asconstant bitrate).
2. Multi-generation robustness: JPEG XS allows for at least 10 encoding-decoding cycles, without significant quality degradation.^[7] This feature allows for example transparently chaining of multiple devices that recompress the signal, without any significant quality degradation taking place.
3. Multi-platform interoperability: The algorithms used in JPEG XS allow for efficient implementations on different platforms, likeCPU,GPU,FPGA andASIC. Each of these platform architectures is best exploited when a specific degree of parallelism is available in the implementation. For instance, a multi-core CPU implementation will leverage a coarse-grained parallelism, while GPU or FPGA will work better with a fine-grained parallelism. Moreover, the choice of parallelism used in the implementation at the encoder will not affect that of the decoder. This means that real-time encoding and decoding between platforms is possible, without sacrificing the low-complexity, low-latency or high-quality properties.
4. Support for mathematicallossless coding (MLS): JPEG XS is also capable of coding images in a mathematically lossless way, to achieve perfect reconstruction at the decoder side (new profile supported by 2nd edition).
5. Support forHigh Dynamic Range (HDR) content: The current version of JPEG XS supports bit-depths of up to 16 bits per component, and it provides several parameterizable non-linear transforms to efficiently compress HDR content.
6. Support for RAW Bayer/CFA compression: JPEG XS has also the capability to compresscolor filter array (CFA) content, such as RAWBayer content produced by digital cameras. A special color transform, called Star-Tetrix, allows for efficient and direct compression of the original RAW sample values, without the need for converting the Bayer samples to RGB samples first.^[8]
7. Accurate flow control: A JPEG XS encoder continuously monitors the amount of bits sent out, and adjusts its rate allocation process to neither overflow nor underflow a normatively^[9] defined decoder input buffer.

JPEG XS is actively used in several application domains.^[10]

Video bandwidth requirements are growing as video resolutions, frame rates, bit depths, and the number of video streams are constantly increasing. Although the capacities of video links and communication channels are also growing, there are applications where the reduced data rate afforded by JPRG-XS is advantageous.

Both the broadcast and pro-AV markets are shifting towards AV-over-IP-based infrastructure, with a preference going toGigabit Ethernet links for remote production and faster fiber-based Ethernet for in-house facilities. Given the available bandwidth and infrastructure cost, relying on uncompressed video may be cost-prohibitive if 4K, 8K, increased bit depths for HDR, and higher framerates need to be supported.

With XS, it is for example possible to repurpose existing SDI cables to transport 4K60 over a single 3G-SDI (at 4:1 compression ratio), and even over a single HD-SDI (at 8:1 compression ratio). Similar scenarios can be used to transport 8K60 content over various SDI cable types (e.g. 6G-SDI and 12G-SDI). Alternatively, XS enables transporting 4K60 content over Gigabit Ethernet and 8K60 over 5G or10 Gigabit Ethernet, which would be impossible without compression.

The following table shows expected compression ranges for some typical use cases.

Related to the transport of video streams is the storage and retrieval of high-resolution streams where bandwidth limitations similarly apply. For instance, video cameras use internal storage likeSSD drives orSD cards to record high-resolution streams of images, yet the maximum data rates of such storage devices may be below that required to store uncompressed video in real time.

JPEG XS has built-in support for the direct compression of RAWBayer/CFA images using the Star-Tetrix Color Transform. This transform takes a RAW Bayer pattern image anddecorrelates the samples into a 4-component image with each component having only a quarter of the resolution.^[11] This means that the total amount of samples to further process and compress remains the same, yet the values are decorrelated similarly to a classical Multiple Component Transform.

Avoiding such conversion prevents information loss and allows this processing step to be done outside of the camera. This is advantageous because it allows deferringdemosaicing the Bayer content from the moment of capturing to the production phase, where choices regarding artistic intent and various settings can be better made; The demosaicing process is irreversible and requires certain choices, like the choice of interpolation algorithm or the level of noise reduction, to be made upfront. The demosaicing process can be power-hungry and will also introduce extra latency and complexity.

The JPEG XS coding system is anISO/IEC suite of standards that consists of the following parts:

RFC 9134^[12] describes a payload format for theReal-Time Transport Protocol (RTP) to carry JPEG XS encoded video. In addition, the recommendation also registers the officialmedia type registration for JPEG XS video asvideo/jxsv, along with its mapping of all parameters into theSession Description Protocol (SDP).

TheRTP payload format for JPEG XS, in turn, enables the use of JPEG XS inSMPTE 2110 environments using SMPTE ST 2110-22 for CBR compressed video transport.

See also TR-08,^[13] published by theVideo Services Forum.

ISO/IEC 13818-1:2023, known as MPEG-TS 9th edition, specifies carriage support for JPEG XS inMPEG transport streams.^[14] See also VSF TR-07^[15] published by theVideo Services Forum.

ANetworked Media Open Specifications (NMOS) standard that enables registration, discovery, and connection management of JPEG XS endpoints uses the AMWA IS-04 and IS-05 NMOS specifications. See AMWA BCP-006-01,^[16] published byAdvanced Media Workflow Association.

Internet Protocol Media Experience (IPMX) is a suite of open standards and specifications to enable the carriage of compressed and uncompressed video, audio, and data over IP networks for the pro AV market. JPEG XS is supported under IPMX via VSF TR-10-8^[17] and TR-10-11.^[18]

The JPEG committee started the standardization activity in 2016 with an open call for a high-performance, low-complexity image coding standard. The best-performing candidate technologies came from intoPIX and Fraunhofer IIS and formed the basis for the new standard. First implementations were demonstrated in April 2018 at theNAB Show and later that year at theInternational Broadcasting Convention. XS was also presented atCES in 2019. JPEG XS was formally standardized as ISO/IEC 21122 by theJoint Photographic Experts Group with the first edition published in 2019. A second edition was published in 2022,^[19] adding support for direct compression of raw CFA Bayer content, lossless compression, and support for 4:2:0 color subsampling. Today, the JPEG committee is still actively working on further improvements to XS, with the third edition published in 2024. This edition adds support for a temporal decorrelation technology in the wavelet domain, called Temporal Differential Coding (TDC).

The JPEG XS standard is a classicalwavelet-based still-image codec without any frame buffer. While the standard^[20] defines JPEG XS based on a hypothetical reference coder, JPEG XS is easier to explain through the steps a typical encoder performs:^[21]

Component up-scaling and optional component decorrelation: In the first step, the DC gain of the input data is removed and it is upscaled to a bit-precision of 20 bits. Optionally, a multi-component generation, identical to the JPEG 2000 RCT, is applied. This transformation is a lossless approximation of an RGB to YUV conversion, generating one luma and two chroma channels.

Wavelet transformation: Input data is spacially decorrelated by a 5/3Daubechies wavelet filter. While a five-stage transformation is performed in the horizontal direction, only 0 to 2 transformations are run in the vertical direction. The reason for this asymmetrical filter is to minimize latency.

Prequantization: The output of the wavelet filter is converted to a sign-magnitude representation and pre-quantized by a dead zonequantizer to 16-bit precision.

Rate control and quantization: The encoder determines by a non-normative process^[21] the rate of each possible quantization setting and then quantizes data by either a dead zone quantizer or a data-dependent uniform quantizer.

Entropy coding: JPEG XS uses minimalisticEntropy encoding for the quantized data which proceeds in up to four passes over horizontal lines of quantized wavelet coefficients. The steps are:

• Significance coding: In the (optional) first pass, the significance of 32 consecutive wavelet coefficients is coded by a single bit.
• Bitplane count coding: In the second pass, the number of non-zero bitplanes of groups of four coefficients each, the so-called "bitplane count", is entropy coded through aGolomb type code. This step may optionally use the bitplane counts of the preceding line as the source for prediction (Differential pulse-code modulation) and then encode only the prediction difference.
• Data coding:The third pass inserts the raw bitplane values into the codestream without further coding.
• Sign coding: In the last optional coding pass, the sign bits of all non-zero coefficients are inserted into the codestream. If this coding pass is not present, sign bits are included in the data coding pass for all coefficients.

Codestream packing: All entropy-coded data are packed into a linear stream of bits (grouped in byte multiples) along with all of the required image metadata. This sequence of bytes is called the codestream and its high-level syntax is based on the typical JPEG markers and marker segments syntax.^[22]

JPEG XS defines profiles (in ISO/IEC 21122-2^[9]) that define subsets of coding tools that conforming decoders shall support, by limiting the permitted parameter values and allowed markers. The following table represents an overview of all the profiles along with their most important properties. Please refer to the standard for a complete specification of each profile.

In addition, JPEG XS defines levels to represent a lower bound on the required throughput that conforming decoders need to support in the decoded image domain (also called the spatial domain). The following table lists the levels as defined by JPEG XS. The maximums are given in the context of the sampling grid, so they refer to a per-pixel value where each pixel represents one or more component values. However, in the context of Bayer data JPEG XS internally interprets the Bayer pattern as an interleaved grid of four components. This means that the number of sampling grid points required to represent a Bayer image is four times smaller than the total number of Bayer sample points. Each group of 2x2 (four) Bayer values gets interpreted as one sampling grid point with four components. Thus sensor resolutions should be divided by four to calculate the respective width, height and amount of sampling grid points. For this reason, all levels also bear double names. Please refer to the standard for a complete specification of each level.

Similarly to the concept of levels, JPEG XS defines sublevels to represent a lower bound on the required throughput that conforming decoders need to support in the encoded image domain. Each sublevel is defined by a nominal bit-per-pixel (Nbpp) value that indicates the maximum amount of bits per pixel for an encoded image of the maximum permissible number of sampling grid points according to the selected conformance level. Thus, decoders conforming to a particular level and sublevel shall conform to the following constraints derived from Nbpp:

• The maximum codestream size in bytes (from SOC to EOC, including all markers) is${\displaystyle S_{sl,max}=\lfloor \frac{L_{max}\times N_{bpp}}{8}\rfloor }$.
• The maximum admissible encoded throughput in bits per second is${\displaystyle R_{t,max}=R_{s,max}\times N_{bpp}}$.

The following table lists the existing sublevels and their respective nominal bpp values. Please refer to the standard for a complete specification of each level.

JPEG XS contains patented technology which is made available for licensing via the JPEG XS Patent Portfolio License (JPEG XS PPL). This license pool covers essential patents owned by Licensors for implementing the ISO/IEC 21122 JPEG XS video coding standard and is available underRAND terms.^[23]

• JPEG
• IEC standards
• ISO standards
• Lossy compression algorithms
• Image compression
• Raster graphics file formats

• Articles with short description
• Short description matches Wikidata