Many Web APIs use media codecs internally.For example, theWeb Audio API, and theWebRTC API.However these APIs do not allow developers to work with individual frames of a video stream and unmixed chunks of encoded audio or video.

Web developers have typically used WebAssembly in order to get around this limitation,and to work with media codecs in the browser.However, this requires additional bandwidth to download codecs that already exist in the browser,reducing performance and power efficiency, and adding additional development overhead.

The WebCodecs API provides access to codecs that are already in the browser.It gives access to raw video frames, chunks of audio data, image decoders, audio and video encoders and decoders.

The WebCodecs API uses an asynchronousprocessing model. Each instanceof an encoder or decoder maintains an internal, independent processing queue. When queueing a substantial amount of work, it's important tokeep this model in mind.

Methods namedconfigure(),encode(),decode(), andflush() operate asynchronously by appending control messagesto the end the queue, while methods namedreset() andclose() synchronously abort all pending work and purge theprocessing queue. Afterreset(), more work may be queued following a call toconfigure(), butclose() is a permanent operation.

Methods namedflush() can be used to wait for the completion of all work that was pending at the timeflush() was called. However, itshould generally only be called once all desired work is queued. It is not intended to force progress at regular intervals. Calling itunnecessarily will affect encoder quality and cause decoders to require the next input to be a key frame.

There is currently no API for demuxing media containers. Developers working with containerized media will need to implement their ownor use third party libraries. E.g.,MP4Box.js orjswebm can beused to demux audio and video data intoEncodedAudioChunk andEncodedVideoChunk objects respectively.

AudioDecoder

DecodesEncodedAudioChunk objects.

VideoDecoder

DecodesEncodedVideoChunk objects.

AudioEncoder

EncodesAudioData objects.

VideoEncoder

EncodesVideoFrame objects.

EncodedAudioChunk

Represents codec-specific encoded audio bytes.

EncodedVideoChunk

Represents codec-specific encoded video bytes.

AudioData

Represents unencoded audio data.

VideoFrame

Represents a frame of unencoded video data.

VideoColorSpace

Represents the color space of a video frame.

ImageDecoder

Unpacks and decodes image data, giving access to the sequence of frames in an animated image.

ImageTrackList

Represents the list of tracks available in the image.

ImageTrack

Represents an individual image track.

In the following example, frames are returned from aMediaStreamTrackProcessor, then encoded.See the full example and read more about it in the articleVideo processing with WebCodecs.

let frame_counter = 0;const track = stream.getVideoTracks()[0];const media_processor = new MediaStreamTrackProcessor(track);const reader = media_processor.readable.getReader();while (true) {  const result = await reader.read();  if (result.done) break;  let frame = result.value;  if (encoder.encodeQueueSize > 2) {    // Too many frames in flight, encoder is overwhelmed    // let's drop this frame.    frame.close();  } else {    frame_counter++;    const insert_keyframe = frame_counter % 150 === 0;    encoder.encode(frame, { keyFrame: insert_keyframe });    frame.close();  }}

• Video processing with WebCodecs
• WebCodecs API Samples
• Real-Time Video Processing with WebCodecs and Streams: Processing Pipelines
• Video Frame Processing on the Web – WebAssembly, WebGPU, WebGL, WebCodecs, WebNN, and WebTransport