Repository files navigation

The WebCodecs API provides low-level access to media codecs, but provides no way of actually packaging (multiplexing)the encoded media into a playable file. This project implements a WebM/Matroska multiplexer in pure TypeScript, which ishigh-quality, fast and tiny, and supports video, audio and subtitles as well as live-streaming.

Demo: Muxing into a file

Demo: Streaming

Note: If you're looking to createMP4 files, check outmp4-muxer, thesister library to webm-muxer.

The following is an example for a common usage of this library:

import{Muxer,ArrayBufferTarget }from'webm-muxer';letmuxer=newMuxer({target:newArrayBufferTarget(),video:{codec:'V_VP9',width:1280,height:720}});letvideoEncoder=newVideoEncoder({output:(chunk,meta)=>muxer.addVideoChunk(chunk,meta),error:e=>console.error(e)});videoEncoder.configure({codec:'vp09.00.10.08',width:1280,height:720,bitrate:1e6});/* Encode some frames... */awaitvideoEncoder.flush();muxer.finalize();let{ buffer} =muxer.target;// Buffer contains final WebM file

This library was created to power the in-game video renderer of the browser gameMarble Blast Web -here youcan find a video completely rendered by it and muxed with this library. Previous efforts at in-browser WebM muxing,such aswebm-writer-js orwebm-muxer.js, were either lacking in functionality or were way tooheavy in terms of byte size, which prompted the creation of this library.

Using NPM, simply install this package using

npm install webm-muxer

You can import all exported classes like so:

import*asWebMMuxerfrom'webm-muxer';// Or, using CommonJS:constWebMMuxer=require('webm-muxer');

Alternatively, you can simply include the library as a script in your HTML, which will add aWebMMuxer object,containing all the exported classes, to the global object, like so:

<scriptsrc="build/webm-muxer.js"></script>

For each WebM file you wish to create, create an instance ofMuxer like so:

import{Muxer}from'webm-muxer';letmuxer=newMuxer(options);

The available options are defined by the following interface:

interfaceMuxerOptions{target:|ArrayBufferTarget| StreamTarget| FileSystemWritableFileStreamTarget,video?:{codec:string,width:number,height:number,frameRate?:number,// Optional, adds metadata to the filealpha?:boolean// If the video contains transparency data},audio?:{codec:string,numberOfChannels:number,sampleRate:number,bitDepth?:number// Mainly necessary for PCM-coded audio},subtitles?:{codec:string},streaming?:boolean,type?:'webm'| 'matroska',firstTimestampBehavior?:'strict'|'offset'|'permissive'}

Codecs officially supported by WebM are:
Video:V_VP8,V_VP9,V_AV1
Audio:A_OPUS,A_VORBIS
Subtitles:S_TEXT/WEBVTT

This option specifies where the data created by the muxer will be written. The options are:

• ArrayBufferTarget: The file data will be written into a single large buffer, which is then stored in the target.

  import{Muxer,ArrayBufferTarget}from'webm-muxer';letmuxer=newMuxer({target:newArrayBufferTarget(),// ...});// ...muxer.finalize();let{ buffer}=muxer.target;

• StreamTarget: This target defines callbacks that will get called whenever there is new data available - this isuseful if you want to stream the data, e.g. pipe it somewhere else. The constructor has the following signature:

  constructor(options:{onData?:(data:Uint8Array,position:number)=>void,onHeader?:(data:Uint8Array,position:number)=>void,onCluster?:(data:Uint8Array,position:number,timestamp:number)=>void,chunked?:boolean,chunkSize?:number});

  onData is called for each new chunk of available data. Theposition argument specifies the offset in bytes atwhich the data has to be written. Since the data written by the muxer is not entirely sequential,make sure torespect this argument.

  When usingchunked: true, data created by the muxer will first be accumulated and only written out once it hasreached sufficient size. This is useful for reducing the total amount of writes, at the cost of latency. It using adefault chunk size of 16 MiB, which can be overridden by manually settingchunkSize to the desired byte length.

  If you want to use this target forlive-streaming, make sure to also setstreaming: true in the muxer options.This will ensure that data is written monotonically (sequentially) and already-written data is never "patched" -necessary for live-streaming, but not recommended for muxing files for later viewing.

  TheonHeader andonCluster callbacks will be called for the file header and each Matroska cluster, respectively.This way, you don't need to parse them out yourself from the data provided byonData.

  import{Muxer,StreamTarget}from'webm-muxer';letmuxer=newMuxer({target:newStreamTarget({onData:(data,position)=>{/* Do something with the data */}}),// ...});

• FileSystemWritableFileStreamTarget: This is essentially a wrapper around a chunkedStreamTarget with the intentionof simplifying the use of this library with the File System Access API. Writing the file directly to disk as it'sbeing created comes with many benefits, such as creating files way larger than the available RAM.

  You can optionally override the defaultchunkSize of 16 MiB.

  constructor(stream:FileSystemWritableFileStream,options?:{chunkSize?:number});

  Usage example:

  import{Muxer,FileSystemWritableFileStreamTarget}from'webm-muxer';letfileHandle=awaitwindow.showSaveFilePicker({suggestedName:`video.webm`,types:[{description:'Video File',accept:{'video/webm':['.webm']}}],});letfileStream=awaitfileHandle.createWritable();letmuxer=newMuxer({target:newFileSystemWritableFileStreamTarget(fileStream),// ...});// ...muxer.finalize();awaitfileStream.close();// Make sure to close the stream

Configures the muxer to only write data monotonically, useful for live-streaming the WebM as it's being muxed; intendedto be used together with thetarget set to typefunction. When enabled, some features such as storing duration andseeking will be disabled or impacted, so don't use this option when you want to write out WebM file for later use.

As WebM is a subset of the more general Matroska multimedia container format, this library muxes both WebM and Matroskafiles. WebM, according to the official specification, supports only a small subset of the codecs supported by Matroska.It is likely, however, that most players will successfully play back a WebM file with codecs other than the onessupported in the spec. To be on the safe side, however, you can set thetype option to'matroska', whichwill internally label the file as a general Matroska file. If you do this, your output file should also have the .mkvextension.

Specifies how to deal with the first chunk in each track having a non-zero timestamp. In the default strict mode,timestamps must start with 0 to ensure proper playback. However, when directly pumping video frames or audio datafrom a MediaTrackStream into the encoder and then the muxer, the timestamps are usually relative to the age ofthe document or the computer's clock, which is typically not what we want. Handling of these timestamps must beset explicitly:

• Use'offset' to offset the timestamp of each video track by that track's first chunk's timestamp. This way, itstarts at 0.
• Use'permissive' to allow the first timestamp to be non-zero.

Then, with VideoEncoder and AudioEncoder set up, send encoded chunks to the muxer using the following methods:

addVideoChunk(chunk:EncodedVideoChunk,meta?:EncodedVideoChunkMetadata,timestamp?: number):void;addAudioChunk(chunk:EncodedAudioChunk,meta?:EncodedAudioChunkMetadata,timestamp?: number):void;

Both methods accept an optional, third argumenttimestamp (microseconds) which, if specified, overridesthetimestamp property of the passed-in chunk.

The metadata comes from the second parameter of theoutput callback given to theVideoEncoder or AudioEncoder's constructor and needs to be passed into the muxer, like so:

letvideoEncoder=newVideoEncoder({output:(chunk,meta)=>muxer.addVideoChunk(chunk,meta),error:e=>console.error(e)});videoEncoder.configure(/* ... */);

Should you have obtained your encoded media data from a source other than the WebCodecs API, you can use these followingmethods to directly send your raw data to the muxer:

addVideoChunkRaw(data:Uint8Array,type:'key'|'delta',timestamp: number,// In microsecondsmeta?:EncodedVideoChunkMetadata):void;addAudioChunkRaw(data:Uint8Array,type:'key'|'delta',timestamp: number,// In microsecondsmeta?:EncodedAudioChunkMetadata):void;

When encoding is finished and all the encoders have been flushed, callfinalize on theMuxer instance to finalizethe WebM file:

muxer.finalize();

When using an ArrayBufferTarget, the final buffer will be accessible through it:

let{ buffer}=muxer.target;

When using a FileSystemWritableFileStreamTarget, make sure to close the stream after callingfinalize:

awaitfileStream.close();

Canonical WebM files can only have a maximum Matroska Cluster length of 32.768 seconds, and each cluster must begin witha video key frame. You therefore need to tell yourVideoEncoder to encode aVideoFrame as a key frame at least every32 seconds, otherwise your WebM file will be incorrect. You can do this by doing:

videoEncoder.encode(frame,{keyFrame:true});

When muxing a file with a videoand an audio track, it is important that the individual chunks inside the WebM filebe stored in monotonically increasing time. This does mean, however, that the multiplexer must buffer chunks of onemedium if the other medium has not yet encoded chunks up to that timestamp. For example, should you first encode allyour video frames and then encode the audio afterwards, the multiplexer will have to hold all those video frames inmemory until the audio chunks start coming in. This might lead to memory exhaustion should your video be very long.When there is only one media track, this issue does not arise. So, when muxing a multimedia file, make sure it issomewhat limited in size or the chunks are encoded in a somewhat interleaved way (like is the case for live media).

This library supports adding a subtitle track to a file. Like video and audio, subtitles also need to be encoded beforethey can be added to the muxer. To do this, this library exports its ownSubtitleEncoder class with a WebCodecs-likeAPI. Currently, it only supports encoding WebVTT files.

Here's a full example using subtitles:

import{Muxer,SubtitleEncoder,ArrayBufferTarget}from'webm-muxer';letmuxer=newMuxer({target:newArrayBufferTarget(),subtitles:{codec:'S_TEXT/WEBVTT'},// ....});letsubtitleEncoder=newSubtitleEncoder({output:(chunk,meta)=>muxer.addSubtitleChunk(chunk,meta),error:e=>console.error(e)});subtitleEncoder.configure({codec:'webvtt'});letsimpleWebvttFile=`WEBVTT00:00:00.000 --> 00:00:10.000Example entry 1: Hello <b>world</b>.`;subtitleEncoder.encode(simpleWebvttFile);// ...muxer.finalize();

You do not need to encode an entire WebVTT file in one go; you can encode individual cues or any number of them at once.Just make sure that the preamble (the part before the first cue) is the first thing to be encoded.

This library can mux WebM files up to a total size of ~4398 GB and with a Matroska Cluster size of ~34 GB.

WebM files are a subset of the more general Matroska media container format. Matroska in turn uses a format known asEBML (think of it like binary XML) to structure its file. This project therefore implements a simple EBML writer tocreate the Matroska elements needed to form a WebM file. Many thanks towebm-writer-js for being the inspiration for most of the core EBMLwriting code.

For development, clone this repository, install everything withnpm install, then runnpm run watch to bundle thecode into thebuild directory. Runnpm run check to run the TypeScript type checker, andnpm run lint to runESLint.