RTCRtpScriptTransformer.readableRead only

AReadableStream on which encoded frames from the WebRTC sender or receiver pipelines may be enqueued.

RTCRtpScriptTransformer.writableRead only

AWritableStream that encoded frames should be piped to.

RTCRtpScriptTransformer.optionsRead only

Options passed from theRTCRtpScriptTransform constructor, which are used to configure transform code based on whether incoming or outgoing frames are being processed.

RTCRtpScriptTransformer.generateKeyFrame()

Requests a video encoder generate a key frame. May be called by a transformer in the sender pipeline when processing outgoing frames.

RTCRtpScriptTransformer.sendKeyFrameRequest()

Requests the sender send a key frame. May be called by a transformer in the receiver pipeline when processing incoming encoded video frames.

ARTCRtpScriptTransformer instance is created as part of construction of an associatedRTCRtpScriptTransform, which specifies the worker in which the transformer is created and options that will be passed to it.

The transformer is made available to a worker through thertctransform eventtransformer property.This event is fired on construction of the associatedRTCRtpScriptTransform and when an encoded frame is enqueued on theRTCRtpScriptTransformer.readable from a codec (outgoing) or from the packetizer (incoming).

The transformer exposes areadable andwritable stream into the worker, along with anoptions object provided to theRTCRtpScriptTransform on construction.When the associatedRTCRtpScriptTransform is assigned to aRTCRtpSender orRTCRtpReceiver, encoded media frames from the WebRTC sender or receiver pipelines are enqueued on thereadable stream.

A WebRTC Encoded Transform must read encoded frames fromtransformer.readable, modify them as needed, and write them totransformer.writable in the same order, and without any duplication.Thetransformer.options allow an appropriate transform function to be used, based on whether the encoded media frames are incoming or outgoing.The transform is usually implemented by piping frames from thereadable through one or moreTransformStream instances to thewritable, transforming them as needed.

The interface also provides methods for a sender to generate get a video encoder to generate a new keyframe, or for a receiver to request a new key frame from the sender's encoder (video encoders commonly send a key frame containing the full information needed to construct an image, and subsequently send delta frames containing just the information that has changed since the previous frame).

These methods are required in cases where a recipient would be unable to decode incoming frames until they receive a new key frame.For example, a new recipient joining a conference call will not be able see video until they have received a new key frame, since delta frames can only be decoded if you have the last key frame and all subsequent delta frames.Similarly, if frames are encrypted for a recipient, they will only be able to decode frames once they have received their first encrypted key frame.

This example shows the code for a WebRTC Encoded Transform running in a worker.

The code usesaddEventListener() to register a handler function for thertctransform event, which makes theRTCRtpScriptTransformer available asevent.transformer.

The handler creates aTransformStream and pipes frames from theevent.transformer.readable through it toevent.transformer.writable.The transform streamtransform() implementation is called for each encoded frame queued on the stream: it can read the data from the frame and in this case negates the bytes and then enqueues the modifiable frame on the stream.

addEventListener("rtctransform", (event) => {  const transform = new TransformStream({    start() {}, // Called on startup.    flush() {}, // Called when the stream is about to be closed.    async transform(encodedFrame, controller) {      // Reconstruct the original frame.      const view = new DataView(encodedFrame.data);      // Construct a new buffer      const newData = new ArrayBuffer(encodedFrame.data.byteLength);      const newView = new DataView(newData);      // Negate all bits in the incoming frame      for (let i = 0; i < encodedFrame.data.byteLength; ++i) {        newView.setInt8(i, ~view.getInt8(i));      }      encodedFrame.data = newData;      controller.enqueue(encodedFrame);    },  });  event.transformer.readable    .pipeThrough(transform)    .pipeTo(event.transformer.writable);});

The only special things to note about theTransformStream above are that it queues encoded media frames (RTCEncodedVideoFrame orRTCEncodedAudioFrame) rather than arbitrary "chunks", and thatwritableStrategy andreadableStrategy properties are not defined (because the queuing strategy is entirely managed by the user agent).

A transform can run in either the incoming or outgoing WebRTC pipelines.This doesn't matter in the code above, because the same algorithm might be used in the sender to negate the frames, and in the receiver to revert them.If the sender and receiver pipelines need to apply a different transform algorithm then information about the current pipeline must be passed from the main thread.This is done by setting anoptions argument in the correspondingRTCRtpScriptTransform constructor, which is then made available to the worker inRTCRtpScriptTransformer.options.

Below we use thetransformer.options to choose either a sender transform or a receiver transform.Note that the properties of the object are arbitrary (provided the values can be serialized) and it is also possible to transfer aMessageChannel and use it tocommunicate with a transform at runtime in order to, for example, share encryption keys.

// Code to instantiate transform and attach them to sender/receiver pipelines.onrtctransform = (event) => {  let transform;  if (event.transformer.options.name === "senderTransform")    transform = createSenderTransform();  // returns a TransformStream (not shown)  else if (event.transformer.options.name === "receiverTransform")    transform = createReceiverTransform();  // returns a TransformStream (not shown)  else return;  event.transformer.readable    .pipeThrough(transform)    .pipeTo(event.transformer.writable);};

Note that the above code is part of more complete examples provided inUsing WebRTC Encoded Transforms.

• Using WebRTC Encoded Transforms
• TransformStream