| RFC 8848 | CLUE Signaling | January 2021 |
| Hanton, et al. | Experimental | [Page] |
This document is about Controlling Multiple Streams for Telepresence (CLUE) signaling. It specifies how the CLUE protocol and the CLUE data channel are used in conjunction with each other and with existing signaling mechanisms, such as SIP and the Session Description Protocol (SDP), to produce a telepresence call.¶
This document is not an Internet Standards Track specification; it is published for examination, experimental implementation, and evaluation.¶
This document defines an Experimental Protocol for the Internet community. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are candidates for any level of Internet Standard; see Section 2 of RFC 7841.¶
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained athttps://www.rfc-editor.org/info/rfc8848.¶
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.¶
To enable devices to participate in a telepresence call, where they select the sourcesthey wish to view, receive those media sources, and display them in anoptimal fashion, Controlling Multiple Streams for Telepresence (CLUE) employstwo principal and interrelated protocol negotiations.SDP[RFC4566], conveyed via SIP[RFC3261], is used to negotiate the specific mediacapabilities that can be delivered to specific addresses on a device.Meanwhile, CLUE protocol messages[RFC8847], transported via a CLUE data channel[RFC8850], are used tonegotiate the Capture Sources available, their attributes, and any constraintsin their use. They also allow the far-end device to specify which Capturesthey wish to receive. It is recommended that those documents be read prior tothis one as this document assumes familiarity with those protocols and henceuses terminology from each with limited introduction.¶
Beyond negotiating the CLUE channel, SDP is also used to negotiate the detailsof supported media streams and the maximum capability of each of thosestreams. As the CLUE Framework[RFC8845]defines a manner in which the Media Provider expresses their maximum EncodingGroup capabilities, SDP is also used to express the encoding limits for eachpotential Encoding.¶
Backwards compatibility is an important consideration of the protocol: it isvital that a CLUE-capable device contacting a device that does not supportCLUE is able to fall back to a fully functional non-CLUE call. The documentalso defines how a non-CLUE call may be upgraded to CLUE mid-call and,similarly, how CLUE functionality can be removed mid-call to return to astandard non-CLUE call.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14[RFC2119][RFC8174] when, and only when, they appear in all capitals, as shown here.¶
This document uses terminology defined in the CLUE Framework[RFC8845].¶
A few additional terms specific to this document are defined as follows:¶
The "sip.clue" media feature tag[RFC3840] indicatessupport for CLUE in SIP[RFC3261] calls. A CLUE-capabledeviceSHOULD include this media feature tag in its REGISTER requests andOPTION responses. ItSHOULD also include the media feature tag in INVITE andUPDATE[RFC3311] requests and responses.¶
Presence of the media feature tag in the contact field of a request orresponse can be used to determine that the far end supports CLUE.¶
This section defines a new SDP Grouping Framework[RFC5888]extension called 'CLUE'.¶
The CLUE extension can be indicated using an SDP session-level'group' attribute. Each SDP media "m=" line that is included in this group,using SDP media-level mid attributes, is CLUE controlled by a CLUE datachannel that is also included in this CLUE group.¶
Currently, only support for a single CLUE group is specified; support formultiple CLUE groups in a single session is outside the scope of thisdocument. A deviceMUST NOT include more than one CLUE group in its SDPmessage unless it is following a specification that defines how multiple CLUEchannels are signaled and is able to either determine that the other side ofthe SDP exchange supports multiple CLUE channels or failgracefully in the event it does not.¶
The CLUE data channel[RFC8850] is abidirectional data channel[RFC8831]used for the transport of CLUE messages, conveyed within an SCTP over DTLSconnection. This channel must be established before CLUE protocol messages canbe exchanged and CLUE-controlled media can be sent.¶
The data channel is negotiated over SDP as described in[RFC8864]. A CLUE-capabledevice wishing to negotiate CLUEMUST also include a CLUE group in their SDPOffer or Answer and include the "mid" of the "m=" line for the data channel inthat group. The CLUE groupMUST include the "mid" of the "m=" line for one(and only one) data channel.¶
Presence of the data channel in the CLUE group in an SDP Offer or Answer alsoserves, along with the "sip.clue" media feature tag, as an indication that thedevice supports CLUE and wishes to upgrade the call to include CLUE-controlledmedia. A CLUE-capable deviceSHOULD include a data channel "m=" line in offersand, when allowed by[RFC3264], answers.¶
CLUE-controlled media lines in an SDP are "m=" lines in which the content ofthe media streams to be sent is negotiated via the CLUE protocol[RFC8847]. For an "m=" lineto be CLUE controlled, its "mid" attribute valueMUST be included in the CLUE group. CLUE-controlled media is controlled by the CLUE protocol as negotiated on theCLUE data channel with a "mid" included in the CLUE group.¶
"m=" lines not specified as being under CLUE control follow normal rules for media streams negotiated in SDP as defined in documents such as[RFC3264].¶
The restrictions on CLUE-controlled media that are defined below always applyto "m=" lines in an SDP Offer or Answer, even if negotiation of the datachannel in SDP failed due to lack of CLUE support by the remote device or forany other reason, or in an offer if the recipient does not include the "mid"of the corresponding "m=" line in their CLUE group.¶
The CLUE Framework[RFC8845] defines theconcept of "Encodings", which represent the sender's encode ability. EachEncoding the Media Provider wishes to signal is done so via an "m=" line ofthe appropriate media type, whichMUST be marked as sendonly with the"a=sendonly" attribute or as inactive with the "a=inactive" attribute.¶
The encoder limits of active (e.g., "a=sendonly") Encodings can then beexpressed using existing SDP syntax. For instance, for H.264, see Table 6 inSection 8.2.2 of [RFC6184] for a list of valid parameters for representingencoder sender stream limits.¶
These Encodings are CLUE controlled and henceMUST include a "mid" in theCLUE group as defined above.¶
In addition to the normal restrictions defined in[RFC3264], thestreamMUST be treated as if the "m=" line direction attribute had been set to"a=inactive" until the Media Provider has received a valid CLUE 'configure'message specifying the Capture to be used for this stream. This means thatRTP packetsMUST NOT be sent until configuration is complete, whilenon-media packets such as STUN, RTCP, and DTLSMUST be sent as per theirrelevant specifications, if negotiated.¶
Every "m=" line representing a CLUE EncodingMUST contain a "label" attributeas defined in[RFC4574]. This label is used to identify theEncoding by the sender in CLUE 'advertisement' messages and by the receiver inCLUE 'configure' messages. Each label used for a CLUE-controlled "m=" lineMUST be different from the label on all other "m=" lines in the CLUE group,unless an "m=" line represents a dependent stream related to another "m=" line(such as a Forward Error Correction (FEC) stream), in which case itMUST have the same label value asthe "m=" line on which it depends.¶
CLUE Encodings are defined in SDP but can be referenced from CLUE protocolmessages -- this is how the protocol defines which Encodings are a part of anEncoding Group (in 'advertisement' messages) and which Encoding is used to encode a specific Capture (in 'configure' messages). The labels on theCLUE-controlled "m=" lines are the references that are used in the CLUEprotocol.¶
Each <encID> (in encodingIDList) in a CLUE 'advertisement' messageSHOULD represent an Encoding defined in SDP; the specific Encoding referencedis a CLUE-controlled "m=" line in the most recent SDP Offer/Answer messagesent by the sender of the 'advertisement' message with a label valuecorresponding to the text content of the <encID>. If the <encID>is not defined in SDP, itMUST be one it anticipates sending in a subsequentSDP Offer/Answer exchange.¶
Each <encodingID> (in captureEncodingType) in a CLUE 'configure' messageMUST represent an Encoding defined in SDP; the specific Encoding referenced isa CLUE-controlled "m=" line in the most recent SDP Offer/Answer messagereceived by the sender of the 'configure' message with a label valuecorresponding to the text content of the <encodingID>.¶
Note that the non-atomic nature of SDP/CLUE protocol interaction may mean thatthere are temporary periods where an <encID>/<encodingID> in aCLUE message does not reference an SDP "m=" line, or where an Encoding represented in SDP is not referenced in a CLUE protocol message. SeeSection 5 for specifics.¶
A receiver who wishes to receive a CLUE stream via a specific Encodingrequires an "a=recvonly" "m=" line that matches the "a=sendonly" Encoding.¶
These "m=" lines are CLUE controlled and henceMUST include their "mid" in theCLUE group. TheyMAY include a "label" attribute, but this is not required by CLUE, as only label values associated with "a=sendonly" Encodings arereferenced by CLUE protocol messages.¶
A CLUE-capable device sending an initial SDP Offer of a SIP session andwishing to negotiate CLUE will include an "m=" line for the data channel toconvey the CLUE protocol, along with a CLUE group containing the "mid" of thedata channel "m=" line.¶
For interoperability with non-CLUE devices, a CLUE-capable device sending aninitial SDP OfferSHOULD NOT include any "m=" line for CLUE-controlled mediabeyond the "m=" line for the CLUE data channel, and itSHOULD include at leastone non-CLUE-controlled media "m=" line.¶
If the device has evidence that the receiver is also CLUE capable, forinstance, due to receiving an initial INVITE with no SDP but including a"sip.clue" media feature tag, the above recommendation is waived, and theinitial offerMAY contain "m=" lines for CLUE-controlled media.¶
With the same interoperability recommendations as for Encodings, the sender ofthe initial SDP OfferMAY also include "a=recvonly" media lines topreallocate "m=" lines to receive media. Alternatively, itMAY wait until CLUEprotocol negotiation has completed before including these lines in a newoffer/answer exchange -- seeSection 5 forrecommendations.¶
If the recipient of an initial offer is CLUE capable, and the offer containsboth an "m=" line for a data channel and a CLUE group containing the "mid" forthat "m=" line, theySHOULD negotiate data channel support for an "m=" lineand include the "mid" of that "m=" line in a corresponding CLUE group.¶
A CLUE-capable recipient that receives an "m=" line for a data channel but nocorresponding CLUE group containing the "mid" of that "m=" lineMAY still include a corresponding data channel "m=" line if there are any other non-CLUEprotocols it can convey over that channel, but the use of the CLUE protocolMUST NOT be negotiated on this channel.¶
If the initial offer contained "a=recvonly" CLUE-controlled media lines, therecipientSHOULD include corresponding "a=sendonly" CLUE-controlled medialines for accepted Encodings, up to the maximum number of Encodings it wishes to advertise. As CLUE-controlled media, the "mid" of these "m=" linesMUST be included in the corresponding CLUE group. The recipientMUST set thedirection of the corresponding "m=" lines of any remaining "a=recvonly"CLUE-controlled media lines received in the offer to "a=inactive".¶
If the initial offer contained "a=sendonly" CLUE-controlled media lines, therecipientMAY include corresponding "a=recvonly" CLUE-controlled media lines,up to the maximum number of Capture Encodings it wishes to receive.Alternatively, itMAY wait until CLUE protocol negotiation has completedbefore including these lines in a new offer/answer exchange -- seeSection 5 for recommendations. The recipientMUST setthe direction of the corresponding "m=" lines of any remaining "a=sendonly"CLUE-controlled media lines received in the offer to "a=inactive".¶
A CLUE-controlled device implementationMAY prefer to render initial,single-stream audio and/or video for the user as rapidly as possible,transitioning to CLUE-controlled media once that has been negotiated.Alternatively, an implementationMAY wish to suppress initial media, onlyproviding media once the final, CLUE-controlled streams have been negotiated.¶
The receiver of the initial offer, if making the call CLUE-enabled with theirSDP Answer, can make their preference clear by their action in accepting orrejecting non-CLUE-controlled media lines. Rejecting these "m=" lines willensure that no non-CLUE-controlled media flows before the CLUE-controlledmedia is negotiated. In contrast, accepting one or more non-CLUE-controlled"m=" lines in this initial answer will enable initial media to flow.¶
If the answerer chooses to send initial non-CLUE-controlled media in a CLUE-enabled call,Section 4.5.4.1 addresses the need todisable it once the CLUE-controlled media is fully negotiated.¶
In the event that both the offer and answer include a data channel "m=" line witha "mid" value included in corresponding CLUE groups, CLUE has been successfullynegotiated, and the call is now CLUE enabled. If not, then the call is notCLUE enabled.¶
In the event of successful CLUE enablement of the call, devicesMUST now beginnegotiation of the CLUE channel; see[RFC8850] for negotiation details. Ifnegotiation is successful, the sending of CLUE protocol messages[RFC8847] can begin.¶
A CLUE-capable deviceMAY choose not to send RTP on the non-CLUE-controlledchannels during the period in which control of the CLUE-controlled media linesis being negotiated (though RTCPMUST still be sent and received as normal).However, a CLUE-capable deviceMUST still be prepared to receive media onnon-CLUE-controlled media lines that have been successfully negotiated asdefined in[RFC3264].¶
If either side of the call wishes to add additional CLUE-controlled "m=" linesto send or receive CLUE-controlled media, theyMAY now send a SIP request witha new SDP Offer following the normal rules of SDP Offer/Answer and anynegotiated extensions.¶
In the event that the negotiation of CLUE fails and the call is notCLUE enabled once the initial offer/answer negotiation completes, then CLUE isnot in use in the call. CLUE-capable devicesMUST either revert tonon-CLUE behavior or terminate the call.¶
Subsequent offer/answer exchangesMAY add additional "m=" lines forCLUE-controlled media or activate or deactivate existing "m=" lines per thestandard SDP mechanisms.¶
In most cases, at least one additional exchange after the initial offer/answerexchange will be required before both sides have added all the Encodings andthe ability to receive Encodings that they desire. DevicesMAY delay adding"a=recvonly" CLUE-controlled "m=" lines until after CLUE protocol negotiationcompletes -- seeSection 5 for recommendations.¶
Once CLUE media has been successfully negotiated, devicesSHOULD ensure thatnon-CLUE-controlled media is deactivated by setting their ports to 0 in caseswhere it corresponds to the media type of CLUE-controlled media that has beensuccessfully negotiated. This deactivation may require an additional SDPexchange or may be incorporated into one that is part of the CLUEnegotiation.¶
A CLUE-capable device that receives an initial SDP Offer from a non-CLUEdeviceSHOULD include a new data channel "m=" line and corresponding CLUEgroup in any subsequent offers it sends, to indicate that it is CLUE capable.¶
If, in an ongoing non-CLUE call, an SDP Offer/Answer exchange completes withboth sides having included a data channel "m=" line in their SDP and with the"mid" for that channel in a corresponding CLUE group, then the call is nowCLUE enabled; negotiation of the data channel and subsequently the CLUEprotocol begins.¶
If, during an ongoing CLUE-enabled call, a device wishes to disable CLUE, it can do so by following the procedures for closing a data channel as defined inSection 6.6.1 of [RFC8864]: sendinga new SDP Offer/Answer exchange and subsequent SCTP Stream Sequence Number (SSN) reset for the CLUEchannel. ItMUST also remove the CLUE group. Without the CLUE group, any "m="lines that were previously CLUE controlled no longer are; implementationsMAYdisable them by setting their ports to 0 orMAY continue to use them -- in thelatter case, how they are used is outside the scope of this document.¶
If a device follows the procedure above, or an SDP Offer/Answer negotiation completes in a fashion in which either the "m=" CLUE data channel line was not successfully negotiated and/or one side did not include the data channel inthe CLUE group, then CLUE for this call is disabled. In the event that thisoccurs, CLUE is no longer enabled. Any active "m=" lines still included in theCLUE group are no longer CLUE controlled, and the implementationMAY eitherdisable them in a subsequent negotiation or continue to use them in some otherfashion. If the data channel is still present but not included in the CLUEgroup semantic, CLUE protocol messagesMUST no longer be sent.¶
In contrast to the specific disablement of the use of CLUE described above,the CLUE channel may fail unexpectedly. Two circumstances where this can occurare:¶
In this circumstance, implementationsSHOULD continue to transmit and receiveCLUE-controlled media on the basis of the last negotiated CLUE messages,until the CLUE protocol is re-established (in the event of a channel error) ordisabled mid-call by an SDP exchange as defined inSection 4.5.4.3. ImplementationsMAY choose to send suchan SDP request to disable CLUE immediately orMAY continue on in acall-preservation mode.¶
Information about media streams in CLUE is split between two message types:SDP, which defines media addresses and limits, and the CLUE channel,which defines properties of Capture Devices available, scene information, andadditional constraints. As a result, certain operations, such as advertisingsupport for a new transmissible Capture with an associated stream, cannot beperformed atomically, as they require changes to both SDP and CLUE messaging.¶
This section defines how the negotiation of the two protocols interact,provides some recommendations on dealing with intermediate stages innon-atomic operations, and mandates additional constraints on whenCLUE-configured media can be sent.¶
To avoid the need to implement interlocking state machines with the potentialto reach invalid states if messages were to be lost, or be rewritten en routeby middleboxes, the state machines in SDP and CLUE operate independently. Thestate of the CLUE channel does not restrict when an implementation may send anew SDP Offer or Answer; likewise, the implementation's ability to send anew CLUE 'advertisement' or 'configure' message is not restricted by theresults of or the state of the most recent SDP negotiation (unless the SDPnegotiation has removed the CLUE channel).¶
The primary implication of this is that a device may receive an SDPOffer/Answer message with a CLUE Encoding for which it does not yet haveCapture information or receive a CLUE 'configure' message specifying aCapture Encoding for which the far end has not negotiated a media stream inSDP.¶
CLUE messages contain an <encID> (in encodingIDList) or <encodingID> (in captureEncodingType), which is used to identify a specific Encoding or captureEncoding in SDP; see[RFC8846] for specifics.The non-atomic nature of CLUE negotiation means that a sender may wish to senda new CLUE 'advertisement' message before the corresponding SDP message. Assuch, the sender of the CLUE messageMAY include an <encID> that doesnot currently match a CLUE-controlled "m=" line label in SDP; a CLUE-capableimplementationMUST NOT reject a CLUE protocol message solely because itcontains <encID> elements that do not match a label in SDP.¶
The current state of the CLUE Participant or Media Provider/Consumerstate machines does not affect compliance with any of the normative language of[RFC3264]. That is, theyMUST NOT delay an ongoing SDPexchange as part of a SIP server or client transaction; an implementationMUST NOT delay an SDP exchange while waiting for CLUE negotiation to complete orfor a 'configure' message to arrive.¶
Similarly, a device in a CLUE-enabled callMUST NOT delay any mandatory statetransitions in the CLUE Participant or Media Provider/Consumer state machinesdue to the presence or absence of an ongoing SDP exchange.¶
A device with the CLUE Participant state machine in the ACTIVE stateMAY choose to delay moving from ESTABLISHED to ADV (Media Providerstate machine) or from ESTABLISHED to WAIT FOR CONF RESPONSE (Media Consumerstate machine) based on the SDP state. See[RFC8847] for CLUE state machine specifics. Similarly, a deviceMAY choose to delay initiating a new SDP exchange based onthe state of their CLUE state machines.¶
While SDP and CLUE message states do not impose constraints on each other,both impose constraints on the sending of media -- CLUE-controlled mediaMUST NOT be sent unless it has been negotiated in both CLUE and SDP: animplementationMUST NOT send a specific CLUE Capture Encoding unless its mostrecent SDP exchange contains an active media channel for that Encoding ANDit has received a CLUE 'configure' message specifying a valid Capture for thatEncoding.¶
CLUE-capable devicesMUST be able to handle states in which CLUE messages makereference to EncodingIDs that do not match the most recently received SDP,irrespective of the order in which SDP and CLUE messages are received. Whilethese mismatches will usually be transitory, a deviceMUST be able to copewith such mismatches remaining indefinitely. However, this document makes somerecommendations on message ordering for these non-atomic transitions.¶
CLUE-capable devicesMUST ensure that any inconsistencies between SDP andCLUE signaling are temporary by sending updated SDP or CLUE messages as soonas the relevant state machines and other constraints permit.¶
Generally, implementations that receive messages withincomplete information will be most efficient if they wait until they have thecorresponding information they lack before sending messages to make changesrelated to that information. For example, an answerer that receives a new SDPOffer with three new "a=sendonly" CLUE "m=" lines for which it has received noCLUE 'advertisement' message providing the corresponding capture informationwould typically include corresponding "a=inactive" lines in its answer, andit would only make a new SDP Offer with "a=recvonly" when and if a new 'advertisement'message arrives with Captures relevant to those Encodings.¶
Because of the constraints of SDP Offer/Answer and because new SDP negotiations are generally more 'costly' than sending a new CLUE message,implementations needing to make changes to both channelsSHOULD prioritizesending the updated CLUE message over sending the new SDP message. The aim isfor the recipient to receive the CLUE changes before the SDP changes, allowingthe recipient to send their SDP Answers without incomplete information andreducing the number of new SDP Offers required.¶
The CLUE Framework[RFC8845] allows forMultiple Content Captures (MCCs): Captures that contain multiple sourceCaptures, whether composited into a single stream or switched based on somemetric.¶
The Captures that contribute to these MCCs may or may not be defined in the'advertisement' message. If they are defined and the MCC is providing them ina switched format, the recipient may wish to determine which originating sourceCapture is currently being provided, so that they can apply geometriccorrections based on that Capture's geometry or take some other action basedon the original Capture information.¶
To do this,[RFC8849] allows for the CaptureID of the originating Capture to be conveyed via RTP or RTCP. A MediaProvider sending switched media for an MCC with defined originating sourcesMUST send the CaptureID in both RTP and RTCP, as describedin the mapping document.¶
Because the RTP/RTCP CaptureID is delivered via a different channel to the'advertisement' message in which in the contents of the MCC are defined, thereis an intrinsic race condition in cases where the contents of an MCC areredefined.¶
When a Media Provider redefines an MCC that involves CaptureIDs, the reception of the relevant CaptureIDs by the recipient will either lead or lag reception and the processing of the new 'advertisement' message by the recipient.As such, a Media ConsumerMUST NOT be disrupted by any of the following scenarios in anyCLUE-controlled media stream it is receiving, whether that stream is for astatic Capture or for an MCC (as any static Capture may be redefined to an MCCin a later 'advertisement' message):¶
A CLUE call may involve sending and/or receiving significant numbers of mediastreams. Conventionally, media streams are sent and received on unique ports.However, each separate port used for this purpose may impose costs that adevice wishes to avoid, such as the need to open that port on firewalls andNATs, the need to collect Interactive Connectivity Establishment (ICE) candidates[RFC8445], etc.¶
The BUNDLE extension[RFC8843] can be used to negotiate the multiplexing of multiple media linesonto a single 5-tuple for sending and receiving media, allowing devices incalls to another BUNDLE-supporting device to potentially avoid some of theabove costs.¶
While CLUE-capable devicesMAY support the BUNDLE extension for this purpose, supporting the extension is not mandatory for a device to be CLUE compliant.¶
A CLUE-capable device that supports BUNDLESHOULD also support rtcp-mux[RFC5761]. However, a CLUE-capable device thatsupports rtcp-mux may or may not support BUNDLE.¶
This specification imposes no additional requirements or restrictions on theusage of BUNDLE when used with CLUE. There is no restriction on combiningCLUE-controlled media lines and non-CLUE-controlled media lines in the sameBUNDLE group or in multiple such groups. However, there are several steps animplementation may wish to take to ameliorate the cost and time requirementsof extra SDP Offer/Answer exchanges between CLUE and BUNDLE.¶
BUNDLE mandates that the initial SDP OfferMUST use a unique address for each"m=" line with a non-zero port. Because CLUE implementations generally willnot include CLUE-controlled media lines, with the exception of the datachannel in the initial SDP Offer, CLUE devices that support large numbers ofstreams can avoid ever having to open large numbers of ports if theysuccessfully negotiate BUNDLE.¶
An implementation that does include CLUE-controlled media lines in its initialSDP Offer while also using BUNDLE must take care to avoid rendering itsCLUE-controlled media lines unusable in the event the far end does notnegotiate BUNDLE if it wishes to avoid the risk of additional SDP exchanges toresolve this issue. This is best achieved by not sending any CLUE-controlledmedia lines in an initial offer with the 'bundle-only' attribute unless it hasbeen established via some other channel that the recipient supports and isable to use BUNDLE.¶
BUNDLE-supporting CLUE-capable devicesMAY include the data channel in thesame BUNDLE group as RTP media. In this case, the deviceMUST be able todemultiplex the various transports -- see Section9.2 of the BUNDLE specification[RFC8843]. Ifthe BUNDLE group includes protocols other than the data channel transportedvia DTLS, the deviceMUST also be able to differentiate the various protocols.¶
This example illustrates a call between two CLUE-capable Endpoints.Alice, initiating the call, is a system with three cameras and three screens.Bob, receiving the call, is a system with two cameras and two screens.A call-flow diagram is presented, followed by a summary of each message.¶
To manage the size of this section, the SDP snippets only illustrate video "m="lines. SIP ACKs are not always discussed. Note that BUNDLE is not in use.¶
+----------+ +-----------+ | Alice | | Bob | | | | | +----+-----+ +-----+-----+ | | | | | SIP INVITE 1 | |--------------------------------->| | | | | | SIP 200 OK 1 | |<---------------------------------| | | | | | SIP ACK 1 | |--------------------------------->| | | | | | | |<########### MEDIA 1 ############>| | 1 video A->B, 1 video B->A | |<################################>| | | | | | | |<================================>| | CLUE DATA CHANNEL ESTABLISHED | |<================================>| | | | | | CLUE OPTIONS | |<*********************************| | | | | | CLUE OPTIONS RESPONSE | |*********************************>| | | | | | CLUE ADVERTISEMENT 1 | |*********************************>| | | | | | CLUE ADVERTISEMENT 2 | |<*********************************| | | | | | CLUE ACK 1 | |<*********************************| | | | | | CLUE ACK 2 | |*********************************>| | | | | | SIP INVITE 2 (+3 sendonly) | |--------------------------------->| | | | | | CLUE CONFIGURE 1 | |<*********************************| | | | | | SIP 200 OK 2 (+2 recvonly) | |<---------------------------------| | | | | | CLUE CONFIGURE RESPONSE 1 | |*********************************>| | | | | | SIP ACK 2 | |--------------------------------->| | | | | | | |<########### MEDIA 2 ############>| | 2 video A->B, 1 video B->A | |<################################>| | | | | | SIP INVITE 3 (+2 sendonly) | |<---------------------------------| | | | | | CLUE CONFIGURE 2 | |*********************************>| | | | | | SIP 200 OK 3 (+2 recvonly) | |--------------------------------->| | | | | | CLUE CONFIGURE RESPONSE 2 | |<*********************************| | | | | | SIP ACK 3 | |<---------------------------------| | | | | | | |<########### MEDIA 3 ############>| | 2 video A->B, 2 video B->A | |<################################>| | | | | | | v v¶
In SIP INVITE 1, Alice sends Bob a SIP INVITE with thebasic audio and video capabilities and data channel included in the SIP body as per[RFC8841]. Alice also includes the "sip.clue"media feature tag in the INVITE. A snippet of the SDP showing the groupingattribute and the video "m=" line are shown below. Alice has included a "CLUE"group and the mid corresponding to a data channel in the group (3).Note that Alice has chosen not to include any CLUE-controlled media in theinitial offer -- the "mid" value of the video line is not included in the "CLUE"group.¶
... a=group:CLUE 3 ... m=video 6002 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=sendrecv a=mid:2 ... m=application 6100 UDP/DTLS/SCTP webrtc-datachannel a=setup:actpass a=sctp-port: 5000 a=dcmap:2 subprotocol="CLUE";ordered=true a=mid:3¶
Bob responds with a similar SDP in SIP 200 OK 1, which also has a "CLUE" groupincluding the "mid" value of a data channel; due to their similarity, no SDPsnippet is shown here. Bob wishes to receive initial media and thus includescorresponding non-CLUE-controlled audio and video lines. Bob also includes the"sip.clue" media feature tag in the 200 OK. Alice and Bob are each now able tosend a single audio and video stream. This is illustrated as MEDIA 1.¶
With the successful initial SDP Offer/Answer exchange complete, Alice and Bobare also free to negotiate the CLUE data channel. This is illustrated as CLUEDATA CHANNEL ESTABLISHED.¶
Once the data channel is established, CLUE protocol negotiation begins. In thiscase, Bob was the DTLS client (sending "a=active" in his SDP Answer) and hence isthe CLUE Channel Initiator. He sends a CLUE OPTIONS message describing hisversion support. On receiving that message, Alice sends her corresponding CLUEOPTIONS RESPONSE.¶
With the OPTIONS phase complete, Alice now sends her CLUE 'advertisement'message (CLUE ADVERTISEMENT 1). She advertises three static Capturesrepresenting her three cameras. She also includes switched Captures suitablefor systems with one or two screens. All of these Captures are in a single CaptureScene, with suitable Capture Scene Views that tell Bob he shouldsubscribe to the three static Captures, the two switched Captures, or the oneswitched Capture. Alice has no simultaneity constraints, so all sixCaptures are included in one simultaneous set. Finally, Alice includes an Encoding Groupwith three Encoding IDs: "enc1", "enc2", and "enc3". These Encoding IDs aren'tcurrently valid but will match the next SDP Offer she sends.¶
Bob received CLUE ADVERTISEMENT 1 but does not yet send a 'configure' message, because he has not yet received Alice's Encoding information; thus, hedoes not know if she will have sufficient resources in order to send him the twostreams he ideally wants at a quality he is happy with. Because Bob is notsending an immediate 'configure' message with the "ack" element set, he mustsend an explicit 'ack' message (CLUE ACK 1) to signal receipt of CLUEADVERTISEMENT 1.¶
Bob also sends his CLUE 'advertisement' message (CLUE ADVERTISEMENT 2) --though the diagram shows that this occurs after Alice sends CLUE ADVERTISEMENT1, Bob sends his 'advertisement' message independently and does not wait forCLUE ADVERTISEMENT 1 to arrive. He advertises two static Captures representinghis cameras. He also includes a single composed Capture for single-screensystems, in which he will composite the two camera views into a single videostream. All three Captures are in a single Capture Scene, with suitableCapture Scene Views that tell Alice she should subscribe to either the twostatic Captures or the single composed Capture. Bob also has no simultaneityconstraints, so he includes all three Captures in one simultaneous set. Bob alsoincludes a single Encoding Group with two Encoding IDs: "foo" and "bar".¶
Similarly, Alice receives CLUE ADVERTISEMENT 2 but does not yet send a'configure' message, because she has not yet received Bob's Encodinginformation; instead, she sends an 'ack' message (CLUE ACK 2).¶
Both sides have now sent their CLUE 'advertisement' messages, and an SDPexchange is required to negotiate Encodings. For simplicity, in this case,Alice is shown sending an INVITE with a new offer; in many implementations,both sides might send an INVITE, which would be resolved by use of the 491Request Pending resolution mechanism from[RFC3261].¶
Alice now sends SIP INVITE 2. She maintains the sendrecv audio, video, and CLUE"m=" lines, and she adds three new sendonly "m=" lines to represent the threeCLUE-controlled Encodings she can send. Each of these "m=" lines has a labelcorresponding to one of the Encoding IDs from CLUE ADVERTISEMENT 1. Each alsohas its mid added to the grouping attribute to show they are controlled by theCLUE data channel. A snippet of the SDP showing the grouping attribute, datachannel, and video "m=" lines are shown below:¶
... a=group:CLUE 3 4 5 6 ... m=video 6002 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=sendrecv a=mid:2 ... m=application 6100 UDP/DTLS/SCTP webrtc-datachannel a=sctp-port: 5000 a=dcmap:2 subprotocol="CLUE";ordered=true a=mid:3 ... m=video 6004 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=mid:4 a=label:enc1 m=video 6006 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=mid:5 a=label:enc2 m=video 6008 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=mid:6 a=label:enc3¶
Bob now has all the information he needs to decide which streams to configure,allowing him to send both a CLUE 'configure' message and his SDP Answer. Assuch, he now sends CLUE CONFIGURE 1. This requests the pair of switchedCaptures that represent Alice's scene, and he configures them with encoder ids"enc1" and "enc2".¶
Bob also sends his SDP Answer as part of SIP 200 OK 2. Alongside his originalaudio, video, and CLUE "m=" lines, he includes three additional "m=" linescorresponding to the three added by Alice: two active recvonly "m= "lines andan inactive "m=" line for the third. He adds their "mid" values to the groupingattribute to show they are controlled by the CLUE data channel. A snippet ofthe SDP showing the grouping attribute and the video "m=" lines are shownbelow (mid 100 represents the CLUE data channel, which is not shown):¶
... a=group:CLUE 11 12 13 100 ... m=video 58722 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=sendrecv a=mid:10 ... m=video 58724 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=recvonly a=mid:11 m=video 58726 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=recvonly a=mid:12 m=video 58728 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=inactive a=mid:13¶
Alice receives Bob's CLUE CONFIGURE 1 message and sends CLUE CONFIGURERESPONSE 1 to acknowledge its reception. She does not yet send the Capture Encodingsspecified, because at this stage, she hasn't processed Bob's answer SDP and thushasn't negotiated the ability for Bob to receive these streams.¶
On receiving SIP 200 OK 2 from Bob, Alice sends her SIP ACK (SIP ACK 2). She isnow able to send the two streams of video Bob requested -- this is illustratedas MEDIA 2.¶
The constraints of offer/answer meant that Bob could not include his Encodinginformation as new "m=" lines in SIP 200 OK 2. As such, Bob now sends SIPINVITE 3 to generate a new offer. Along with all the streams from SIP 200 OK 2,Bob also includes two new sendonly streams. Each stream has a labelcorresponding to the Encoding IDs in his CLUE ADVERTISEMENT 2 message. He alsoadds their "mid" values to the grouping attribute to show they are controlled bythe CLUE data channel. A snippet of the SDP showing the grouping attribute andthe video "m=" lines are shown below (mid 100 represents the CLUE datachannel, which is not shown):¶
... a=group:CLUE 11 12 14 15 100 ... m=video 58722 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=sendrecv a=mid:10 ... m=video 58724 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=recvonly a=mid:11 m=video 58726 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=recvonly a=mid:12 m=video 0 RTP/AVP 96 a=mid:13 m=video 58728 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=label:foo a=mid:14 m=video 58730 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=label:bar a=mid:15¶
Having received this, Alice now has all the information she needs to sendher CLUE 'configure' message and her SDP Answer. In CLUE CONFIGURE 2, sherequests the two static Captures from Bob to be sent on Encodings "foo" and"bar".¶
Alice also sends SIP 200 OK 3, matching two recvonly "m=" lines to Bob's newsendonly lines. She includes their "mid" values in the grouping attribute toshow they are controlled by the CLUE data channel. Alice then deactivatesthe initial non-CLUE-controlled media, as bidirectional CLUE-controlled mediais now available. A snippet of the SDP showing the grouping attribute and thevideo "m=" lines are shown below (mid 3 represents the data channel, notshown):¶
... a=group:CLUE 3 4 5 7 8 ... m=video 0 RTP/AVP 96 a=mid:2 ... m=video 6004 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=mid:4 a=label:enc1 m=video 6006 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016 a=sendonly a=mid:5 a=label:enc2 m=video 0 RTP/AVP 96 a=mid:6 m=video 6010 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=recvonly a=mid:7 m=video 6012 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=recvonly a=mid:8¶
Bob receives Alice's CLUE CONFIGURE 2 message and sends CLUE CONFIGURERESPONSE 2 to acknowledge its reception. Bob does not yet send the Capture Encodingsspecified, because he hasn't yet received and processed Alice's SDP Answerand negotiated the ability to send these streams.¶
Finally, on receiving SIP 200 OK 3, Bob is now able to send the two streams ofvideo Alice requested -- this is illustrated as MEDIA 3.¶
Both sides of the call are now sending multiple video streams with theirsources defined via CLUE negotiation. As the call progresses, either side cansend a new 'advertisement' or 'configure' message or the new SDP Offers/Answers toadd, remove, or change what they have available or want to receive.¶
In this brief example, Alice is a CLUE-capable Endpoint making a call to Bob,who is not CLUE capable (i.e., is not able to use the CLUE protocol).¶
+----------+ +-----------+ | Alice | | Bob | | | | | +----+-----+ +-----+-----+ | | | | | SIP INVITE 1 | |--------------------------------->| | | | | | 200 0K 1 | |<---------------------------------| | | | | | SIP ACK 1 | |--------------------------------->| | | | | | | |<########### MEDIA 1 ############>| | 1 video A->B, 1 video B->A | |<################################>| | | | | | | | | v v¶
In SIP INVITE 1, Alice sends Bob a SIP INVITE including thebasic audio and video capabilities and data channel in the SDP body as per[RFC8841]. Alice also includes the "sip.clue"media feature tag in the INVITE. A snippet of the SDP showing the groupingattribute and the video "m=" line are shown below. Alice has included a "CLUE"group and the mid corresponding to a data channel in the group (3).Note that Alice has chosen not to include any CLUE-controlled media in theinitial offer -- the "mid" value of the video line is not included in the "CLUE"group.¶
... a=group:CLUE 3 ... m=video 6002 RTP/AVP 96 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=108000;max-fs=3600 a=sendrecv a=mid:2 ... m=application 6100 UDP/DTLS/SCTP webrtc-datachannel a=sctp-port: 5000 a=dcmap:2 subprotocol="CLUE";ordered=true a=mid:3¶
Bob is not CLUE capable and hence does not recognize the "CLUE" semantic for the grouping attribute, nor does he support the data channel. IN SIP 200 OK 1, heresponds with an answer that includes audio and video, but with the data channelzeroed.¶
From the lack of a CLUE group, Alice understands that Bob does not supportCLUE, or does not wish to use it. Both sides are now able to send a singleaudio and video stream to each other. At this point, Alice begins to send herfallback video: in this case, it's likely a switched view from whichever camerashows the current loudest participant on her side.¶
This document registers the following semantics with IANA in the"Semantics for the 'group' SDP Attribute" subregistry (under the"Session Description Protocol (SDP) Parameters" registry) per[RFC5888]:¶
| Semantics | Token | Mux Category | Reference |
|---|---|---|---|
| CLUE-controlled "m=" line | CLUE | NORMAL | RFC 8848 |
This specification registers a new media feature tag in the SIP[RFC3261] tree per the procedures defined in[RFC2506] and[RFC3840].¶
This feature tag is most useful in a communications application for describing the capabilities of a device to use the CLUE control protocol to negotiate the use of multiple media streams.¶
CLUE makes use of a number of protocols and mechanisms, either defined by CLUEor long-standing. The Security Considerations section of the CLUE Framework document[RFC8845] addresses theneed to secure these mechanisms by following the recommendations of theindividual protocols.¶
Beyond the need to secure the constituent protocols, the use of CLUE doesimpose additional security concerns. One area of increased risk involves thepotential for a malicious party to subvert a CLUE-capable device to attack athird party by driving large volumes of media (particularly video) traffic atthem by establishing a connection to the CLUE-capable device and directing themedia to the victim. While this is a risk for all media devices, aCLUE-capable device may allow the attacker to configure multiple media streamsto be sent, significantly increasing the volume of traffic directed at thevictim.¶
This attack can be prevented by ensuring that the media recipient intends toreceive the media packets. As such, all CLUE-capable devicesMUST support key negotiation and receiver intent assurance via DTLS / Secure Real-time Transport Protocol (SRTP)[RFC5763] on CLUE-controlled RTP "m=" lines, and theyMUST use it or some other mechanism that provides receiver intent assurance.All CLUE-controlled RTP "m" lines must be secured and implemented usingmechanisms such as SRTP[RFC3711]. CLUE implementationsMAY choose not to require the use of SRTP to secure legacy(non-CLUE-controlled) media for backwards compatibility with older SIP clientsthat are incapable of supporting it.¶
CLUE also defines a new media feature tag that indicates CLUE support. Thistag may be present even in non-CLUE calls, which increases the metadataavailable about the sending device; this can help an attacker differentiatebetween multiple devices and identify otherwise anonymized usersvia the fingerprint of features their device supports. To prevent this, SIPsignaling used to set up CLUE sessionsSHOULD always be encrypted usingTLS[RFC5630].¶
The CLUE protocol also carries additional information that could be used tohelp fingerprint a particular user or to identify the specific version ofsoftware being used. The CLUE Framework[RFC8847] provides detailsabout these issues and how to mitigate them.¶
Besides the authors, the team focusing on this document consists of:Roni Even,Simon Pietro Romano, andRoberta Presta.¶
Christian Groves,Jonathan Lennox, andAdam Roach have contributed detailedcomments and suggestions.¶