| RFC 9506 | Host-to-Network Flow Measurements | October 2023 |
| Cociglio, et al. | Informational | [Page] |
This document describes protocol-independent methods called ExplicitHost-to-Network Flow Measurement Techniques that can be applicable totransport-layer protocols between the client and server. These methods employ just afew marking bits inside the header of each packet for performance measurementsand require the client and server to collaborate. Both endpoints cooperate by markingpackets and, possibly, mirroring the markings on the round-trip connection. Thetechniques are especially valuable when applied to protocols that encrypttransport headers since they enable loss and delay measurements by passive,on-path network devices. This document describes several methods that can beused separately or jointly depending of the availability of marking bits,desired measurements, and properties of the protocol to which the methods areapplied.¶
This document is not an Internet Standards Track specification; it is published for informational purposes.¶
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are candidates for any level of Internet Standard; see Section 2 of RFC 7841.¶
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained athttps://www.rfc-editor.org/info/rfc9506.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Packet loss and delay are hard and pervasive problems of day-to-day networkoperation. Proactively detecting, measuring, and locating them is crucial tomaintaining high QoS and timely resolution of end-to-end throughput issues.¶
Detecting and measuring packet loss and delay allows network operators toindependently confirm trouble reports and, ideally, be proactively notified ofdeveloping problems on the network. Locating the cause of packet loss orexcessive delay is the first step to resolving problems and restoring QoS.¶
Network operators wishing to perform quantitative measurement ofpacket loss and delay have been heavily relying on information present in theclear in transport-layer headers (e.g., TCP sequence and acknowledgmentnumbers). By passively observing a network path at multiple points within one'snetwork, operators have been able to either quickly locate the source theproblem within their network or reliably attribute it to an upstream ordownstream network.¶
With encrypted protocols, the transport-layer headers are encrypted and passivepacket loss and delay observations are not possible, as also noted in[TRANSPORT-ENCRYPT]. Nevertheless, accurate measurement of packet loss anddelay experienced by encrypted transport-layer protocols is highly desired,especially by network operators who own or control the infrastructure between theclient and server.¶
The measurement of loss and delay experienced by connections using an encryptedprotocol cannot be based on a measurement of loss and delay experienced byconnections between the same or similar endpoints that use an unencryptedprotocol because different protocols may utilize the network differently and berouted differently by the network. Therefore, it is necessary to directlymeasure the packet loss and delay experienced by users of encrypted protocols.¶
The Alternate-Marking method[AltMark] defines a consolidated method toperform packet loss, delay, and jitter measurements on live traffic. However,as mentioned in[IPv6AltMark],[AltMark] mainly applies to a network-layer-controlled domain managed with a Network Management System (NMS), where the Customer Premises Equipment (CPE)or the Provider Edge (PE) routers are the starting or the ending nodes.[AltMark] providesmeasurement within a controlled domain in which the packets aremarked. Therefore, applying[AltMark] to end-to-end transport-layerconnections is not easy because packet identification and marking by networknodes is prevented when encrypted transport-layer headers (e.g., QUIC, TCP withTLS) are being used.¶
This document defines Explicit Host-to-Network Flow Measurement Techniques thatare specifically designed for encrypted transport protocols. According to thedefinitions of[IPPM-METHODS], these measurement methods can be classified asHybrid. They are to be embedded into a transport-layer protocol and areexplicitly intended for exposing delay and loss rate information to on-pathmeasurement devices. Unlike[AltMark], most of these methods requirecollaborative endpoint nodes. Since these measurement techniques make performanceinformation directly visible to the path, they do not rely on an external NMS.¶
The Explicit Host-to-Network Flow Measurement Techniques described in thisdocument are applicable to any transport-layer protocol connecting a client and aserver. In this document, the client and the server are also referred to as theendpoints of the transport-layer protocol.¶
The different methods described in this document can be used alone or incombination. Each technique uses few bits and exposes a specific measurement.It is assumed that the endpoints are collaborative in the sense of themeasurements, indeed both the client and server need to cooperate.¶
Following the recommendation in[RFC8558] of making path signals explicit,this document proposes adding some dedicated measurement bits to the clearportion of the transport protocol headers. These bits can be added to anunencrypted portion of a transport-layer header, e.g., UDP surplus space (see[UDP-OPTIONS] and[UDP-SURPLUS]) or reserved bits in a QUIC v1 header, asalready done with the latency Spin bit (seeSection 17.4 of [QUIC-TRANSPORT]). Note that this document does not recommend the use of anyspecific bits, as these would need to be chosen by the specific protocolimplementations (seeSection 5).¶
The Spin bit, Delay bit, and loss bits explained in this document are inspired by[AltMark],[QUIC-MANAGEABILITY],[QUIC-SPIN],[TSVWG-SPIN],and[IPPM-SPIN].¶
Additional details about the performance measurements for QUIC are described inthe paper[ANRW19-PM-QUIC].¶
This section introduces bits that can be used for round-trip latencymeasurements. Whenever this section of the specification refers to packets, itis referring only to packets with protocol headers that include the latencybits.¶
In[QUIC-TRANSPORT],Section 17.4 introduces an explicit, per-flowtransport-layer signal for hybrid measurement of RTT. This signal consists of aSpin bit that toggles once per RTT.Section 4 of [QUIC-SPIN] discusses anadditional two-bit Valid Edge Counter (VEC) to compensate for loss andreordering of the Spin bit and to increase fidelity of the signal in less thanideal network conditions.¶
This document introduces a standalone single-bit delay signal that can be usedby passive observers to measure the RTT of a network flow, avoiding the Spin bitambiguities that arise as soon as network conditions deteriorate.¶
This section is a small recap of the Spin bit working mechanism. For acomprehensive explanation of the algorithm, seeSection 3.8.2 of [QUIC-MANAGEABILITY].¶
The Spin bit is a signal generated by Alternate-Marking[AltMark], where thesize of the alternation changes with the flight size each RTT.¶
The latency Spin bit is a single-bit signal that toggles once per RTT,enabling latency monitoring of a connection-oriented communicationfrom intermediate observation points.¶
A "Spin bit period" is a set of packets with the same Spin bit value sent during oneRTT time interval. A "Spin bit period value" is the value of the Spin bit shared byall packets in a Spin bit period.¶
The client and server maintain an internal per-connection spin value (i.e., 0 or1) used to set the Spin bit on outgoing packets. Both endpoints initialize thespin value to 0 when a new connection starts. Then:¶
The computed spin value is used by the endpoints for setting the Spinbit on outgoing packets. This mechanism allows the endpointsto generate a square wave such that, by measuring the distance in timebetween pairs of consecutive edges observed in the same direction, apassive on-path observer can compute the round-trip network delay of thatnetwork flow.¶
Spin bit enables round-trip latency measurement by observing a single directionof the traffic flow.¶
Note that packet reordering can cause spurious edges that require heuristics tocorrect. The Spin bit performance deteriorates as soon as network impairmentsarise as explained inSection 2.2.¶
The Delay bit has been designed to overcome accuracy limitations experienced bythe Spin bit under difficult network conditions:¶
Unlike the Spin bit, which is set in every packet transmitted on the network,the Delay bit is set only once per round trip.¶
When the Delay bit is used, a single packet with a marked bit (the Delay bit)bounces between a client and a server during the entire connection lifetime.This single packet is called the "delay sample".¶
An observer placed at an intermediate point, observing a single direction oftraffic and tracking the delay sample and the relative timestamp, can measure theround-trip delay of the connection.¶
The delay sample lifetime comprises two phases: initialization and reflection.The initialization is the generation of the delay sample, while thereflection realizes the bounce behavior of this single packet between the twoendpoints.¶
The next figure describes the elementary Delay bit mechanism.¶
+--------+ - - - - - +--------+ | | -----------> | | | Client | | Server | | | <----------- | | +--------+ - - - - - +--------+ (a) No traffic at beginning. +--------+ 0 0 1 - - +--------+ | | -----------> | | | Client | | Server | | | <----------- | | +--------+ - - - - - +--------+ (b) The Client starts sending data and sets the first packet as the delay sample. +--------+ 0 0 0 0 0 +--------+ | | -----------> | | | Client | | Server | | | <----------- | | +--------+ - - - 1 0 +--------+ (c) The Server starts sending data and reflects the delay sample. +--------+ 0 1 0 0 0 +--------+ | | -----------> | | | Client | | Server | | | <----------- | | +--------+ 0 0 0 0 0 +--------+ (d) The Client reflects the delay sample. +--------+ 0 0 0 0 0 +--------+ | | -----------> | | | Client | | Server | | | <----------- | | +--------+ 0 0 0 1 0 +--------+ (e) The Server reflects the delay sample and so on.
Only the client is actively involved in the Generation Phase. It maintains aninternal per-flow timestamp variable (ds_time) updated every time a delaysample is transmitted.¶
When connection starts, the client generates a new delay sample initializing theDelay bit of the first outgoing packet to 1. Then it updates theds_timevariable with the timestamp of its transmission.¶
The server initializes the Delay bit to 0 at the beginning of the connection,and its only task during the connection is described inSection 2.2.2.¶
In absence of network impairments, the delay sample should bounce between the clientand server continuously for the entire duration of the connection. However, that ishighly unlikely for two reasons:¶
To deal with these problems, the client generates a new delay sample if morethan a predetermined time (T_Max) has elapsed since the last delay sampletransmission (including reflections). Note thatT_Max should be greater thanthe max measurable RTT on the network. SeeSection 2.2.3 for details.¶
Reflection is the process that enables the bouncing of the delay sample betweena client and a server. The behavior of the two endpoints is almost the same.¶
ds_time variable when the outgoing delay sample is actually forwarded.¶In both cases, if the outgoing delay sample is being transmitted with a delaygreater than a predetermined threshold after the reception of the incoming delaysample (1 ms by default), the delay sample is not reflected, and the outgoingDelay bit is kept at 0.¶
By doing so, the algorithm can reject measurements that would overestimate thedelay due to lack of traffic at the endpoints. Hence, the maximum estimationerror would amount to twice the threshold (e.g., 2 ms) per measurement.¶
T_Max SelectionThe internalds_time variable allows a client to identify delay sample losses.Considering that a lost delay sample is regenerated at the end of an explicittime (T_Max) since the last generation, this same value can be used by anobserver to reject a measure and start a new one.¶
In other words, if the difference in time between two delay samples is greateror equal thanT_Max, then these cannot be used to produce a delay measure.Therefore, the value ofT_Max must also be known to the on-path network probes.¶
There are two alternatives to selecting theT_Max value so that both the client andobservers know it. The first one requires thatT_Max is known a priori(T_Max_p) and therefore set within the protocol specifications that implementsthe marking mechanism (e.g., 1 second, which usually is greater than the maxexpected RTT). The second alternative requires a dynamic mechanism able toadapt the duration of theT_Max to the delay of the connection (T_Max_c).¶
For instance, the client and observers could use the connection RTT as a basis forcalculating an effectiveT_Max. They should use a predetermined initial valueso thatT_Max = T_Max_p (e.g., 1 second) and then, when a valid RTT ismeasured, changeT_Max accordingly so thatT_Max = T_Max_c. In any case, theselectedT_Max should be large enough to absorb any possible variations in theconnection delay. This also helps to prevent the mechanism from failing when theobserver cannot recognize sudden changes in RTT exceedingT_Max.¶
T_Max_c could be computed as two times the measuredRTT plus a fixed amountof time (100 ms) to prevent lowT_Max values in the case of very small RTTs.The resulting formula is:T_Max_c = 2RTT + 100 ms. IfT_Max_c is greater thanT_Max_p, thenT_Max_c is forced to theT_Max_p value.Note that the value of 100 ms is provided as an example, and it may be chosendifferently depending on the specific scenarios. For instance, an implementer mayconsider using existing protocol-specific values if appropriate.¶
Note that the observer'sT_Max should always be less than or equal to theclient'sT_Max to avoid considering as a valid measurement what is actuallythe client'sT_Max. To obtain this result, the client waits for twoconsecutive incoming samples and computes the two related RTTs. Then it takesthe largest of them as the basis of theT_Max_c formula. At this point,observers have already measured a valid RTT and then computed theirT_Max_c.¶
When the Delay bit is used, a passive observer can use delay samples directlyand avoid inherent ambiguities in the calculation of the RTT as can be seen inSpin bit analysis.¶
The delay sample generation process ensures that only one packet marked with theDelay bit set to 1 runs back and forth between two endpoints per round-triptime. To determine the RTT measurement of a flow, an on-path passive observercomputes the time difference between two delay samples observed in a singledirection.¶
To ensure a valid measurement, the observer must verify that the distance intime between the two samples taken into account is less thanT_Max.¶
=======================|======================> = ********** -----Obs----> ********** = = * Client * * Server * = = ********** <------------ ********** = <============================================== (a) client-server RTT ==============================================> = ********** ------------> ********** = = * Client * * Server * = = ********** <----Obs----- ********** = <======================|======================= (b) server-client RTT
An observer that is able to observe both forward and return traffic directionscan use the delay samples to measure "upstream" and "downstream" RTT components,also known as the half-RTT measurements. It does this by measuring the timebetween a delay sample observed in one direction and the delay sample previouslyobserved in the opposite direction.¶
As with RTT measurement, the observer must verify that the distance in timebetween the two samples taken into account is less thanT_Max.¶
Note that upstream and downstream sections of paths between the endpoints andthe observer (i.e., observer-to-client vs. client-to-observer andobserver-to-server vs. server-to-observer) may have different delaycharacteristics due to the difference in network congestion and other factors.¶
=======================> = ********** ------|-----> ********** = * Client * Obs * Server * = ********** <-----|------ ********** <======================= (a) client-observer half-RTT =======================> ********** ------|-----> ********** = * Client * Obs * Server * = ********** <-----|------ ********** = <======================= (b) observer-server half-RTT
Intra-domain RTT is the portion of the entire RTT used by a flow to traverse thenetwork of a provider. To measure intra-domain RTT, two observers capable ofobserving traffic in both directions must be employed simultaneously at the ingressand egress of the network to be measured. Intra-domain RTT is the differencebetween the two computed upstream (or downstream) RTT components.¶
=========================================> = =====================> = = ********** ---|--> ---|--> ********** = = * Client * Obs Obs * Server * = = ********** <--|--- <--|--- ********** = <===================== <========================================= (a) client-observer RTT components (half-RTTs) ==================> ********** ---|--> ---|--> ********** * Client * Obs Obs * Server * ********** <--|--- <--|--- ********** <================== (b) the intra-domain RTT resulting from the subtraction of the above RTT components
An on-path observer maintains an internal per-flow variable to keep track ofthe time at which the last delay sample has been observed. The flow characterizationshould be part of the protocol.¶
If the observer is unidirectional or in case of asymmetric routing, then upon detecting adelay sample:¶
T_Max - K, then the two delaysamples can be used to calculate RTT measurement.K is a protectionthreshold to absorb differences inT_Max computation and delay variationsbetween two consecutive delay samples (e.g.,K = 10% T_Max).¶If the observer can observe both forward and return traffic flows, and it isable to determine which direction contains the client and the server (e.g., byobserving the connection handshake), then upon detecting a delay sample:¶
T_Max - K, then the two delay samples canbe used to measure the observer-client half-RTT or the observer-serverhalf-RTT, according to the direction of the last delay sample observed.¶Note that the accuracy can be influenced by what the observer is capable ofobserving. Additionally, the type of measurement differs, as described in theprevious sections.¶
The Spin and Delay bit algorithms work independently. If both marking methods areused in the same connection, observers can choose the best measurement betweenthe two available:¶
This section introduces bits that can be used for loss measurements.Whenever this section of the specification refers to packets, it isreferring only to packets with protocol headers that include the lossbits -- the only packets whose loss can be measured.¶
Loss measurements enabled by T, Q, and L bits can be implemented by those lossbits alone (T bit requires a working Spin bit). Two-bit combinations Q+L and Q+Renable additional measurement opportunities discussed below.¶
Each endpoint maintains appropriate counters independently and separately foreach identifiable flow (or each sub-flow for multipath connections).¶
Since loss is reported independently for each flow, all bits (except for the L bit)require a certain minimum number of packets to be exchanged per flow before anysignal can be measured. Therefore, loss measurements work best for flows thattransfer more than a minimal amount of data.¶
The round-Trip loss bit is used to mark a variable number of packets exchangedtwice between the endpoints realizing a two round-trip reflection. A passiveon-path observer, observing either direction, can count and compare the numberof marked packets seen during the two reflections, estimating the loss rateexperienced by the connection. The overall exchange comprises:¶
Packets belonging to the first round trip (first and second train)represent the Generation Phase, while those belonging to the secondround trip (third and fourth train) represent the Reflection Phase.¶
A passive on-path observer can count and compare the number of markedpackets seen during the two round trips (i.e., the first and thirdor the second and the fourth trains of packets, depending on whichdirection is observed) and estimate the loss rate experienced by theconnection. This process is repeated continuously to obtain moremeasurements as long as the endpoints exchange traffic. Thesemeasurements can be called round-trip losses.¶
Since the packet rates in two directions may be different, the number of markedpackets in the train is determined by the direction with the lowest packet rate.SeeSection 3.1.2 for details on packet generation.¶
Since the measurements are performed on a portion of the traffic exchangedbetween the client and the server, the observer calculates the end-to-end Round-Trip Packet Loss (RTPL) that, statistically, will correspond to the loss rateexperienced by the connection along the entire network path.¶
=======================|======================> = ********** -----Obs----> ********** = = * Client * * Server * = = ********** <------------ ********** = <============================================== (a) client-server RTPL ==============================================> = ********** ------------> ********** = = * Client * * Server * = = ********** <----Obs----- ********** = <======================|======================= (b) server-client RTPL
This methodology also allows the half-RTPL measurement andthe Intra-domain RTPL measurement in a way similar to RTT measurement.¶
=======================> = ********** ------|-----> ********** = * Client * Obs * Server * = ********** <-----|------ ********** <======================= (a) client-observer half-RTPL =======================> ********** ------|-----> ********** = * Client * Obs * Server * = ********** <-----|------ ********** = <======================= (b) observer-server half-RTPL
=========================================> =====================> = ********** ---|--> ---|--> ********** = = * Client * Obs Obs * Server * = = ********** <--|--- <--|--- ********** = = <===================== = <========================================= (a) observer-server RTPL components (half-RTPLs) ==================> ********** ---|--> ---|--> ********** * Client * Obs Obs * Server * ********** <--|--- <--|--- ********** <================== (b) the intra-domain RTPL resulting from the subtraction of the above RTPL components
The round-Trip loss signal requires a working Spin bit signal to separate trainsof marked packets (packets with T bit set to 1). A "pause" of at least oneempty Spin bit period between each phase of the algorithm serves as such aseparator for the on-path observer. The connection between T bit and Spin bithelps the observer correlate packet trains.¶
The client maintains a "generation token" count that is set to zero at thebeginning of the session and is incremented every time a packet is received(marked or unmarked). The client also maintains a "reflection counter" thatstarts at zero at the beginning of the session.¶
The client is in charge of launching trains of marked packets and does soaccording to the algorithm:¶
The generation token counter should be capped to limit the effects of asubsequent sudden reduction in the other endpoint's packet rate that couldprevent that endpoint from reflecting collected packets. Acap value of 1 is recommended.¶
A server maintains a "marking counter" that starts at zero and is incrementedevery time a marked packet arrives. When the server transmits a packet and the"marking counter" is positive, the server marks the packet and decrements the"marking counter". If the "marking counter" is zero, the outgoing packet istransmitted unmarked.¶
Note that a choice of 2 RTT (two Spin bit periods) for the Generation Phase is atrade-off between the percentage of marked packets (i.e., the percentage oftraffic monitored) and the measurement delay. Using this value, the algorithmproduces a measurement approximately every 6 RTT (2 generations, ~2reflections, 2 pauses), marking ~1/3 of packets exchanged in the slowerdirection (seeSection 3.1.4). Choosing a Generation Phase of 1 RTT, we wouldproduce measurements every 4 RTT, monitoring ~1/4 of packets in theslower direction.¶
It is worth mentioning that problems can happen in some cases, especially if therate suddenly changes, but the mechanism described hereworked well with normal traffic conditions in the implementation.¶
The on-path observer counts marked packets and separates different trains bydetecting Spin bit periods (at least one) with no marked packets. The Round-Trip Packet Loss (RTPL) is the difference between the size of the Generationtrain and the Reflection train.¶
In the following example, packets are represented by two bits (first oneis the Spin bit, second one is the round-Trip loss bit):¶
Generation Pause Reflection Pause ____________________ ______________ ____________________ ________ | | | | | 01 01 00 01 11 10 11 00 00 10 10 10 01 00 01 01 10 11 10 00 00 10
Note that 5 marked packets have been generated, of which 4 have been reflected.¶
A cycle of the round-Trip loss signaling algorithm contains 2 RTTs of Generationphase, 2 RTTs of Reflection Phase, and 2 Pause Phases at least 1 RTT induration each. Hence, the loss signal is delayed by about 6 RTTs since the lossevents.¶
The observer can only detect the loss of marked packets that occurs after itsinitial observation of the Generation Phase and before its subsequentobservation of the Reflection Phase. Hence, if the loss occurs on the path thatsends packets at a lower rate (typically ACKs in such asymmetric scenarios),2/6 (1/3) of the packets will be sampled for loss detection.¶
If the loss occurs on the path that sends packets at a higher rate,lowPacketRate/(3*highPacketRate) of the packets will be sampled for lossdetection. For protocols that use ACKs, the portion of packets sampled for lossin the higher rate direction during unidirectional data transfer is1/(3*packetsPerAck), where the value ofpacketsPerAck can vary by protocol,by implementation, and by network conditions.¶
The sQuare bit (Q bit) takes its name from the square wave generated by itssignal. This method is based on the Alternate-Marking method[AltMark], andthe Q bit represents the "packet color" that can be switched between 0 and 1 in order to mark consecutive blocksof packets with different colors. This method does not require cooperation fromboth endpoints.¶
[AltMark] introduces two variations of the Alternate-Marking method dependingon whether the color is switched according to a fixed timer or after a fixednumber of packets. Cooperating and synchronized observers on either end of a network segment can use the fixed-timer method to measure packet loss on thesegment by comparing packet counters for the same packet blocks. The time length ofthe blocks can be chosen depending on the desired measurement frequency, but itmust be long enough to guarantee the proper operation with respect to clock errorsand network delay issues.¶
The Q bit method described in this document chooses the color-switching methodbased on a fixed number of packets for each block. This approach has theadvantage that it does not require cooperating or synchronized observers ornetwork elements. Each probe can measure packet loss autonomously withoutrelying on an external NMS. For the purpose of thepacket loss measurement, all blocks have the same number of packets, and it isnecessary to detect only the loss event and not to identify the exact block withlosses.¶
Following the method based on fixed number of packets, the square wave signal isgenerated by the switching of the Q bit: every outgoing packet contains the Qbit value, which is initialized to 0 and inverted after sending N packets (asQuare Block or simply Q Block). Hence, Q Period is 2*N.¶
Observation points can estimate upstream losses by watching a single directionof the traffic flow and counting the number of packets in each observed Q Block,as described inSection 3.2.2.¶
The length of the block must be known to the on-path network probes. There aretwo alternatives to selecting the Q Block length. The first one requires thatthe length is known a priori and therefore set within the protocolspecifications that implement the marking mechanism. The second requires thesender to select it.¶
In this latter scenario, the sender is expected to choose N (Q Blocklength) based on the expected amount of loss and reordering on thepath. The choice of N strikes a compromise -- the observation couldbecome too unreliable in case of packet reordering and/or severe lossif N is too small, while short flows may not yield a useful upstreamloss measurement if N is too large (seeSection 3.2.2).¶
The value of N should be at least 64 and be a power of 2. This requirement allowsan observer to infer the Q Block length by observing one period of the squaresignal. It also allows the observer to identify flows that set the loss bits toarbitrary values (seeSection 6).¶
If the sender does not have sufficient information to make an informed decisionabout Q Block length, the sender should use N=64, since this value has beenextensively tried in large-scale field tests and yielded good results.Alternatively, the sender may also choose a random power-of-2 N for each flow,increasing the chances of using a Q Block length that gives the best signal forsome flows.¶
The sender must keep the value of N constant for a given flow.¶
Blocks of N (Q Block length) consecutive packets are sent with the samevalue of the Q bit, followed by another block of N packets with aninverted value of the Q bit. Hence, knowing the value of N, anon-path observer can estimate the amount of upstream loss afterobserving at least N packets. The upstream loss rate (uloss) is oneminus the average number of packets in a block of packets with thesame Q value (p) divided by N (uloss=1-avg(p)/N).¶
The observer needs to be able to tolerate packet reordering that canblur the edges of the square signal, as explained inSection 3.2.3.¶
=====================> ********** -----Obs----> ********** * Client * * Server * ********** <------------ ********** (a) in client-server channel (uloss_up) ********** ------------> ********** * Client * * Server * ********** <----Obs----- ********** <===================== (b) in server-client channel (uloss_down)
Packet reordering can produce spurious edges in the square signal. To addressthis, the observer should look for packets with the current Q bit value up to Xpackets past the first packet with a reverse Q bit value. The value of X, a"Marking Block Threshold", should be less thanN/2.¶
The choice of X represents a trade-off between resiliency to reordering andresiliency to loss. A very large Marking Block Threshold will be able toreconstruct Q Blocks despite a significant amount of reordering, but it mayerroneously coalesce packets from multiple Q Blocks into fewer Q Blocks if lossexceeds 50% for some Q Blocks.¶
Burst losses can affect the accuracy of Q measurements. Generally, burst losses can beabsorbed and correctly measured if smaller than the established Q Blocklength. If the entire Q Block length of packets is lost in a burst, however, theobserver may be left completely unaware of the loss.¶
To improve burst loss resilience, an observer may consider a received Q Blocklarger than the selected Q Block length as an indication of a burst lossevent. The observer would then compute the loss as three times the Q Block lengthminus the measured block length. By doing so, the observer can detect burstlosses of less than two blocks (e.g., less than 128 packets for a Q Block lengthof 64 packets). A burst loss of two or more consecutive periods would stillremain unnoticed by the observer (or underestimated if a period longer than QBlock length were formed).¶
The Loss Event bit uses an Unreported Loss counter maintained by the protocolthat implements the marking mechanism. To use the Loss Event bit, the protocolmust allow the sender to identify lost packets. This is true of protocols suchas QUIC, partially true for TCP and Stream Control Transmission Protocol (SCTP) (losses of pure ACKs are not detected),and is not true of protocols such as UDP and IPv4/IPv6.¶
The Unreported Loss counter is initialized to 0, and the L bit of every outgoingpacket indicates whether the Unreported Loss counter is positive (L=1 if thecounter is positive, and L=0 otherwise).¶
The value of the Unreported Loss counter is decremented every time a packetwith L=1 is sent.¶
The value of the Unreported Loss counter is incremented for every packet thatthe protocol declares lost, using whatever loss detection machinery the protocolemploys. If the protocol is able to rescind the loss determination later, apositive Unreported Loss counter may be decremented due to the rescission.In general, it should not become negative due to the rescission, but it can happenin few cases.¶
This loss signaling is similar to loss signaling in[ConEx], except that the LossEvent bit is reporting the exact number of lost packets, whereas the signal mechanismin[ConEx] is reporting an approximate number of lost bytes.¶
For protocols, such as TCP[TCP], that allow network devices to change datasegmentation, it is possible that only a part of the packet is lost. In thesecases, the sender must increment the Unreported Loss counter by the fraction of thepacket data lost (so the Unreported Loss counter may become negative when a packetwith L=1 is sent after a partial packet has been lost).¶
Observation points can estimate the end-to-end loss, as determined by theupstream endpoint, by counting packets in this direction with the L bit equal to1, as described inSection 3.3.1.¶
The Loss Event bit allows an observer to estimate the end-to-end loss rate bycounting packets with L bit values of 0 and 1 for a given flow. The end-to-endloss ratio is the fraction of packets with L=1.¶
The assumption here is that upstream loss affects packets with L=0 and L=1equally. If some loss is caused by tail-drop in a network device, this may be asimplification. If the sender's congestion controller reduces the packet sendrate after loss, there may be a sufficient delay before sending packets with L=1that they have a greater chance of arriving at the observer.¶
The Loss Event bit allows an observer to characterize the loss profile, since thedistribution of observed packets with the L bit set to 1 roughly corresponds to thedistribution of packets lost between 1 RTT and 1 retransmission timeout (RTO) before (seeSection 3.3.2.1). Hence, observing random single instances of the L bit setto 1 indicates random single packet loss, while observing blocks of packetswith the L bit set to 1 indicates loss affecting entire blocks of packets.¶
Combining L and Q bits allows a passive observer watching a single direction oftraffic to accurately measure:¶
Upstream loss is calculated by observing packets that did not suffer theupstream loss (Section 3.2.2). End-to-end loss, however, is calculated byobserving subsequent packets after the sender's protocol detected the loss.Hence, end-to-end loss is generally observed with a delay of between 1 RTT (lossdeclared due to multiple duplicate acknowledgments) and 1 RTO (loss declareddue to a timeout) relative to the upstream loss.¶
The flow RTT can sometimes be estimated by timing the protocol handshakemessages. This RTT estimate can be greatly improved by observing a dedicatedprotocol mechanism for conveying RTT information, such as the Spin bit (seeSection 2.1) or Delay bit (seeSection 2.2).¶
Whenever the observer needs to perform a computation that uses both upstream andend-to-end loss rate measurements, it should consider the upstream loss rate leading theend-to-end loss rate by approximately 1 RTT. If the observer is unable toestimate RTT of the flow, it should accumulate loss measurements over timeperiods of at least 4 times the typical RTT for the observed flows.¶
If the calculated upstream loss rate exceeds the end-to-end loss rate calculatedinSection 3.3.1, then either the Q Period is too short for the amount ofpacket reordering or there is observer loss, described inSection 3.3.2.3. Ifthis happens, the observer should adjust the calculated upstream loss rate tomatch end-to-end loss rate, unless the following applies.¶
In case of a protocol, such as TCP or SCTP, that does not track losses of pureACK packets, observing a direction of traffic dominated by pure ACK packetscould result in measured upstream loss that is higher than measured end-to-endloss if said pure ACK packets are lost upstream. Hence, if the measurement isapplied to such protocols, and the observer can confirm that pure ACK packetsdominate the observed traffic direction, the observer should adjust thecalculated end-to-end loss rate to match upstream loss rate.¶
Because downstream loss affects only those packets that did not suffer upstreamloss, the end-to-end loss rate (eloss) relates to the upstream loss rate(uloss) and downstream loss rate (dloss) as(1-uloss)(1-dloss)=1-eloss.Hence,dloss=(eloss-uloss)/(1-uloss).¶
A typical deployment of a passive observation system includes a network tapdevice that mirrors network packets of interest to a device that performsanalysis and measurement on the mirrored packets. The observer loss is the lossthat occurs on the mirror path.¶
Observer loss affects the upstream loss rate measurement since it causes theobserver to account for fewer packets in a block of identical Q bit values (seeSection 3.2.2). The end-to-end loss rate measurement, however, is unaffectedby the observer loss since it is a measurement of the fraction of packets withthe L bit value of 1, and the observer loss would affect all packets equally(seeSection 3.3.1).¶
The need to adjust the upstream loss rate down to match the end-to-end loss rate asdescribed inSection 3.3.2.1 is an indication of the observer loss, whosemagnitude is between the amount of such adjustment and the entirety of theupstream loss measured inSection 3.2.2. Alternatively, a high apparentupstream loss rate could be an indication of significant packet reordering,possibly due to packets belonging to a single flow being multiplexed overseveral upstream paths with different latency characteristics.¶
R bit requires a deployment alongside Q bit. Unlike the square signal for whichpackets are transmitted in blocks of fixed size, the number of packets inReflection square blocks (also an Alternate-Marking signal) variesaccording to these rules:¶
The Reflection square value is initialized to 0 and is applied to the R bit ofevery outgoing packet. The Reflection square value is toggled for the firsttime when the completion of a Q Block is detected in the incoming square signal(produced by the other endpoint using the Q bit). The number of packetsdetected within this first Q Block (p), is used to generate a reflectionsquare signal that toggles everyM=p packets (at first). This new signalproduces blocks of M packets (marked using the R bit) and each of them iscalled "Reflection Block" (Reflection Block).¶
The M value is then updated every time a completed Q Block in theincoming square signal is received, following this formula:M=round(avg(p)).¶
The parameteravg(p), the average number of packets in a markingperiod, is computed based on all the Q Blocks received since thebeginning of the current Reflection Block.¶
The transmission of a Reflection Block is considered complete (and the signal toggled)when the number of packets transmitted in that block is at least the latestcomputed M value.¶
To ensure a proper computation of the M value, endpoints implementing the R bitmust identify the boundaries of incoming Q Blocks. The same approach describedinSection 3.2.3 should be used.¶
By looking at the R bit, unidirectional observation points have an indication ofloss experienced by the entire unobserved channel plus the loss on the pathfrom the sender to them.¶
Since the Q Block is sent in one direction, and the corresponding reflected RBlock is sent in the opposite direction, the reflected R signal is transmittedwith the packet rate of the slowest direction. Namely, if the observed directionis the slowest, there can be multiple Q Blocks transmitted in the unobserveddirection before a complete Reflection Block is transmitted in the observed direction. Ifthe unobserved direction is the slowest, the observed direction can be sending RBlocks of the same size repeatedly before it can update the signal to accountfor a newly completed Q Block.¶
The use of the rounding function used in the M computation introduces errorsthat can be minimized by storing the rounding applied each time M is computedand using it during the computation of the M value in the following Reflection Block.¶
This can be achieved by introducing the newr_avg parameter in the computation ofM. The new formula isMr=avg(p)+r_avg; M=round(Mr); r_avg=Mr-M where theinitial value ofr_avg is equal to 0.¶
When a protocol implementing the marking mechanism is able to detect whenpackets are received out of order, it can improve resilience to packetreordering beyond what is possible by using methods described inSection 3.2.3.¶
This can be achieved by updating the size of the current Reflection Block while it isbeing transmitted. The Reflection Block size is then updated every time anincoming reordered packet of the previous Q Block is detected. This can bedone if and only if the transmission of the current Reflection Block is inprogress and no packets of the following Q Block have been received.¶
Burst losses can affect the accuracy of R measurements similar to how they affectaccuracy of Q measurements. Therefore, recommendations inSection 3.2.3.1 applyequally to improving burst loss resilience for R measurements.¶
Since both sQuare and Reflection square bits are toggled at most every N packets(except for the first transition of the R bit as explained before), an on-pathobserver can count the number of packets of each marking block and, knowing thevalue of N, can estimate the amount of loss experienced by the connection. Anobserver can calculate different measurements depending on whether it is able toobserve a single direction of the traffic or both directions.¶
Except for the very first block in which there is nothing to reflect(a complete Q Block has not been yet received), packets arecontinuously R-bit marked into alternate blocks of size lower or equalthan N. By knowing the value of N, an on-path observer can estimate theamount of loss that has occurred in the whole opposite channel plus the lossfrom the sender up to it in the observation channel. As for theprevious metric, thethree-quarters connection loss rate (tqloss) isone minus the average number of packets in a block of packets with thesame R value (t) divided byN (tqloss=1-avg(t)/N).¶
=======================> = ********** -----Obs----> ********** = * Client * * Server * = ********** <------------ ********** <============================================ (a) in client-server channel (tqloss_up) ============================================> ********** ------------> ********** = * Client * * Server * = ********** <----Obs----- ********** = <======================= (b) in server-client channel (tqloss_down)
The following metrics derive from this last metric and the upstreamloss produced by the Q bit.¶
End-to-end loss in the unobserved direction (eloss_unobserved) relates to the"three-quarters" connection loss (tqloss) and upstream loss in the observeddirection (uloss) as(1-eloss_unobserved)(1-uloss)=1-tqloss. Hence,eloss_unobserved=(tqloss-uloss)/(1-uloss).¶
********** -----Obs----> ********** * Client * * Server * ********** <------------ ********** <========================================== (a) in client-server channel (eloss_down) ==========================================> ********** ------------> ********** * Client * * Server * ********** <----Obs----- ********** (b) in server-client channel (eloss_up)
If the observer is able to observe both directions of traffic, it is able tocalculate two "half round-trip" loss measurements -- loss from the observer tothe receiver (in a given direction) and then back to the observer in theopposite direction. For both directions, "half round-trip" loss (hrtloss)relates to "three-quarters" connection loss (tqloss_opposite) measured in theopposite direction and the upstream loss (uloss) measured in the givendirection as(1-uloss)(1-hrtloss)=1-tqloss_opposite. Hence,hrtloss=(tqloss_opposite-uloss)/(1-uloss).¶
=======================> = ********** ------|-----> ********** = * Client * Obs * Server * = ********** <-----|------ ********** <======================= (a) client-observer half round-trip loss (hrtloss_co) =======================> ********** ------|-----> ********** = * Client * Obs * Server * = ********** <-----|------ ********** = <======================= (b) observer-server half round-trip loss (hrtloss_os)
If the observer is able to observe both directions of traffic, it is able tocalculate two downstream loss measurements using either end-to-end loss andupstream loss, similar to the calculation inSection 3.3.2.2, or "halfround-trip" loss and upstream loss in the opposite direction.¶
For the latter,dloss=(hrtloss-uloss_opposite)/(1-uloss_opposite).¶
=====================> ********** ------|-----> ********** * Client * Obs * Server * ********** <-----|------ ********** (a) in client-server channel (dloss_up) ********** ------|-----> ********** * Client * Obs * Server * ********** <-----|------ ********** <===================== (b) in server-client channel (dloss_down)
While the primary focus of this document is on exposing packet loss anddelay, modern networks can report congestion before they are forced todrop packets, as described in[ECN]. When transport protocols keepECN-Echo feedback under encryption, this signal cannot be observed bythe network operators. When tasked with diagnosing networkperformance problems, knowledge of a congestion downstream of anobservation point can be instrumental.¶
If downstream congestion information is desired, this information can besignaled with an additional bit.¶
The Unreported ECN-Echo counter operates identically to Unreported Loss counter(Section 3.3), except it counts packets delivered by the network with Congestion Experienced (CE)markings, according to the ECN-Echo feedback from the receiver.¶
This ECN-Echo signaling is similar to ECN signaling in[ConEx]. The ECN-Echomechanism in QUIC provides the number of packets received with CE marks. Forprotocols like TCP, the method described in[ConEx-TCP] can be employed. Asstated in[ConEx-TCP], such feedback can be further improved using a methoddescribed in[ACCURATE-ECN].¶
A network observer can count packets with the CE codepoint and determine theupstream CE-marking rate directly.¶
Observation points can also estimate ECN-reported end-to-end congestion bycounting packets in this direction with an E bit equal to 1.¶
The upstream CE-marking rate and end-to-end ECN-reported congestion can provideinformation about the downstream CE-marking rate. The presence of E bits along with Lbits, however, can somewhat confound precise estimates of upstream anddownstream CE markings if the flow contains packets that are notECN capable.¶
Some protocols, such as QUIC, support separate ECN-Echo counters. For example,Section 13.4.1 of [QUIC-TRANSPORT] describes separate counters for ECT(0),ECT(1), and ECN-CE. To better support such protocols, multiple E bits can beused, one per a corresponding ECN-Echo counter.¶
This section summarizes the marking methods described in this document, whichproposes a toolkit of techniques that can be used separately, partly, or alltogether depending on the need.¶
For the delay measurement, it is possible to use the Spin bit and/or the Delaybit. A unidirectional or bidirectional observer can be used.¶
| Method | # of bits | Available Delay Metrics | Impairments Resiliency | # of meas. | |
|---|---|---|---|---|---|
| UniDir Observer | BiDir Observer | ||||
| S: Spin Bit | 1 | RTT | x2, Half RTT | low | very high |
| D: Delay Bit | 1 | RTT | x2, Half RTT | high | medium |
| SD: Spin Bit & Delay Bit * | 2 | RTT | x2, Half RTT | high | very high |
For the Loss measurement, each row inTable 2represents a loss-marking method. For each method, the table specifiesthe number of bits required in the header, the available metrics usinga unidirectional or bidirectional observer, applicable protocols,measurement fidelity, and delay.¶
| Method | Bits | Available Loss Metrics | Prto | Measurement Aspects | ||
|---|---|---|---|---|---|---|
| UniDir Observer | BiDir Observer | Fidelity | Delay | |||
| T: Round-Trip Loss Bit | $1 | RT | x2, Half RT | * | Rate by sampling 1/3 to 1/(3*ppa) of pkts over 2 RTT | ~6 RTT |
| Q: sQuare Bit | 1 | Upstream | x2 | * | Rate over N pkts (e.g., 64) | N pkts (e.g., 64) |
| L: Loss Event Bit | 1 | E2E | x2 | # | Loss shape (and rate) | Min: RTT, Max: RTO |
| QL: sQuare + Loss Ev. Bits | 2 | Upstream | x2 | # | see Q | see Q |
| Downstream | x2 | # | see Q|L | see L | ||
| E2E | x2 | # | see L | see L | ||
| QR: sQuare + Ref. Sq. Bits | 2 | Upstream | x2 | * | Rate over N*ppa pkts (see Q bit for N) | see Q |
| 3/4 RT | x2 | * | N*ppa pkts (see Q bit for N) | |||
| !E2E | E2E, Downstream, Half RT | * | ||||
By combining the information of the two tables above, it can be deduced that thesolutions with 3 bits (i.e., QL or QR + S or D) or 4 bits (i.e., QL or QR + SD)allow having more complete and resilient measurements.¶
The methodologies described in the previous sections are transport agnostic andcan be applied in various situations. The choice of the methods also dependson the specific protocol. For example, QL is a good combination; however, ifa protocol does not support, or cannot set, the L bit, QR is the onlyviable solution.¶
This document describes several measurement methods, but it is not expected thatall methods will be implemented together. For example, only some of the methodsdescribed in this document (i.e., sQuare bit and Spin bit) are utilized in[CORE-COAP-PM]. Also, the binding of a delay signal to QUIC ispartially described inSection 17.4 of [QUIC-TRANSPORT], which adds only theSpin bit to the first byte of the short packet header, leaving two reserved bitsfor future use (seeSection 17.2.2 of [QUIC-TRANSPORT]).¶
All signals discussed in this document have been implemented in successfulexperiments for both QUIC and TCP. The application scenarios considered allowthe monitoring of the interconnections inside a data center (Intra-DC), betweendata centers (Inter-DC), as well as end-to-end large-scale data transfers. Forthe application of the methods described in this document, it is assumed thatthe monitored flows follow stable paths and traverse the same measurementpoints.¶
The specific implementation details and the choice of the bits used for theexperiments with QUIC and TCP are out of scope for this document. Aspecification defining the specific protocol application is expected to discussthe implementation details depending on which bits will be implemented in theprotocol, e.g.,[CORE-COAP-PM]. If bits used for specificmeasurements can also be used for other purposes by a protocol, thespecification is expected to address ways for on-path observers to disambiguatethe signals or to discuss limitations on the conditions under which theobservers can expect a valid signal.¶
Accurate loss and delay information is not required for the operation of anyprotocol, though its presence for a sufficient number of flows is important forthe operation of networks.¶
The delay and loss bits are amenable to "greasing" described in[RFC8701] ifthe protocol designers are not ready to dedicate (and ossify) bits used for lossreporting to this function. The greasing could be accomplished similarly to thelatency Spin bit greasing inSection 17.4 of [QUIC-TRANSPORT]. For example,the protocol designers could decide that a fraction of flows should not encodeloss and delay information, and instead, the bits would be set to arbitraryvalues. Setting any of the bits described in this document to arbitrary valueswould make the corresponding delay and loss information resemble noise ratherthan the expected signal for the flow, and the observers would need to be readyto ignore such flows.¶
The methods described in this document are transport agnostic and potentiallyapplicable to any transport-layer protocol, and especially valuable for encryptedprotocols. These methods can be applied to both limited domains and the Internet,depending on the specific protocol application.¶
Passive loss and delay observations have been a part of the network operationsfor a long time, so exposing loss and delay information to the network does notadd new security concerns for protocols that are currently observable.¶
In the absence of packet loss, Q and R bits signals do not provide anyinformation that cannot be observed by simply counting packets transiting anetwork path. In the presence of packet loss, Q and R bits will disclosethe loss, but this is information about the environment and not the endpointstate. The L bit signal discloses internal state of the protocol's loss-detection machinery, but this state can often be gleaned by timing packets andobserving the congestion controller response.¶
The measurements described in this document do not imply that new packets injectedinto the network can cause potential harm to the network itself and to datatraffic. The measurements could be harmed by an attacker altering the markingof the packets or injecting artificial traffic. Authentication techniques may beused where appropriate to guard against these traffic attacks.¶
Hence, loss bits do not provide a viable new mechanism to attack data integrityand secrecy.¶
The measurement fields introduced in this document are intended to be includedin the packets. However, it is worth mentioning that it may be possible to use thisinformation as a covert channel.¶
This document does not define a specific application, and the describedtechniques can generally apply to different communication protocols operating indifferent security environments. A specification defining a specific protocolapplication is expected to address the respective security considerations andmust consider specifics of the protocol and its expected operating environment.For example, security considerations for QUIC, discussed inSection 21 of [QUIC-TRANSPORT] andSection 9 of [QUIC-TLS], consider a possibility ofactive and passive attackers in the network as well as attacks on specific QUICmechanisms.¶
A defense against an optimistic ACK attack, described inSection 21.4 of [QUIC-TRANSPORT], involves a sender randomly skipping packet numbers to detecta receiver acknowledging packet numbers that have never been received. The Q bitsignal may inform the attacker which packet numbers were skipped on purpose andwhich had been actually lost (and are, therefore, safe for the attacker toacknowledge). To use the Q bit for this purpose, the attacker must first receiveat least an entire Q Block of packets, which renders the attack ineffectiveagainst a delay-sensitive congestion controller.¶
A protocol that is more susceptible to an optimistic ACK attack with the losssignal provided by the Q bit and that uses a loss-based congestion controller shouldshorten the current Q Block by the number of skipped packets numbers. For example,skipping a single packet number will invert the square signal one outgoingpacket sooner.¶
Similar considerations apply to the R bit, although a shortened Reflection Block alongwith a matching skip in packet numbers does not necessarily imply a lostpacket, since it could be due to a lost packet on the reverse path along with adeliberately skipped packet by the sender.¶
Theoretically, delay measurements can be used to roughly evaluate the distanceof the client from the server (using the RTT) or from any intermediate observer(using the client-observer half-RTT). As described in[RTT-PRIVACY], connectionRTT measurements for geolocating endpoints are usually inferior to even themost basic IP geolocation databases. It is the variability within RTTmeasurements (the jitter) that is most informative, as it can provide insightinto the operating environment of the endpoints as well as the state of thenetworks (queuing delays) used by the connection.¶
Nevertheless, to further mask the actual RTT of the connection, the Delay bitalgorithm can be slightly modified by, for example, delaying the client-sidereflection of the delay sample by a fixed, randomly chosen time value. Thiswould lead an intermediate observer to measure a delay greater than the realone.¶
This Additional Delay should be randomly selected by the client and keptconstant for a certain amount of time across multiple connections. This ensuresthat the client-server jitter remains the same as if no Additional Delay hadbeen inserted. For example, a new Additional Delay value could be generatedwhenever the client's IP address changes.¶
Despite the Additional Delay, this Hidden Delay technique still allows anaccurate measurement of the RTT components (observer-server) and all theintra-domain measurements used to distribute the delay in the network.Furthermore, unlike the Delay bit, the Hidden Delay bit does not require theuse of the client reflection threshold (1 ms by default). Removing thisthreshold may lead to increasing the number of valid measurements produced bythe algorithm.¶
Note that the Hidden Delay bit does not affect an observer's ability to measureaccurate RTT using other means, such as timing packets exchanged during theconnection establishment.¶
To minimize unintentional exposure of information, loss bits provide an explicitloss signal -- a preferred way to share information per[RFC8558].¶
New protocols commonly have specific privacy goals, and loss reporting mustensure that loss information does not compromise those privacy goals. Forexample,[QUIC-TRANSPORT] allows changing Connection IDs in the middle of aconnection to reduce the likelihood of a passive observer linking old and newsub-flows to the same device (seeSection 5.1 of [QUIC-TRANSPORT]). A QUICimplementation would need to reset all counters when it changes the destination(IP address or UDP port) or the Connection ID used for outgoing packets. Itwould also need to avoid incrementing the Unreported Loss counter for loss ofpackets sent to a different destination or with a different Connection ID.¶
It is also worth highlighting that, if these techniques are not widely deployed,an endpoint that uses them may be fingerprinted based on their usage.However, since there is no release of user data, the techniques seem unlikely tosubstantially increase the existing privacy risks.¶
Furthermore, if there is experimental traffic with these bits set on the network,a network operator could potentially prioritize this marked traffic by placing itin a priority queue. This may result in the delivery of better service, whichcould potentially mislead an experiment intended to benchmark the network.¶
This document has no IANA actions.¶
The authors would like to thank the QUIC WG for their contributions,Christian Huitema for implementing Q and L bits in his picoquic stack, andIke Kunze forproviding constructive reviews and helpful suggestions.¶
The following people provided valuable contributions to this document:¶