4.1.Design Requirements

This section gives an illustrative example of how an overlay application canbenefit from the extension defined in this document.¶

Assume that an application has control over a set of flows, which may go throughshared links/nodes and share bottlenecks. The application seeks to schedule thetraffic among multiple flows to get better performance. The constraints offeasible rate allocations of those flows will benefit the scheduling. However,Cost Maps as defined in[RFC7285] can not reveal such information.¶

Specifically, consider a network as shown inFigure 1. The network has 7switches (sw1 to sw7) forming a dumb-bell topology. Switches "sw1", "sw2", "sw3"and "sw4" are access switches, and sw5-sw7 form the backbone. End hosts eh1 toeh4 are connected to access switches sw1 to sw4 respectively. Assume that thebandwidth of link eh1 -> sw1 and link sw1 -> sw5 is 150 Mbps, and the bandwidthof the other links is 100 Mbps.¶

                              +-----+                              |     |                            --+ sw6 +--                           /  |     |  \     PID1 +-----+         /   +-----+   \          +-----+  PID2     eh1__|     |_       /               \     ____|     |__eh2192.0.2.2 | sw1 | \   +--|--+         +--|--+ /    | sw2 | 192.0.2.3          +-----+  \  |     |         |     |/     +-----+                    \_| sw5 +---------+ sw7 |     PID3 +-----+   / |     |         |     |\     +-----+  PID4     eh3__|     |__/  +-----+         +-----+ \____|     |__eh4192.0.2.4 | sw3 |                                  | sw4 | 192.0.2.5          +-----+                                  +-----+bw(eh1--sw1) = bw(sw1--sw5) = 150 Mbpsbw(eh2--sw2) = bw(eh3--sw3) = bw(eh4--sw4) = 100 Mbpsbw(sw1--sw5) = bw(sw3--sw5) = bw(sw2--sw7) = bw(sw4--sw7) = 100 Mbpsbw(sw5--sw6) = bw(sw5--sw7) = bw(sw6--sw7) = 100 Mbps

The base ALTO topology abstraction of the network is shown inFigure 2.Assume the cost map returns an hypothetical cost type representing the availablebandwidth between a source and a destination.¶

                          +----------------------+                 {eh1}    |                      |     {eh2}                 PID1     |                      |     PID2                   +------+                      +------+                          |                      |                          |                      |                 {eh3}    |                      |     {eh4}                 PID3     |                      |     PID4                   +------+                      +------+                          |                      |                          +----------------------+

Now assume the application wants to maximize the total rate of the traffic amonga set of <source, destination> pairs, say "eh1 -> eh2" and "eh1 -> eh4". Let "x"denote the transmission rate of "eh1 -> eh2" and "y" denote the rate of "eh1 ->eh4". The objective function is¶

    max(x + y).

With the ALTO Cost Map, the cost between PID1 and PID2 and between PID1 and PID4 willboth be 100 Mbps. The client can get a capacity region of¶

    x <= 100 Mbps,    y <= 100 Mbps.

With this information, the client may mistakenly think it can achieve a maximumtotal rate of 200 Mbps. However, this rate is infeasible, as there are only twopotential cases:¶

• Case 1: "eh1 -> eh2" and "eh1 -> eh4" take different path segments from "sw5" to "sw7". Forexample, if "eh1 -> eh2" uses path "eh1 -> sw1 -> sw5 -> sw6 -> sw7 -> sw2 -> eh2"and "eh1 -> eh4" uses path "eh1 -> sw1 -> sw5 -> sw7 -> sw4 -> eh4", then the sharedbottleneck links are "eh1 -> sw1" and "sw1 -> sw5". In this case, the capacityregion is:¶

      x     <= 100 Mbps    y     <= 100 Mbps    x + y <= 150 Mbps

  ¶

  and the real optimal total rate is 150 Mbps.¶

• Case 2: "eh1 -> eh2" and "eh1 -> eh4" take the same path segment from "sw5" to "sw7".For example, if "eh1 -> eh2" uses path "eh1 -> sw1 -> sw5 -> sw7 -> sw2 -> eh2"and "eh1 -> eh4" also uses path "eh1 -> sw1 -> sw5 -> sw7 -> sw4 -> eh4", then theshared bottleneck link is "sw5 -> sw7". In this case, the capacity region is:¶

      x     <= 100 Mbps    y     <= 100 Mbps    x + y <= 100 Mbps

  ¶

  and the real optimal total rate is 100 Mbps.¶

Clearly, with more accurate and fine-grained information, the application cangain a better prediction of its traffic and may orchestrate its resourcesaccordingly. However, to provide such information, the network needs to exposeabstract information beyond the simple cost map abstraction. In particular:¶

• The ALTO server must expose abstract information about the network paths that aretraversed by the traffic between a source and a destination beyond a simplenumerical value, which allows the overlay application to distinguish betweenCases 1 and 2 and to compute the optimal total rate accordingly.¶
• The ALTO server must allow the client to distinguish the common ANE shared by"eh1 -> eh2" and "eh1 -> eh4", e.g., "eh1 - sw1" and "sw1 - sw5" in Case 1.¶
• The ALTO server must expose abstract information on the properties of theANEs used by "eh1 -> eh2" and "eh1 -> eh4". For example, an ALTO server caneither expose the available bandwidth between "eh1 - sw1", "sw1 - sw5", "sw5 -sw7", "sw5 - sw6", "sw6 - sw7", "sw7 - sw2", "sw7 - sw4", "sw2 - eh2", "sw4 -eh4" in Case 1, or expose 3 abstract elements "A", "B" and "C", whichrepresent the linear constraints that define the same capacity region in Case1.¶

In general, we can conclude that to support the multiple flow schedulinguse case, the ALTO framework must be extended to satisfy the followingadditional requirements:¶

AR1:

An ALTO server must provide the ANEs that are important to assess the QoE ofthe overlay application on the path of a <source, destination> pair.¶

AR2:

An ALTO server must provide information to identify how ANEs are shared on thepaths of different <source, destination> pairs.¶

AR3:

An ALTO server must provide information on the properties that are importantto assess the QoE of the application for ANEs.¶

The extension defined in this document specifies a solution to expose suchabstract information.¶

4.2.Sample Use Cases

While the multiple flow scheduling problem is used to help identify theadditional requirements, the extension defined in this document can be appliedto a wide range of applications. This section highlights some use cases that arereported.¶

                                  Data schema      Queries                                       |             |                                       \             /    +-------------+                   +-----------------+    | ALTO Client | <===============> |  Job Optimizer  |    +-------------+                   +-----------------+PV       |   ^ PV                                    |Request  |   | Response                              |         |   |                  On-demand resource   |(Data    |   | (Network         allocation, demand   |Transfer |   | Resource         vector, etc.         |Intents) |   | Constraints)     (Non-ALTO interfaces)|         v   |                                       v    +-------------+                   +-----------------+    | ALTO Server | <===============> | Network Manager |    +-------------+                   +-----------------+                                        /      |      \                                        |      |      |                                       WAN    DC1    DC2

               On-going transfers   New transfer requests                             \----\        |                                  |        |                                  v        v   +-------------+               +---------------+   | ALTO Client | <===========> | Data Transfer |   +-------------+               |   Scheduler   |     ^ |      ^ | PV request     +---------------+     | |      | \--------------\     | |      \--------------\ |     | v       PV response   | v   +-------------+          +-------------+   | ALTO Server |          | ALTO Server |   +-------------+          +-------------+         ||                       ||     +---------+              +---------+     | Network |              | Network |     | Manager |              | Manager |     +---------+              +---------+      .                           .     .             _~_  __         . . .    .             (   )(  )             .___  ~v~v~       /--(         )------------(   ) (     )-----/    (       )            (     )  ~w~w~            ~^~^~^~              ~v~v~ Site 1        Non-blocking Core        Site 2

Site 1:[c] . ........................................> [d]  +---+ 10 Gbps +---+ 10 Gbps +----+ 50 Gbps  | A |---------| B |---------| GW |--------- Core  +---+         +---+         +----+ ................... .                 . .                 v[a]               [b]Site 2:[d] <........................................ [c]  +---+ 5 Gbps +---+ 10 Gbps +----+ 20 Gbps  | X |--------| Y |---------| GW |--------- Core  +---+        +---+         +----+             ....................             .                  .             .                  v            [e]                [f]

a: { b: [ane1] },c: { d: [ane1, ane2, ane3] }ane1: bw = 10 Gbps (link: A->B)ane2: bw = 10 Gbps (link: B->GW)ane3: bw = 50 Gbps (link: GW->Core)

c: { d: [anei, aneii, aneiii] }e: { f: [aneiv] }anei: bw = 5 Gbps (link Y->X)aneii: bw = 10 Gbps (link GW->Y)aneiii: bw = 20 Gbps (link Core->GW)aneiv: bw = 10 Gbps (link Y->GW)

      +-------------+              +----------------------+      | ALTO Client | <==========> | Application Provider |      +-------------+              +----------------------+PV         |   ^ PV                      |Request    |   | Response                | Resource allocation,           |   |                         | service establishment,(End hosts |   | (Edge nodes             | etc.and cloud  |   | and metrics)            |servers)   |   |                         |           v   |                         v      +-------------+             +---------------------+      | ALTO Server | <=========> | Cloud-Edge Provider |      +-------------+             +---------------------+       ____________________________________/\___________      /                                                 \      |           (((o                                  |                     |                    /_\             _~_            __   __  a               (/\_/\)          (   )          (  )~(  )_   \      /------(      )---------(     )----\\---(          )   _|_   /        (______)         (___)          (          )   |_| -/         Site-radio     Access CO       (__________)  /---\          Edge Node 1         |             Cloud DCOn premise                           |                           /---------/           (((o           /              |          / Site-radio  /_\        /Edge Node 2(/\_/\)-----/          /(_____)\   ___   /         \   ---b--|_| -/           \--|_|--c  /---\               /---\On premise          On premise

a: { b: [ane1, ane2, ane3, ane4, ane5],     c: [ane1, ane2, ane3, ane4, ane6],     DC: [ane1, ane2, ane3] }b: { c: [ane5, ane4, ane6], DC: [ane5, ane4, ane3] }ane1: latency=5ms cpu=2 memory=8G storage=10T(on premise, a)ane2: latency=20ms cpu=4 memory=8G storage=10T(Site-radio Edge Node 1)ane3: latency=100ms cpu=8 memory=128G storage=100T(Access CO)ane4: latency=20ms cpu=4 memory=8G storage=10T(Site-radio Edge Node 2)ane5: latency=5ms cpu=2 memory=8G storage=10T(on premise, b)ane6: latency=5ms cpu=2 memory=8G storage=10T(on premise, c)

5.1.Abstract Network Element (ANE)

This extension introduces ANE as an indirect and network-agnostic way to specifya component or an aggregation of components of a network whose properties havean impact on the end-to-end performance for application traffic betweenendpoints.¶

ANEs allow ALTO servers to focus on common properties of different types ofnetwork components. For example, the throughput of a flow can be constrained bydifferent components in a network: the capacity of a physical link, the maximumthroughput of a firewall, the reserved bandwidth of an MPLS tunnel, etc. See theexample below, assume the throughput of the firewall is 100 Mbps and thecapacity for link (A, B) is also 100 Mbps, they result in the same constraint onthe total throughput of f1 and f2. Thus, they are identical when treated as anANE.¶

   f1 |      ^                  f1      |      |                 ----------------->    +----------+                +---+     +---+    | Firewall |                | A |-----| B |    +----------+                +---+     +---+      |      |                 ----------------->      v      | f2               f2

When an ANE is defined by an ALTO server, it is assigned an identifier by theALTO server, i.e., a string of type ANEName as specified inSection 6.1,and a set of associated properties.¶

5.2.Path Vector Cost Type

For an ALTO client to correctly interpret the Path Vector, this extensionspecifies a new cost type called the Path Vector cost type.¶

The Path Vector cost type must convey both the interpretation and semantics inthe "cost-mode" and "cost-metric" respectively. Unfortunately, a single"cost-mode" value cannot fully specify the interpretation of a Path Vector,which is a compound data type. For example, in programming languages such as C++where there existed a JSON array type named JSONArray, a Path Vector will havethe type ofJSONArray<ANEName>.¶

Instead of extending the "type system" of ALTO, this document takes a simpleand backward compatible approach. Specifically, the "cost-mode" of the PathVector cost type is "array", which indicates the value is a JSON array. Then, anALTO client must check the value of the "cost-metric". If the value is"ane-path", it means that the JSON array should be further interpreted as a pathof ANENames.¶

The Path Vector cost type is specified inSection 6.5.¶

5.3.Multipart Path Vector Response

For a basic ALTO information resource, a response contains only one type ofALTO resources, e.g., Network Map, Cost Map, or Property Map. Thus, only oneround of communication is required: An ALTO client sends a request to an ALTOserver, and the ALTO server returns a response, as shown inFigure 8.¶

  ALTO client                              ALTO server       |-------------- Request ---------------->|       |<------------- Response ----------------|

The extension defined in this document, on the other hand, involves two types ofinformation resources: Path Vectors conveyed in an InfoResourceCostMap (definedin Section 11.2.3.6 of[RFC7285]) or an InfoResourceEndpointCostMap (definedin Section 11.5.1.6 of[RFC7285]), and ANE properties conveyed in anInfoResourceProperties (defined in Section 7.6 of[I-D.ietf-alto-unified-props-new]).¶

Instead of two consecutive message exchanges, the extension defined in thisdocument enforces one round of communication. Specifically, the ALTO client mustinclude the source and destination pairs and the requested ANE properties in asingle request, and the ALTO server must return a single response containingboth the Path Vectors and properties associated with the ANEs in the PathVectors, as shown inFigure 9. Since the two parts are bundled together in oneresponse message, their orders are interchangeable. SeeSection 7.2.6 andSection 7.3.6 for details.¶

  ALTO client                              ALTO server       |------------- PV Request -------------->|       |<----- PV Response (Cost Map Part) -----|       |<--- PV Response (Property Map Part) ---|

This design is based on the following considerations:¶

1. ANEs may be constructed on demand, and potentially based on therequested properties (SeeSection 5.1 for more details). If sources anddestinations are not in the same request as the properties, an ALTO servereither cannot construct ANEs on-demand, or must wait until both requests arereceived.¶
2. As ANEs may be constructed on demand, mappings of each ANE to its underlyingnetwork devices and resources can be specific to the request. In orderto respond to the Property Map request correctly, an ALTO server must storethe mapping of each Path Vector request until the client fully retrieves theproperty information. The "stateful" behavior may substantially harm theserver scalability and potentially lead to Denial-of-Service attacks.¶

One approach to realize the one-round communication is to define a new mediatype to contain both objects, but this violates modular design. This documentfollows the standard-conforming usage of "multipart/related" media type definedin[RFC2387] to elegantly combine the objects. Path Vectors are encoded in anInfoResourceCostMap or an InfoResourceEndpointCostMap, and the Property Map isencoded in an InfoResourceProperties. They are encapsulated as parts of amultipart message. The modular composition allows ALTO servers and clients toreuse the data models of the existing information resources. Specifically, thisdocument addresses the following practical issues using "multipart/related".¶

6.1.ANE Name

An ANE Name is encoded as a JSON string with the same format as that of the typePIDName (Section 10.1 of[RFC7285]).¶

The type ANEName is used in this document to indicate a string of thisformat.¶

6.2.ANE Entity Domain

The ANE entity domain associates property values with the Abstract NetworkElements in a Property Map. Accordingly, the ANE entity domain always depends ona Property Map.¶

It must be noted that the term "domain" here does not refer to a network domain.Rather, it is inherited from the "entity domain" defined in Sec 3.2 in[I-D.ietf-alto-unified-props-new] that represents the set of valid entitiesdefined by an ALTO information resource (called the defining informationresource).¶

application/alto-propmap+json

6.3.ANE Property Name

An ANE Property Name is encoded as a JSON string with the same format as that ofEntity Property Name (Section 5.2.2 of[I-D.ietf-alto-unified-props-new]).¶

6.4.Initial ANE Property Types

Two initial ANE property types are specified, "max-reservable-bandwidth" and"persistent-entity-id".¶

Note that these property types do not depend on any information resource. Assuch, the EntityPropertyName MUST only have the EntityPropertyType part.¶

                     20 Mbps  20 Mbps          10 Mbps +---+   +---+    +---+             /----| B |---| D |----| E |---- H2       +---+/     +---+   +---+    +---+H1 ----| A | 15 Mbps|       +---+\     +---+             \----| C |          15 Mbps +---+

6.5.Path Vector Cost Type

This document defines a new cost type, which is referred to as the Path Vectorcost type. An ALTO server MUST offer this cost type if it supports the extensiondefined in this document.¶

6.6.Part Resource ID and Part Content ID

A Part Resource ID is encoded as a JSON string with the same format as that of thetype ResourceID (Section 10.2 of[RFC7285]).¶

Even though the client-id assigned to a Path Vector request and the PartResource ID MAY contain up to 64 characters by their own definition, theirconcatenation (seeSection 5.3.2) MUST also conform to the same lengthconstraint. The same requirement applies to the resource ID of the Path Vectorresource, too. Thus, it is RECOMMENDED to limit the length of resource ID andclient ID related to a Path Vector resource to 31 characters.¶

A Part Content ID conforms to the format of msg-id as specified in[RFC2387] and[RFC5322]. Specifically, it has the following format:¶

"<" PART-RESOURCE-ID "@" DOMAIN-NAME ">"¶

PART-RESOURCE-ID:

PART-RESOURCE-ID has the same format as the Part Resource ID. It is used toidentify whether a part message is a Path Vector or a Property Map.¶

DOMAIN-NAME:

DOMAIN-NAME has the same format as dot-atom-text specified in Section 3.2.3 of[RFC5322]. It must be the domain name of the ALTO server.¶

7.1.Notations

This document uses the same syntax and notations as introduced in Section 8.2 ofRFC 7285[RFC7285] to specify the extensions to existing ALTO resources andservices.¶

7.2.Multipart Filtered Cost Map for Path Vector

This document introduces a new ALTO resource called multipart Filtered Cost Mapresource, which allows an ALTO server to provide other ALTO resources associatedwith the Cost Map resource in the same response.¶

object {  [EntityPropertyName ane-property-names<0..*>;]} PVReqFilteredCostMap : ReqFilteredCostMap;

   POST /costmap/pv HTTP/1.1   Host: alto.example.com   Accept: multipart/related;type=application/alto-costmap+json,           application/alto-error+json   Content-Length: 201   Content-Type: application/alto-costmapfilter+json   {     "cost-type": {       "cost-mode": "array",       "cost-metric": "ane-path"     },     "pids": {       "srcs": [ "PID1" ],       "dsts": [ "PID2" ]     },     "ane-property-names": [ "max-reservable-bandwidth" ]   }

object {  [EntityPropertyName ane-property-names<0..*>;]} PVFilteredCostMapCapabilities : FilteredCostMapCapabilities;

HTTP/1.1 200 OKContent-Length: 859Content-Type: multipart/related; boundary=example-1;              type=application/alto-costmap+json--example-1Content-ID: <costmap@alto.example.com>Content-Type: application/alto-costmap+json{  "meta": {    "vtag": {      "resource-id": "filtered-cost-map-pv.costmap",      "tag": "fb20b76204814e9db37a51151faaaef2"    },    "dependent-vtags": [      {        "resource-id": "my-default-networkmap",        "tag": "75ed013b3cb58f896e839582504f6228"      }    ],    "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" }  },  "cost-map": {    "PID1": { "PID2": ["ANE1"] }  }}--example-1Content-ID: <propmap@alto.example.com>Content-Type: application/alto-propmap+json{  "meta": {    "dependent-vtags": [      {        "resource-id": "filtered-cost-map-pv.costmap",        "tag": "fb20b76204814e9db37a51151faaaef2"      }    ]  },  "property-map": {    ".ane:ANE1": { "max-reservable-bandwidth": 100000000 }  }}

7.3.Multipart Endpoint Cost Service for Path Vector

This document introduces a new ALTO resource called multipart Endpoint CostService, which allows an ALTO server to provide other ALTO resources associatedwith the Endpoint Cost Service resource in the same response.¶

object {  [EntityPropertyName ane-property-names<0..*>;]} PVReqEndpointcost : ReqEndpointcostMap;

POST /ecs/pv HTTP/1.1Host: alto.example.comAccept: multipart/related;type=application/alto-endpointcost+json,        application/alto-error+jsonContent-Length: 227Content-Type: application/alto-endpointcostparams+json{  "cost-type": {    "cost-mode": "array",    "cost-metric": "ane-path"  },  "endpoints": {    "srcs": [ "ipv4:192.0.2.2" ],    "dsts": [ "ipv4:192.0.2.18" ]  },  "ane-property-names": [ "max-reservable-bandwidth" ]}

HTTP/1.1 200 OKContent-Length: 845Content-Type: multipart/related; boundary=example-1;              type=application/alto-endpointcost+json--example-1Content-ID: <ecs@alto.example.com>Content-Type: application/alto-endpointcost+json{  "meta": {    "vtag": {      "resource-id": "ecs-pv.ecs",      "tag": "ec137bb78118468c853d5b622ac003f1"    },    "dependent-vtags": [      {        "resource-id": "my-default-networkmap",        "tag": "677fe5f4066848d282ece213a84f9429"      }    ],    "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" }  },  "cost-map": {    "ipv4:192.0.2.2": { "ipv4:192.0.2.18": ["ANE1"] }  }}--example-1Content-ID: <propmap@alto.example.com>Content-Type: application/alto-propmap+json{  "meta": {    "dependent-vtags": [      {        "resource-id": "ecs-pv.ecs",        "tag": "ec137bb78118468c853d5b622ac003f1"      }    ]  },  "property-map": {    ".ane:ANE1": { "max-reservable-bandwidth": 100000000 }  }}

8.1.Sample Setup

                                     ----- L1                                    /        PID1   +----------+ 10 Gbps +----------+    PID3 192.0.2.0/28+-+ +------+ +---------+          +--+192.0.2.32/28               | | MEC1 | |         |          |   2001:db8::3:0/16               | +------+ |   +-----+          |        PID2   |          |   |     +----------+192.0.2.16/28+-+          |   |         NET3               |          |   | 15 Gbps               |          |   |        \               +----------+   |         -------- L2                   NET1       |                            +----------+                            | +------+ |   PID4                            | | MEC2 | +--+192.0.2.48/28                            | +------+ |   2001:db8::4:0/16                            +----------+                                NET2

In this document,Figure 10 is used to illustrate the message contents. Thereare 3 sub-networks (NET1, NET2 and NET3) and two interconnection links (L1 andL2). It is assumed that each sub-network has sufficiently large bandwidth to bereserved.¶

8.2.Information Resource Directory

To give a comprehensive example of the extension defined in this document, weconsider the network inFigure 10. Assume that the ALTO server provides thefollowing information resources:¶

• "my-default-networkmap": A Network Map resource which contains the PIDs in thenetwork.¶
• "filtered-cost-map-pv": A Multipart Filtered Cost Map resource for Path Vector,which exposes the "max-reservable-bandwidth" property for the PIDs in"my-default-networkmap".¶
• "ane-props": A filtered Unified Property resource that exposes theinformation for persistent ANEs in the network.¶
• "endpoint-cost-pv": A Multipart Endpoint Cost Service for Path Vector, whichexposes the "max-reservable-bandwidth" and the "persistent-entity-id" properties.¶
• "update-pv": An Update Stream service, which provides the incremental updateservice for the "endpoint-cost-pv" service.¶
• "multicost-pv": A Multipart Endpoint Cost Service with both Multi-Cost andPath Vector.¶

Below is the Information Resource Directory of the example ALTO server. Toenable the extension defined in this document, the "path-vector" cost type(Section 6.5) is defined in the "cost-types" of the "meta" field, and isincluded in the "cost-type-names" of resources "filtered-cost-map-pv" and"endpoint-cost-pv".¶

{  "meta": {    "cost-types": {      "path-vector": {        "cost-mode": "array",        "cost-metric": "ane-path"      },      "num-rc": {        "cost-mode": "numerical",        "cost-metric": "routingcost"      }    }  },  "resources": {    "my-default-networkmap": {      "uri": "https://alto.example.com/networkmap",      "media-type": "application/alto-networkmap+json"    },    "filtered-cost-map-pv": {      "uri": "https://alto.example.com/costmap/pv",      "media-type": "multipart/related;                     type=application/alto-costmap+json",      "accepts": "application/alto-costmapfilter+json",      "capabilities": {        "cost-type-names": [ "path-vector" ],        "ane-property-names": [ "max-reservable-bandwidth" ]      },      "uses": [ "my-default-networkmap" ]    },    "ane-props": {      "uri": "https://alto.example.com/ane-props",      "media-type": "application/alto-propmap+json",      "accepts": "application/alto-propmapparams+json",      "capabilities": {        "mappings": {          ".ane": [ "cpu" ]        }      }    },    "endpoint-cost-pv": {      "uri": "https://alto.exmaple.com/endpointcost/pv",      "media-type": "multipart/related;                     type=application/alto-endpointcost+json",      "accepts": "application/alto-endpointcostparams+json",      "capabilities": {        "cost-type-names": [ "path-vector" ],        "ane-property-names": [          "max-reservable-bandwidth", "persistent-entity-id"        ]      },      "uses": [ "ane-props" ]    },    "update-pv": {      "uri": "https://alto.example.com/updates/pv",      "media-type": "text/event-stream",      "uses": [ "endpoint-cost-pv" ],      "accepts": "application/alto-updatestreamparams+json",      "capabilities": {        "support-stream-control": true      }    },    "multicost-pv": {      "uri": "https://alto.exmaple.com/endpointcost/mcpv",      "media-type": "multipart/related;                     type=application/alto-endpointcost+json",      "accepts": "application/alto-endpointcostparams+json",      "capabilities": {        "cost-type-names": [ "path-vector", "num-rc" ],        "max-cost-types": 2,        "testable-cost-type-names": [ "num-rc" ],        "ane-property-names": [          "max-reservable-bandwidth", "persistent-entity-id"        ]      },      "uses": [ "ane-props" ]    }  }}

8.3.Multipart Filtered Cost Map

The following examples demonstrate the request to the "filtered-cost-map-pv"resource and the corresponding response.¶

The request uses the "path-vector" cost type in the "cost-type" field. The"ane-property-names" field is missing, indicating that the client only requestsfor the Path Vector but not the ANE properties.¶

The response consists of two parts. The first part returns the array of ANENamefor each source and destination pair. There are two ANEs, where "L1" representsthe interconnection link L1, and "L2" represents the interconnection link L2.¶

The second part returns an empty Property Map. Note that the ANE entries areomitted since they have no properties (See Section 3.1 of[I-D.ietf-alto-unified-props-new]).¶

POST /costmap/pv HTTP/1.1Host: alto.example.comAccept: multipart/related;type=application/alto-costmap+json,        application/alto-error+jsonContent-Length: 153Content-Type: application/alto-costmapfilter+json{  "cost-type": {    "cost-mode": "array",    "cost-metric": "ane-path"  },  "pids": {    "srcs": [ "PID1" ],    "dsts": [ "PID3", "PID4" ]  }}

HTTP/1.1 200 OKContent-Length: 855Content-Type: multipart/related; boundary=example-1;              type=application/alto-costmap+json--example-1Content-ID: <costmap@alto.example.com>Content-Type: application/alto-costmap+json{  "meta": {    "vtag": {      "resource-id": "filtered-cost-map-pv.costmap",      "tag": "d827f484cb66ce6df6b5077cb8562b0a"    },    "dependent-vtags": [      {        "resource-id": "my-default-networkmap",        "tag": "c04bc5da49534274a6daeee8ea1dec62"      }    ],    "cost-type": {      "cost-mode": "array",      "cost-metric": "ane-path"    }  },  "cost-map": {    "PID1": {      "PID3": [ "L1" ],      "PID4": [ "L1", "L2" ]    }  }}--example-1Content-ID: <propmap@alto.example.com>Content-Type: application/alto-propmap+json{  "meta": {    "dependent-vtags": [      {        "resource-id": "filtered-cost-map-pv.costmap",        "tag": "d827f484cb66ce6df6b5077cb8562b0a"      }    ]  },  "property-map": {  }}

8.4.Multipart Endpoint Cost Service Resource

The following examples demonstrate the request to the "endpoint-cost-pv"resource and the corresponding response.¶

The request uses the Path Vector cost type in the "cost-type" field, andqueries the Maximum Reservable Bandwidth ANE property and the Persistent Entityproperty for two IPv4 source and destination pairs (192.0.2.34 -> 192.0.2.2 and192.0.2.34 -> 192.0.2.50) and one IPv6 source and destination pair(2001:db8::3:1 -> 2001:db8::4:1).¶

The response consists of two parts. The first part returns the array of ANENamefor each valid source and destination pair. As one can see inFigure 10, flow192.0.2.34 -> 192.0.2.2 traverses NET2, L1 and NET1, and flows 192.0.2.34 ->192.0.2.50 and 2001:db8::3:1 -> 2001:db8::4:1 traverse NET2, L2 and NET3.¶

The second part returns the requested properties of ANEs. Assume NET1, NET2 and NET3 hassufficient bandwidth and their "max-reservable-bandwidth" values are set to a sufficientlylarge number (50 Gbps in this case). On the other hand, assume there are noprior reservation on L1 and L2, and their "max-reservable-bandwidth" values arethe corresponding link capacity (10 Gbps for L1 and 15 Gbps for L2).¶

Both NET1 and NET2 have a mobile edge deployed, i.e., MEC1 in NET1 and MEC2 inNET2. Assume the ANEName for MEC1 and MEC2 are "MEC1" and "MEC2" and theirproperties can be retrieved from the Property Map "ane-props". Thus, the"persistent-entity-id" property of NET1 and NET3 are "ane-props.ane:MEC1" and"ane-props.ane:MEC2" respectively.¶

POST /endpointcost/pv HTTP/1.1Host: alto.example.comAccept: multipart/related;        type=application/alto-endpointcost+json,        application/alto-error+jsonContent-Length: 362Content-Type: application/alto-endpointcostparams+json{  "cost-type": {    "cost-mode": "array",    "cost-metric": "ane-path"  },  "endpoints": {    "srcs": [      "ipv4:192.0.2.34",      "ipv6:2001:db8::3:1"    ],    "dsts": [      "ipv4:192.0.2.2",      "ipv4:192.0.2.50",      "ipv6:2001:db8::4:1"    ]  },  "ane-property-names": [    "max-reservable-bandwidth",    "persistent-entity-id"  ]}

HTTP/1.1 200 OKContent-Length: 1432Content-Type: multipart/related; boundary=example-2;              type=application/alto-endpointcost+json--example-2Content-ID: <ecs@alto.example.com>Content-Type: application/alto-endpointcost+json{  "meta": {    "vtags": {      "resource-id": "endpoint-cost-pv.ecs",      "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef"    },    "cost-type": {      "cost-mode": "array",      "cost-metric": "ane-path"    }  },  "endpoint-cost-map": {    "ipv4:192.0.2.34": {      "ipv4:192.0.2.2":   [ "NET3", "L1", "NET1" ],      "ipv4:192.0.2.50":   [ "NET3", "L2", "NET2" ]    },    "ipv6:2001:db8::3:1": {      "ipv6:2001:db8::4:1": [ "NET3", "L2", "NET2" ]    }  }}--example-2Content-ID: <propmap@alto.example.com>Content-Type: application/alto-propmap+json{  "meta": {    "dependent-vtags": [      {        "resource-id": "endpoint-cost-pv.ecs",        "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef"      },      {        "resource-id": "ane-props",        "tag": "bf3c8c1819d2421c9a95a9d02af557a3"      }    ]  },  "property-map": {    ".ane:NET1": {      "max-reservable-bandwidth": 50000000000,      "persistent-entity-id": "ane-props.ane:MEC1"    },    ".ane:NET2": {      "max-reservable-bandwidth": 50000000000,      "persistent-entity-id": "ane-props.ane:MEC2"    },    ".ane:NET3": {      "max-reservable-bandwidth": 50000000000    },    ".ane:L1": {      "max-reservable-bandwidth": 10000000000    },    ".ane:L2": {      "max-reservable-bandwidth": 15000000000    }  }}

Under certain scenarios where the traversal order is not crucial, an ALTO serverimplementation may choose to not follow strictly the physical traversal orderand may even obfuscate the order intentionally to preserve its own privacy orconform to its own policies.For example, an ALTO server may choose to aggregate NET1 and L1 as a new ANEwith ANE name "AGGR1", and aggregate NET2 and L2 as a new ANE with ANE name"AGGR2". The "max-reservable-bandwidth" of "AGGR1" takes the value of L1, whichis smaller than that of NET1, and the "persistent-entity-id" of "AGGR1" takesthe value of NET1. The properties of "AGGR2" are computed in a similar way andthe obfuscated response is as shown below. Note that the obfuscation of PathVector responses is implementation-specific and is out of the scope of thisdocument, and developers may refer toSection 11 for further references.¶

HTTP/1.1 200 OKContent-Length: 1263Content-Type: multipart/related; boundary=example-2;              type=application/alto-endpointcost+json--example-2Content-ID: <ecs@alto.example.com>Content-Type: application/alto-endpointcost+json{  "meta": {    "vtags": {      "resource-id": "endpoint-cost-pv.ecs",      "tag": "bb975862fbe3422abf4dae386b132c1d"    },    "cost-type": {      "cost-mode": "array",      "cost-metric": "ane-path"    }  },  "endpoint-cost-map": {    "ipv4:192.0.2.34": {      "ipv4:192.0.2.2":   [ "NET3", "AGGR1" ],      "ipv4:192.0.2.50":   [ "NET3", "AGGR2" ]    },    "ipv6:2001:db8::3:1": {      "ipv6:2001:db8::4:1": [ "NET3", "AGGR2" ]    }  }}--example-2Content-ID: <propmap@alto.example.com>Content-Type: application/alto-propmap+json{  "meta": {    "dependent-vtags": [      {        "resource-id": "endpoint-cost-pv.ecs",        "tag": "bb975862fbe3422abf4dae386b132c1d"      },      {        "resource-id": "ane-props",        "tag": "bf3c8c1819d2421c9a95a9d02af557a3"      }    ]  },  "property-map": {    ".ane:AGGR1": {      "max-reservable-bandwidth": 10000000000,      "persistent-entity-id": "ane-props.ane:MEC1"    },    ".ane:AGGR2": {      "max-reservable-bandwidth": 15000000000,      "persistent-entity-id": "ane-props.ane:MEC2"    },    ".ane:NET3": {      "max-reservable-bandwidth": 50000000000    }  }}

8.5.Incremental Updates

In this example, an ALTO client subscribes to the incremental update for themultipart Endpoint Cost Service resource "endpoint-cost-pv".¶

POST /updates/pv HTTP/1.1Host: alto.example.comAccept: text/event-streamContent-Type: application/alto-updatestreamparams+jsonContent-Length: 112{  "add": {    "ecspvsub1": {      "resource-id": "endpoint-cost-pv",      "input": <ecs-input>    }  }}

Based on the server-side process defined in[RFC8895], the ALTO server willsend the "control-uri" first using Server-Sent Event (SSE), followed by the fullresponse of the multipart message.¶

HTTP/1.1 200 OKConnection: keep-aliveContent-Type: text/event-streamevent: application/alto-updatestreamcontrol+jsondata: {"control-uri": "https://alto.example.com/updates/streams/123"}event: multipart/related;boundary=example-3;       type=application/alto-endpointcost+json,ecspvsub1data: --example-3data: Content-ID: <ecsmap@alto.example.com>data: Content-Type: application/alto-endpointcost+jsondata:data: <endpoint-cost-map-entry>data: --example-3data: Content-ID: <propmap@alto.example.com>data: Content-Type: application/alto-propmap+jsondata:data: <property-map-entry>data: --example-3--

When the contents change, the ALTO server will publish the updates for each nodein this tree separately, based on Section 6.7.3 of[RFC8895].¶

event: application/merge-patch+json, ecspvsub1.ecsmap@alto.example.comdata: <Merge patch for endpoint-cost-map-update>event: application/merge-patch+json, ecspvsub1.propmap@alto.example.comdata: <Merge patch for property-map-update>

8.6.Multi-cost

The following examples demonstrate the request to the "multicost-pv" resourceand the corresponding response.¶

The request asks for two cost types: the first is the Path Vector cost type, andthe second is a numerical routing cost. It also queries the Maximum ReservableBandwidth ANE property and the Persistent Entity property for two IPv4 source anddestination pairs (192.0.2.34 -> 192.0.2.2 and 192.0.2.34 -> 192.0.2.50) and oneIPv6 source and destination pair (2001:db8::3:1 -> 2001:db8::4:1).¶

The response consists of two parts. The first part returns a JSONArray thatcontains two JSONValue for each requested source and destination pair: the firstJSONValue is a JSONArray of ANENames, which is the value of the Path Vector costtype, and the second JSONValue is a JSONNumber which is the value of the routingcost. The second part contains a Property Map that maps the ANEs to theirrequested properties.¶

POST /endpointcost/mcpv HTTP/1.1Host: alto.example.comAccept: multipart/related;        type=application/alto-endpointcost+json,        application/alto-error+jsonContent-Length: 433Content-Type: application/alto-endpointcostparams+json{  "multi-cost-types": [    { "cost-mode": "array", "cost-metric": "ane-path" },    { "cost-mode": "numerical", "cost-metric": "routingcost" }  ],  "endpoints": {    "srcs": [      "ipv4:192.0.2.34",      "ipv6:2001:db8::3:1"    ],    "dsts": [      "ipv4:192.0.2.2",      "ipv4:192.0.2.50",      "ipv6:2001:db8::4:1"    ]  },  "ane-property-names": [    "max-reservable-bandwidth",    "persistent-entity-id"  ]}

HTTP/1.1 200 OKContent-Length: 1350Content-Type: multipart/related; boundary=example-4;              type=application/alto-endpointcost+json--example-4Content-ID: <ecs@alto.example.com>Content-Type: application/alto-endpointcost+json{  "meta": {    "vtags": {      "resource-id": "endpoint-cost-pv.ecs",      "tag": "84a4f9c14f9341f0983e3e5f43a371c8"    },    "multi-cost-types": [      { "cost-mode": "array", "cost-metric": "ane-path" },      { "cost-mode": "numerical", "cost-metric": "routingcost" }    ]  },  "endpoint-cost-map": {    "ipv4:192.0.2.34": {      "ipv4:192.0.2.2":   [[ "NET3", "AGGR1" ], 3],      "ipv4:192.0.2.50":   [[ "NET3", "AGGR2" ], 2]    },    "ipv6:2001:db8::3:1": {      "ipv6:2001:db8::4:1": [[ "NET3", "AGGR2" ], 2]    }  }}--example-4Content-ID: <propmap@alto.example.com>Content-Type: application/alto-propmap+json{  "meta": {    "dependent-vtags": [      {        "resource-id": "endpoint-cost-pv.ecs",        "tag": "84a4f9c14f9341f0983e3e5f43a371c8"      },      {        "resource-id": "ane-props",        "tag": "be157afa031443a187b60bb80a86b233"      }    ]  },  "property-map": {    ".ane:AGGR1": {      "max-reservable-bandwidth": 10000000000,      "persistent-entity-id": "ane-props.ane:MEC1"    },    ".ane:AGGR2": {      "max-reservable-bandwidth": 15000000000,      "persistent-entity-id": "ane-props.ane:MEC2"    },    ".ane:NET3": {      "max-reservable-bandwidth": 50000000000    }  }}

9.1.Compatibility with Legacy ALTO Clients/Servers

The multipart Filtered Cost Map resource and the multipart Endpoint CostService resource has no backward compatibility issue with legacy ALTO clients andservers. Although these two types of resources reuse the media types defined inthe base ALTO protocol for the accept input parameters, they have differentmedia types for responses. If the ALTO server provides these two types ofresources, but the ALTO client does not support them, the ALTO client willignore the resources without incurring any incompatibility problem.¶

9.2.Compatibility with Multi-Cost Extension

The extension defined in this document is compatible with the multi-costextension[RFC8189]. Such a resource has a media type of either"multipart/related; type=application/alto-costmap+json" or "multipart/related;type=application/alto-endpointcost+json". Its "cost-constraints" field musteither be "false" or not present and the Path Vector cost type must be presentin the "cost-type-names" capability field but must not be present in the"testable-cost-type-names" field, as specified inSection 7.2.4 andSection 7.3.4.¶

9.3.Compatibility with Incremental Update

This extension is compatible with the incremental update extension[RFC8895].ALTO clients and servers MUST follow the specifications given in Sections 5.2and 6.7.3 of[RFC8895] to support incremental updates for a Path Vectorresource.¶

9.4.Compatibility with Cost Calendar

The extension specified in this document is compatible with the Cost Calendarextension[RFC8896]. When used together with the Cost Calendar extension, thecost value between a source and a destination is an array of Path Vectors, wherethe k-th Path Vector refers to the abstract network paths traversed in the k-thtime interval by traffic from the source to the destination.¶

When used with time-varying properties, e.g., maximum reservable bandwidth, aproperty of a single ANE may also have different values in different timeintervals. In this case, if such an ANE has different property values in twotime intervals, it MUST be treated as two different ANEs, i.e., with differententity identifiers. However, if it has the same property values in two timeintervals, it MAY use the same identifier.¶

This rule allows the Path Vector extension to represent both changes of ANEs andchanges of the ANEs' properties in a uniform way. The Path Vector part iscalendared in a compatible way, and the Property Map part is not affected by thecalendar extension.¶

The two extensions combined together can provide the historical networkcorrelation information for a set of source and destination pairs. A networkbroker or client may use this information to derive other resource requirementssuch as Time-Block-Maximum Bandwidth, Bandwidth-Sliding-Window, andTime-Bandwidth-Product (TBP) (See[SENSE] for details).¶

10.1.Constraint Tests for General Cost Types

The constraint test is a simple approach to query the data. It allows users tofilter the query result by specifying some boolean tests. This approach isalready used in the ALTO protocol.[RFC7285] and[RFC8189] allow ALTOclients to specify the "constraints" and "or-constraints" tests to betterfilter the result.¶

However, the current syntax can only be used to test scalar cost types, andcannot easily express constraints on complex cost types, e.g., the Path Vectorcost type defined in this document.¶

In practice, developing a bespoke language for general-purpose boolean tests canbe a complex undertaking, and it is conceivable that there are some existingimplementations already (the authors have not done an exhaustive search todetermine whether there are such implementations). One avenue to develop such alanguage may be to explore extending current query languages like XQuery[XQuery] or JSONiq[JSONiq] and integrating these with ALTO.¶

Filtering the Path Vector results or developing a more sophisticated filteringmechanism is beyond the scope of this document.¶

10.2.General Multi-Resource Query

Querying multiple ALTO information resources continuously is a generalrequirement. Enabling such a capability, however, must address generalissues like efficiency and consistency. The incremental update extension[RFC8895] supports submitting multiple queries in a single request, and allowsflexible control over the queries. However, it does not cover the caseintroduced in this document where multiple resources are needed for a singlerequest.¶

This extension gives an example of using a multipart message to encode theresponses from two specific ALTO information resources: a Filtered Cost Map oran Endpoint Cost Service, and a Property Map. By packing multiple resources in asingle response, the implication is that servers may proactively push relatedinformation resources to clients.¶

Thus, it is worth looking into the direction of extending the SSE mechanism asused in the incremental update extension[RFC8895], or upgrading to HTTP/2[I-D.ietf-httpbis-http2bis] and HTTP/3[I-D.ietf-quic-http], whichprovides the ability to multiplex queries and to allow servers proactively sendrelated information resources.¶

Defining a general multi-resource query mechanism is out of the scope of thisdocument.¶

12.1.ALTO Cost Metric Registry

This document registers a new entry to the ALTO Cost Metric Registry, asinstructed by Section 14.2 of[RFC7285]. The new entryis as shown below inTable 1.¶

12.2.ALTO Cost Mode Registry

This document registers a new entry to the ALTO Cost Mode Registry, asinstructed by Section 4 of[I-D.bw-alto-cost-mode]. The new entryis as shown below inTable 2.¶

12.3.ALTO Entity Domain Type Registry

This document registers a new entry to the ALTO Domain Entity Type Registry, asinstructed by Section 12.2 of[I-D.ietf-alto-unified-props-new]. The new entryis as shown below inTable 3.¶

Identifier:

SeeSection 6.2.1.¶

Entity Identifier Encoding:

SeeSection 6.2.2.¶

Hierarchy:

None¶

Inheritance:

None¶

Media Type of Defining Resource:

SeeSection 6.2.4.¶

Mapping to ALTO Address Type:

This entity type does not map to ALTO address type.¶

Security Considerations:

In some usage scenarios, ANE addresses carried in ALTO Protocol messages mayreveal information about an ALTO client or an ALTO service provider.Applications and ALTO service providers using addresses of ANEs will be madeaware of how (or if) the addressing scheme relates to private information andnetwork proximity, in further iterations of this document.¶

12.4.ALTO Entity Property Type Registry

Two initial entries "max-reservable-bandwidth" and "persistent-entity-id" areregistered to the ALTO Domain "ane" in the "ALTO Entity Property Type Registry",as instructed by Section 12.3 of[I-D.ietf-alto-unified-props-new]. The twonew entries are shown below inTable 4 and their details can befound inSection 12.4.1 andSection 12.4.2.¶

B.1.Changes since -20

Reivision -21¶

• changes the normative requirement on protecting confidentiality of PVinformation with softer language¶

B.2.Changes since -19

Revision -20¶

• changes the IANA registry information¶
• adopts the comments from IESG reviews¶

B.3.Changes since -18

Revision -19¶

• adds detailed examples for use cases¶
• clarify terms with ambiguous meanings¶

B.4.Changes since -17

Revision -18¶

• changes the specification for content-id to conform to[RFC2387] and[RFC5322]¶
• adds IPv6 examples¶

B.5.Changes since -16

Revision -17¶

• adds items for media type of defining resources in IANA considerations¶

B.6.Changes since -15

Revision -16¶

• resolves the compatibility with the Multi-Cost extension (RFC 8189)¶
• adds media types of defining resources for ANE property types (for IANAregistration)¶

B.7.Changes since -14

Revision -15¶

• fixes the IDNits warnings,¶
• fixes grammar issues,¶
• addresses the comments in the AD review.¶

B.8.Changes since -13

Revision -14¶

• addresses the comments in the chair review,¶
• fixes most issues raised by IDNits.¶

B.9.Changes since -12

Revision -13¶

• changes the abstract based on the chairs' reviews¶
• integrates Richard's responds to WGLC reviews¶

B.10.Changes since -11

Revision -12¶

• clarifies the definition of ANEs in a similar way as how Network Elements isdefined in[RFC2216]¶
• restructures several paragraphs that are not clear (Sec 3, Path Vector bullet, Sec 4.2, Sec 5.1.3, Sec 6.2.4, Sec 6.4.2, Sec 9.3)¶
• uses "ALTO Entity Domain Type Registry"¶

B.11.Changes since -10

Revision -11¶

• replaces "part" with "components" in the abstract;¶
• identifies additional requirements (AR) derived from the flow schedulingexample, and introduces how the extension addresses the additionalrequirements¶
• fixes the inconsistent use of "start" parameter in multipart responses;¶
• specifies explicitly how to handle "cost-constraints";¶
• uses the latest IANA registration mechanism defined in[I-D.ietf-alto-unified-props-new];¶
• renames "persistent-entities" to "persistent-entity-id";¶
• makes "application/alto-propmap+json" as the media type of defining resourcesfor the "ane" domain;¶
• updates the examples;¶
• adds the discussion on ephemeral and persistent ANEs.¶

B.12.Changes since -09

Revision -10¶

• revises the introduction which¶

  • extends the scope where the PV extension can be applied beyond the "pathcorrelation" information¶
• brings back the capacity region use case to better illustrate the problem¶
• revises the overview to explain and defend the concepts and decision choices¶
• fixes inconsistent terms, typos¶

B.13.Changes since -08

This revision¶

• fixes a few spelling errors¶
• emphasizes that abstract network elements can be generated on demand in bothintroduction and motivating use cases¶

B.14.Changes Since Version -06

• We emphasize the importance of the path vector extension in two aspects:¶

  1. It expands the problem space that can be solved by ALTO, from preferencesof network paths to correlations of network paths.¶
  2. It is motivated by new usage scenarios from both application's andnetwork's perspectives.¶
• More use cases are included, in addition to the original capacity region usecase.¶
• We add more discussions to fully explore the design space of the path vectorextension and justify our design decisions, including the concept of abstractnetwork element, cost type (reverted to -05), newer capabilities and themultipart message.¶
• Fix the incremental update process to be compatible with SSE -16 draft, whichuses client-id instead of resource-id to demultiplex updates.¶
• Register an additional ANE property (i.e., persistent-entities) to cover alluse cases mentioned in the draft.¶