Repository files navigation

This is the explainer for the Observable API proposal for more ergonomic andcomposable event handling.

This proposal adds a.when() method toEventTarget that becomes a betteraddEventListener(); specifically it returns anewObservable that adds a new event listener to the targetwhen itssubscribe() method is called. The Observable calls the subscriber'snext() handler with each event.

Observables turn event handling, filtering, and termination, into an explicit, declarative flowthat's easier to understand andcomposethan today's imperative version, which often requires nested calls toaddEventListener() andhard-to-follow callback chains.

// Filtering and mapping:element.when('click').filter((e)=>e.target.matches('.foo')).map((e)=>({x:e.clientX,y:e.clientY})).subscribe({next:handleClickAtPoint});

// Automatic, declarative unsubscription via the takeUntil method:element.when('mousemove').takeUntil(document.when('mouseup')).subscribe({next:e=>…});// Since reduce and some other terminators return promises, they also play// well with async functions:awaitelement.when('mousemove').takeUntil(element.when('mouseup')).reduce((soFar,e)=>…);

// Imperativeconstcontroller=newAbortController();element.addEventListener('mousemove',e=>{console.log(e);element.addEventListener('mouseup',e=>{controller.abort();});},{signal:controller.signal});

Tracking all link clicks within a container(example):

container.when('click').filter((e)=>e.target.closest('a')).subscribe({next:(e)=>{// …},});

Find the maximum Y coordinate while the mouse is held down(example):

constmaxY=awaitelement.when('mousemove').takeUntil(element.when('mouseup')).map((e)=>e.clientY).reduce((soFar,y)=>Math.max(soFar,y),0);

Multiplexing aWebSocket, such that a subscription message is send on connection,and an unsubscription message is send to the server when the user unsubscribes.

constsocket=newWebSocket('wss://example.com');functionmultiplex({ startMsg, stopMsg, match}){if(socket.readyState!==WebSocket.OPEN){returnsocket.when('open').flatMap(()=>multiplex({ startMsg, stopMsg, match}));}else{socket.send(JSON.stringify(startMsg));returnsocket.when('message').filter(match).takeUntil(socket.when('close')).takeUntil(socket.when('error')).map((e)=>JSON.parse(e.data)).finally(()=>{socket.send(JSON.stringify(stopMsg));});}}functionstreamStock(ticker){returnmultiplex({startMsg:{ ticker,type:'sub'},stopMsg:{ ticker,type:'unsub'},match:(data)=>data.ticker===ticker,});}constgoogTrades=streamStock('GOOG');constnflxTrades=streamStock('NFLX');constgoogController=newAbortController();googTrades.subscribe({next:updateView},{signal:googController.signal});nflxTrades.subscribe({next:updateView, ...});// And the stream can disconnect later, which// automatically sends the unsubscription message// to the server.googController.abort();

// Imperativefunctionmultiplex({ startMsg, stopMsg, match}){conststart=(callback)=>{constteardowns=[];if(socket.readyState!==WebSocket.OPEN){constopenHandler=()=>start({ startMsg, stopMsg, match})(callback);socket.addEventListener('open',openHandler);teardowns.push(()=>{socket.removeEventListener('open',openHandler);});}else{socket.send(JSON.stringify(startMsg));constmessageHandler=(e)=>{constdata=JSON.parse(e.data);if(match(data)){callback(data);}};socket.addEventListener('message',messageHandler);teardowns.push(()=>{socket.send(JSON.stringify(stopMsg));socket.removeEventListener('message',messageHandler);});}constfinalize=()=>{teardowns.forEach((t)=>t());};socket.addEventListener('close',finalize);teardowns.push(()=>socket.removeEventListener('close',finalize));socket.addEventListener('error',finalize);teardowns.push(()=>socket.removeEventListener('error',finalize));returnfinalize;};returnstart;}functionstreamStock(ticker){returnmultiplex({startMsg:{ ticker,type:'sub'},stopMsg:{ ticker,type:'unsub'},match:(data)=>data.ticker===ticker,});}constgoogTrades=streamStock('GOOG');constnflxTrades=streamStock('NFLX');constunsubGoogTrades=googTrades(updateView);constunsubNflxTrades=nflxTrades(updateView);// And the stream can disconnect later, which// automatically sends the unsubscription message// to the server.unsubGoogTrades();

Here we're leveraging observables to match a secret code, which is a pattern ofkeys the user might hit while using an app:

constpattern=['ArrowUp','ArrowUp','ArrowDown','ArrowDown','ArrowLeft','ArrowRight','ArrowLeft','ArrowRight','b','a','Enter',];constkeys=document.when('keydown').map(e=>e.key);keys.flatMap(firstKey=>{if(firstKey===pattern[0]){returnkeys.take(pattern.length-1).every((k,i)=>k===pattern[i+1]);}}).filter(matched=>matched).subscribe(()=>console.log('Secret code matched!'));

constpattern=[...];// Imperativedocument.addEventListener('keydown',e=>{constkey=e.key;if(key===pattern[0]){leti=1;consthandler=(e)=>{constnextKey=e.key;if(nextKey!==pattern[i++]){document.removeEventListener('keydown',handler)}elseif(pattern.length===i){console.log('Secret code matched!');document.removeEventListener('keydown',handler)}};document.addEventListener('keydown',handler);}},{once:true});

Observables are first-class objects representing composable, repeated events.They're like Promises but for multiple events, and specifically withEventTarget integration, they are to events whatPromises are to callbacks. They can be:

• Created by script or by platform APIs, and passed to anyone interested inconsuming events viasubscribe()
• Fed tooperators likeObservable.map(), to be composed &transformed without a web of nested callbacks

Better yet, the transition from event handlers ➡️ Observables is simpler thanthat of callbacks ➡️ Promises, since Observables integrate nicely on top ofEventTarget, the de facto way of subscribing to events from the platformandcustom script.As a result, developers can use Observables without migrating tons of code onthe platform, since it's an easy drop-in wherever you're handling events today.

The proposed API shape can be found inhttps://wicg.github.io/observable/#core-infrastructure.

The creator of an Observable passes in a callback that gets invokedsynchronously wheneversubscribe() is called. Thesubscribe() method can becalledany number of times, and the callback it invokes sets up a new"subscription" by registering the caller ofsubscribe() as a Observer. Withthis in place, the Observable can signal any number of events to the Observervia thenext() callback, optionally followed by a single call to eithercomplete() orerror(), signaling that the stream of data is finished.

constobservable=newObservable((subscriber)=>{leti=0;setInterval(()=>{if(i>=10)subscriber.complete();elsesubscriber.next(i++);},2000);});observable.subscribe({// Print each value the Observable produces.next:console.log,});

While custom Observables can be useful on their own, the primary use case theyunlock is with event handling. Observables returned by the newEventTarget#when() method are created natively with an internal callback thatuses the sameunderlyingmechanism asaddEventListener(). Therefore callingsubscribe() essentially registers anew event listener whose events are exposed through the Observer handlerfunctions and are composable with the variouscombinators available to all Observables.

Observables can be created by their native constructor, as demonstrated above,or by theObservable.from() static method. This method constructs a nativeObservable from objects that are any of the following,in this order:

• Observable (in which case it just returns the given object)
• AsyncIterable (anything withSymbol.asyncIterator)
• Iterable (anything withSymbol.iterator)
• Promise (or any thenable)

Furthermore, any method on the platform that wishes to accept an Observable as aWeb IDL argument, or return one from a callback whose return type isObservable can do so with any of the above objects as well, that getautomatically converted to an Observable. We can accomplish this in one of twoways that we'll finalize in the Observable specification:

1. By making theObservable type a special Web IDL type that performs thisECMAScript Object ➡️ Web IDL conversion automatically, like Web IDL does forother types.
2. Require methods and callbacks that work with Observables to specify the typeany, and have the corresponding spec prose immediately invoke a conversionalgorithm that the Observable specification will supply. This is similar towhat the Streams Standarddoes with async iterablestoday.

The conversation in#60 leanstowards option (1).

Crucially, Observables are "lazy" in that they do not start emitting data untilthey are subscribed to, nor do they queue any databefore subscription. Theycan also start emitting data synchronously during subscription, unlike Promiseswhich always queue microtasks when invoking.then() handlers. Consider thisexample:

el.when('click').subscribe({next:()=>console.log('One')});el.when('click').find(()=>{…}).then(()=>console.log('Three'));el.click();console.log('Two');// Logs "One" "Two" "Three"

By usingAbortController, you can unsubscribe from an Observable even as itsynchronously emits dataduring subscription:

// An observable that synchronously emits unlimited data during subscription.letobservable=newObservable((subscriber)=>{leti=0;while(true){subscriber.next(i++);}});letcontroller=newAbortController();observable.subscribe({next:(data)=>{if(data>100)controller.abort();}},{signal:controller.signal},});

It is critical for an Observable subscriber to be able to register an arbitraryteardown callback to clean up any resources relevant to the subscription. Theteardown can be registered from within the subscription callback passed into theObservable constructor. When run (upon subscribing), the subscription callbackcan register a teardown function viasubscriber.addTeardown().

If the subscriber has already been aborted (i.e.,subscriber.signal.aborted istrue), then the given teardown callback is invoked immediately from withinaddTeardown(). Otherwise, it is invoked synchronously:

• Fromcomplete(), after the subscriber's complete handler (if any) isinvoked
• Fromerror(), after the subscriber's error handler (if any) is invoked
• The signal passed to the subscription is aborted by the user.

We propose the following operators in addition to theObservable interface:

• catch()
  • LikePromise#catch(), it takes a callback which gets fired after the sourceobservable errors. It will then map to a new observable, returned by the callback,unless the error is rethrown.
• takeUntil(Observable)
  • Returns an observable that mirrors the one that this method is called on,until the input observable emits its first value
• finally()
  • LikePromise.finally(), it takes a callback which gets fired after theobservable completes in any way (complete()/error()).
  • Returns anObservable that mirrors the source observable exactly. The callbackpassed tofinally is fired when a subscription to the resulting observable is terminatedforany reason. Either immediately after the source completes or errors, or when the consumerunsubscribes by aborting the subscription.

Versions of the above are often present in userland implementations ofobservables as they are useful for observable-specific reasons, but in additionto these we offer a set of common operators that follow existing platformprecedent and can greatly increase utility and adoption. These exist on otheriterables, and are derived from TC39'siterator helpersproposal which adds thefollowingmethods toIterator.prototype:

• map()
• filter()
• take()
• drop()
• flatMap()
• reduce()
• toArray()
• forEach()
• some()
• every()
• find()

And the following method statically on theIterator constructor:

• from()

We expect userland libraries to provide more niche operators that integrate withtheObservable API central to this proposal, potentially shipping natively ifthey get enough momentum to graduate to the platform. But for this initialproposal, we'd like to restrict the set of operators to those that follow theprecedent stated above, similar to how web platform APIs that are declaredSetlike andMaplike have native propertiesinspired by TC39'sMap andSetobjects. Therefore we'd consider most discussion of expanding this set asout-of-scope for theinitial proposal, suitable for discussion in an appendix.Any long tail of operators couldconceivably follow along if there is supportfor the native Observable API presented in this explainer.

Note that the operatorsevery(),find(),some(), andreduce() returnPromises whose scheduling differs from that of Observables, which sometimesmeans event handlers that calle.preventDefault() will run too late. See theConcerns section which goes into more detail.

To illustrate how Observables fit into the current landscape of other reactiveprimitives, see the below table which is an attempt at combiningtwoothertables that classify reactiveprimitives by their interaction with producers & consumers:

Observables were first proposed to the platform inTC39in May of 2015. The proposal failed to gain traction, in part due to some opposition thatthe API was suitable to be a language-level primitive. In an attempt to renew the proposalat a higher level of abstraction, a WHATWGDOM issue wasfiled in December of 2017. Despite ampledeveloper demand,lots of discussion, and no strong objectors, the DOM Observables proposal sat mostly still for severalyears (with some flux in the API design) due to a lack of implementer prioritization.

Later in 2019,an attempt at reviving theproposal was made back at the original TC39 repository, which involved some API simplifications andadded support for the synchronous "firehose" problem.

This repository is an attempt to again breathe life into the Observable proposal with the hopeof shipping a version of it to the Web Platform.

Inprior discussion,Ben Lesh has listed several custom userland implementations ofobservable primitives, of which RxJS is the most popular with "47,000,000+ downloadsper week."

• RxJS: Started as a reference implementation of the TC39 proposal, is nearly identical to this proposal's observable.
• Relay: A mostly identical contract with the addition ofstart andunsubscribe events for observation and acquiring theSubscription prior to the return.
• tRPC: A nearly identical implemention of observable to this proposal.
• XState: uses an observable interface in several places in their library, in particular for theirActor type, to allowsubscriptions to changes in state, as shown in theiruseActor hook. Using an identical observable is also adocumented part of access state machine changes when using XState with SolidJS.
• SolidJS: An identical interface to this proposal is exposed for users to use.
• Apollo GraphQL: Actually re-exporting fromzen-observable astheir own thing, giving some freedom to reimplement on their own or pivot to something like RxJS observable at some point.
• zen-observable: A reference implementation of the TC39 observable proposal. Nearly identical to this proposal.
• React Router: Uses a{ subscribe(callback: (value: T) => void): () => void } pattern in theirRouter andDeferredData code. This was pointed out by maintainers as being inspired by Observable.
• Preact Uses a{ subscribe(callback: (value: T) => void): () => void } interface for their signals.
• TanStack: Uses a subscribable interface that matches{ subscribe(callback: (value: T) => void): () => void } inseveral places
• Redux: Implements an observable that is nearly identical to this proposal's observable as a means of subscribing to changes to a store.
• Svelte: Supportssubscribing to observables that fit this exact contract, and also exports and uses asubscribable contract for stores like{ subscribe(callback: (value: T) => void): () => void }.
• Dexie.js: Has anobservable implementation that is used for creatinglive queries to IndexedDB.
• MobX: Usessimilar interface to Observable internally for observation:{ observe_(callback: (value: T)): () => void }.

• Svelte: Directly supports implicit subscription and unsubscription to observables simply by binding to them in templates.
• Angular: Directly supports implicit subscription and unsubscription to observables using their| async "async pipe" functionality in templates.
• Vue: maintains adedicated library specifically for using Vue with RxJS observables.
• Cycle.js: A UI framework built entirely around observables

Given the extensive prior art in this area, there exists a public"Observable Contract".

Additionally many JavaScript APIs been trying to adhere to the contract defined by theTC39 proposal from 2015.To that end, there is a library,symbol-observable,that ponyfills (polyfills)Symbol.observable to help with interoperability between observable types that adheres to exactlythe interface defined here.symbol-observable has 479 dependent packages on npm, and is downloaded more than 13,000,000 timesper week. This means that there are a minimum of 479 packages on npm that are using the observable contract in some way.

This is similar to howPromises/A+ specification that was developed beforePromises wereadopted into ES2015 as a first-class language primitive.

One of the mainconcernsexpressed in the original WHATWG DOM thread has to do with Promise-ifying APIs on Observable,such as the proposedfirst(). The potential footgun here with microtask scheduling and eventintegration. Specifically, the following innocent-looking code would notalways work:

element.when('click').first().then((e)=>{e.preventDefault();// Do something custom...});

IfObservable#first() returns a Promise that resolves when the first event is fired on anEventTarget, then the user-supplied Promise.then() handler will run:

• ✅ Synchronously after event firing, for events triggered by the user
• ❌ Asynchronously after event firing, for all events triggered by script (i.e.,element.click())
  • This meanse.preventDefault() will have happened too late and effectively been ignored

Two things mitigate this concern. First, there is a very simple workaround toalways avoid thecase where youre.preventDefault() might run too late:

element.when('click').map((e)=>(e.preventDefault(),e)).first();

...or if Observable had a.do() method (seewhatwg/dom#544 (comment)):

element.when('click').do((e)=>e.preventDefault()).first();

...or bymodifying the semantics offirst() to take a callback that produces a value that the returned Promise resolves to:

el.when('submit').first((e)=>e.preventDefault()).then(doMoreStuff);

Second, this "quirk" already exists in today's thriving Observable ecosystem, and there are no seriousconcerns or reports from that community that developers are consistently running into this. This givessome confidence that baking this behavior into the web platform will not be dangerous.

There's been much discussion about which standards venue should ultimately host an Observablesproposal. The venue is not inconsequential, as it effectively decides whether Observables becomes alanguage-level primitive likePromises, that ship in all JavaScript browser engines, or a web platformprimitive with likely (but technicallyoptional) consideration in other environments like Node.js(seeAbortController for example).

Observables purposefully integrate frictionlessly with the main event-emitting interface(EventTarget) and cancellation primitive (AbortController) that live in the Web platform. Asproposed here, observables join this existing strongly-connected component from theDOMStandard: Observables depend on AbortController/AbortSignal, whichdepend on EventTarget, and EventTarget depends on both Observables and AbortController/AbortSignal.Because we feel that Observables fits in best where its supporting primitives live, the WHATWGstandards venue is probably the best place to advance this proposal. Additionally, non-WebECMAScript embedders like Node.js and Deno would still be able to adopt Observables, and are evenlikely to, given their commitment to Web platformaborting andevents.

This does not preclude future standardization of event-emitting and cancellation primitives in TC39in the future, something Observables could theoretically be layered on top of later. But for now, weare motivated to make progress in WHATWG.

In attempt to avoid relitigating this discussion, we'd urge the reader to see the followingdiscussion comments:

• whatwg/dom#544 (comment)
• whatwg/dom#544 (comment)
• whatwg/dom#544 (comment)
• whatwg/dom#544 (comment)
• whatwg/dom#544 (comment)

This section bares a collection of web standards and standards positions issuesused to track the Observable proposal's life outside of this repository.

• Mozilla standardsposition
• WebKit standardsposition
• Chrome Status
• WinterCG
• Node.js
• W3C TAG review

Observables are designed to make event handling more ergonomic and composable.As such, their impact on end users is indirect, largely coming in the form ofusers having to download less JavaScript to implement patterns that developerscurrently use third-party libraries for. As statedabove in theexplainer, thereis a thriving userland Observables ecosystem which results in loads of excessivebytes being downloaded every day.

In an attempt to codify the strong userland precedent of the Observable API,this proposal would save dozens of custom implementations from being downloadedevery day.

Additionally, as an API likeEventTarget,AbortController, and one relatedtoPromises, it enables developers to build less-complicated event handlingflows by constructing them declaratively, which may enable them to build moresound user experiences on the Web.

• Dominic Farolino
• Ben Lesh