Repository files navigation

ECMAScript Stage-3 (conditional on editor review) Proposal. J. S. Choi, 2021.

• Specification available
• Experimental polyfills (donot use in production code yet):
  • array-from-async
  • core-js

Since its standardization in JavaScript,Array.from has become one ofArray’s most frequently used built-in methods. However, no similarfunctionality exists for async iterators.

constarr=[];for(constvofiterable){arr.push(v);}// This does the same thing.constarr=Array.from(iterable);

Such functionality would also be useful fordumping the entirety of anasync iterator into asingle data structure, especially inunittests or incommand-line interfaces. (Several real-world examples areincluded in a following section.)

constarr=[];forawait(constvofasyncIterable){arr.push(v);}// We should add something that does the same thing.constarr=await??????????(asyncIterable);

There is anit-all NPM library that performs only this taskand which gets about 50,000 weekly downloads.This of course doesnot include any codethat uses ad-hocfor await–of loops with empty arrays.Further demonstrating the demand for such functionality,severalStack Overflow questions have been askedby various developers, asking how to convert async iterators to arrays.

There are severalreal-world examples listedlater in this explainer.

(Aformal draft specification is available.)

Array.fromAsync is tofor await
asArray.from is tofor.

Similarly toArray.from, Array.fromAsync would be a static method of theArray built-in class, with one required argument and two optional arguments:(items, mapfn, thisArg).

But, instead of converting a sync iterable to an array, Array.fromAsync canconvert an async iterable to apromise that (if everything goes well) willresolve to a new array. Before the promise resolves, it will create an asynciterator from the input, lazily iterate over it, and add each yielded value tothe new array. (The promise is immediately returned after the Array.fromAsyncfunction call, no matter what.)

asyncfunction*asyncGen(n){for(leti=0;i<n;i++)yieldi*2;}// `arr` will be `[0, 2, 4, 6]`.constarr=[];forawait(constvofasyncGen(4)){arr.push(v);}// This is equivalent.constarr=awaitArray.fromAsync(asyncGen(4));

If the argument is a sync iterable (and not an async iterable), then the returnvalue is still a promise that will resolve to an array. If the sync iteratoryields promises, then each yielded promise is awaited before its value is addedto the new array. (Values that are not promises are also awaited toprevent Zalgo.) All of this matches the behavior offor await.

function*genPromises(n){for(leti=0;i<n;i++)yieldPromise.resolve(i*2);}// `arr` will be `[ 0, 2, 4, 6 ]`.constarr=[];forawait(constvofgenPromises(4)){arr.push(v);}// This is equivalent.constarr=awaitArray.fromAsync(genPromises(4));

Likefor await, Array.fromAsynclazily iterates over a sync-but-not-asyncinput. Whenever a developer needs to dump a synchronous input that yieldspromises into an array, the developer needs to choose carefully betweenArray.fromAsync and Promise.all, which have complementary control flows:

Also likefor await, when given a sync-but-not-async iterable input, thenArray.fromAsync will catchonly the first rejection that its iterationreaches, and only if that rejection doesnot occur in a microtask beforethe iteration reaches and awaits for it. For more information, see§ Errors.

// `arr` will be `[ 0, 2, 4, 6 ]`.// `genPromises(4)` is lazily iterated,// and its four yielded promises are awaited in sequence.constarr=awaitArray.fromAsync(genPromises(4));// `arr` will also be `[ 0, 2, 4, 6 ]`.// However, `genPromises(4)` is eagerly iterated// (into an array of four promises),// and the four promises are awaited in parallel.constarr=awaitPromise.all(Array.from(genPromises(4)));

Array.fromAsync’s valid inputs are a superset of Array.from’s valid inputs.This includes non-iterable array-likes: objects that have a length property aswell as indexed elements (similarly to Array.prototype.values). The returnvalue is still a promise that will resolve to an array. If the array-likeobject’s elements are promises, then each accessed promise is awaited beforeits value is added to the new array.

OneTC39 representative’s opinion: “[Array-likes are] verymuch not obsolete, and it’s very nice that things aren’t forced to implementthe iterator protocol to be transformable into an Array.”

constarrLike={length:4,0:Promise.resolve(0),1:Promise.resolve(2),2:Promise.resolve(4),3:Promise.resolve(6),}// `arr` will be `[ 0, 2, 4, 6 ]`.constarr=[];forawait(constvofArray.from(arrLike)){arr.push(v);}// This is equivalent.constarr=awaitArray.fromAsync(arrLike);

As it does with sync-but-not-async iterable inputs, Array.fromAsync lazilyiterates over the values of array-like inputs, and it awaits each value.The developer must choose between using Array.fromAsync and Promise.all (see§ Sync-iterable inputs and§ Errors).

Array.fromAsync is a generic factory method. It does not require that its thisreceiver be the Array constructor. fromAsync can be transferred to or inheritedby any other constructor. In that case, the final result will be the datastructure created by that constructor (with no arguments), and with each valueyielded by the input being assigned to the data structure’s numeric properties.(Symbol.species is not involved at all.) If the this receiver is not aconstructor, then fromAsync creates an array as usual. This matches thebehavior of Array.from.

asyncfunction*asyncGen(n){for(leti=0;i<n;i++)yieldi*2;}functionData(n){}Data.from=Array.from;Data.fromAsync=Array.fromAsync;// d will be a `new Data(0)`, with its `0` property assigned to `0`, its `1`// property assigned to `2`, etc.constd=newData(0);leti=0;forawait(constvofasyncGen(4)){d[i++]=v;}// This is equivalent.constd=awaitData.fromAsync(asyncGen(4));

Array.fromAsync has two optional parameters:mapfn andthisArg.

mapfn is an optional mapping callback, which is called on each value yielded from the input,along with its index integer (starting from 0).Each result of the mapping callback is, in turn, awaited then added to the array.

However, whenmapfn is given and the input is a sync iterable (or non-iterable array-like),then each value from the input is awaited before being given tomapfn.(The values from the input arenot awaited if the input is an async iterable.)This matches the behavior offor await.

Whenmapfn is not given, each value yielded from asynchronousinputs is not awaited, and each value yielded from synchronous inputs isawaited only once, before the value is added to the result array.This also matches the behavior offor await.

This means that:

Array.fromAsync(input)

…is not equivalent to:

Array.fromAsync(input,x=>x)

…at least wheninput is an async iterable.

This is because, whenever input is an async iterable that yields promise items,Array.fromAsync(input) will not resolve those promise items,butArray.fromAsync(input, x => x) will resolve thembecause the result of thex => x mapping function is awaited.

For example:

functioncreateAsyncIter(){leti=0;return{[Symbol.asyncIterator](){return{asyncnext(){if(i>2)return{done:true};i++;return{value:Promise.resolve(i),done:false}}}}};}// This prints `[Promise.resolve(1), Promise.resolve(2), Promise.resolve(3)]`:console.log(awaitArray.fromAsync(createAsyncIter()));// This prints `[1, 2, 3]`:console.log(awaitArray.fromAsync(createAsyncIter(),x=>x));

See alsoissue #19.

thisArg is athis-binding receiver value for the mapping callback. Bydefault, this is undefined. These optional parameters match the behavior ofArray.from. Their exclusion would be surprising to developers who are alreadyused to Array.from.

asyncfunction*asyncGen(n){for(leti=0;i<n;i++)yieldi*2;}// `arr` will be `[ 0, 4, 16, 36 ]`.constarr=[];forawait(constvofasyncGen(4)){arr.push(await(v**2));}// This is equivalent.constarr=awaitArray.fromAsync(asyncGen(4),v=>v**2);

Like other promise-based APIs, Array.fromAsync will always immediately return apromise. Array.fromAsync will never synchronously throw an error andsummonZalgo.

When Array.fromAsync’s input throws an error while creating its async or synciterator, then Array.fromAsync’s returned promise will reject with that error.

consterr=newError;constbadIterable={[Symbol.iterator](){throwerr;}};// This returns a promise that will reject with `err`.Array.fromAsync(badIterable);

When Array.fromAsync’s input is iterable but the input’s iterator throws whileiterating, then Array.fromAsync’s returned promise will reject with that error.

consterr=newError;asyncfunction*genErrorAsync(){throwerr;}// This returns a promise that will reject with `err`.Array.fromAsync(genErrorAsync());

consterr=newError;function*genError(){throwerr;}// This returns a promise that will reject with `err`.Array.fromAsync(genError());

When Array.fromAsync’s input is synchronous only (i.e., the input is not anasync iterable), and when one of the input’s values is a promise thateventually rejects or has rejected, then iteration stops and Array.fromAsync’sreturned promise will reject with the first such error.

In this case, Array.fromAsync will catch and handle that first input rejectiononly if that rejection doesnot occur in a microtask before theiteration reaches and awaits for it.

consterr=newError;function*genRejection(){yieldPromise.reject(err);}// This returns a promise that will reject with `err`. There is **no**// unhandled promise rejection, because the rejection occurs in the same// microtask.Array.fromAsync(genZeroThenRejection());

Just like withfor await, Array.fromAsync willnot catch any rejectionsby the input’s promises whenever those rejections occurbefore the ticks inwhich Array.fromAsync’s iteration reaches those promises.

This is because – likefor await – Array.fromAsynclazily iterates overits input andsequentially awaits each yielded value. Whenever a developerneeds to dump a synchronous input that yields promises into an array, thedeveloper needs to choose carefully between Array.fromAsync and Promise.all,which have complementary control flows (see§ Sync-iterableinputs).

For example, when a synchronous input contains two promises, the latter ofwhich will reject before the former promise resolves, then Array.fromAsync willnot catch that rejection, because it lazily reaches the rejecting promise onlyafter it already has rejected.

constnumOfMillisecondsPerSecond=1000;constslowError=newError;constfastError=newError;functionwaitThenReject(value){returnnewPromise((resolve,reject)=>{setTimeout(()=>reject(value),numOfMillisecondsPerSecond);});}function*genRejections(){// Slow promise.yieldwaitAndReject(slowError);// Fast promise.yieldPromise.reject(fastError);}// This returns a promise that will reject with `slowError`. There is **no**// unhandled promise rejection: the iteration is lazy and will stop early at the// slow promise, so the fast promise will never be created.Array.fromAsync(genSlowRejectThenFastReject());// This returns a promise that will reject with `slowError`. There **is** an// unhandled promise rejection with `fastError`: the iteration eagerly creates// and dumps both promises into an array, but Array.fromAsync will// **sequentially** handle only the slow promise.Array.fromAsync([ ...genSlowRejectThenFastReject()]);// This returns a promise that will reject with `fastError`. There is **no**// unhandled promise rejection: the iteration eagerly creates and dumps both// promises into an array, but Promise.all will handle both promises **in// parallel**.Promise.all([ ...genSlowRejectThenFastReject()]);

When Array.fromAsync’s input has at least one value, and when Array.fromAsync’smapping callback throws an error when given any of those values, thenArray.fromAsync’s returned promise will reject with the first such error.

consterr=newError;functionbadCallback(){throwerr;}// This returns a promise that will reject with `err`.Array.fromAsync([0],badCallback);

When Array.fromAsync’s input is null or undefined, or when Array.fromAsync’smapping callback is neither undefined nor callable, then Array.fromAsync’sreturned promise will reject with a TypeError.

// These return promises that will reject with TypeErrors.Array.fromAsync(null);Array.fromAsync([],1);

Array.fromAsync tries to matchfor await’s behavior as much as possible.

Previously,for await did not close sync iterables when ityields a rejected promise.

function*createIter(){try{yieldPromise.resolve(console.log("a"));yieldPromise.reject("x");}finally{console.log("finalized");}}// Prints "a" and then prints "finalized".// There is an uncaught "x" rejection.for(constxofcreateIter()){console.log(awaitx);}// Prints "a" and then prints "finalized".// There is an uncaught "x" rejection.Array.from(createIter());// Prints "a" and does *not* print "finalized".// There is an uncaught "x" rejection.forawait(constxofcreateIter()){console.log(x);}// Prints "a" and does *not* print "finalized".// There is an uncaught "x" rejection.Array.fromAsync(createIter());

TC39 has recently changedfor await’s behavior here.In the latest version of the language,for await now will close sync iterators when async wrappers yield rejections (seetc39/ecma262#2600).All of the JavaScript engines are already updating to this new behavior.

Array.fromAsync matches this new behavior offor await.Both will close any given sync iteratorwhen the sync iterator yields a rejected promise as its next value.

Theiterator-helpers andasync-iterator-helpers proposals defineIterator.toArray and AsyncIterator.toArray. The following pairs of lines areequivalent:

// Array.fromArray.from(iterable)Iterator(iterable).toArray()Array.from(iterable,mapfn)Iterator(iterable).map(mapfn).toArray()// Array.fromAsyncArray.fromAsync(asyncIterable)AsyncIterator(asyncIterable).toArray()Array.fromAsync(asyncIterable,mapfn)AsyncIterator(asyncIterable).map(mapfn).toArray()

Iterator.toArray overlaps with Array.from, and AsyncIterator.toArray overlapswith Array.fromAsync. This is okay: they all can coexist.

Aco-champion of iterable-helpers agreesthat we should have both or that we should prefer Array.fromAsync: “Iremembered why it’s better for a buildable structure to consume an iterablethan for an iterable to consume a buildable protocol. Sometimes buildingsomething one element at a time is the same as building it [more than one]element at a time, but sometimes it could be slow to build that way or producea structure with equivalent semantics but different performance properties.”

The following built-ins also resemble Array.from:

TypedArray.from()newSetObject.fromEntries()newMap

We are deferring any async versions of these methods to future proposals.Seeissue #8 andproposal-setmap-offrom.

In the future, standardizing an async spread operator (like[ 0, await ...v ]) may be useful. This proposal leaves that idea to aseparate proposal.

Therecord/tuple proposal puts forward two new data types with APIs thatrespectivelyresemble those ofArray andObject. TheTupleconstructor, too, would probably need anfromAsync method. Whether theRecord constructor gets afromEntriesAsync method will depend on whetherObject.fromEntriesAsync will also be added in a separate proposal.

Only minor formatting changes have been made to the status-quo examples.

constall=require('it-all');// Add the default assets to the repo.constresults=awaitall(addAll(globSource(initDocsPath,{recursive:true,}),{preload:false},),);constdir=results.filter(file=>file.path==='init-docs').pop()print('to get started, enter:\n');print(`\tjsipfs cat`+`/ipfs/${dir.cid}/readme\n`,);

// Add the default assets to the repo.constresults=awaitArray.fromAsync(addAll(globSource(initDocsPath,{recursive:true,}),{preload:false},),);constdir=results.filter(file=>file.path==='init-docs').pop()print('to get started, enter:\n');print(`\tjsipfs cat`+`/ipfs/${dir.cid}/readme\n`,);

constall=require('it-all');constresults=awaitall(node.contentRouting.findProviders('a cid'),);expect(results).to.be.an('array').with.lengthOf(1).that.deep.equals([result]);

constresults=awaitArray.fromAsync(node.contentRouting.findProviders('a cid'),);expect(results).to.be.an('array').with.lengthOf(1).that.deep.equals([result]);

asyncfunctiontoArray(items){constresult=[];forawait(constitemofitems){result.push(item);}returnresult;}it('empty-pipeline',async()=>{constpipeline=newPipeline();constresult=awaittoArray(pipeline.execute([1,2,3,4,5]));assert.deepStrictEqual(result,[1,2,3,4,5],);});

it('empty-pipeline',async()=>{constpipeline=newPipeline();constresult=awaitArray.fromAsync(pipeline.execute([1,2,3,4,5]));assert.deepStrictEqual(result,[1,2,3,4,5],);});