Movatterモバイル変換

[0]ホーム
URL:

21. Iterables and iterators
Table of contents
Please support this book:buy it (PDF, EPUB, MOBI) ordonate
(Ad, please don’t block.)

21.Iterables and iterators

21.1Overview

ES6 introduces a new mechanism for traversing data:iteration. Two concepts are central to iteration:

Expressed as interfaces in TypeScript notation, these roles look like this:

interfaceIterable{[Symbol.iterator]():Iterator;}interfaceIterator{next():IteratorResult;}interfaceIteratorResult{value:any;done:boolean;}

21.1.1Iterable values

The following values are iterable:

Plain objects are not iterable (why is explained ina dedicated section).

21.1.2Constructs supporting iteration

Language constructs that access data via iteration:

21.2Iterability

The idea of iterability is as follows.

It’s not practical for every consumer to support all sources, especially because it should be possible to create new sources (e.g. via libraries). Therefore, ES6 introduces the interfaceIterable. Data consumers use it, data sources implement it:

Given that JavaScript does not have interfaces,Iterable is more of a convention:

Let’s see what consumption looks like for an Arrayarr. First, you create an iterator via the method whose key isSymbol.iterator:

> const arr = ['a', 'b', 'c'];> const iter = arr[Symbol.iterator]();

Then you call the iterator’s methodnext() repeatedly to retrieve the items “inside” the Array:

> iter.next(){ value: 'a', done: false }> iter.next(){ value: 'b', done: false }> iter.next(){ value: 'c', done: false }> iter.next(){ value: undefined, done: true }

As you can see,next() returns each item wrapped in an object, as the value of the propertyvalue. The boolean propertydone indicates when the end of the sequence of items has been reached.

Iterable and iterators are part of a so-calledprotocol (interfaces plus rules for using them) for iteration. A key characteristic of this protocol is that it is sequential: the iterator returns values one at a time. That means that if an iterable data structure is non-linear (such as a tree), iteration will linearize it.

21.3Iterable data sources

I’ll use thefor-of loop (see Chap. “Thefor-of loop”) to iterate over various kinds of iterable data.

21.3.1Arrays

Arrays (and Typed Arrays) are iterables over their elements:

for(constxof['a','b']){console.log(x);}// Output:// 'a'// 'b'

21.3.2Strings

Strings are iterable, but they iterate over Unicode code points, each of which may comprise one or two JavaScript characters:

for(constxof'a\uD83D\uDC0A'){console.log(x);}// Output:// 'a'// '\uD83D\uDC0A' (crocodile emoji)

You have just seen that primitive values can be iterable. A value doesn’t have to be an object in order to be iterable. That’s because all values are coerced to objects before the iterator method (property keySymbol.iterator) is accessed.

21.3.3Maps

Maps are iterables over their entries. Each entry is encoded as a [key, value] pair, an Array with two elements. The entries are always iterated over deterministically, in the same order in which they were added to the map.

constmap=newMap().set('a',1).set('b',2);for(constpairofmap){console.log(pair);}// Output:// ['a', 1]// ['b', 2]

Note that WeakMaps are not iterable.

21.3.4Sets

Sets are iterables over their elements (which are iterated over in the same order in which they were added to the Set).

constset=newSet().add('a').add('b');for(constxofset){console.log(x);}// Output:// 'a'// 'b'

Note that WeakSets are not iterable.

21.3.5arguments

Even though the special variablearguments is more or less obsolete in ECMAScript 6 (due to rest parameters), it is iterable:

functionprintArgs(){for(constxofarguments){console.log(x);}}printArgs('a','b');// Output:// 'a'// 'b'

21.3.6DOM data structures

Most DOM data structures will eventually be iterable:

for(constnodeofdocument.querySelectorAll('div')){···}

Note that implementing this functionality is work in progress. But it is relatively easy to do so, because the symbolSymbol.iterator can’t clash with existing property keys.

21.3.7Iterable computed data

Not all iterable content does have to come from data structures, it could also be computed on the fly. For example, all major ES6 data structures (Arrays, Typed Arrays, Maps, Sets) have three methods that return iterable objects:

Let’s see what that looks like.entries() gives you a nice way to get both Array elements and their indices:

constarr=['a','b','c'];for(constpairofarr.entries()){console.log(pair);}// Output:// [0, 'a']// [1, 'b']// [2, 'c']

21.3.8Plain objects are not iterable

Plain objects (as created by object literals) are not iterable:

for(constxof{}){// TypeErrorconsole.log(x);}

Why aren’t objects iterable over properties, by default? The reasoning is as follows. There are two levels at which you can iterate in #"leanpub-endnotes.html#fn-iteration_1" rel="footnote">1. But we only do that in ES5 because we have no better alternative. In ECMAScript 6, we have the built-in data structureMap.

21.3.8.1How to iterate over properties

The proper (and safe) way to iterate over properties is via a tool function. For example, viaobjectEntries(),whose implementation is shown later (future ECMAScript versions may have something similar built in):

constobj={first:'Jane',last:'Doe'};for(const[key,value]ofobjectEntries(obj)){console.log(`${key}:${value}`);}// Output:// first: Jane// last: Doe

21.4Iterating language constructs

The following ES6 language constructs make use of the iteration protocol:

The next sections describe each one of them in detail.

21.4.1Destructuring via an Array pattern

Destructuring via Array patterns works for any iterable:

constset=newSet().add('a').add('b').add('c');const[x,y]=set;// x='a'; y='b'const[first,...rest]=set;// first='a'; rest=['b','c'];

21.4.2Thefor-of loop

for-of is a new loop in ECMAScript 6. It’s basic form looks like this:

for(constxofiterable){···}

For more information, consult Chap. “Thefor-of loop”.

Note that the iterability ofiterable is required, otherwisefor-of can’t loop over a value. That means that non-iterable values must be converted to something iterable. For example, viaArray.from().

21.4.3Array.from()

Array.from() converts iterable and Array-like values to Arrays. It is also available for typed Arrays.

> Array.from(new Map().set(false, 'no').set(true, 'yes'))[[false,'no'], [true,'yes']]> Array.from({ length: 2, 0: 'hello', 1: 'world' })['hello', 'world']

For more information onArray.from(), consultthe chapter on Arrays.

21.4.4The spread operator (...)

The spread operator inserts the values of an iterable into an Array:

>constarr=['b','c'];>['a',...arr,'d']['a','b','c','d']

That means that it provides you with a compact way to convert any iterable to an Array:

constarr=[...iterable];

The spread operator also turns an iterable into the arguments of a function, method or constructor call:

>Math.max(...[-1,8,3])8

21.4.5Maps and Sets

The constructor of a Map turns an iterable over [key, value] pairs into a Map:

> const map = new Map([['uno', 'one'], ['dos', 'two']]);> map.get('uno')'one'> map.get('dos')'two'

The constructor of a Set turns an iterable over elements into a Set:

>constset=newSet(['red','green','blue']);>set.has('red')true>set.has('yellow')false

The constructors ofWeakMap andWeakSet work similarly. Furthermore, Maps and Sets are iterable themselves (WeakMaps and WeakSets aren’t), which means that you can use their constructors to clone them.

21.4.6Promises

Promise.all() andPromise.race() accept iterables over Promises:

Promise.all(iterableOverPromises).then(···);Promise.race(iterableOverPromises).then(···);

21.4.7yield*

yield* is an operator that is only available inside generators. It yields all items iterated over by an iterable.

function*yieldAllValuesOf(iterable){yield*iterable;}

The most important use case foryield* is to recursively call a generator (which produces something iterable).

21.5Implementing iterables

In this section, I explain in detail how to implement iterables. Note thatES6 generators are usually much more convenient for this task than doing so “manually”.

The iteration protocol looks as follows.

An object becomesiterable (“implements” the interfaceIterable) if it has a method (own or inherited) whose key isSymbol.iterator. That method must return aniterator, an object thatiterates over theitems “inside” the iterable via its methodnext().

In TypeScript notation, the interfaces for iterables and iterators look as follows2.

interfaceIterable{[Symbol.iterator]():Iterator;}interfaceIterator{next():IteratorResult;return?(value?:any):IteratorResult;}interfaceIteratorResult{value:any;done:boolean;}

return() is an optional method that we’ll get to later3. Let’s first implement a dummy iterable to get a feeling for how iteration works.

constiterable={[Symbol.iterator](){letstep=0;constiterator={next(){if(step<=2){step++;}switch(step){case1:return{value:'hello',done:false};case2:return{value:'world',done:false};default:return{value:undefined,done:true};}}};returniterator;}};

Let’s check thatiterable is, in fact, iterable:

for(constxofiterable){console.log(x);}// Output:// hello// world

The code executes three steps, with the counterstep ensuring that everything happens in the right order. First, we return the value'hello', then the value'world' and then we indicate that the end of the iteration has been reached. Each item is wrapped in an object with the properties:

You can omitdone if it isfalse andvalue if it isundefined. That is, theswitch statement could be written as follows.

switch(step){case1:return{value:'hello'};case2:return{value:'world'};default:return{done:true};}

As is explained in thethe chapter on generators, there are cases where you want even the last item withdone: true to have avalue. Otherwise,next() could be simpler and return items directly (without wrapping them in objects). The end of iteration would then be indicated via a special value (e.g., a symbol).

Let’s look at one more implementation of an iterable. The functioniterateOver() returns an iterable over the arguments that are passed to it:

functioniterateOver(...args){letindex=0;constiterable={[Symbol.iterator](){constiterator={next(){if(index<args.length){return{value:args[index++]};}else{return{done:true};}}};returniterator;}}returniterable;}// Using `iterateOver()`:for(constxofiterateOver('fee','fi','fo','fum')){console.log(x);}// Output:// fee// fi// fo// fum

21.5.1Iterators that are iterable

The previous function can be simplified if the iterable and the iterator are the same object:

functioniterateOver(...args){letindex=0;constiterable={[Symbol.iterator](){returnthis;},next(){if(index<args.length){return{value:args[index++]};}else{return{done:true};}},};returniterable;}

Even if the original iterable and the iterator are not the same object, it is still occasionally useful if an iterator has the following method (which also makes it an iterable):

[Symbol.iterator](){returnthis;}

All built-in ES6 iterators follow this pattern (via a common prototype, seethe chapter on generators). For example, the default iterator for Arrays:

> const arr = [];> const iterator = arr[Symbol.iterator]();> iterator[Symbol.iterator]() === iteratortrue

Why is it useful if an iterator is also an iterable?for-of only works for iterables, not for iterators. Because Array iterators are iterable, you can continue an iteration in another loop:

constarr=['a','b'];constiterator=arr[Symbol.iterator]();for(constxofiterator){console.log(x);// abreak;}// Continue with same iterator:for(constxofiterator){console.log(x);// b}

One use case for continuing an iteration is that you can remove initial items (e.g. a header) before processing the actual content viafor-of.

21.5.2Optional iterator methods:return() andthrow()

Two iterator methods are optional:

21.5.2.1Closing iterators viareturn()

As mentioned before, the optional iterator methodreturn() is about letting an iterator clean up if it wasn’t iterated over until the end. Itcloses an iterator. Infor-of loops, premature (orabrupt, in spec language) termination can be caused by:

In each of these cases,for-of lets the iterator know that the loop won’t finish. Let’s look at an example, a functionreadLinesSync that returns an iterable of text lines in a file and would like to close that file no matter what happens:

functionreadLinesSync(fileName){constfile=···;return{···next(){if(file.isAtEndOfFile()){file.close();return{done:true};}···},return(){file.close();return{done:true};},};}

Due toreturn(), the file will be properly closed in the following loop:

// Only print first linefor(constlineofreadLinesSync(fileName)){console.log(x);break;}

Thereturn() method must return an object. That is due to how generators handle thereturn statement and will be explained inthe chapter on generators.

The following constructs close iterators that aren’t completely “drained”:

Alater section has more information on closing iterators.

21.6More examples of iterables

In this section, we look at a few more examples of iterables. Most of these iterables are easier to implement via generators.The chapter on generators shows how.

21.6.1Tool functions that return iterables

Tool functions and methods that return iterables are just as important as iterable data structures. The following is a tool function for iterating over the own properties of an object.

functionobjectEntries(obj){letindex=0;// In ES6, you can use strings or symbols as property keys,// Reflect.ownKeys() retrieves bothconstpropKeys=Reflect.ownKeys(obj);return{[Symbol.iterator](){returnthis;},next(){if(index<propKeys.length){constkey=propKeys[index];index++;return{value:[key,obj[key]]};}else{return{done:true};}}};}constobj={first:'Jane',last:'Doe'};for(const[key,value]ofobjectEntries(obj)){console.log(`${key}:${value}`);}// Output:// first: Jane// last: Doe

Another option is to use an iterator instead of an index to traverse the Array with the property keys:

functionobjectEntries(obj){letiter=Reflect.ownKeys(obj)[Symbol.iterator]();return{[Symbol.iterator](){returnthis;},next(){let{done,value:key}=iter.next();if(done){return{done:true};}return{value:[key,obj[key]]};}};}

21.6.2Combinators for iterables

Combinators4 are functions that combine existing iterables to create new ones.

21.6.2.1take(n, iterable)

Let’s start with the combinator functiontake(n, iterable), which returns an iterable over the firstn items ofiterable.

functiontake(n,iterable){constiter=iterable[Symbol.iterator]();return{[Symbol.iterator](){returnthis;},next(){if(n>0){n--;returniter.next();}else{return{done:true};}}};}constarr=['a','b','c','d'];for(constxoftake(2,arr)){console.log(x);}// Output:// a// b

This version oftake() doesn’t close the iteratoriter. How to do that isshown later, afterI explain what closing an iterator actually means.

21.6.2.2zip(...iterables)

zip turnsn iterables into an iterable ofn-tuples (encoded as Arrays of lengthn).

functionzip(...iterables){constiterators=iterables.map(i=>i[Symbol.iterator]());letdone=false;return{[Symbol.iterator](){returnthis;},next(){if(!done){constitems=iterators.map(i=>i.next());done=items.some(item=>item.done);if(!done){return{value:items.map(i=>i.value)};}// Done for the first time: close all iteratorsfor(constiteratorofiterators){if(typeofiterator.return==='function'){iterator.return();}}}// We are donereturn{done:true};}}}

As you can see, the shortest iterable determines the length of the result:

constzipped=zip(['a','b','c'],['d','e','f','g']);for(constxofzipped){console.log(x);}// Output:// ['a', 'd']// ['b', 'e']// ['c', 'f']

21.6.3Infinite iterables

Some iterable may never bedone.

functionnaturalNumbers(){letn=0;return{[Symbol.iterator](){returnthis;},next(){return{value:n++};}}}

With an infinite iterable, you must not iterate over “all” of it. For example, by breaking from afor-of loop:

for(constxofnaturalNumbers()){if(x>2)break;console.log(x);}

Or by only accessing the beginning of an infinite iterable:

const[a,b,c]=naturalNumbers();// a=0; b=1; c=2;

Or by using a combinator.take() is one possibility:

for(constxoftake(3,naturalNumbers())){console.log(x);}// Output:// 0// 1// 2

The “length” of the iterable returned byzip() is determined by its shortest input iterable. That means thatzip() andnaturalNumbers() provide you with the means to number iterables of arbitrary (finite) length:

constzipped=zip(['a','b','c'],naturalNumbers());for(constxofzipped){console.log(x);}// Output:// ['a', 0]// ['b', 1]// ['c', 2]

21.7FAQ: iterables and iterators

21.7.1Isn’t the iteration protocol slow?

You may be worried about the iteration protocol being slow, because a new object is created for each invocation ofnext(). However, memory management for small objects is fast in modern engines and in the long run, engines can optimize iteration so that no intermediate objects need to be allocated. Athread on es-discuss has more information.

21.7.2Can I reuse the same object several times?

In principle, nothing prevents an iterator from reusing the same iteration result object several times – I’d expect most things to work well. However, there will be problems if a client caches iteration results:

constiterationResults=[];constiterator=iterable[Symbol.iterator]();letiterationResult;while(!(iterationResult=iterator.next()).done){iterationResults.push(iterationResult);}

If an iterator reuses its iteration result object,iterationResults will, in general, contain the same object multiple times.

21.7.3Why doesn’t ECMAScript 6 have iterable combinators?

You may be wondering why ECMAScript 6 does not haveiterable combinators, tools for working with iterables or for creating iterables. That is because the plans are to proceed in two steps:

Eventually, one such library or pieces from several libraries will be added to the JavaScript standard library.

If you want to get an impression of what such a library could look like, take a look at the standard Python moduleitertools.

21.7.4Aren’t iterables difficult to implement?

Yes, iterables are difficult to implement – if you implement them manually.The next chapter will introducegenerators that help with this task (among other things).

21.8The ECMAScript 6 iteration protocol in depth

The iteration protocol comprises the following interfaces (I have omittedthrow() fromIterator, which is only supported byyield* and optional there):

interfaceIterable{[Symbol.iterator]():Iterator;}interfaceIterator{next():IteratorResult;return?(value?:any):IteratorResult;}interfaceIteratorResult{value:any;done:boolean;}

The spec hasa section on the iteration protocol.

21.8.1Iteration

Rules fornext():

21.8.1.1TheIteratorResult

The propertydone of an iterator result doesn’t have to betrue orfalse, truthy or falsy is enough. All built-in language mechanisms let you omitdone: false.

21.8.1.2Iterables that return fresh iterators versus those that always return the same iterator

Some iterables produce a new iterator each time they are asked for one. For example, Arrays:

functiongetIterator(iterable){returniterable[Symbol.iterator]();}constiterable=['a','b'];console.log(getIterator(iterable)===getIterator(iterable));// false

Other iterables return the same iterator each time. For example, generator objects:

function*elements(){yield'a';yield'b';}constiterable=elements();console.log(getIterator(iterable)===getIterator(iterable));// true

Whether an iterable produces a fresh iterators or not matter when you iterate over the same iterable multiple times. For example, via the following function:

functioniterateTwice(iterable){for(constxofiterable){console.log(x);}for(constxofiterable){console.log(x);}}

With fresh iterators, you can iterate over the same iterable multiple times:

iterateTwice(['a','b']);// Output:// a// b// a// b

If the same iterator is returned each time, you can’t:

iterateTwice(elements());// Output:// a// b

Note that each iterator in the standard library is also an iterable. Its method[Symbol.iterator]() returnthis, meaning that it always returns the same iterator (itself).

21.8.2Closing iterators

The iteration protocol distinguishes two ways of finishing an iterator:

Rules for callingreturn():

Rules for implementingreturn():

The following code illustrates that thefor-of loop callsreturn() if it is aborted before it receives adone iterator result. That is,return() is even called if you abort after receiving the last value. This is subtle and you have to be careful to get it right when you iterate manually or implement iterators.

functioncreateIterable(){letdone=false;constiterable={[Symbol.iterator](){returnthis;},next(){if(!done){done=true;return{done:false,value:'a'};}else{return{done:true,value:undefined};}},return(){console.log('return() was called!');},};returniterable;}for(constxofcreateIterable()){console.log(x);// There is only one value in the iterable and// we abort the loop after receiving itbreak;}// Output:// a// return() was called!
21.8.2.1Closable iterators

An iterator isclosable if it has a methodreturn(). Not all iterators are closable. For example, Array iterators are not:

> let iterable = ['a', 'b', 'c'];> const iterator = iterable[Symbol.iterator]();> 'return' in iteratorfalse

Generator objects are closable by default. For example, the ones returned by the following generator function:

function*elements(){yield'a';yield'b';yield'c';}

If you invokereturn() on the result ofelements(), iteration is finished:

> const iterator = elements();> iterator.next(){ value: 'a', done: false }> iterator.return(){ value: undefined, done: true }> iterator.next(){ value: undefined, done: true }

If an iterator is not closable, you can continue iterating over it after an abrupt exit (such as the one in line A) from afor-of loop:

functiontwoLoops(iterator){for(constxofiterator){console.log(x);break;// (A)}for(constxofiterator){console.log(x);}}functiongetIterator(iterable){returniterable[Symbol.iterator]();}twoLoops(getIterator(['a','b','c']));// Output:// a// b// c

Conversely,elements() returns a closable iterator and the second loop insidetwoLoops() doesn’t have anything to iterate over:

twoLoops(elements());// Output:// a
21.8.2.2Preventing iterators from being closed

The following class is a generic solution for preventing iterators from being closed. It does so by wrapping the iterator and forwarding all method calls exceptreturn().

classPreventReturn{constructor(iterator){this.iterator=iterator;}/** Must also be iterable, so that for-of works */[Symbol.iterator](){returnthis;}next(){returnthis.iterator.next();}return(value=undefined){return{done:false,value};}// Not relevant for iterators: `throw()`}

If we usePreventReturn, the result of the generatorelements() won’t be closed after the abrupt exit in the first loop oftwoLoops().

function*elements(){yield'a';yield'b';yield'c';}functiontwoLoops(iterator){for(constxofiterator){console.log(x);break;// abrupt exit}for(constxofiterator){console.log(x);}}twoLoops(elements());// Output:// atwoLoops(newPreventReturn(elements()));// Output:// a// b// c

There is another way of making generators unclosable: All generator objects produced by the generator functionelements() have the prototype objectelements.prototype. Viaelements.prototype, you can hide the default implementation ofreturn() (which resides in a prototype ofelements.prototype) as follows:

// Make generator object unclosable// Warning: may not work in transpilerselements.prototype.return=undefined;twoLoops(elements());// Output:// a// b// c
21.8.2.3Handling clean-up in generators viatry-finally

Some generators need to clean up (release allocated resources, close open files, etc.) after iteration over them is finished. Naively, this is how we’d implement it:

function*genFunc(){yield'a';yield'b';console.log('Performing cleanup');}

In a normalfor-of loop, everything is fine:

for(constxofgenFunc()){console.log(x);}// Output:// a// b// Performing cleanup

However, if you exit the loop after the firstyield, execution seemingly pauses there forever and never reaches the cleanup step:

for(constxofgenFunc()){console.log(x);break;}// Output:// a

What actually happens is that, whenever one leaves afor-of loop early,for-of sends areturn() to the current iterator. That means that the cleanup step isn’t reached because the generator function returns beforehand.

Thankfully, this is easily fixed, by performing the cleanup in afinally clause:

function*genFunc(){try{yield'a';yield'b';}finally{console.log('Performing cleanup');}}

Now everything works as desired:

for(constxofgenFunc()){console.log(x);break;}// Output:// a// Performing cleanup

The general pattern for using resources that need to be closed or cleaned up in some manner is therefore:

function*funcThatUsesResource(){constresource=allocateResource();try{···}finally{resource.deallocate();}}
21.8.2.4Handling clean-up in manually implemented iterators
constiterable={[Symbol.iterator](){functionhasNextValue(){···}functiongetNextValue(){···}functioncleanUp(){···}letreturnedDoneResult=false;return{next(){if(hasNextValue()){constvalue=getNextValue();return{done:false,value:value};}else{if(!returnedDoneResult){// Client receives first `done` iterator result// => won’t call `return()`cleanUp();returnedDoneResult=true;}return{done:true,value:undefined};}},return(){cleanUp();}};}}

Note that you must callcleanUp() when you are going to return adone iterator result for the first time. You must not do it earlier, because thenreturn() may still be called. This can be tricky to get right.

21.8.2.5Closing iterators you use

If you use iterators, you should close them properly. In generators, you can letfor-of do all the work for you:

/** * Converts a (potentially infinite) sequence of * iterated values into a sequence of length `n` */function*take(n,iterable){for(constxofiterable){if(n<=0){break;// closes iterable}n--;yieldx;}}

If you manage things manually, more work is required:

function*take(n,iterable){constiterator=iterable[Symbol.iterator]();while(true){const{value,done}=iterator.next();if(done)break;// exhaustedif(n<=0){// Abrupt exitmaybeCloseIterator(iterator);break;}yieldvalue;n--;}}functionmaybeCloseIterator(iterator){if(typeofiterator.return==='function'){iterator.return();}}

Even more work is necessary if you don’t use generators:

functiontake(n,iterable){constiter=iterable[Symbol.iterator]();return{[Symbol.iterator](){returnthis;},next(){if(n>0){n--;returniter.next();}else{maybeCloseIterator(iter);return{done:true};}},return(){n=0;maybeCloseIterator(iter);}};}

21.8.3Checklist

Next:22. Generators
[8]ページ先頭
©2009-2025 Movatter.jp