This chapter explains new regular expression features in ECMAScript 6. It helps if you are familiar with ES5 regular expression features and Unicode. Consult the following two chapters of “Speaking JavaScript” if necessary:
/y (sticky)RegExp.prototype.exec(str)RegExp.prototype.test(str)String.prototype.search(regex)String.prototype.match(regex)String.prototype.split(separator, limit)String.prototype.replace(search, replacement)/u (unicode).) matches code points, not code unitsflagsRegExp() can be used as a copy constructorexec()The following regular expression features are new in ECMAScript 6:
/y (sticky) anchors each match of a regular expression to the end of the previous match./u (unicode) handles surrogate pairs (such as\uD83D\uDE80) as code points and lets you use Unicode code point escapes (such as\u{1F680}) in regular expressions.flags gives you access to the flags of a regular expression, just likesource already gives you access to the pattern in ES5:RegExp() to make a copy of a regular expression:/y (sticky)The new flag/y changes two things while matching a regular expressionre against a string:
re.lastIndex: The match must start atre.lastIndex (the index after the previous match). This behavior is similar to the^ anchor, but with that anchor, matches must always start at index 0.re.lastIndex is set to the index after the match. This behavior is similar to the/g flag. Like/g,/y is normally used to match multiple times.The main use case for this matching behavior is tokenizing, where you want each match to immediately follow its predecessor. An example of tokenizing via a sticky regular expression andexec() is given later.
Let’s look at how various regular expression operations react to the/y flag. The following tables give an overview. I’ll provide more details afterwards.
Methods of regular expressions (re is the regular expression that a method is invoked on):
| Flags | Start matching | Anchored to | Result if match | No match | re.lastIndex | |
|---|---|---|---|---|---|---|
| exec() | – | 0 | – | Match object | null | unchanged |
| /g | re.lastIndex | – | Match object | null | index after match | |
| /y | re.lastIndex | re.lastIndex | Match object | null | index after match | |
| /gy | re.lastIndex | re.lastIndex | Match object | null | index after match | |
| test() | (Any) | (like exec()) | (like exec()) | true | false | (like exec()) |
Methods of strings (str is the string that a method is invoked on,r is the regular expression parameter):
| Flags | Start matching | Anchored to | Result if match | No match | r.lastIndex | |
|---|---|---|---|---|---|---|
| search() | –, /g | 0 | – | Index of match | -1 | unchanged |
| /y, /gy | 0 | 0 | Index of match | -1 | unchanged | |
| match() | – | 0 | – | Match object | null | unchanged |
| /y | r.lastIndex | r.lastIndex | Match object | null | index after | |
| match | ||||||
| /g | After prev. | – | Array with matches | null | 0 | |
| match (loop) | ||||||
| /gy | After prev. | After prev. | Array with matches | null | 0 | |
| match (loop) | match | |||||
| split() | –, /g | After prev. | – | Array with strings | [str] | unchanged |
| match (loop) | between matches | |||||
| /y, /gy | After prev. | After prev. | Arr. w/ empty strings | [str] | unchanged | |
| match (loop) | match | between matches | ||||
| replace() | – | 0 | – | First match replaced | No repl. | unchanged |
| /y | 0 | 0 | First match replaced | No repl. | unchanged | |
| /g | After prev. | – | All matches replaced | No repl. | unchanged | |
| match (loop) | ||||||
| /gy | After prev. | After prev. | All matches replaced | No repl. | unchanged | |
| match (loop) | match |
RegExp.prototype.exec(str)If/g is not set, matching always starts at the beginning, but skips ahead until a match is found.REGEX.lastIndex is not changed.
constREGEX=/a/;REGEX.lastIndex=7;// ignoredconstmatch=REGEX.exec('xaxa');console.log(match.index);// 1console.log(REGEX.lastIndex);// 7 (unchanged)
If/g is set, matching starts atREGEX.lastIndex and skips ahead until a match is found.REGEX.lastIndex is set to the position after the match. That means that you receive all matches if you loop untilexec() returnsnull.
constREGEX=/a/g;REGEX.lastIndex=2;constmatch=REGEX.exec('xaxa');console.log(match.index);// 3console.log(REGEX.lastIndex);// 4 (updated)// No match at index 4 or laterconsole.log(REGEX.exec('xaxa'));// null
If only/y is set, matching starts atREGEX.lastIndex and is anchored to that position (no skipping ahead until a match is found).REGEX.lastIndex is updated similarly to when/g is set.
constREGEX=/a/y;// No match at index 2REGEX.lastIndex=2;console.log(REGEX.exec('xaxa'));// null// Match at index 3REGEX.lastIndex=3;constmatch=REGEX.exec('xaxa');console.log(match.index);// 3console.log(REGEX.lastIndex);// 4
Setting both/y and/g is the same as only setting/y.
RegExp.prototype.test(str)test() works the same asexec(), but it returnstrue orfalse (instead of a match object ornull) when matching succeeds or fails:
constREGEX=/a/y;REGEX.lastIndex=2;console.log(REGEX.test('xaxa'));// falseREGEX.lastIndex=3;console.log(REGEX.test('xaxa'));// trueconsole.log(REGEX.lastIndex);// 4
String.prototype.search(regex)search() ignores the flag/g andlastIndex (which is not changed, either). Starting at the beginning of the string, it looks for the first match and returns its index (or-1 if there was no match):
constREGEX=/a/;REGEX.lastIndex=2;// ignoredconsole.log('xaxa'.search(REGEX));// 1
If you set the flag/y,lastIndex is still ignored, but the regular expression is now anchored to index 0.
constREGEX=/a/y;REGEX.lastIndex=1;// ignoredconsole.log('xaxa'.search(REGEX));// -1 (no match)
String.prototype.match(regex)match() has two modes:
/g is not set, it works likeexec()./g is set, it returns an Array with the string parts that matched, ornull.If the flag/g is not set,match() captures groups likeexec():
{constREGEX=/a/;REGEX.lastIndex=7;// ignoredconsole.log('xaxa'.match(REGEX).index);// 1console.log(REGEX.lastIndex);// 7 (unchanged)}{constREGEX=/a/y;REGEX.lastIndex=2;console.log('xaxa'.match(REGEX));// nullREGEX.lastIndex=3;console.log('xaxa'.match(REGEX).index);// 3console.log(REGEX.lastIndex);// 4}
If only the flag/g is set thenmatch() returns all matching substrings in an Array (ornull). Matching always starts at position 0.
constREGEX=/a|b/g;REGEX.lastIndex=7;console.log('xaxb'.match(REGEX));// ['a', 'b']console.log(REGEX.lastIndex);// 0
If you additionally set the flag/y, then matching is still performed repeatedly, while anchoring the regular expression to the index after the previous match (or 0).
constREGEX=/a|b/gy;REGEX.lastIndex=0;// ignoredconsole.log('xab'.match(REGEX));// nullREGEX.lastIndex=1;// ignoredconsole.log('xab'.match(REGEX));// nullconsole.log('ab'.match(REGEX));// ['a', 'b']console.log('axb'.match(REGEX));// ['a']
String.prototype.split(separator, limit)The complete details ofsplit()are explained in Speaking JavaScript.
For ES6, it is interesting to see how things change if you use the flag/y.
With/y, the string must start with a separator:
Subsequent separators are only recognized if they immediately follow the first separator:
That means that the string before the first separator and the strings between separators are always empty.
As usual, you can use groups to put parts of the separators into the result array:
String.prototype.replace(search, replacement)Without the flag/g,replace() only replaces the first match:
constREGEX=/a/;// One matchconsole.log('xaxa'.replace(REGEX,'-'));// 'x-xa'
If only/y is set, you also get at most one match, but that match is always anchored to the beginning of the string.lastIndex is ignored and unchanged.
constREGEX=/a/y;// Anchored to beginning of string, no matchREGEX.lastIndex=1;// ignoredconsole.log('xaxa'.replace(REGEX,'-'));// 'xaxa'console.log(REGEX.lastIndex);// 1 (unchanged)// One matchconsole.log('axa'.replace(REGEX,'-'));// '-xa'
With/g set,replace() replaces all matches:
constREGEX=/a/g;// Multiple matchesconsole.log('xaxa'.replace(REGEX,'-'));// 'x-x-'
With/gy set,replace() replaces all matches, but each match is anchored to the end of the previous match:
constREGEX=/a/gy;// Multiple matchesconsole.log('aaxa'.replace(REGEX,'-'));// '--xa'
The parameterreplacement can also be a function,consult “Speaking JavaScript” for details.
The main use case for sticky matching istokenizing, turning a text into a sequence of tokens. One important trait about tokenizing is that tokens are fragments of the text and that there must be no gaps between them. Therefore, sticky matching is perfect here.
functiontokenize(TOKEN_REGEX,str){constresult=[];letmatch;while(match=TOKEN_REGEX.exec(str)){result.push(match[1]);}returnresult;}constTOKEN_GY=/\s*(\+|[0-9]+)\s*/gy;constTOKEN_G=/\s*(\+|[0-9]+)\s*/g;
In a legal sequence of tokens, sticky matching and non-sticky matching produce the same output:
If, however, there is non-token text in the string then sticky matching stops tokenizing, while non-sticky matching skips the non-token text:
The behavior of sticky matching during tokenizing helps with error handling.
If you wanted to manually implement sticky matching, you’d do it as follows: The functionexecSticky() works likeRegExp.prototype.exec() in sticky mode.
functionexecSticky(regex,str){// Anchor the regex to the beginning of the stringletmatchSource=regex.source;if(!matchSource.startsWith('^')){matchSource='^'+matchSource;}// Ensure that instance property `lastIndex` is updatedletmatchFlags=regex.flags;// ES6 feature!if(!regex.global){matchFlags=matchFlags+'g';}constmatchRegex=newRegExp(matchSource,matchFlags);// Ensure we start matching `str` at `regex.lastIndex`constmatchOffset=regex.lastIndex;constmatchStr=str.slice(matchOffset);letmatch=matchRegex.exec(matchStr);// Translate indices from `matchStr` to `str`regex.lastIndex=matchRegex.lastIndex+matchOffset;match.index=match.index+matchOffset;returnmatch;}
/u (unicode)The flag/u switches on a special Unicode mode for a regular expression. That mode has two features:
\u{1F42A} for specifying characters via code points. Normal Unicode escapes such as\u03B1 only have a range of four hexadecimal digits (which equals the basic multilingual plane).A section in the chapter on Unicode has more information on escape sequences. I’ll explain the consequences of feature 2 next. Instead of Unicode code point escapes (e.g.,\u{1F680}), I’m using two UTF-16 code units (e.g.,\uD83D\uDE80). That makes it clear that surrogate pairs are grouped in Unicode mode and works in both Unicode mode and non-Unicode mode.
In non-Unicode mode, a lone surrogate in a regular expression is even found inside (surrogate pairs encoding) code points:
In Unicode mode, surrogate pairs become atomic units and lone surrogates are not found “inside” them:
Actual lone surrogate are still found:
In Unicode mode, you can put code points into character classes and they won’t be interpreted as two characters, anymore.
.) matches code points, not code unitsIn Unicode mode, the dot operator matches code points (one or two code units). In non-Unicode mode, it matches single code units. For example:
In Unicode mode, quantifiers apply to code points (one or two code units). In non-Unicode mode, they apply to single code units. For example:
flagsIn ECMAScript 6, regular expressions have the following data properties:
sourceflagsglobal,ignoreCase,multiline,sticky,unicodelastIndexAs an aside,lastIndex is the only instance property now, all other data properties are implemented via internal instance properties and getters such asget RegExp.prototype.global.
The propertysource (which already existed in ES5) contains the regular expression pattern as a string:
The propertyflags is new, it contains the flags as a string, with one character per flag:
You can’t change the flags of an existing regular expression (ignoreCase etc. have always been immutable), butflags allows you to make a copy where the flags are changed:
functioncopyWithIgnoreCase(re){returnnewRegExp(re.source,re.flags.includes('i')?re.flags:re.flags+'i');}
The next section explains another way to make modified copies of regular expressions.
RegExp() can be used as a copy constructorIn ES6 there are two variants of the constructorRegExp() (the second one is new):
new RegExp(pattern : string, flags = '')pattern. Ifflags is missing, the empty string'' is used.new RegExp(regex : RegExp, flags = regex.flags)regex is cloned. Ifflags is provided then it determines the flags of the copy.The following interaction demonstrates the latter variant:
Therefore, theRegExp constructor gives us another way to change flags:
functioncopyWithIgnoreCase(re){returnnewRegExp(re,re.flags.includes('i')?re.flags:re.flags+'i');}
exec()The following functionexecAll() is an iterable version ofexec() that fixes several issues with usingexec() to retrieve all matches of a regular expression:
exec() until it returnsnull).exec() mutates the regular expression, which means that side effects can become a problem./g must be set. Otherwise, only the first match is returned.function*execAll(regex,str){// Make sure flag /g is set and regex.index isn’t changedconstlocalCopy=copyAndEnsureFlag(regex,'g');letmatch;while(match=localCopy.exec(str)){yieldmatch;}}functioncopyAndEnsureFlag(re,flag){returnnewRegExp(re,re.flags.includes(flag)?re.flags:re.flags+flag);}
UsingexecAll():
conststr='"fee" "fi" "fo" "fum"';constregex=/"([^"]*)"/;// Access capture of group #1 via destructuringfor(const[,group1]ofexecAll(regex,str)){console.log(group1);}// Output:// fee// fi// fo// fum
The following string methods now delegate some of their work to regular expression methods:
String.prototype.match callsRegExp.prototype[Symbol.match].String.prototype.replace callsRegExp.prototype[Symbol.replace].String.prototype.search callsRegExp.prototype[Symbol.search].String.prototype.split callsRegExp.prototype[Symbol.split].For more information, consult Sect. “String methods that delegate regular expression work to their parameters” in the chapter on strings.
If you want to know in more detail how the regular expression flag/u works, I recommend the article “Unicode-aware regular expressions in ECMAScript 6” by Mathias Bynens.