This is great! I'd only suggest adding some complicated tests involving nested levels of backticks / interpolation. That's usually where things get tricky to lex.

@jashkenas more complicated thanthis?

Could somebody explain with words:

• When a ``` JS block ends.
• When a ````` JS block ends.

Do backslashes inside JS block affect where the block ends?

I'm probably not the expert in the details of it — but I'd imagine that the idea is to be sensible about escaping the backticks in the same way that we try to be sensible about escaping quotes in a heredoc.

So:AJS block ends at the first unescaped and uninterpolated that follows.

So:AJS block ends at the first unescaped and uninterpolated that follows.

@jashkenas What do you mean by uninterpolated? I would’ve answered this way:

• AJS block ends at the first unescaped that follows.
• AJS block ends at the first that follows.

`\`a\n\``

↓

`a\n`;

Is the above correct? Just checking if I understand the rules or not.

@lydell — The above looks correct to me.

@GeoffreyBooth: By uninterpolated, I mean this:

before #{"str`ing"} after

The backtick inside the interpolation doesn't kill the expression halfway. Ideally the same would be true for triple backticks.

Edit:Hah, nevermind. I just tested it and I guess we don't allow interpolations in JS. Forget it.

@jashkenas I wasn’t aware that we supported interpolation between backticks, or wanted to.

@lydell Your example, if saved into a file that is compiled viacoffee -bp test.coffee, outputs:

\`a\n\`;

So clearly somehow my “escaping works!” test is flawed. I guess some escaping is happening in the test runner itself? Anyway it appears we’re not escaping backticks (that is, outputting just ``` for ```), we’re just skipping over escaped backticks until we find a non-escaped one to end the token on. I can’t see any benefit to our current behavior.

@jashkenas I wasn’t aware that we supported interpolation between backticks, or wanted to.

Yes. We don't. My mistake.

Might be worth noting that the main problem in the issues you linked was for tagged template literals, which will hopefully soon be merged.

@connec Yes, but this should get fixed anyway. This future-proofs us against any other weird things ECMAScript wants to use backticks for.

I figured out why my test passed when you wouldn’t expect it to: JavaScript converts ``` into ``` wherever it sees it:

So the test should compare a string against the expected JavaScript output, not the executed JavaScript.

I’ll update this PR soon, but just so we’re clear on the intended behavior:

• AJS block ends at the first unescaped that follows. Escaped backticks are converted to simple backticks (i.e. ``` becomes ```).
• AJS block ends at the first that follows. Escapedsingle backticks are converted to simple backticks (i.e.\`` becomes ```) so if you really want a triple-backtick in the generated JavaScript, you would write````.

?

That sounds fine to me. I wonder if we can leverage the code paths used to handle heredocs and heregexen? This is an area of the parser I've not looked at.

Okay, it took quite some doing to get `````hello````` to match out to the last backtick, and not one before it (i.e. treating the ``` as not part of the closing triple backticks).@connec or anyone else, can you please check the new regex?

Escaped backticks are now replaced with just backticks in the JavaScript code, so you can write code like this:

`let greeting = \`hello, ${name}\``

I looked into somehow getting dangling backslashes at the end of a line to not stop the capture (the specific bug in#3760) but I ultimately don’t think it’s worth the effort: a backslash before a newline is invalid JavaScript, so it will never compile correctly.

I think this is everything?

I think I'd approach this problem something like this:

JSTOKEN=///^ `(?!``) ((?:[^`\\]|\\[\s\S]           )*) `///HERE_JSTOKEN=///^ ```     ((?:[^`\\]|\\[\s\S]| `(?!``) )*) ```///test= (chunk)->match=HERE_JSTOKEN.exec(chunk)orJSTOKEN.exec(chunk)console.log chunk, matchif matchscript= match[1].replace(/\\+(`|$)/g, (string)->      string[-Math.ceil(string.length/2)..]    )return scriptreturnnullconsole.log([test('``````')# == ''test('`\\\\`')# == '\\'test('`\\\\\\`a`')# == '\\`a'test('```\\```')# == null])

@lydell I'm not sure I follow; aside from more readable regexes, what problems with the PR does your version solve?

You should be able to output a backslash before a backtick, or at the end. For example\\\``` → `\`;` and\`` →\;. (We should add tests for even crazier numbers of backslashes as well, and tests for invalid stuff like\.)

@lydell I would think that\\\``` is supposed to become `\\`;`, and it does in this PR. Likewise, I think\`` should become\\;, as it does. It’s not our job to escape all JavaScript escaped characters; if the user types `\` between backticks, it should just be output as `\`. That’s the current behavior before this PR, so changing it could break backward compatibility. A backslash is normally escaped in JavaScript, so it’s probably pretty common to find `\` in backticked code.

Technically speaking, converting ``` to ``` is also breaking backward compatibility, but I think this is okay since the failure to do this conversion is more of a bug than a feature. The only time a backtick is ever escaped in JavaScript is within a template literal, but people basically haven’t been able to write those yet in CoffeeScript because of the lack of any way to escape backticks.

I only meant that backslashes should be handled specially before backticks and at the end of the block (which my code does). In other words,\\ and\n, for example, are untouched. Otherwise you wouldn't be able to expressanything inside a JS block (which is the point of this PR, isn't it?). For example, a template literal ending in a backslash:\`a\\\``` →a```

@lydell can you please explain

script= match[1].replace/\\+(`|$)/g, (string)->      string[-Math.ceil(string.length/2)..]

? I replaced my regex with yours and adjusted the code as appropriate, but when I add this newreplace function the first JavaScript literal test fails.

Okay, I’ll concede to the preferences of the crowd. I’ve updated thereplace function per@lydell.

@connec, I added your examples as a new test, though I had to escape all the backslashes for them to be parsed correctly as strings. Since that made them hard to understand, I included all the before-and-afters as a big comment just above the test. To double-check it was really working, I created atest.coffee file and pasted your code in there:

`hello`# hello;`\`hello\``                                   #`hello`;`\`Escaping backticks in JS:\\\`hello\\\`\`` #`Escaping backticks in JS:\`hello\``;`Single backslash: \`# Single backslash: \ ;`Single backslash atEOS: \\`# Single backslash at EOS: \;`Double backslash atEOS: \\\\`# Double backslash at EOS: \\;`Double backslash: \\\`# Double backslash: \\ ;

It becomes:

hello;`hello`;`Escaping backticks in JS: \`hello\``;Singlebackslash: \;SinglebackslashatEOS: \;DoublebackslashatEOS: \\;Double backslash: \\\;

The only difference is the last one: you had it outputting only two backslashes, instead of the three I get. But there’s no backtick following these backslashes, so I don’t think the lexer should be affecting them.

Is everyone happy with this?

`Double backslash: \\\ `                      # Double backslash: \\ ;

The above is nothing but wrong and misleading.

Do we have any tests for that'\\\n' becomes'\\\n' (in other words, making sure that we don't mess up other backslashes)?

@lydell Other backslashes are unaffected. I added a test.

OK, cool.

I would definitely remove theDouble backslash: \\\ ; test, or rename it toTriple backslash: \\\ ;, because as@lydell says it's currently misleading - Whoops, looks like that's already changed.

This seems good to me. I'm still slightly dubious about only escaping ```s, but it makes sense that the literal JS should be as close as possible to what's written.

I don't think that I can mentally expand these regexes well enough to comment on them materially — but if@lydell and@connec sign off on this, then I'm cool with it too.

@lydell and@connec, do you have any other notes or should we merge this in?

LGTM

LGTM