Commit1025014

committed

Our tokenizer currently never outputs adjacent Character tokens; expect this.

1 parente4d4b15 commit1025014Copy full SHA for 1025014

File tree

-1

lines changed

-1

lines changed

Lines changed: 0 additions & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -182,7 +182,6 @@ def runTokenizerTest(test):`
`182`	`182`	`parser=TokenizerTestParser(test['initialState'],`
`183`	`183`	`test['lastStartTag'])`
`184`	`184`	`tokens=parser.parse(test['input'])`
`185`		`-tokens=concatenateCharacterTokens(tokens)`
`186`	`185`	`received=normalizeTokens(tokens)`
`187`	`186`	`errorMsg="\n".join(["\n\nInitial state:",`
`188`	`187`	`test['initialState'],`

Comments

(0)