Commit9b1096e

committed

Avoid O(n^2) string concatenation in concatCharacterTokens()

1 parent8f7f9f0 commit9b1096eCopy full SHA for 9b1096e

File tree

-10

lines changed

-10

lines changed

Lines changed: 7 additions & 10 deletions

Original file line number	Diff line number	Diff line change
`@@ -62,21 +62,18 @@ def getTreeWalker(treeType, implementation=None, **kwargs):`
`62`	`62`
`63`	`63`
`64`	`64`	`defconcatenateCharacterTokens(tokens):`
`65`		`-charactersToken=None`
	`65`	`+pendingCharacters=[]`
`66`	`66`	`fortokenintokens:`
`67`	`67`	`type=token["type"]`
`68`	`68`	`iftypein ("Characters","SpaceCharacters"):`
`69`		`-ifcharactersTokenisNone:`
`70`		`-charactersToken= {"type":"Characters","data":token["data"]}`
`71`		`-else:`
`72`		`-charactersToken["data"]+=token["data"]`
	`69`	`+pendingCharacters.append(token["data"])`
`73`	`70`	`else:`
`74`		`-ifcharactersTokenisnotNone:`
`75`		`-yieldcharactersToken`
`76`		`-charactersToken=None`
	`71`	`+ifpendingCharacters:`
	`72`	`+yield{"type":"Characters","data":"".join(pendingCharacters)}`
	`73`	`+pendingCharacters=[]`
`77`	`74`	`yieldtoken`
`78`		`-ifcharactersTokenisnotNone:`
`79`		`-yieldcharactersToken`
	`75`	`+ifpendingCharacters:`
	`76`	`+yield{"type":"Characters","data":"".join(pendingCharacters)}`
`80`	`77`
`81`	`78`
`82`	`79`	`defpprint(tokens):`

Comments

(0)