Commitbfe955c

committed

fixup! Generate tokenizer tests from testdata files

1 parent8e1bef2 commitbfe955cCopy full SHA for bfe955c

File tree

-1

lines changed

-1

lines changed

Lines changed: 2 additions & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,6 @@`
`1`	`1`	`from __future__importabsolute_import,division,unicode_literals`
`2`	`2`
	`3`	`+importcodecs`
`3`	`4`	`importjson`
`4`	`5`	`importwarnings`
`5`	`6`	`importre`
`@@ -185,7 +186,7 @@ def capitalize(s):`
`185`	`186`
`186`	`187`	`classTokenizerFile(pytest.File):`
`187`	`188`	`defcollect(self):`
`188`		`-withopen(str(self.fspath),"rb")asfp:`
	`189`	`+withcodecs.open(str(self.fspath),"r",encoding="utf-8")asfp:`
`189`	`190`	`tests=json.load(fp)`
`190`	`191`	`if'tests'intests:`
`191`	`192`	`fori,testinenumerate(tests['tests']):`

Comments

(0)