Commitda509e2

authored

Skip decoding special tokens (#1244)

1 parenta64be9b commitda509e2Copy full SHA for da509e2

File tree

-0

lines changed

-0

lines changed

Lines changed: 2 additions & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -314,6 +314,7 @@ def stream(self, input, timeout=None, **kwargs):`
`314`	`314`	`self.tokenizer,`
`315`	`315`	`timeout=timeout,`
`316`	`316`	`skip_prompt=True,`
	`317`	`+skip_special_tokens=True`
`317`	`318`	`)`
`318`	`319`	`if"chat_template"inkwargs:`
`319`	`320`	`input=self.tokenizer.apply_chat_template(`
`@@ -338,6 +339,7 @@ def stream(self, input, timeout=None, **kwargs):`
`338`	`339`	`streamer=TextIteratorStreamer(`
`339`	`340`	`self.tokenizer,`
`340`	`341`	`timeout=timeout,`
	`342`	`+skip_special_tokens=True`
`341`	`343`	`)`
`342`	`344`	`input=self.tokenizer(input,return_tensors="pt",padding=True).to(`
`343`	`345`	`self.model.device`

Comments

(0)