Commitb6ea77f

Mesh TensorFlow Team

committed

Enable multi-file inference

PiperOrigin-RevId: 475338276

1 parentd229a44 commitb6ea77fCopy full SHA for b6ea77f

File tree

+29

-0

lines changed

+29

-0

lines changed

Lines changed: 29 additions & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -1500,6 +1500,35 @@ def input_fn(params):`
`1500`	`1500`	`write_lines_to_file(decodes,output_filename)`
`1501`	`1501`
`1502`	`1502`
	`1503`	`+@gin.configurable`
	`1504`	`+defdecode_from_files(`
	`1505`	`+estimator,`
	`1506`	`+vocabulary,`
	`1507`	`+model_type,`
	`1508`	`+batch_size,`
	`1509`	`+sequence_length,`
	`1510`	`+checkpoint_path=None,`
	`1511`	`+input_filenames=gin.REQUIRED,`
	`1512`	`+output_filenames=gin.REQUIRED,`
	`1513`	`+eos_id=1,`
	`1514`	`+repeats=1,`
	`1515`	`+):`
	`1516`	`+"""Decodes from multiple files and writes to output_filenames."""`
	`1517`	`+iflen(input_filenames)!=len(output_filenames):`
	`1518`	`+raiseValueError("Input and output filename lists must have equal length.")`
	`1519`	`+forinput_filename,output_filenameinzip(input_filenames,output_filenames):`
	`1520`	`+decode_from_file(estimator=estimator,`
	`1521`	`+vocabulary=vocabulary,`
	`1522`	`+model_type=model_type,`
	`1523`	`+batch_size=batch_size,`
	`1524`	`+sequence_length=sequence_length,`
	`1525`	`+checkpoint_path=checkpoint_path,`
	`1526`	`+input_filename=input_filename,`
	`1527`	`+output_filename=output_filename,`
	`1528`	`+eos_id=eos_id,`
	`1529`	`+repeats=repeats)`
	`1530`	`+`
	`1531`	`+`
`1503`	`1532`	`@gin.configurable`
`1504`	`1533`	`defdecode_from_dataset(estimator,`
`1505`	`1534`	`vocabulary,`

Comments

(0)