Commit9f37705

authored

Add static graph param (lucidrains#226)

* Add static graph param* use static graph param

1 parentc3df46e commit9f37705Copy full SHA for 9f37705

File tree

-1

lines changed

-1

lines changed

Lines changed: 1 addition & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -307,6 +307,7 @@ class DecoderTrainConfig(BaseModel):`
`307`	`307`	`wd:SingularOrIterable[float]=0.01`
`308`	`308`	`warmup_steps:Optional[SingularOrIterable[int]]=None`
`309`	`309`	`find_unused_parameters:bool=True`
	`310`	`+static_graph:bool=True`
`310`	`311`	`max_grad_norm:SingularOrIterable[float]=0.5`
`311`	`312`	`save_every_n_samples:int=100000`
`312`	`313`	`n_sample_images:int=6# The number of example images to produce when sampling the train and test dataset`

Lines changed: 1 addition & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -556,7 +556,7 @@ def initialize_training(config: TrainDecoderConfig, config_path):`
`556`	`556`	`torch.manual_seed(config.seed)`
`557`	`557`
`558`	`558`	`# Set up accelerator for configurable distributed training`
`559`		`-ddp_kwargs=DistributedDataParallelKwargs(find_unused_parameters=config.train.find_unused_parameters)`
	`559`	`+ddp_kwargs=DistributedDataParallelKwargs(find_unused_parameters=config.train.find_unused_parameters,static_graph=config.train.static_graph)`
`560`	`560`	`init_kwargs=InitProcessGroupKwargs(timeout=timedelta(seconds=60*60))`
`561`	`561`	`accelerator=Accelerator(kwargs_handlers=[ddp_kwargs,init_kwargs])`
`562`	`562`

Comments

(0)