- Notifications
You must be signed in to change notification settings - Fork1.4k
distributed
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
parent directory.. | ||||
distributed_data_parallel.py andrun.sh show an example using Amp withapex.parallel.DistributedDataParallel ortorch.nn.parallel.DistributedDataParalleland the Pytorch multiprocess launcher script,torch.distributed.launch.The use ofAmp with DistributedDataParallel does not need to change from ordinarysingle-process use. The only gotcha is that wrapping your model withDistributedDataParallel mustcome after the call toamp.initialize. Test via
bash run.sh
This is intended purely as an instructional example, not a performance showcase.