Led by Corey Adams and Huihuo Zheng from ALCF
This section of the workshop will introduce to you the methods we use to run distributed deep learning training on ALCF resources like Theta and ThetaGPU.
We show distributed training using two frameworks:
- Horovod (for TensorFlow and PyTorch), and
- DistributedDataParallel (DDP) (for PyTorch only).