Distributed Training with Determined#
Learn how to perform optimized distributed training with Determined to speed up the training of a single trial.
In Concepts of Distributed Training, you’ll learn about the following topics:
How Determined distributed training works
Reducing computation and communication overhead
Training effectively with large batch sizes
Model characteristics that affect performance
Debugging performance bottlenecks
Visit Implementing Distributed Training to discover how to implement distributed training, including the following:
Connectivity considerations for multi-machine training
Configuration including slots per trial and global batch size
Considerations for concurrent data downloads
Details to be aware regarding scheduler behavior
Accelerating inference workloads
Learn how Configuration Templates can help reduce redundancy.
Discover how Determined aims to support reproducible machine learning experiments in Reproducibility.
In Optimizing Training, you’ll learn about out-of-the box tools you can use for instrumenting training.