In this guide, you’ll learn how to use the DeepSpeed API.
Visit the API reference
DeepSpeed is a Microsoft library that supports large-scale, distributed
learning with sharded optimizer state training and pipeline parallelism. Determined supports
DeepSpeed with the
DeepSpeedTrial provides a way to use an automated training
loop with DeepSpeed.
Determined DeepSpeed documentation:
Advanced Usage discusses advanced topics like using multiple model engines, manual gradient aggregation, custom data loaders, and custom model parallelism.
DeepSpeed Autotune: User Guide demonstrates how to use DeepSpeed Autotune to take full advantage of your hardware and model.