DeepSpeed – Efficient Training Scalability for Deep Learning Models

Location: Founders Café

Deep Learning (DL) is driving unprecedented progress across Artificial Intelligence domains, including natural language processing, vision, speech, and multimodal. Sustaining this rapid pace of AI revolution, however, requires practical solutions to the extreme demands of scaling on the compute, memory, communication, and storage components of modern computing hardware. To address this challenge, we created a deep learning optimization library called DeepSpeed to make distributed model training efficient, effective, and easy on commodity hardware. This talk will focus on DeepSpeed optimizations for improving compute, communication, and I/O of extreme-scale model training.