NVIDIA–UNITO Workshop “Data Parallelism: How to Train Deep Learning Models on Multiple GPUs”
Friday, 8 May 2026
Computer Science Department, University of Torino
Bruno Casella, research fellow at the Computer Science Department, and NVIDIA DLI Certified Instructor, presents “Data Parallelism: How to Train Deep Learning Models on Multiple GPUs”.
Overview
Some of the most common challenges faced when training modern deep learning models involve leveraging increasingly large datasets and increasingly complex models. As a result, a growing and significant amount of computational power is required to train models effectively and efficiently. Learning how to distribute data across multiple GPUs during the training phase opens the door to new and important deep learning applications.
Goals
From participating in this workshop:
– You will understand how the Data Parallelism technique across multiple GPUs is applied during the training of deep learning models.
– You will achieve maximum throughput during training, maximizing the utilization of multiple GPUs.
– You will distribute training across multiple GPUs using PyTorch Distributed Data Parallel.
– You will understand and apply specific algorithmic considerations for multi-GPU training.
Venue
Computer Science Department, Via Pessinetto, 12, 10149 Torino TO, L staircase, third floor
Agenda
9.30-10.00 Introduction and motivation
10.00-11.00 Lab 1: Neural network training and stochastic gradient descent
11.00-11.15 Coffee break
11.15-11.45 Introduction to data parallelism using PyTorch Distributed Data Parallel
11.45-13.00 Lab 2: Scaling to multiple GPUs with DistributedDataParallel
13.00-14.30 Lunch
14.30-15.00 Challenges of Data Parallel using multiple GPUs
15.00-16.30 Lab 3: Large scale optimizations.
16.30-16.45: Closing remarks.
Slides are available at this link.
Setup
To get started, follow these steps:
1. Create an NVIDIA Developer account at: https://learn.nvidia.com/dli-event.
2. Visit https://learn.nvidia.com/dli-event and enter the event code provided by the instructor.
3. You are ready to get started.
For more info: https://learn.nvidia.com/courses/course-detail?course_id=course-v1:DLI+C-MG-01+V3
Participation is free. Registration is mandatory.
