The Relationship Between Hyperparameters and Distributed Training Algorithms for Large-Scale AI Models

Study how training hyperparameters interact with distributed training algorithms and how these interactions affect performance, convergence, and scalability.

The candidate will: 1. Review optimization hyperparameters and distributed training algorithms. 2. Select representative models, datasets, and distributed configurations. 3. Design a systematic experimental methodology. 4. Investigate the effects of batch size, learning rate, optimizer settings, and other relevant parameters. 5. Analyze performance, convergence, and scalability results. 6. Formulate practical guidelines for hyperparameter selection in distributed training.

3 months.

• Knowledge of Python
• Knowledge of deep learning and optimization
• Basic knowledge of distributed training

1. Gianluca Mittone, & Marco Aldinucci (2026). Accelerating Sharded Data Parallelism at Scale with Federated Learning. In Euro-Par 2026: Parallel Processing - 32nd European Conference on Parallel and Distributed Processing, Pisa, Italy, August 24-28, 2026, Proceedings, Part II (pp. 437–451). Springer.