AI-Based Cross-HPC Job Queue Time Prediction Using Real Production Data

Develop and evaluate AI-based methods for predicting job queue times across HPC systems using real production workload data.

The candidate will: 1. Review HPC scheduling systems and queue-time prediction methods. 2. Inspect and preprocess real production workload data. 3. Identify relevant workload, system, and scheduling features. 4. Develop baseline statistical and machine learning models. 5. Investigate cross-system prediction and generalization. 6. Evaluate prediction accuracy, robustness, and practical applicability.

3 months according to the student's background.

• Knowledge of Python
• Knowledge of machine learning and statistics
• Basic understanding of HPC systems and job schedulers

1. J. Park, “Queue waiting time prediction for large-scale high-performance computing system,” in 17th International Conference on High Performance Computing & Simulation, HPCS 2019, Dublin, Ireland, July 15-19, 2019. IEEE, 2019, pp. 850–855
2. N. Brown, G. Gibb, E. Belikov, and R. Nash, “Predicting batch queue job wait times for informed scheduling of urgent HPC workloads,” CoRR, vol. abs/2204.13543, 2022
3. J. Park, M. Kwon, and T. Hong, “Queue congestion prediction for large-scale high performance computing systems using a hidden markov model,” J. Supercomput., vol. 78, no. 10, pp. 12 202–12 223, 2022.
4. C. Vercellino, A. Scionti, G. Varavallo, P. Viviani, G. Vitali, and O. Terzo, “A machine learning approach for an HPC use case: the jobs queuing time prediction,” Future Gener. Comput. Syst., vol. 143, pp. 215–230, 2023.
5. N. Okafor, B. Lusch, and V. Vishwanath, “Queue wait time prediction in high performance computing (HPC) systems,” J. Supercomput., vol. 82,no. 2, p. 90, 2026.
6. C. Whitton, W. M. Jones, C. S. Walker, V. Job, S. T. Senator, and N. DeBardeleben, “Are we there yet? predicting the queue wait times for HPC jobs,” in IEEE International Conference on Cluster Computing, CLUSTER 2025, Edinburgh, United Kingdom, September 2-5, 2025. IEEE, 2025, pp. 1–12.