Alberto Mulone is a Ph.D. student in Computer Science at the University of Turin. He received his Master’s degree with honours from the same institution in 2023, with a thesis assessing the advantages of hybrid workflows for executing a variant calling pipeline across cloud and HPC environments.
Eva Sciacca, Nicola Tuccari, Marco Edoardo Santimaria, Valentina Cesare, Fabio Vitello, Alberto Mulone, Doriana Medić, Iacopo Colonnelli
SN Computer Science
icsc space streamflow
Modern astrophysics and cosmology generate petabyte-scale datasets that demand advanced tools for scalable visualization and analysis. The Visualization Interface for the Virtual Observatory (VisIVO) provides multi-dimensional data exploration, but its integration with heterogeneous computing infrastructures and data-intensive workflows has remained challenging. Here we present a portable and reproducible approach that combines VisIVO with the StreamFlow workflow engine and the CAPIO middleware. StreamFlow enables hybrid execution across cloud-HPC environments, while CAPIO injects transparent I/O streaming into legacy file-based pipelines without modifying application code. Together, these technologies accelerate high–performance visualization, improve reproducibility, and reduce I/O bottlenecks. We demonstrate our approach on large-scale cosmological simulations produced with ChaNGa and GADGET-based codes, including studies of massive neutrinos in the DEMNUni suite. Our results show that streaming-enhanced workflows deliver up to 50% performance gains and enable interactive, scalable visualization across distributed infrastructures, paving the way for exascale-ready scientific discovery.
@article{2026:sn:visivo,
abstract = {Modern astrophysics and cosmology generate petabyte-scale datasets that demand advanced tools for scalable visualization and analysis. The Visualization Interface for the Virtual Observatory (VisIVO) provides multi-dimensional data exploration, but its integration with heterogeneous computing infrastructures and data-intensive workflows has remained challenging. Here we present a portable and reproducible approach that combines VisIVO with the StreamFlow workflow engine and the CAPIO middleware. StreamFlow enables hybrid execution across cloud-HPC environments, while CAPIO injects transparent I/O streaming into legacy file-based pipelines without modifying application code. Together, these technologies accelerate high--performance visualization, improve reproducibility, and reduce I/O bottlenecks. We demonstrate our approach on large-scale cosmological simulations produced with ChaNGa and GADGET-based codes, including studies of massive neutrinos in the DEMNUni suite. Our results show that streaming-enhanced workflows deliver up to 50{\%} performance gains and enable interactive, scalable visualization across distributed infrastructures, paving the way for exascale-ready scientific discovery.},
author = {Sciacca, Eva and Tuccari, Nicola and Santimaria, Marco Edoardo and Cesare, Valentina and Vitello, Fabio and Mulone, Alberto and Medi{\'{c}}, Doriana and Colonnelli, Iacopo},
doi = {10.1007/s42979-026-05076-4},
issn = {2661-8907},
journal = {SN Computer Science},
keywords = {streamflow, space, icsc},
month = jun,
number = 5,
pages = 527,
title = {Hybrid Streaming Workflows for Scalable Visualization in Astronomy and Cosmology},
url = {https://link.springer.com/content/pdf/10.1007/s42979-026-05076-4.pdf},
volume = 7,
year = {2026},
bdsk-url-1 = {https://link.springer.com/content/pdf/10.1007/s42979-026-05076-4.pdf},
bdsk-url-2 = {https://doi.org/10.1007/s42979-026-05076-4}}
Alberto Mulone, Doriana Medić, Iacopo Colonnelli, Marco Aldinucci
Future Generation Computer Systems
eupex icsc space streamflow
In large-scale distributed systems, failures are routine events whose occurrences increase with the number of computational tasks and execution locations. The advantage of representing an application as a workflow is the possibility of exploiting Workflow Management System (WMS) features such as portability, scalability, and, crucially, reliability. Among these, reliability is essential for ensuring robust execution in dynamic and failure-prone environments. In recent years, the emergence of hybrid workflows has posed new and intriguing challenges by increasing the possibility of distributing computations involving heterogeneous and independent environments. Consequently, the number of possible points of failure during the execution increased, creating a need for sophisticated fault tolerance mechanisms capable of addressing the specific requirements of hybrid systems. This work introduces a formal framework for a fault tolerance mechanism in hybrid workflows, enabling failure recovery through a rollback approach. The framework is rigorously defined by adapting and extending an existing workflow semantics tailored for hybrid execution. Our method leverages provenance data from workflow execution up to the point of failure, and creates a recovery workflow that spans multiple infrastructures. The rollback approach provides a robust and reliable strategy to ensure resilience against step failures and potential data loss. We then implement this mechanism in the StreamFlow WMS, and evaluate it using two case studies: the 1000 Genomes workflow and a synthetic workflow featuring iterative patterns. Experiments showcase the conceptual validity of our approach and assess the overhead introduced by the mechanism, including data availability checks.
@article{25:fgcs:ft,
abstract = {In large-scale distributed systems, failures are routine events whose occurrences increase with the number of computational tasks and execution locations. The advantage of representing an application as a workflow is the possibility of exploiting Workflow Management System (WMS) features such as portability, scalability, and, crucially, reliability. Among these, reliability is essential for ensuring robust execution in dynamic and failure-prone environments. In recent years, the emergence of hybrid workflows has posed new and intriguing challenges by increasing the possibility of distributing computations involving heterogeneous and independent environments. Consequently, the number of possible points of failure during the execution increased, creating a need for sophisticated fault tolerance mechanisms capable of addressing the specific requirements of hybrid systems. This work introduces a formal framework for a fault tolerance mechanism in hybrid workflows, enabling failure recovery through a rollback approach. The framework is rigorously defined by adapting and extending an existing workflow semantics tailored for hybrid execution. Our method leverages provenance data from workflow execution up to the point of failure, and creates a recovery workflow that spans multiple infrastructures. The rollback approach provides a robust and reliable strategy to ensure resilience against step failures and potential data loss. We then implement this mechanism in the StreamFlow WMS, and evaluate it using two case studies: the 1000 Genomes workflow and a synthetic workflow featuring iterative patterns. Experiments showcase the conceptual validity of our approach and assess the overhead introduced by the mechanism, including data availability checks.},
author = {Alberto Mulone and Doriana Medi\'{c} and Iacopo Colonnelli and Marco Aldinucci},
doi = {10.1016/j.future.2025.108188},
issn = {0167-739X},
journal = {{F}uture {G}eneration {C}omputer {S}ystems},
keywords = {icsc, eupex, space, streamflow},
pages = {108188},
title = {A formal framework for fault tolerance in hybrid scientific workflows},
url = {https://iris.unito.it/retrieve/b9adc822-4455-4fe1-a8c1-8a66c084335e/1-s2.0-S0167739X25004820-main.pdf},
volume = {176},
year = {2026},
bdsk-url-1 = {https://iris.unito.it/retrieve/b9adc822-4455-4fe1-a8c1-8a66c084335e/1-s2.0-S0167739X25004820-main.pdf},
bdsk-url-2 = {https://doi.org/10.1016/j.future.2025.108188}}
Iacopo Colonnelli, Doriana Medić, Alberto Mulone, Viviana Bono, Luca Padovani, Marco Aldinucci
Formal Methods. FM 2024
eupex icsc
In the ever-evolving landscape of scientific computing, properly supporting the modularity and complexity of modern scientific applications requires new approaches to workflow execution, like seamless interoperability between different workflow systems, distributed-by-design workflow models, and automatic optimisation of data movements. In order to address this need, this article introduces SWIRL, an intermediate representation language for scientific workflows. In contrast with other product-agnostic workflow languages, SWIRL is not designed for human interaction but to serve as a low-level compilation target for distributed workflow execution plans. The main advantages of SWIRL semantics are low-level primitives based on the send/receive programming model and a formal framework ensuring the consistency of the semantics and the specification of translating workflow models represented by Directed Acyclic Graphs (DAGs) into SWIRL workflow descriptions. Additionally, SWIRL offers rewriting rules designed to optimise execution traces, accompanied by corresponding equivalence. An open-source SWIRL compiler toolchain has been developed using the ANTLR Python3 bindings.
@inproceedings{24:fm:swirl,
abstract = {In the ever-evolving landscape of scientific computing, properly supporting the modularity and complexity of modern scientific applications requires new approaches to workflow execution, like seamless interoperability between different workflow systems, distributed-by-design workflow models, and automatic optimisation of data movements. In order to address this need, this article introduces SWIRL, an intermediate representation language for scientific workflows. In contrast with other product-agnostic workflow languages, SWIRL is not designed for human interaction but to serve as a low-level compilation target for distributed workflow execution plans. The main advantages of SWIRL semantics are low-level primitives based on the send/receive programming model and a formal framework ensuring the consistency of the semantics and the specification of translating workflow models represented by Directed Acyclic Graphs (DAGs) into SWIRL workflow descriptions. Additionally, SWIRL offers rewriting rules designed to optimise execution traces, accompanied by corresponding equivalence. An open-source SWIRL compiler toolchain has been developed using the ANTLR Python3 bindings.},
address = {Milano, Italy},
author = {Iacopo Colonnelli and Doriana Medi\'{c} and Alberto Mulone and Viviana Bono and Luca Padovani and Marco Aldinucci},
booktitle = {Formal Methods. FM 2024},
doi = {10.1007/978-3-031-71162-6_12},
editor = {Andr{\'e} Platzer and Kristin Yvonne Rozier and Matteo Pradella and Matteo Rossi},
keywords = {icsc, eupex},
month = sep,
pages = {226--244},
publisher = {Springer Nature Switzerland},
series = {Lecture Notes in Computer Science},
title = {Introducing {SWIRL}: An Intermediate Representation Language for Scientific Workflows},
url = {https://iris.unito.it/retrieve/b39a6f09-a8d3-4974-abf6-c109916694fa/PDFEditoriale.pdf},
volume = {14933},
year = {2025},
bdsk-url-1 = {https://iris.unito.it/retrieve/b39a6f09-a8d3-4974-abf6-c109916694fa/PDFEditoriale.pdf},
bdsk-url-2 = {https://doi.org/10.1007/978-3-031-71162-6_12}}
1st workshop about High-Performance e-Science (HiPES)
eupex icsc streamflow
In large distributed systems, failures are a daily event occurring frequently, especially with growing numbers of computation tasks and locations on which they are deployed. The advantage of representing an application as a workflow is possibility to utilize the Workflow Management Systems which are reliable systems guaranteeing the correct execution of the application and providing the features such as portability, scalability, and fault tolerance. Over recent years, the emergence of hybrid workflows has posed new and intriguing challenges by increasing the possibility of distributing computations involving heterogeneous and independent environments. As a consequence, the number of possible points of failure in the execution augmented, creating different important challenges interesting to study.
@inproceedings{24:madrid:hipes,
abstract = {In large distributed systems, failures are a daily event occurring frequently, especially with growing numbers of computation tasks and locations on which they are deployed. The advantage of representing an application as a workflow is possibility to utilize the Workflow Management Systems which are reliable systems guaranteeing the correct execution of the application and providing the features such as portability, scalability, and fault tolerance. Over recent years, the emergence of hybrid workflows has posed new and intriguing challenges by increasing the possibility of distributing computations involving heterogeneous and independent environments. As a consequence, the number of possible points of failure in the execution augmented, creating different important challenges interesting to study.},
address = {Madrid, Spain},
annote = {accepted},
author = {Alberto Mulone and Doriana Medi\'{c} and Marco Aldinucci},
booktitle = {1st workshop about {H}igh-{P}erformance e-{S}cience (HiPES)},
doi = {10.1007/978-3-031-90203-1_13},
editor = {Caino-Lores, Silvina and Zeinalipour, Demetris and Doudali, Thaleia Dimitra and Singh, David E. and Garz{\'o}n, Gracia Ester Mart{\'\i}n and Sousa, Leonel and Andrade, Diego and Cucinotta, Tommaso and D'Ambrosio, Donato and Diehl, Patrick and Dolz, Manuel F. and Jukan, Admela and Montella, Raffaele and Nardelli, Matteo and Garcia-Gasulla, Marta and Neuwirth, Sarah},
isbn = {978-3-031-90203-1},
keywords = {streamflow, icsc, eupex},
month = aug,
pages = {141--153},
publisher = {Springer Nature Switzerland},
title = {A Fault Tolerance mechanism for Hybrid Scientific Workflows},
volume = {15386},
year = {2024},
bdsk-url-1 = {https://doi.org/10.1007/978-3-031-90203-1_13}}
Iacopo Colonnelli, Robert Birke, Giulio Malenza, Gianluca Mittone, Alberto Mulone, Jeroen Galjaard, Lydia Y. Chen, Sanzio Bassini, Gabriella Scipione, Jan Martinovič, Vit Vondrák, Marco Aldinucci
Procedia Computer Science
fl hpc icsc space streamflow
In a decade, AI frontier research transitioned from the researcher's workstation to thousands of high-end hardware-accelerated compute nodes. This rapid evolution shows no signs of slowing down in the foreseeable future. While top cloud providers may be able to keep pace with this growth rate, obtaining and efficiently exploiting computing resources at that scale is a daunting challenge for universities and SMEs. This work introduces the Cross-Facility Federated Learning (XFFL) framework to bridge this compute divide, extending the opportunity to efficiently exploit multiple independent data centres for extreme-scale deep learning tasks to data scientists and domain experts. XFFL relies on hybrid workflow abstractions to decouple tasks from environment-specific technicalities, reducing complexity and enhancing reusability. In addition, Federated Learning (FL) algorithms eliminate the need to move large amounts of data between different facilities, reducing time-to-solution and preserving data privacy. The XFFL approach is empirically evaluated by training a full LLaMAv2 7B instance on two facilities of the EuroHPC JU, showing how the increased computing power completely compensates for the additional overhead introduced by two data centres.
@article{24:eurohpc:xffl,
abstract = {In a decade, AI frontier research transitioned from the researcher's workstation to thousands of high-end hardware-accelerated compute nodes. This rapid evolution shows no signs of slowing down in the foreseeable future. While top cloud providers may be able to keep pace with this growth rate, obtaining and efficiently exploiting computing resources at that scale is a daunting challenge for universities and SMEs. This work introduces the Cross-Facility Federated Learning (XFFL) framework to bridge this compute divide, extending the opportunity to efficiently exploit multiple independent data centres for extreme-scale deep learning tasks to data scientists and domain experts. XFFL relies on hybrid workflow abstractions to decouple tasks from environment-specific technicalities, reducing complexity and enhancing reusability. In addition, Federated Learning (FL) algorithms eliminate the need to move large amounts of data between different facilities, reducing time-to-solution and preserving data privacy. The XFFL approach is empirically evaluated by training a full LLaMAv2 7B instance on two facilities of the EuroHPC JU, showing how the increased computing power completely compensates for the additional overhead introduced by two data centres.},
address = {Bruxelles, Belgium},
author = {Colonnelli, Iacopo and Birke, Robert and Malenza, Giulio and Mittone, Gianluca and Mulone, Alberto and Galjaard, Jeroen and Chen, Lydia Y. and Bassini, Sanzio and Scipione, Gabriella and Martinovi\v{c}, Jan and Vondr\'{a}k, Vit and Aldinucci, Marco},
booktitle = {Proceedings of the First EuroHPC user day},
date-modified = {2025-09-23 17:53:31 +0200},
doi = {10.1016/j.procs.2024.07.003},
issn = {1877-0509},
journal = {Procedia Computer Science},
keywords = {streamflow, space, icsc, fl, hpc},
pages = {3--12},
publisher = {Elsevier},
title = {Cross-Facility Federated Learning},
url = {https://www.sciencedirect.com/science/article/pii/S1877050924016909},
volume = {240},
year = {2024},
bdsk-url-1 = {https://www.sciencedirect.com/science/article/pii/S1877050924016909},
bdsk-url-2 = {https://doi.org/10.1016/j.procs.2024.07.003}}
Iacopo Colonnelli, Doriana Medic, Alberto Mulone, Viviana Bono, Luca Padovani, Marco Aldinucci
Zenodo
@misc{DBLP:data/11/ColonnelliMMBPA24,
author = {Iacopo Colonnelli and
Doriana Medic and
Alberto Mulone and
Viviana Bono and
Luca Padovani and
Marco Aldinucci},
title = {Artifact for paper "Introducing {SWIRL:} An Intermediate Representation
Language for Scientific Workflows" (Version 1)},
publisher = {Zenodo},
year = {2024},
month = jun,
howpublished = {\url{https://doi.org/10.5281/zenodo.12523000}},
note = {Accessed on YYYY-MM-DD.},
url = {https://doi.org/10.5281/zenodo.12523000},
doi = {10.5281/ZENODO.12523000},
timestamp = {Thu, 26 Jun 2025 01:00:00 +0200},
biburl = {https://dblp.org/rec/data/11/ColonnelliMMBPA24.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
Alberto Mulone, Sherine Awad, Davide Chiarugi, Marco Aldinucci
47th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2023
across icsc streamflow
In recent years we have understood the importance of analyzing and sequencing human genetic variation. A relevant aspect that emerged from the Covid-19 pandemic was the need to obtain results very quickly; this involved using High-Performance Computing (HPC) environments to execute the Next Generation Sequencing (NGS) pipeline. However, HPC is not always the most suitable environment for the entire execution of a pipeline, especially when it involves many heterogeneous tools. The ability to execute parts of the pipeline on different environments can lead to higher performance but also cheaper executions. This work shows the design and optimization process that led us to a state-of-the-art Variant Calling hybrid workflow based on the StreamFlow Workflow Management System (WfMS). We also compare StreamFlow with Snakemake, an established WfMS targeting HPC facilities, observing comparable performance on single environments and satisfactory improvements with a hybrid cloud-HPC configuration.
@inproceedings{23:mulone:wide:vcp,
abstract = {In recent years we have understood the importance of analyzing and sequencing human genetic variation. A relevant aspect that emerged from the Covid-19 pandemic was the need to obtain results very quickly; this involved using High-Performance Computing (HPC) environments to execute the Next Generation Sequencing (NGS) pipeline. However, HPC is not always the most suitable environment for the entire execution of a pipeline, especially when it involves many heterogeneous tools. The ability to execute parts of the pipeline on different environments can lead to higher performance but also cheaper executions. This work shows the design and optimization process that led us to a state-of-the-art Variant Calling hybrid workflow based on the StreamFlow Workflow Management System (WfMS). We also compare StreamFlow with Snakemake, an established WfMS targeting HPC facilities, observing comparable performance on single environments and satisfactory improvements with a hybrid cloud-HPC configuration.},
address = {Torino, Italy},
author = {Alberto Mulone and Sherine Awad and Davide Chiarugi and Marco Aldinucci},
booktitle = {47th {IEEE} Annual Computers, Software, and Applications Conference, {COMPSAC} 2023},
doi = {10.1109/COMPSAC57700.2023.00288},
editor = {Hossain Shahriar and Yuuichi Teranishi and Alfredo Cuzzocrea and Moushumi Sharmin and Dave Towey and A. K. M. Jahangir Alam Majumder and Hiroki Kashiwazaki and Ji{-}Jiang Yang and Michiharu Takemoto and Nazmus Sakib and Ryohei Banno and Sheikh Iqbal Ahamed},
keywords = {across, icsc, streamflow},
pages = {1858--1863},
publisher = {{IEEE}},
title = {Porting the {V}ariant {C}alling {P}ipeline for {NGS} data in cloud-{HPC} environment},
url = {https://iris.unito.it/bitstream/2318/1919364/1/paper.pdf},
year = {2023},
bdsk-url-1 = {https://iris.unito.it/bitstream/2318/1919364/1/paper.pdf},
bdsk-url-2 = {https://doi.org/10.1109/COMPSAC57700.2023.00288}}
Giulio Malenza, Alberto Mulone, Daniele Ibba, Marco Edoardo Santimaria, Lorenzo Bosio, Iacopo Colonnelli, Robert Birke, Marco Aldinucci
icsc
Computational fluid dynamics (CFD) simulations based on OpenFOAM are widely adopted in both academic and industrial settings. However, their execution typically relies on rigid, monolithic bash scripts that limit portability, scalability, and efficient exploitation of heterogeneous computing resources. This work introduces a workflow-oriented execution model for OpenFOAM that enables more flexible simulation orchestration and improved utilization of heterogeneous architectures. As a case study, we reimplement the well-known OpenFOAM Motorbike tutorial—a benchmark for steady incompressible Reynolds-averaged Navier–Stokes (RANS) simulations of airflow around a motorcycle and rider. The original tutorial follows a strictly sequential execution model defined in a single Allrun script, encompassing mesh generation, parallel decomposition, solver execution, and result reconstruction. We translate this bash-based pipeline into the Common Workflow Language (CWL), a widely adopted, implementation-agnostic workflow specification, and execute it using StreamFlow. This workflow management system supports modular execution, flexible orchestration, and improved resource utilization across heterogeneous platforms. The original execution steps—ranging from environment setup and mesh preparation to parallel solver runs and post-processing—are reorganized into discrete CWL workflow components. The workflow is structured into three main stages: (i) prep, responsible for mesh preparation; (ii) decompose, handling mesh decomposition and refinement; and (iii) foam_run, executing potential flow initialization, the main solver, and mesh reconstruction. Each stage is encapsulated in reusable CWL components, enabling clear separation of concerns and simplified workflow reconfiguration. The resulting workflow is executed on CINECA systems using the official OpenFOAM-v2412 distribution deployed via Docker/Singularity. This study demonstrates how traditional OpenFOAM tutorials can be transformed into portable, reproducible, and extensible workflows. By decoupling simulation phases and explicitly exposing orchestration strategies, the proposed approach lays the groundwork for systematic investigation of performance portability and scalability on heterogeneous and distributed computing environments.
@misc{26:malenza:Workflow_OpenFOAM,
abstract = {Computational fluid dynamics (CFD) simulations based on OpenFOAM are widely adopted in both academic and industrial settings. However, their execution typically relies on rigid, monolithic bash scripts that limit portability, scalability, and efficient exploitation of heterogeneous computing resources. This work introduces a workflow-oriented execution model for OpenFOAM that enables more flexible simulation orchestration and improved utilization of heterogeneous architectures. As a case study, we reimplement the well-known OpenFOAM Motorbike tutorial---a benchmark for steady incompressible Reynolds-averaged Navier--Stokes (RANS) simulations of airflow around a motorcycle and rider. The original tutorial follows a strictly sequential execution model defined in a single Allrun script, encompassing mesh generation, parallel decomposition, solver execution, and result reconstruction. We translate this bash-based pipeline into the Common Workflow Language (CWL), a widely adopted, implementation-agnostic workflow specification, and execute it using StreamFlow. This workflow management system supports modular execution, flexible orchestration, and improved resource utilization across heterogeneous platforms. The original execution steps---ranging from environment setup and mesh preparation to parallel solver runs and post-processing---are reorganized into discrete CWL workflow components. The workflow is structured into three main stages: (i) prep, responsible for mesh preparation; (ii) decompose, handling mesh decomposition and refinement; and (iii) foam_run, executing potential flow initialization, the main solver, and mesh reconstruction. Each stage is encapsulated in reusable CWL components, enabling clear separation of concerns and simplified workflow reconfiguration. The resulting workflow is executed on CINECA systems using the official OpenFOAM-v2412 distribution deployed via Docker/Singularity. This study demonstrates how traditional OpenFOAM tutorials can be transformed into portable, reproducible, and extensible workflows. By decoupling simulation phases and explicitly exposing orchestration strategies, the proposed approach lays the groundwork for systematic investigation of performance portability and scalability on heterogeneous and distributed computing environments.},
address = {Bologna, IT},
author = {Giulio Malenza and Alberto Mulone and Daniele Ibba and Marco Edoardo Santimaria and Lorenzo Bosio and Iacopo Colonnelli and Robert Birke and Marco Aldinucci},
date-modified = {2026-09-19 19:46:55 +0200},
keywords = {icsc},
month = mar,
title = {Exploring workflow solutions on the {OpenFOAM} framework},
url = {https://datacloud.di.unito.it/index.php/s/HwXHg7GicwHcx59},
year = 2026,
bdsk-url-1 = {https://datacloud.di.unito.it/index.php/s/HwXHg7GicwHcx59}}
Gianluca Mittone, Alberto Mulone, Iacopo Colonnelli, Robert Birke, Marco Aldinucci
eupilot fl icsc space streamflow
Big-tech companies pre-train SOTA LLMs on special-purpose, private HPCs, while public research centres lack the resources to compete. We advocate a new take on large model training, e.g., LLMs, called xFFL, which leverages federated learning as an enabling technique to exploit geographically distributed computing power to bridge such digital divide. This work introduces a proof-of-concept federated training of LLaMA-3 8B on three EuroHPC Top500 facilities, proving the viability of leveraging cross-facility publicly available computational power to sustain SOTA LLM workloads.
@misc{25:cineca:xffl,
abstract = {Big-tech companies pre-train SOTA LLMs on special-purpose, private HPCs, while public research centres lack the resources to compete. We advocate a new take on large model training, e.g., LLMs, called xFFL, which leverages federated learning as an enabling technique to exploit geographically distributed computing power to bridge such digital divide. This work introduces a proof-of-concept federated training of LLaMA-3 8B on three EuroHPC Top500 facilities, proving the viability of leveraging cross-facility publicly available computational power to sustain SOTA LLM workloads.},
address = {Bologna, Italy},
author = {Mittone, Gianluca and Mulone, Alberto and Colonnelli, Iacopo and Birke, Robert and Aldinucci, Marco},
howpublished = {AI & HPC User Forums and HPC User Day},
keywords = {icsc, eupilot, space, fl, streamflow},
month = mar,
title = {Cross-Facility Federated Learning},
url = {https://datacloud.di.unito.it/index.php/s/oXoeHLytc9CZ5KR},
year = 2025,
bdsk-url-1 = {https://datacloud.di.unito.it/index.php/s/oXoeHLytc9CZ5KR}}
Iacopo Colonnelli, Robert Birke, Giulio Malenza, Gianluca Mittone, Alberto Mulone, Marco Aldinucci
eupex icsc space
@misc{24:ic:elise:xffl,
address = {Helsinki, Finland},
author = {Iacopo Colonnelli and Robert Birke and Giulio Malenza and Gianluca Mittone and Alberto Mulone and Marco Aldinucci},
keywords = {icsc,space,eupex},
month = jun,
note = {Invited talk},
title = {Cross-Facility Federated Learning - Part II},
url = {https://datacloud.di.unito.it/index.php/s/7HonBpcWPxotXLX},
year = 2024,
bdsk-url-1 = {https://datacloud.di.unito.it/index.php/s/7HonBpcWPxotXLX}}
In large distributed systems, failures are a daily event occurring frequently, especially with growing numbers of computation tasks and locations on which they are deployed. The advantage of representing an application as a workflow is possibility to utilize the Workflow Management Systems which are reliable systems guaranteeing the correct execution of the application and providing the features such as portability, scalability, and fault tolerance. Over recent years, the emergence of hybrid workflows has posed new and intriguing challenges by increasing the possibility of distributing computations involving heterogeneous and independent environments. As a consequence, the number of possible points of failure in the execution augmented, creating different important challenges interesting to study.
@misc{24:madrid:hipes:talk,
abstract = {In large distributed systems, failures are a daily event occurring frequently, especially with growing numbers of computation tasks and locations on which they are deployed. The advantage of representing an application as a workflow is possibility to utilize the Workflow Management Systems which are reliable systems guaranteeing the correct execution of the application and providing the features such as portability, scalability, and fault tolerance. Over recent years, the emergence of hybrid workflows has posed new and intriguing challenges by increasing the possibility of distributing computations involving heterogeneous and independent environments. As a consequence, the number of possible points of failure in the execution augmented, creating different important challenges interesting to study.},
address = {Madrid, Spain},
author = {Alberto Mulone and Doriana Medi\'{c} and Marco Aldinucci},
howpublished = {1st workshop about {H}igh-{P}erformance e-{S}cience (HiPES)},
keywords = {streamflow, icsc, eupex},
month = aug,
title = {A Fault Tolerance mechanism for Hybrid Scientific Workflows},
url = {https://datacloud.di.unito.it/index.php/s/9Ddj6fGgmDbLDXj},
year = 2024,
bdsk-url-1 = {https://datacloud.di.unito.it/index.php/s/9Ddj6fGgmDbLDXj}}
Gianluca Mittone, Alberto Mulone, Iacopo Colonnelli, Robert Birke, Marco Aldinucci
eupilot fl icsc space streamflow
Big-tech companies pre-train SOTA LLMs on special-purpose, private HPCs, while public research centres lack the resources to compete. We advocate a new take on large model training, e.g., LLMs, called xFFL, which leverages federated learning as an enabling technique to exploit geographically distributed computing power to bridge such digital divide. This work introduces a proof-of-concept federated training of LLaMA-3 8B on three EuroHPC Top500 facilities, proving the viability of leveraging cross-facility publicly available computational power to sustain SOTA LLM workloads.
@misc{24:mittone:TPC,
abstract = {Big-tech companies pre-train SOTA LLMs on special-purpose, private HPCs, while public research centres lack the resources to compete. We advocate a new take on large model training, e.g., LLMs, called xFFL, which leverages federated learning as an enabling technique to exploit geographically distributed computing power to bridge such digital divide. This work introduces a proof-of-concept federated training of LLaMA-3 8B on three EuroHPC Top500 facilities, proving the viability of leveraging cross-facility publicly available computational power to sustain SOTA LLM workloads.},
address = {Atlanta, GE, USA},
author = {Mittone, Gianluca and Mulone, Alberto and Colonnelli, Iacopo and Birke, Robert and Aldinucci, Marco},
howpublished = {Accelerating the Development and Use of Generative AI for Science and Engineering: The Trillion Parameter Consortium ({TPC})},
keywords = {icsc, eupilot, space, fl, streamflow},
month = nov,
title = {Enabling Cross-Facility LLMs Pre-Training},
url = {https://datacloud.di.unito.it/index.php/s/DRgm8ebBkKQgD2d},
year = 2024,
bdsk-url-1 = {https://datacloud.di.unito.it/index.php/s/DRgm8ebBkKQgD2d}}
Alberto Mulone, Sherine Awad, Davide Chiarugi, Marco Aldinucci
across icsc streamflow
In recent years we have understood the importance of analyzing and sequencing human genetic variation. A relevant aspect that emerged from the Covid-19 pandemic was the need to obtain results very quickly; this involved using High-Performance Computing (HPC) environments to execute the Next Generation Sequencing (NGS) pipeline. However, HPC is not always the most suitable environment for the entire execution of a pipeline, especially when it involves many heterogeneous tools. The ability to execute parts of the pipeline on different environments can lead to higher performance but also cheaper executions. This work shows the design and optimization process that led us to a state-of-the-art Variant Calling hybrid workflow based on the StreamFlow Workflow Management System (WfMS). We also compare StreamFlow with Snakemake, an established WfMS targeting HPC facilities, observing comparable performance on single environments and satisfactory improvements with a hybrid cloud-HPC configuration.
@misc{23:mulone:wide:talk,
abstract = {In recent years we have understood the importance of analyzing and sequencing human genetic variation. A relevant aspect that emerged from the Covid-19 pandemic was the need to obtain results very quickly; this involved using High-Performance Computing (HPC) environments to execute the Next Generation Sequencing (NGS) pipeline. However, HPC is not always the most suitable environment for the entire execution of a pipeline, especially when it involves many heterogeneous tools. The ability to execute parts of the pipeline on different environments can lead to higher performance but also cheaper executions. This work shows the design and optimization process that led us to a state-of-the-art Variant Calling hybrid workflow based on the StreamFlow Workflow Management System (WfMS). We also compare StreamFlow with Snakemake, an established WfMS targeting HPC facilities, observing comparable performance on single environments and satisfactory improvements with a hybrid cloud-HPC configuration.},
address = {Torino, Italy},
author = {Alberto Mulone and Sherine Awad and Davide Chiarugi and Marco Aldinucci},
howpublished = {47th {IEEE} Annual Computers, Software, and Applications Conference, {COMPSAC} 2023},
keywords = {across, icsc, streamflow},
month = jun,
title = {Porting the {V}ariant {C}alling {P}ipeline for {NGS} data in cloud-{HPC} environment},
url = {https://datacloud.di.unito.it/index.php/s/zNLj3LCZNsNxHwy},
year = 2023,
bdsk-url-1 = {https://datacloud.di.unito.it/index.php/s/zNLj3LCZNsNxHwy}}