Talks | Parallel Computing
2024
Robert Birke
The impact of the advances in generative models on applications and systems Miscellaneous
8th GDR RSD / ASF Winter School on Distributed Systems & Networks 2024, 2024, (Keynote talk).
Abstract | Links | BibTeX | Tags: ai, eupilot, textarossa
@misc{24:ASF:WINTER,
title = {The impact of the advances in generative models on applications and systems},
author = {Robert Birke},
url = {https://datacloud.di.unito.it/index.php/s/QYTCMfWp4sY5qx4},
year = {2024},
date = {2024-01-01},
address = {Le Pleynet, France},
abstract = {Generative models have achieved unprecedented quality levels across a wide range of data types. This advance often stems from the ever increasing data and compute used to train larger and larger models. One major use case of such synthetic data is in privacy-compliant data sharing. Gartner predicts that synthetic data will reduce by 2025 the need for real data by 70% for analytics and machine learning. We will look at generative models, with a special focus on tabular data, and the issue of democratization of large model training.},
howpublished = {8th GDR RSD / ASF Winter School on Distributed Systems & Networks 2024},
note = {Keynote talk},
keywords = {ai, eupilot, textarossa},
pubstate = {published},
tppubtype = {misc}
}
Generative models have achieved unprecedented quality levels across a wide range of data types. This advance often stems from the ever increasing data and compute used to train larger and larger models. One major use case of such synthetic data is in privacy-compliant data sharing. Gartner predicts that synthetic data will reduce by 2025 the need for real data by 70% for analytics and machine learning. We will look at generative models, with a special focus on tabular data, and the issue of democratization of large model training.
2023
Bruno Casella, Samuele Fonio
Architecture-Based FedAvg for Vertical Federated Learning Miscellaneous
2023, (https://iris.unito.it/bitstream/2318/1949730/1/HALF_HVL_for_DML_ICC23___Taormina-2.pdf).
Abstract | Links | BibTeX | Tags: ai, epi, fl, icsc
@misc{23:casella:architecturalfedavg,
title = {Architecture-Based FedAvg for Vertical Federated Learning},
author = {Bruno Casella and Samuele Fonio},
url = {https://datacloud.di.unito.it/index.php/s/kJQxnqG4d2ZSicK},
doi = {10.1109/ICCVW60793.2023.00362},
year = {2023},
date = {2023-01-01},
booktitle = {Proceedings of the 3rd Workshop on Distributed Machine Learning for the Intelligent Computing Continuum (DML-ICC), IEEE/ACM UCC 2023, Taormina, Italy, 4 December 2023},
abstract = {Federated Learning (FL) has emerged as a promising solution to address privacy concerns by collaboratively training Deep Learning (DL) models across distributed parties. This work proposes an architecture-based aggregation strategy in Vertical FL, where parties hold data with different attributes but shared instances. Our approach leverages the identical architectural parts, i.e. neural network layers, of different models to selectively aggregate weights, which is particularly relevant when collaborating with institutions holding different types of datasets, i.e., image, text, or tabular datasets. In a scenario where two entities train DL models, such as a Convolutional Neural Network (CNN) and a Multi-Layer Perceptron (MLP), our strategy computes the average only for architecturally identical segments. This preserves data-specific features learned from demographic and clinical data. We tested our approach on two clinical datasets, i.e., the COVID-CXR dataset and the ADNI study. Results show that our method achieves comparable results with the centralized scenario, in which all the data are collected in a single data lake, and benefits from FL generalizability. In particular, compared to the non-federated models, our proposed proof-of-concept model exhibits a slight performance loss on the COVID-CXR dataset (less than 8%), but outperforms ADNI models by up to 12%. Moreover, communication costs between training rounds are minimized by exchanging only the dense layer parameters.},
note = {https://iris.unito.it/bitstream/2318/1949730/1/HALF_HVL_for_DML_ICC23___Taormina-2.pdf},
keywords = {ai, epi, fl, icsc},
pubstate = {published},
tppubtype = {misc}
}
Federated Learning (FL) has emerged as a promising solution to address privacy concerns by collaboratively training Deep Learning (DL) models across distributed parties. This work proposes an architecture-based aggregation strategy in Vertical FL, where parties hold data with different attributes but shared instances. Our approach leverages the identical architectural parts, i.e. neural network layers, of different models to selectively aggregate weights, which is particularly relevant when collaborating with institutions holding different types of datasets, i.e., image, text, or tabular datasets. In a scenario where two entities train DL models, such as a Convolutional Neural Network (CNN) and a Multi-Layer Perceptron (MLP), our strategy computes the average only for architecturally identical segments. This preserves data-specific features learned from demographic and clinical data. We tested our approach on two clinical datasets, i.e., the COVID-CXR dataset and the ADNI study. Results show that our method achieves comparable results with the centralized scenario, in which all the data are collected in a single data lake, and benefits from FL generalizability. In particular, compared to the non-federated models, our proposed proof-of-concept model exhibits a slight performance loss on the COVID-CXR dataset (less than 8%), but outperforms ADNI models by up to 12%. Moreover, communication costs between training rounds are minimized by exchanging only the dense layer parameters.