Available Thesis (Bachelor and MSc) UNITO 🇮🇹🏴󠁧󠁢󠁥󠁮󠁧󠁿

Referents: Marco Aldinucci, Bruno Casella (bruno.casella@unito.it)

Keywords: federated learning, decentralized learning, continual learning, experience replay, medical imaging

Abstract: Federated learning encompasses a family of strategies for distributed training over multiple nodes, each with its own private dataset, which typically communicates with a central node by sending local model updates used to train the main model. In this scenario, no data is explicitly shared between nodes, thus addressing the required privacy issues.
However, the presence of a central node that aggregates local updates simplifies the communication protocol when the number of clients is very large (thousands or millions) but introduces several downsides: it represents a single point of failure; it can become a bottleneck when the number of clients increases; in general, it may not always be available or desirable in collaborative learning scenarios.
In decentralized federated learning, the central node is replaced by peer-to-peer communication between clients: there is no longer a global shared model as in standard FL, but the communication protocol is designed so that all local models approximately converge to the same solution. Decentralized learning is particularly suitable for application in the medical domain, where the number of nodes (i.e., institutions) is relatively low; however, research is still ongoing, and no effective solutions have been established.

Task: we aim to investigate some unexplored properties of the FedER algorithm, a decentralized federated learning strategy based on continual learning principles designed for medical imaging data, which outperforms server-based federated learning approaches and yields performance similar to a standard (non-federated) training settings. In particular, the goal of this thesis is to explore the possibility of model heterogeneity in FedER. Unlike all other existing methods based on parameter averaging, the FedER approach does not strictly require that all nodes share the same model architecture. Model heterogeneity could, therefore, be employed to create a shared ensemble and combine different feature learning capabilities.

Bibliography:
– McMahan B., et al., “Communication-Efficient Learning of Deep Networks from Decentralized Data”, https://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017
– Pennisi M., et al. “FedER: Federated Learning through Experience Replay and privacy-preserving data synthesis”, Computer Vision and Image Understanding (CVIU), 2024, https://www.sciencedirect.com/science/article/pii/S107731422300262X

Workload: Under the guidance of a team of researchers, the student will first be required to study recent works on the subject before being practically introduced to the task by experimenting with existing code.
Literature: 20%
Code implementation: 50%
Experiments: 15%
Working towards a publication: 15%

Requirements: The candidate should have good programming skills (Python and PyTorch) and be motivated. A previous knowledge of the topic would be preferred but not required.

Referents: Marco Aldinucci, Bruno Casella (bruno.casella@unito.it)

Keywords: federated learning, video classification, resource constraints

Abstract: Video classification is a computationally intensive task due to the large amount of data and complex processing involved. Each video consists of multiple frames, often in high resolution, and each frame must be analyzed both individually and in the context of temporal dynamics. This requires significant memory and processing power, especially when working with deep learning models like 3D convolutional networks or transformer-based architectures that process both spatial and temporal information. The need to handle long sequences of frames and the variety of actions or events within videos further increase computational demands. Developing methods that require fewer resources, such as lightweight models or efficient pretraining techniques, could make video classification more accessible and reduce the strain on computational infrastructure while still maintaining strong performance. ROCKET (RandOm Convolutional KErnel Transform) is a framework designed for time series classification that stands out for its efficiency and low computational requirements. Unlike traditional deep learning models, ROCKET uses random convolutional kernels to extract features from time series data, followed by a simple linear classifier, such as logistic regression, to perform classification. This approach is both fast and scalable, as it eliminates the need for complex training procedures while still achieving state-of-the-art performance on many tasks. Given its lightweight nature and ability to handle large datasets efficiently, ROCKET could be an ideal solution for video classification tasks, particularly when computational resources are limited.

Task: By treating each video frame or sequence of frames as a time series, ROCKET’s efficient feature extraction can be adapted to video data, potentially reducing the high resource demands typically associated with deep learning-based video classification models. This makes it a promising method for scenarios where minimizing computational cost is crucial.
Two possible solutions:
– adapting ROCKET kernels from time series to videos: this means modifying ROCKET kernels, originally developed to deal with 1D arrays, in such a way they can manage 3D structures like videos.
– converting 3D signals to 1D arrays: this can be done by extracting some info (mean, median, activations) from each frame of the video, and merging them to create a time series.
A federated learning approach can also be applied.

Bibliography:
– Dempster A., “ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels”, Data Mining and Knowledge Discovery, 2020, https://link.springer.com/article/10.1007/s10618-020-00701-z

Workload: Under the guidance of a team of researchers, the student will first be required to study recent works on the subject before being practically introduced to the task by experimenting with existing code.
Literature: 20%
Code implementation: 50%
Experiments: 15%
Working towards a publication: 15%

Requirements: The candidate should have good programming skills (Python and PyTorch) and be motivated. A previous knowledge of the topic would be preferred but not required.

Referents: Marco Aldinucci, Bruno Casella (bruno.casella@unito.it)

Keywords: federated learning, decentralized learning

Abstract: Federated learning encompasses a family of strategies for distributed training over multiple nodes, each with its own private dataset, which typically communicates with a central node by sending local model updates used to train the main model. In this scenario, no data is explicitly shared between nodes, thus addressing the required privacy issues.
However, the presence of a central node that aggregates local updates simplifies the communication protocol when the number of clients is very large (thousands or millions) but introduces several downsides: it represents a single point of failure; it can become a bottleneck when the number of clients increases; in general, it may not always be available or desirable in collaborative learning scenarios.
In decentralized federated learning, the central node is replaced by peer-to-peer communication between clients: there is no longer a global shared model as in standard FL, but the communication protocol is designed so that all local models approximately converge to the same solution. Decentralized learning is particularly suitable for application in the medical domain, where the number of nodes (i.e., institutions) is relatively low; however, research is still ongoing, and no effective solutions have been established.

Task: we aim to investigate some unexplored properties of the FedER algorithm, a decentralized federated learning strategy based on continual learning principles designed for medical imaging data, which outperforms server-based federated learning approaches and yields performance similar to a standard (non-federated) training settings. FedER, by leveraging continual learning and generative adversarial concepts proposes
a principled way for training local models that approximately converge to the same decisions, without the need of a shared model architecture and of central coordination. In particular, the convergence is evidenced by empirical results. The goal of this thesis is to provide a theoretical proof of this property.

Bibliography:
– McMahan B., et al., “Communication-Efficient Learning of Deep Networks from Decentralized Data”, https://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017
– Pennisi M., et al., “FedER: Federated Learning through Experience Replay and privacy-preserving data synthesis”, Computer Vision and Image Understanding (CVIU), 2024, https://www.sciencedirect.com/science/article/pii/S107731422300262X
– Li X., et al., “On the Convergence of FedAvg on Non-IID Data”, International Conference on Learning Representations (ICLR), 2020, https://iclr.cc/virtual_2020/poster_HJxNAnVtDS.html

Workload: Under the guidance of a team of researchers, the student will first be required to study recent works on the subject before being practically introduced to the task by experimenting with existing code.
Literature: 20%
Code implementation: 50%
Experiments: 15%
Working towards a publication: 15%

Requirements: The candidate should have high math and good programming skills (Python and PyTorch) and be motivated. A previous knowledge of the topic would be preferred but not required.

Referenti: Marco Aldinucci, Gianluca Mittone (gianluca.mittone@unito.it)

Descrizione: Con il termine Federated Learning (FL) si intende una tecnica di Machine Learning (ML) in cui il processo di apprendimento è effettuato a partire da dati partizionati su più device.

La peculiarità di questo metodo è quella di produrre un unico modello di ML efficace su tali dati senza il bisogno di dover raggruppare quest’ultimi in un solo luogo, con tutta una serie di vantaggi sulla privacy degli utenti e sui costi della gestione dei dati stessi.

Questo approccio, fin’ora utilizzato solo su Deep Neural Networks (DNN), sta venendo espanso dalla comunità scientifica anche a modelli non-deep, quali Alberi di Decisione, modelli Naive Bayes, Regressione Logistica e così via.

In questa tesi verrà esplorato l’algoritmo per non-deep FL AdaBoost.F, sviluppato nel
nostro dipartimento, nella sua implementazione nel framework per FL di Intel OpenFL al fine di indagarne le prestazioni sia di apprendimento sia computazionali.

Referenti: Marco Aldinucci, Alberto Riccardo Martinelli (albertoriccardo.martinelli@unito.it)

Descrizione: Le GPU sono ormai utilizzate in molti campi per risolvere problemi scientifici complessi in modo efficiente.

Scrivere codice portabile ed efficiente è un task notoriamente difficile e nel caso delle gpu questo problema è ancora più marcato. Negli ultimi anni sono stati molti gli sforzi per creare dei tool per facilitare la scrittura di codice parallelo per GPU.

OpenAcc e OpenMP sono I tool maggiormente usati per questo compito.

Il candidato dovrà effettuare uno studio approfondito di questi tool (confrontandoli anche con approcci di più basso livello) per capirne I punti di forza e di debolezza.
Inoltre dovrà scrivere degli esempi con entrambi I tool per avere un confronto completo sia sull’usabilità che sulle prestazioni.
Al candidato verrà fornito Spray-web, un codice scientifico che simula la dispersione di inquinanti in un ambiente 3D. Esistono diverse versioni di questo codice: OpenMP, OpenACC, CUDA e FPGA. Il candidato potrà testare le varie versioni per avere esperienza con un codice reale che gli permetta di avere una comprensione ancora più completa su tali tool.