Proceedings of the 2nd Italian Conference on Big Data and Data Science (ITADATA 2023), Naples, Italy, September 11-13, 2023 — 2023
The SWH-Analytics Framework
Alessia Antelmi, Massimo Torquati, Daniele Gregori, Francesco Polzella, Gianmarco Spinatelli, Marco Aldinucci
Abstract
The Software Heritage (SWH) dataset serves as a vast repository for open-source code, with the ambitious goal of preserving all publicly available open-source projects. Despite being designed to effectively archive project files, its size of nearly 1 petabyte presents challenges in efficiently supporting Big Data MapReduce or AI systems. To address this disparity and enable seamless custom analytics on the SWH dataset, we present the SWH-Analytics (SWHA) architecture. This development environment quickly and transparently runs custom analytic applications on open-source software data preserved over time by SWH.
@inproceedings{Antelmi_ITADATA_2023,
abstract = {The Software Heritage (SWH) dataset serves as a vast repository for open-source code, with the ambitious goal of preserving all publicly available open-source projects. Despite being designed to effectively archive project files, its size of nearly 1 petabyte presents challenges in efficiently supporting Big Data MapReduce or AI systems. To address this disparity and enable seamless custom analytics on the SWH dataset, we present the SWH-Analytics (SWHA) architecture. This development environment quickly and transparently runs custom analytic applications on open-source software data preserved over time by SWH.},
author = {Antelmi, Alessia and Torquati, Massimo and Gregori, Daniele and Polzella, Francesco and Spinatelli, Gianmarco and Aldinucci, Marco},
booktitle = {Proceedings of the 2nd Italian Conference on Big Data and Data Science ({ITADATA} 2023), Naples, Italy, September 11-13, 2023},
editor = {Bena, Nicola and Di Martino, Beniamino and Maratea, Antonio and Sperduti, Alessandro and Di Nardo, Emanuel and Ciaramella, Angelo and Montella, Raffaele and Ardagna, Claudio A.},
keywords = {admire, icsc, analytics},
publisher = {CEUR-WS.org},
series = {{CEUR} Workshop Proceedings},
title = {The SWH-Analytics Framework},
url = {https://ceur-ws.org/Vol-3606/paper76.pdf},
volume = {3606},
year = {2023},
bdsk-url-1 = {https://ceur-ws.org/Vol-3606/paper76.pdf}}