FutureHPC living lab on Software & Integration in Torino
The National Research Centre HPC, Big Data e Quantum Computing
Living lab on HPC Software & Integration (SWI)
Location:
Co-located with the Computer Science Department of the University of Torino, Corso Svizzera 185, Torino.
Full operativity:
June 5, 2023
The long-term goal is to raise industrial readiness levels for adopting advanced digital technologies as an integral part of the innovation process. Meanwhile, we believe that the innovation process is directly linked with the capability of the industry to understand and possibly control the digital stack end-to-end, from hardware selection to development tools to applications, where the most prominent competitive advantage for the industry is innovating is in the lowest levels of the software stack, not in the applications and service layers build on commodity tools. A paradigmatic example is Steve Jobs’s quote from Alan Key when launching the iPhone (2007): “People who are serious about software should make their hardware.” Similarly, We believe people serious about cloud services should make their development tools and system software.
Most of the national and regional industries acquire development tools as services from OTT (such as cloud services for simulation and Big Data management),
restricting themselves to using off-the-shelf technologies and dropping the possibility of making profits in the most profitable business segments, which are not end-users applications but key technologies and tools; they include innovative platforms for trusted and privacy-preserving data analytics and management looking to next-generation edge/distributed systems.
The Software & Integration (SWI) lab aims to build a school for experts on advanced software at the confluence of HPC, AI, and cloud. The primary method is learning by doing. Above all, developing state-of-the-art software tools up to the pre-commercial procurement level, which is (overseas) a crucial element of the value chain of industrial districts growing around solid research centres and universities.
Eventually, the University of Torino supported the physical realization of the SWI living lab with 1.35M€ by way of the funding of the National Research Centre HPC, BigData, and Quantum Computing (NextGenerationEU).
Design goals
The SWI lab design aims to be a national lighthouse to serve as a contamination lab between academia and industry in the area of software for cloud-HPC. Its design revolves around several objectives supporting this goal:
1. Modular
We designed SWI to be move it and reproduce it in other institutions and buildings. Above all, we aim to define a reusable model for innovation.
2. Green
We designed all the devices, both the data centre and the offices, to match the world’s best energy efficiency benchmarks. We selected software-defined systems so that we program them to minimise energy waste (from lighting to data centre).
3. Wellbeing
The working space includes spaces for collaborative work, informal discussions and relaxation, supporting co-creation opportunities. For example, a domotic system dynamically control the indoor temperature and humidity, shift the lighting colour temperature to match the hours of the day and air temperature, and filter the air for viruses and particulates and formaldehyde to preserve and enhance the cognitive performance of the researchers.
4. Attractive
We designed the office to facilitate cooperation but also guarantees space to work in insulation. Therefore, we choose the furniture according to very high-quality design standards and inclusive principles supporting the sustainable vision of the SWI lab.
5. Sustainable
The SWI lab benefits from the Univ. of Torino co-funding for offices (power, networking, etc.) and ICT systems (HPC4AI). It has a clear sustainability path based on public funding and industrial investments. The lab is focused on projects carrying industrial needs and requiring IP developed at the university to 1) trigger a technological transfer and 2) form next-generation industrial researchers with academic tutorship and training.
6. State of the art
The SWI lab invests in state-of-the-art systems and prototypes to provide academic partners and industries with a real testing validation environment for next-generation software running on current and next-generation systems. We designed our systems using our system software components to prove they work. For example, our Openstack cloud is running with zero downtime in 3 years. The SWI systems will be accessible by all FutureHPC spoke partners
SWI lab equipment
The HPC facility is managed as an accelerator of a cloud system serving as a multi-tenant, robust, secure front-end for end-users (and system software developers). HPC4AI exploit an OpenStack cloud (equipped with a K8S PaaS over virtualized resources). We manage the modular HPC system with (an extended version) of SLURM and SPACK, and more in general, we configured or designed all of the software equipping HPC4AI.
- StreamFlow WMS: a cloud-HPC implementation of the Common Workflow Language open standard.
- CAPIO burst-buffers
- JupyterWorkflow and Dossier: an extended JupyterHub for cloud-HPC
- BookedSLURM supports resource booking from web calendars and workflow scheduling.
- OpenDeepHealth is a PaaS for deploying virtualized, secure, private, multitenant instances of K8S (with keycloack, pfsense, capsule).
- Pytorch for RISC-V
The SWI lab benefits from the University of Torino’s existing HPC4AI implements a Tier-3 250KW green data centre (PUE <1.1 – see online monitoring) with 16 racks (22 KW/rack) and a cloud-HPC system described in Fig. 5. We eqiipped the SWI lab with two Dyson air cleaning systems (PM1-10, viruses including Covid-19, formaldehyde, etc.). We monitor HPC4AI and the SWI lab (Fig. 4b) through 6 independent monitoring stations for all common air pollutants listed by World Health Organization (PM 1/2.5/4/10, CO2, tVOC, O3, NH3, CO, NO2, CH2O) with historical time-series publicly available. AWARD360
HPC4AI is a facility designed (in 2018) to experiment with new systems and prototypes in the compute continuum. As defined in the HPC4AI paper, it has been envisioned as a cloud-HPC system to modernise and raise the level of abstraction of HPC computing services (e.g. making them insulated, composable, and declaratively defined).
Industrial contracts & partnerships
In July 2023, the SWI lab attracted six industrial contracts (value 600k€) finalized for the technological transfer of some research products developed by the Parallel Computing group at the University of Torino. The involved industries are ENI, IntesaSanPaolo, Sogei, Leonardo Company, Thales Alenia Space, iFAB, and UnipolSai.
The partnership with CTE NEXT
With the “Casa delle Tecnologie Emergenti” CTE NEXT, we want to turn Turin into a widespread technology transfer centre developed on emerging technologies in relevant sectors for the Turin area: smart mobility, 4.0 industry and innovative urban services. The implementations of 5G connectivity, such as IoT, Big Data, Artificial Intelligence and Blockchain, will also be quintessential. The SWI living lab and HPC4AI computational resources are one of the laboratories of CTE NEXT.
iFAB: Federated Learning as a Service
Federated Learning is an emerging technique to train a distributed AI model with data from multiple institutions enforcing zero data sharing among different institutions, i.e. privacy and security in data management. Federate Learning as a Service (FLaaS) enacts a novel business model for data: renting data access with close-to-zero information transfer among federated partners. FLaaS is also imagined as a protocol to federate supercomputers.
With ENI: Cross-Platform Full Waveform Inversion
The project aims at modernizing seismic HPC by removing the direct embedding of computation stages in the application business code. The approach under validation aims at describing the application by way of the composition of steps by way of portable and open-standard workflow systems (i.e., streamflow). We will develop some fo the stages exploiting a high-level approach (e.g., DeVito Codes). Overall, the activity aims at enhancing the traditional co-design process with portability and performance portability.
The National Research Centre HPC, Big Data and Quantum Computing
Spoke 1: Future HPC & BIG DATA
The FutureHPC spoke is built upon five scientific flagship projects and two national physical laboratories called spoke living labs. The flagship projects are participated by the ten academic partners and ten large industries that are part of the FutureHPC spoke. The spoke living labs are realized by the spoke leader and co-leader institutions: the Universities of Bologna and Torino. The living lab at the Univ. of Bologna is HWS, which focuses on hardware technologies for HPC (P.I. Luca Benini). The living lab at the Univ. of Torino is called SWI and is focused on HPC software technologies (P.I. Marco Aldinucci). The HSW and SWI laboratories aim to become an integral part of a world-class national federated center with expertise in hardware and software co-design and to strengthen Italian leadership in the EuroHPC Joint Undertaking and the data infrastructure ecosystem for science and industry.
In collaboration with other spokes, the SWI lab is organizing the Euromicro/IEEE Parallel and Distributed processing conference, Torino, Processing conf 12-14 March 2024.
Academic partners:
Industrial partners:
Future HPC laboratories
FutureHPC spoke has planned two national laboratories for ICSC:
- Software and Integration Lab SWI-Lab in Torino
- Hardware and System Lab HWS-Lab, in Bologna
