Thursday, March 14th 2024
The SEA Projects Prepare Europe for Exascale Supercomputing
The HPC research projects DEEP-SEA, IO-SEA and RED-SEA are wrapping up this month after a three-year project term. The three projects worked together to develop key technologies for European Exascale supercomputers, based on the Modular Supercomputing Architecture (MSA), a blueprint architecture for highly efficient and scalable heterogeneous Exascale HPC systems. To achieve this, the three projects collaborated on system software and programming environments, data management and storage, as well as interconnects adapted to this architecture. The results of their joint work will be presented at a co-design workshop and poster session at the EuroHPC Summit (Antwerp, 18-21 March, www.eurohpcsummit.eu).
The future of supercomputing is Exascale
DEEP-SEA, IO-SEA and RED-SEA are part of a larger family of research and innovation projects co-funded by the EuroHPC Joint Undertaking to prepare key technology and solutions for the "Exascale era" in supercomputing. This upcoming Exascale generation will be capable of performing at least 1018 floating point operations per second, and enable novel and more accurate scientific applications. Exascale supercomputers are a lot more complex than previous generations of slower systems and combine a wide range of different hardware components used for specific purposes. They must integrate resources such as CPUs, GPUs and other accelerators as well as communication, memory and storage technologies into one system in a manner that supports traditional HPC simulations as well as artificial intelligence and big data analysis.
A Modular Supercomputing Architecture for European Exascale supercomputers
The MSA was designed for this very purpose. It is based on a novel system design to integrate heterogeneous resources and technologies and fulfil the requirements of a wide range of application areas, from compute-intensive, highly scalable simulation codes to data-intensive artificial intelligence workflows. Different workload components run on the module best suited to them, which improves time-to-solution and energy efficiency. The Jülich Supercomputing Centre's (JSC) new JUPITER supercomputer will be based on this architecture. JUPITER, the installation of which starts in 2024, is co-funded by the EuroHPC JU, the German Ministry of Education and Research, and the Ministry of Culture and Research of the German state of North Rhine-Westphalia.
To implement, extend and enhance the MSA, the three SEA projects worked on different tasks: DEEP-SEA, spearheaded by the JSC, focused on the programming environment and the HPC software stack. IO-SEA, led by CEA, prototyped a novel data management and storage platform. RED-SEA, headed by Eviden, prepared the next generation of the European BXI (Bull eXascale Interconnect) and its related software stack. While the three SEA projects are independent from each other, they are interlinked and collaborate closely to make efficient systems implementing the MSA possible. They build upon the results of other, previous projects co-funded by the European commission and draw on a vast network of ongoing partner projects, institutions and private companies from all over Europe.
The EuroHPC Summit 2024 - A great opportunity to showcase the project results
The integrated software stack developed by DEEP-SEA is one of the key results of the projects, alongside the establishment of the MSA in large, operational systems like JSC's JUWELS and the MeluXina system in Luxembourg, and soon in the JUPITER system. The stack integrates "best of breed" programming models, APIs, tools and libraries that are also relevant for non-modular HPC systems. In particular, the abstraction of optimisation cycles, the support of hierarchical storage systems and the use of modern CI/CD techniques will bring direct benefits for such systems. This also applies to the pioneering work in the area of malleability. The stack is freely available as open source to all interested parties.
IO-SEA developed a storage and I/O stack suitable for the MSA that deals with data movements across hierarchical storage systems including novel non-volatile memory, solid state devices, hard disk drives and tape storage. IO-SEA also developed the concept of short lived "ephemeral storage services" that are created on demand and enable users to flexibly operate the system. The project also introduced extensive use and exploitation of instrumentation and telemetry across the I/O stack and new I/O APIs that expose the actual semantics of data.
The RED-SEA project has laid the groundwork for the development and industrialization of the third generation of the European BXI interconnect. The project helped to bring BXI to the Exascale level, with improved scalability and resilience to reach well over 100K end-points, while significantly improving the latency and the bandwidth available for each end-point. RED-SEA also addressed real-world network performance with congestion management and adaptive routing mechanisms. Finally, RED-SEA opened up BXI to Ethernet compatibility, by designing a seamless interface between BXI and Ethernet via a new low-latency Gateway solution.
The EuroHPC Summit, which takes place from 18 to 21 March in Antwerp, Belgium, offers interested parties the opportunity to learn about the results. The projects will co-host the workshop on "Co-Designing the Future of European HPC: Eco-friendly Technologies, Systems and Software" that will highlight technology areas developed in Europe as a result of successful co-design, including the SEA projects outcomes, and present their results in the poster session.
(Quote DEEP-SEA)
Prof. Thomas Lippert, director of JSC: "Based on the groundbreaking results of the previous DEEP projects, the solid, comprehensive HPC SW stack developed by DEEP-SEA makes systems based on the modular supercomputing architecture (MSA) easily usable for HPC application developers and end users and is therefore of prime importance for the future of MSA in general and especially for the upcoming JUPITER Exascale system at JSC and other modular systems all around the world."
(Quote IO-SEA)
Dr. Philippe Deniel, Scientific Coordinator of the IO-SEA Project: "The Exascale era is a major step forward for mass storage, which will have to face many new challenges as the currently used storage models will not scale up and will turn I/O operations into critical bottlenecks. The IO-SEA software stack introduces new features and paradigms in order to prevent this kind of situations, offering a non disruptive integration path for applications. The stack will be used in the future by the EUPEX community. Furthermore, the stack is planned to be integrated into the second EuroHPC Exascale machine hosted in France."
(Quote RED-SEA)
Laurent Cargemel, Head of Strategy, Innovation and R&D of BDS at Eviden: "The interconnect network is the heart and arteries of a supercomputer: the larger the system, the more critical its role becomes. And nowadays, the interconnect network must also support new levels of complexity, with mixed workflows and heterogeneous systems. RED-SEA brings a European solution to these challenges, a solution that will be used in the EUPEX Pilot system, and potentially in the second EuroHPC Exascale machine hosted in France."
About the SEA projects
The SEA projects DEEP-SEA, RED-SEA and IO-SEA realise the joint vision of the Jülich Supercomputing Centre (JSC), the French Alternative Energies and Atomic Energy Commission (CEA), Eviden and ParTec AG for a Modular Supercomputer Architecture (MSA) for future Exascale high-performance computing (HPC) systems. The four core partners have brought together the best European academic and industrial expertise to develop the technologies for the next generation of supercomputers. DEEP-SEA focuses on the programming environment & software stack for Exascale systems. IO-SEA provides a data management and storage platform for these systems, and RED-SEA enhances the European interconnect technology BXI and its related low-level software.
The SEA projects have received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreements n° 955606, 95811, and, 955776 and support from France, the Czech Republic, Germany, Spain, Ireland, Sweden, Switzerland, Italy and Greece.
More info at sea-projects.eu.
DEEP-SEA, IO-SEA and RED-SEA are part of a larger family of research and innovation projects co-funded by the EuroHPC Joint Undertaking to prepare key technology and solutions for the "Exascale era" in supercomputing. This upcoming Exascale generation will be capable of performing at least 1018 floating point operations per second, and enable novel and more accurate scientific applications. Exascale supercomputers are a lot more complex than previous generations of slower systems and combine a wide range of different hardware components used for specific purposes. They must integrate resources such as CPUs, GPUs and other accelerators as well as communication, memory and storage technologies into one system in a manner that supports traditional HPC simulations as well as artificial intelligence and big data analysis.
A Modular Supercomputing Architecture for European Exascale supercomputers
The MSA was designed for this very purpose. It is based on a novel system design to integrate heterogeneous resources and technologies and fulfil the requirements of a wide range of application areas, from compute-intensive, highly scalable simulation codes to data-intensive artificial intelligence workflows. Different workload components run on the module best suited to them, which improves time-to-solution and energy efficiency. The Jülich Supercomputing Centre's (JSC) new JUPITER supercomputer will be based on this architecture. JUPITER, the installation of which starts in 2024, is co-funded by the EuroHPC JU, the German Ministry of Education and Research, and the Ministry of Culture and Research of the German state of North Rhine-Westphalia.
To implement, extend and enhance the MSA, the three SEA projects worked on different tasks: DEEP-SEA, spearheaded by the JSC, focused on the programming environment and the HPC software stack. IO-SEA, led by CEA, prototyped a novel data management and storage platform. RED-SEA, headed by Eviden, prepared the next generation of the European BXI (Bull eXascale Interconnect) and its related software stack. While the three SEA projects are independent from each other, they are interlinked and collaborate closely to make efficient systems implementing the MSA possible. They build upon the results of other, previous projects co-funded by the European commission and draw on a vast network of ongoing partner projects, institutions and private companies from all over Europe.
The EuroHPC Summit 2024 - A great opportunity to showcase the project results
The integrated software stack developed by DEEP-SEA is one of the key results of the projects, alongside the establishment of the MSA in large, operational systems like JSC's JUWELS and the MeluXina system in Luxembourg, and soon in the JUPITER system. The stack integrates "best of breed" programming models, APIs, tools and libraries that are also relevant for non-modular HPC systems. In particular, the abstraction of optimisation cycles, the support of hierarchical storage systems and the use of modern CI/CD techniques will bring direct benefits for such systems. This also applies to the pioneering work in the area of malleability. The stack is freely available as open source to all interested parties.
IO-SEA developed a storage and I/O stack suitable for the MSA that deals with data movements across hierarchical storage systems including novel non-volatile memory, solid state devices, hard disk drives and tape storage. IO-SEA also developed the concept of short lived "ephemeral storage services" that are created on demand and enable users to flexibly operate the system. The project also introduced extensive use and exploitation of instrumentation and telemetry across the I/O stack and new I/O APIs that expose the actual semantics of data.
The RED-SEA project has laid the groundwork for the development and industrialization of the third generation of the European BXI interconnect. The project helped to bring BXI to the Exascale level, with improved scalability and resilience to reach well over 100K end-points, while significantly improving the latency and the bandwidth available for each end-point. RED-SEA also addressed real-world network performance with congestion management and adaptive routing mechanisms. Finally, RED-SEA opened up BXI to Ethernet compatibility, by designing a seamless interface between BXI and Ethernet via a new low-latency Gateway solution.
The EuroHPC Summit, which takes place from 18 to 21 March in Antwerp, Belgium, offers interested parties the opportunity to learn about the results. The projects will co-host the workshop on "Co-Designing the Future of European HPC: Eco-friendly Technologies, Systems and Software" that will highlight technology areas developed in Europe as a result of successful co-design, including the SEA projects outcomes, and present their results in the poster session.
(Quote DEEP-SEA)
Prof. Thomas Lippert, director of JSC: "Based on the groundbreaking results of the previous DEEP projects, the solid, comprehensive HPC SW stack developed by DEEP-SEA makes systems based on the modular supercomputing architecture (MSA) easily usable for HPC application developers and end users and is therefore of prime importance for the future of MSA in general and especially for the upcoming JUPITER Exascale system at JSC and other modular systems all around the world."
(Quote IO-SEA)
Dr. Philippe Deniel, Scientific Coordinator of the IO-SEA Project: "The Exascale era is a major step forward for mass storage, which will have to face many new challenges as the currently used storage models will not scale up and will turn I/O operations into critical bottlenecks. The IO-SEA software stack introduces new features and paradigms in order to prevent this kind of situations, offering a non disruptive integration path for applications. The stack will be used in the future by the EUPEX community. Furthermore, the stack is planned to be integrated into the second EuroHPC Exascale machine hosted in France."
(Quote RED-SEA)
Laurent Cargemel, Head of Strategy, Innovation and R&D of BDS at Eviden: "The interconnect network is the heart and arteries of a supercomputer: the larger the system, the more critical its role becomes. And nowadays, the interconnect network must also support new levels of complexity, with mixed workflows and heterogeneous systems. RED-SEA brings a European solution to these challenges, a solution that will be used in the EUPEX Pilot system, and potentially in the second EuroHPC Exascale machine hosted in France."
About the SEA projects
The SEA projects DEEP-SEA, RED-SEA and IO-SEA realise the joint vision of the Jülich Supercomputing Centre (JSC), the French Alternative Energies and Atomic Energy Commission (CEA), Eviden and ParTec AG for a Modular Supercomputer Architecture (MSA) for future Exascale high-performance computing (HPC) systems. The four core partners have brought together the best European academic and industrial expertise to develop the technologies for the next generation of supercomputers. DEEP-SEA focuses on the programming environment & software stack for Exascale systems. IO-SEA provides a data management and storage platform for these systems, and RED-SEA enhances the European interconnect technology BXI and its related low-level software.
The SEA projects have received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreements n° 955606, 95811, and, 955776 and support from France, the Czech Republic, Germany, Spain, Ireland, Sweden, Switzerland, Italy and Greece.
More info at sea-projects.eu.
Comments on The SEA Projects Prepare Europe for Exascale Supercomputing
There are no comments yet.