Data Entry: Please note that the research database will be replaced by UNIverse by the end of October 2023. Please enter your data into the system https://universe-intern.unibas.ch. Thanks

Login for users with Unibas email account...

Login for registered users without Unibas email account...

 
DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning
Third-party funded project
Project title DAPHNE: Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and Machine Learning
Principal Investigator(s) Ciorba, Florina M.
Project Members Eleliemy, Ahmed Hamdy Mohamed
Organisation / Research unit Departement Mathematik und Informatik / High Performance Computing (Ciorba)
Department Departement Mathematik und Informatik / High Performance Computing (Ciorba)
Project Website https://daphne-eu.eu
Project start 01.10.2020
Probable end 30.09.2024
Status Active
Abstract

The DAPHNE project aims to define and build an open and extensible system infrastructure for integrated data analysis pipelines, including data management and processing, high-performance computing (HPC), and machine learning (ML) training and scoring. Key observations are that (1) systems of these areas share many compilation and runtime techniques, (2) there is a trend towards complex data analysis pipelines that combine these systems, and (3) the used, increasingly heterogeneous, hardware infrastructure converges as well. Yet, the programming paradigms, cluster resource management, as well as data formats and representations differ substantially. Therefore, this project aims – with a joint consortium of experts from the data management, ML systems, and HPC communities – at systematically investigating the necessary system infrastructure, language abstractions, compilation and runtime techniques, as well as systems and tools necessary to increase the productivity when building such data analysis pipelines, and eliminating unnecessary performance bottlenecks.

Keywords System Architecture, APIs and DSL, Hierarchical Scheduling and Task Planning, Use Cases and Benchmarking
Financed by Commission of the European Union

Published results ()

  ID Autor(en) Titel ISSN / ISBN Erschienen in Art der Publikation
4631194  Ihde, N.; Marten, P.; Eleliemy, A.; Poerwawinata, G.; Silva, P.; Tolovski, I.; Ciorba, F. M.; Rabl, T.   A Survey of Big Data, HPC and Machine Learning Benchmarks      Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
4661212  Nina Ihde; Paula Marten; A. Eleliemy; Gabrielle Poerwawinata; P. Silva; Ilin Tolovski; Florina M. Ciorba; Tilmann Rabl  A Survey of Big Data, High Performance Computing, and Machine Learning Benchmarks      Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
4661202  Patrick Damme; Marius Birkenbach; Constantinos Bitsakos; Matthias Boehm; Philippe Bonnet; Florina M. Ciorba; Mark Dokter; Pawel Dowgiallo; A. Eleliemy; Christian Faerber; Georgios I. Goumas; Dirk Habich; Niclas Hedam; Marlies Hofer; Wen-Hung Kevin Huang; Kevin Innerebner; Vasileios Karakostas; Roman Kern; Tomavz Kosar; Alexander Krause; D. Krems; Andreas Laber; Wolfgang Lehner; Eric Mier; Marcus Paradies; Bernhard Peischl; Gabrielle Poerwawinata; Stratos Psomadakis; Tilmann Rabl; Piotr Ratuszniak; Pedro Silva; Nikolai Skuppin; Andreas Starzacher; Benjamin Steinwender; Ilin Tolovski; Pinar Tözün; Wojciech Ulatowski; Yuanyuan Wang; Izajasz P. Wrosz; A. Zamuda; Ce Zhang; Xiao Xiang Zhu  DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines      Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
4626035  Müller Korndörfer, Jonas H.; Eleliemy, Ahmed; Mohammed, Ali; Ciorba M., Florina  LB4OMP: A Dynamic Load Balancing Library for Multithreaded Applications  1045-9219  IEEE Transactions on parallel and distributed systems  Publication: JournalArticle (Originalarbeit in einer wissenschaftlichen Zeitschrift) 
   

MCSS v5.8 PRO. 0.414 sec, queries - 0.000 sec ©Universität Basel  |  Impressum   |    
29/03/2024