Print Email Facebook Twitter Exploring HPC and Big Data Convergence Title Exploring HPC and Big Data Convergence: A Graph Processing Study on Intel Knights Landing Author Uta, Alexandru (Vrije Universiteit Amsterdam) Varbanescu, A.L. (Universiteit van Amsterdam) Musaafir, Ahmed (Vrije Universiteit Amsterdam) Lemaire, Chris (Student TU Delft) Iosup, A. (TU Delft Data-Intensive Systems; Vrije Universiteit Amsterdam) Contributor O'Conner, L. (editor) Torres, H. (editor) Date 2018 Abstract The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters. Subject Big DataGraph ProcessingHPCHPC Big Data convergenceIntel Knights LandingPerformance evaluation To reference this document use: http://resolver.tudelft.nl/uuid:d1e87a58-67dc-497f-b300-ed7614e9ead9 DOI https://doi.org/10.1109/CLUSTER.2018.00019 Publisher IEEE, Piscataway, NJ ISBN 978-1-5386-8320-0 Source 2018 IEEE International Conference on Cluster Computing (CLUSTER) Event 2018 IEEE International Conference on Cluster Computing, CLUSTER 2018, 2018-09-10 → 2018-09-13, Belfast, United Kingdom Bibliographical note Accepted author manuscript Part of collection Institutional Repository Document type conference paper Rights © 2018 Alexandru Uta, A.L. Varbanescu, Ahmed Musaafir, Chris Lemaire, A. Iosup Files PDF 47774685_CLUSTER_2018_Exp ... rgence.pdf 691.31 KB Close viewer /islandora/object/uuid:d1e87a58-67dc-497f-b300-ed7614e9ead9/datastream/OBJ/view