Exploring HPC and Big Data Convergence

A Graph Processing Study on Intel Knights Landing

Conference paper (2018)

Authors

Alexandru Uta Vrije Universiteit Amsterdam
A.L. Varbanescu Universiteit van Amsterdam
Ahmed Musaafir Vrije Universiteit Amsterdam
Chris Lemaire Student
A. Iosup Data-Intensive Systems - EEMCS , Vrije Universiteit Amsterdam
Research Group
Data-Intensive Systems (EEMCS) (TU Delft)
Copyright: © 2018 Alexandru Uta, A.L. Varbanescu, Ahmed Musaafir, Chris Lemaire, A. Iosup
DOI
help_outline
https://doi.org/10.1109/CLUSTER.2018.00019
Big Data HPC Performance evaluation Graph Processing HPC Big Data convergence Intel Knights Landing
More Info
expand_more

Published Date

2018

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Data-Intensive Systems

Abstract

The question 'Can big data and HPC infrastructure converge?' has important implications for many operators and clients of modern computing. However, answering it is challenging. The hardware is currently different, and fast evolving: big data uses machines with modest numbers of fat cores per socket, large caches, and much memory, whereas HPC uses machines with larger numbers of (thinner) cores, non-trivial NUMA architectures, and fast interconnects. In this work, we investigate the convergence of big data and HPC infrastructure for one of the most challenging application domains, the highly irregular graph processing. We contrast through a systematic, experimental study of over 300,000 core-hours the performance of a modern multicore, Intel Knights Landing (KNL) and of traditional big data hardware, in processing representative graph workloads using state-of-the-art graph analytics platforms. The experimental results indicate KNL is convergence-ready, performance-wise, but only after extensive and expert-level tuning of software and hardware parameters.