Self-adaptive Executors for Big Data Processing

The demand for additional performance due to the rapid increase in the size and importance of data-intensive applications has considerably elevated the complexity of computer architecture. In response, systems offer pre-determined behaviors based on heuristics and then expose a large number of configuration parameters for operators to adjust...

SparkJNI: A Reference Design for a Heterogeneous Apache Spark Framework

The digital era's requirements pose many challenges related to deployment, implementation and efficient resource utilization in modern hybrid computing infrastructures. In light of the recent improvements in computing units, the defacto structure of a high-performance computing cluster, ordinarily consisted of CPUs only, is superseeded by...

Scalable machine learning algorithms on a big data infrastructure

Two currently popular topics in computer science are machine learning and big data. Often the two are combined to obtain powerful machines with learning capabilities or high throughput data analysis programs among others. This research analyses which machine learning techniques qualify to be efficiently implemented on a scalable big data...

Evaluation of different storage systems for Apache Spark and Apache Hadoop

Big Data systems have been used for multiple years to solve problems that require scale. A framework takes care of scalability and resiliency issues, and allows the user to focus on relevant computation, in the form of map and reduce functions. In these Big Data systems, we currently see a shift from the traditional use of the Hard Disk Drive ...