Hadoop in the browser

Hadoop in the browser: Simulating MapReduce in the browser

Bilen, H.
Zwart, M.D.

Abeel, T.E.P.M.F. (mentor)

Electrical Engineering, Mathematics and Computer Science

Intelligent Systems

Web Information Systems

2016-06-24

With the current increase of user generated data, the need for tools to process large quantities of data is increasing. One of the courses of the Computer Science BSc curriculum is the Big Data Processing course. The Big Data Processing course teaches students ways of doing so. A popular and teached method is using MapReduce, a programming model to process large quantities of data. Big Data Processing students currently write their implementations for MapReduce related assignments of the lab in the Cloudera Virtual Machine (VM). This VM is slow, cannot be used interactively and it cannot be used to teach all MapReduce principles like memory separation. Since there are no existing solutions that solves these problems we have decided to write our own. Writing our own solution required diverse knowledge including but not limited to JavaScript, ECMAScript 6, WebWorkers, HTML5 (&CSS), PHP, MySQL, API development, Wordpress, Linux, QUnit and Selenium. Our solution, Trifle, overcomes previously mentioned and other shortcomings. Trifle is a web-based solution that simulates MapReduce within the browser. The framework uses JavaScript together with WebWorkers and our front-end is written using Wordpress. By simulating MapReduce within the browser we managed to create a solution that requires no cluster, is easy to use, works multi-platform and most importantly enables lecturers to teach some MapReduce principles that could not be taught before. Furthermore, we integrated a submissions system that can be used to do interactive lectures in which the lecturer can see problems in real-time and explain obstacles. A user test we have done verifies that Trifle helps to better grasp the idea behind MapReduce.

hadoop
mapreduce
big data
javascript
active learning

http://resolver.tudelft.nl/uuid:481fa36a-b688-4ec9-a41d-8760611999ae

Student theses

bachelor thesis

(c) 2016 The Author(s)