Designing DAWN: A Data Analytics Workflow Notation

Big Data is a popular research and business topic. Due to the potential value that lies in Big Data, much effort is put in attempts to improve Big Data analysis methods. ORTEC is a company that provides data analysis, optimisation, and forecasting solutions. In an attempt to make Big Data analysis easier to use for its customers, ORTEC is developing the Big Data Portal. The intention is to create an easy-to-use cloud service that offers an all-in Big Data analysis solution. The Big Data Portal allows users to design workflows for their data analysis work. These workflows need to be designed in the portal through means of a notation that is both easy to use and can be extended with the latest in Big Data analysis tools. Furthermore, the portal should process the analysis work in an efficient manner, to minimise costs. In this design research DAWN, a Data Analytics Workflow Notation, was developed. This design considers the goals set for the Big Data Portal, as well as theoretical base for the workflow notation. The formal definition is derived from nested relational calculus as a theoretical base. In this work, DAWNis shown to be easily visualisable for graphically editing the workflow. Furthermore, DAWN is flexible as it enables the addition of new user defined functions. The compiler for DAWN, presented in this work, shows portability to other workflow execution systems. The nested relational calculus base is shown to enable optimisations in the workflows for cost reduction in the execution.

Subject

Big Data
Data analytics
Workflow
Notation
Nested Relational Calculus
Optimisation
Design research

To reference this document use:

http://resolver.tudelft.nl/uuid:5ccea299-4d16-458a-b7b3-f34d2fbe19e2

Part of collection

Student theses

Document type

master thesis

Rights

Files

PDF

Thesis_-_Mick_de_Lange.pdf

1.02 MB

Close viewer