Building a scalable development cluster at Adyen

Master thesis (2019)

Authors

G. Weterings Electrical Engineering, Mathematics and Computer Science

Contributors

Maurício Aniche Software Engineering - EEMCS (supervisor 1)
A.E. Zaidman Software Engineering - EEMCS (supervisor 1)
Przemysław Pawełczak Embedded Systems - EEMCS (supervisor 2)
Bert Wolters Adyen B.V. (coach)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2019 Gijs Weterings
Software engineering Scalability Containerization Developer tools Orchestration
More Info
expand_more

Published Date

19-07-2019

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Software systems today are growing to incredible proportions. Software impacts everything in our society today, and its impact on the world keeps growing every day. However, developing large software systems is becoming an increasingly complex task, due to the immense complexity and size. For a software engineer to stay productive, it is vital they can work effectively on a system, being able to focus on the problem at hand. However, large software systems throw up a lot of roadblocks on the way, with complex and slow build processes impacting the developer's productivity to higher and higher degrees as the software system grows. To help developers stay productive, we need new, more powerful ways of assisting them during their activities.

In this thesis, we present our development cluster as a part of the solution to developing software at scale. The cluster provides a high-performance infrastructure that can be used by developers to build and deploy their applications during development. By moving these build and deploy processes to a cluster during development, we can benefit from more powerful computing resources which help developers work more effectively. We evaluate our development cluster in a number of different categories, comparing build speed, system startup and general developer workflows. Additionally, we evaluate how well our solution scales and what the impact on network load is for a company integrating with this system.

This move to cloud-based development brings along new challenges, but also many new possibilities in terms of tooling, developer collaboration and software engineering research. We are convinced our cluster can help scale software development efforts in industry, as well as bring new ways of doing research on software engineering.