Server-Side Experimentation

Common challenges from 12 organisations in The Netherlands

Conference paper (2020)

Authors

K.S. Anderson Software Engineering - EEMCS
Tom van den Berg Online Dialogue
A. van Deursen Software Technology
Department
Software Technology
Copyright: © 2020 K.S. Anderson, Tom van den Berg, A. van Deursen
A/B Testing Infrastructure Continuous Experimentation
More Info
expand_more

Published Date

2020

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Department

Software Technology

Abstract

Background: experimentation is widely adopted within industry. Many large organisations have invested in their own infrastructure to be able to run experiments server-side. Objective: we wanted to investigate why organisations switch to server-side experimen- tation and which challenges they encounter in doing so. Method: a qualitative virtual discussion based on a survey was conducted. Twelve Dutch organisations, represented by twelve interviewees, participated in the study. Results: organisations switch to server- side experimentation to run higher quality, more advanced and cheaper experiments. Challenges organisations face are: a shortage of development resources, no standardized process and a lack of a culture of experimentation. Conclusions: this is the first study that explores why organisations from The Netherlands in transition to continuous experimentation choose for investing in server-side experimentation infrastructure, and the challenges they encounter.