Improving User Engagement to Reduce Dropout Rates in Long Web Surveys
Exploring the Effectiveness of Achievement Primes Amongst Intrinsically and Extrinsically Motivated Respondents
T.A.R. van Tussenbroek (TU Delft - Electrical Engineering, Mathematics and Computer Science)
G.M. Allen – Coach (TU Delft - Web Information Systems)
Ujwal Gadiraju – Mentor (TU Delft - Web Information Systems)
Frank Broz – Graduation committee member (TU Delft - Interactive Intelligence)
More Info
expand_more
Data and code produced for the project
https://osf.io/pxfba/?view_only=77cda642e7a84a9cba63b1714c326fdbOther than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Web surveys have increasingly been used to collect data from respondents over the years. They offer several advantages compared to other methods of obtaining data. Researchers benefit from a broad demographic representation to make generalized conclusions, and satisfaction surveys allow employees to explain shortcomings or improvements anonymously. Both examples demand comprehensive information, thereby requiring a lengthy survey. However, dropout increases with the length of a survey, which is a big problem on web surveys as it decreases the statistical significance of the results. Proposed solutions, such as reducing the number of questions or rewarding respondents with an incentive, may not always be feasible due to the preciseness of information required or limited financial capabilities.
Achievement primes have been shown to reduce dropout on short surveys targeting extrinsically motivated respondents without additional costs or the need to reduce survey length. As repeated exposure to primes reinforces the stimuli, long surveys may also benefit from achievement primes. In this study, respondents are exposed to a questionnaire of more than 15 minutes on health whilst working behind a computer containing either no prime, passive achievement primes, or active achievement primes. Besides extrinsically motivated respondents, recruited via the crowdworking platform Prolific, intrinsically motivated respondents are also targeted in this study, recruited via snowball sampling.
Through a 2 times 3 factorial design, we discovered no statistical difference in dropout, perceived workload, and user engagement across the three questionnaire variants when evaluating intrinsically (N=88) and extrinsically motivated respondents (N=140) individually. By comparing intrinsically with extrinsically motivated respondents, we discovered extrinsically motivated respondents were more engaged and dropped out less.