Improving User Engagement to Reduce Dropout Rates in Long Web Surveys

Abstract

Web surveys have increasingly been used to collect data from respondents over the years. They offer several advantages compared to other methods of obtaining data. Researchers benefit from a broad demographic representation to make generalized conclusions, and satisfaction surveys allow employees to explain shortcomings or improvements anonymously. Both examples demand comprehensive information, thereby requiring a lengthy survey. However, dropout increases with the length of a survey, which is a big problem on web surveys as it decreases the statistical significance of the results. Proposed solutions, such as reducing the number of questions or rewarding respondents with an incentive, may not always be feasible due to the preciseness of information required or limited financial capabilities.

Achievement primes have been shown to reduce dropout on short surveys targeting extrinsically motivated respondents without additional costs or the need to reduce survey length. As repeated exposure to primes reinforces the stimuli, long surveys may also benefit from achievement primes. In this study, respondents are exposed to a questionnaire of more than 15 minutes on health whilst working behind a computer containing either no prime, passive achievement primes, or active achievement primes. Besides extrinsically motivated respondents, recruited via the crowdworking platform Prolific, intrinsically motivated respondents are also targeted in this study, recruited via snowball sampling.

Through a 2 times 3 factorial design, we discovered no statistical difference in dropout, perceived workload, and user engagement across the three questionnaire variants when evaluating intrinsically (N=88) and extrinsically motivated respondents (N=140) individually. By comparing intrinsically with extrinsically motivated respondents, we discovered extrinsically motivated respondents were more engaged and dropped out less.