Second Workshop on Educational A/B Testing at Scale

Ritter, Steve; Heffernan, Neil; Williams, Joseph Jay; Lomas, J.D.; Bicknell, Klinton

doi:10.1145/3430895.3460876

Second Workshop on Educational A/B Testing at Scale

Title

Second Workshop on Educational A/B Testing at Scale

Author

Ritter, Steve (Carnegie Learning, Inc.)
Heffernan, Neil (Worcester Polytechnic Institute)
Williams, Joseph Jay (University of Toronto)
Lomas, J.D. (TU Delft Design Aesthetics)
Bicknell, Klinton (Duolingo)

Date

2021

Abstract

The emerging discipline of Learning Engineering is focused on putting into place tools and processes that use the science of learning as a basis for improving educational outcomes. An important part of Learning Engineering focuses on improving the effectiveness of educational software. In many software domains, A/B testing has become a prominent technique to achieve the software's goals. Many large companies (Amazon, Google, Facebook, etc.) run thousands of AB tests and present at the Annual Conference on Digital Experimentation (CODE), but that venue is too broad to address AB testing issues specific to EdTech platforms. We see a need to address issues with running large-scale A/B tests within the educational context, where the use of A/B testing lags other industries. This workshop will explore ways in which A/B testing in educational contexts differs from other domains and proposals to overcome current challenges so that this approach can become a more useful tool in the learning engineer's toolbox.

Subject

a/b testing
educational software
experimentation

To reference this document use:

http://resolver.tudelft.nl/uuid:8cfee08e-4ae5-4f69-bdf2-053ee1fcfdc9

DOI

https://doi.org/10.1145/3430895.3460876

Publisher

Association for Computing Machinery (ACM)

ISBN

9781450382151

Source

L@S 2021 - Proceedings of the 8th ACM Conference on Learning @ Scale