Should we use the NASA-TLX in HCI?
A review of theoretical and methodological issues around Mental Workload Measurement
Ebrahim Babaei (University of Melbourne)
Tilman Dingler (TU Delft - Human-Centred Artificial Intelligence)
B. Tag (University of New South Wales)
Eduardo Velloso (University of Sydney)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Mental Workload (MWL) is a construct widely used in HCI to assess the cognitive demand users must exert to perform a task. Research in human factors, however, has suggested several issues regarding its definitions, scales, and applications. This paper, first, introduces debates surrounding the MWL concept and its most popular measure, the NASA-TLX. We present a systematic review of CHI papers involving MWL and highlight severe issues in its application. Finally, through a validation experiment, we assess the convergent validity and sensitivity of two MWL instruments—NASA-TLX and MRQ. Our findings reveal disagreements in the definitions of MWL and severe drawbacks in NASA-TLX and its applications. Our validation study also presents evidence for a lack of convergent validity and sensitivity of MWL subjective scales in HCI tasks. Our findings recommend caution when employing NASA-TLX in user studies and highlight the need for an MWL definition that is agreed upon within the HCI community.
help_outline