Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods

Comparison of Questionnaire Based and User Model Based Usability Evaluation Methods

Li, M. (TU Delft Applied Ergonomics and Design; Xi’an Jiaotong University)
Albayrak, A. (TU Delft Applied Ergonomics and Design)
Zhang, Yu (Xi’an Jiaotong University)
van Eijk, D.J. (TU Delft Applied Ergonomics and Design)
Yang, Zengyao (Xi’an Jiaotong University)

Bagnara, Sebastiano (editor)
Tartaglia, Riccardo (editor)
Albolino, Sara (editor)
Alexander, Thomas (editor)
Fujita, Yushi (editor)

2019

The usability now serves as a fundamental quality of a computational device, e.g. smartphone. Moreover, the smartphone has firmly embedded into our daily life as an indispensable part, so the context and style that user may interact with them are largely different from a decade ago. Nowadays, testing usability with end user has become a common sense. Thus, how valid a usability evaluation method could assess the ‘extent to which a product can be used by specified users’ (ISO 9241-11) to facilitate software design becomes an interesting question to explore. In this research, three usability evaluation methods are compared. Among these methods, IsoMetrics is a standard questionnaire aiming at offer usability data for summative and formative evaluation; SUMI aims to assess quality of software product from end users perspective; User Model Checklist is a method based on user’s cognition-motor chain in specific tasks. The coverage and amount of usability issues, user’s effort of evaluation and software developer’s feedback on evaluation result are compared under a simulated usability test on SMS function with a smartphone. The result indicate that User Model Checklist could cover 90.4% of the usability issues found by IsoMetrics and SUMI, while 26.3% usability issues found by User Model Checklist could not be covered by IsoMetrics and SUMI. Users put highest effort on accomplish IsoMetrics and lowest effort on User Model Checklist. Moreover, the feedbacks from the developers show that the User Model Checklist requires lower usability knowledge, offers clearer improvement points and supports detailed design better.

IsoMetrics
SUMI
Usability evaluation comparison
User Model Checklist

http://resolver.tudelft.nl/uuid:01057dec-59bb-4a8f-9adb-8e594674afca

https://doi.org/10.1007/978-3-319-96071-5_110

Springer, Cham

2020-08-12

978-3-319-96070-8

Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) - Volume VII: Ergonomics in Design, Design for All, Activity Theories for Work Analysis and Design, Affective Design, VII

IEA 2018: 20th Congress of the International Ergonomics Association, 2018-08-26 → 2018-08-30, Florence, Italy

Advances in Intelligent Systems and Computing, 2194-5357, 824

Accepted author manuscript

Institutional Repository

conference paper

© 2019 M. Li, A. Albayrak, Yu Zhang, D.J. van Eijk, Zengyao Yang