Title
The 19 Unifying Questionnaire Constructs of Artificial Social Agents: An IVA Community Analysis
Author
Fitrianie, S. (TU Delft Interactive Intelligence) 
Bruijnes, M. (TU Delft Interactive Intelligence)
Richards, Deborah (Macquarie University)
Bönsch, Andrea (Rheinisch-Westfälische Technische Hochschule)
Brinkman, W.P. (TU Delft Interactive Intelligence)
Date
2020
Abstract
In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 80 researchers worldwide, researching the IVA community interests and practises in evaluating human interaction with an artificial social agent (ASA). The effort is driven by previous IVA workshops and plenary IVA discussions related to the methodological crisis on the evaluation of ASAs. A previous literature review showed a continuous practise of creating new questionnaires instead of reusing validated questionnaires. We address this issue by examining questionnaire measurement constructs used in empirical studies between 2013 to 2018 published in the IVA conference. We identified 189 constructs used in 89 questionnaires that are reported across 81 studies. Although these constructs have different names, they often measure the same thing. In this paper, we, therefore, present a unifying set of 19 constructs that captures more than 80% of the 189 constructs initially identified. We established this set in two steps. First, 49 researchers classified the constructs in broad theoretically based categories. Next, 23 researchers grouped the constructs in each category on their similarity. The resulting 19 groups form a unifying set of constructs, which will be the basis for the future questionnaire instrument of human-ASA interaction.
Subject
Artificial social agent
evaluation instrument
measurement construct
questionnaire
user study
To reference this document use:
http://resolver.tudelft.nl/uuid:521411a1-579c-4054-878d-31ba3815b628
DOI
https://doi.org/10.1145/3383652.3423873
Publisher
Association for Computing Machinery (ACM)
Embargo date
2021-11-29
ISBN
9781450375863
Source
Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
Event
20th ACM International Conference on Intelligent Virtual Agents, IVA 2020, 2020-10-20 → 2020-10-22, Virtual, Online, United Kingdom
Series
Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2020 S. Fitrianie, M. Bruijnes, Deborah Richards, Andrea Bönsch, W.P. Brinkman