Question answering with the RoboTutor Nao

Master thesis (2017)

Authors

S. Ercan Electrical Engineering, Mathematics and Computer Science

Contributors

K.V. Hindriks (mentor)
M.A. Neerincx (graduation committee member)
D.J. Broekens (graduation committee member)
C. Lofi (graduation committee member)
Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Copyright: © 2017 Selman Ercan
More Info
expand_more

Published Date

17-08-2017

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

This thesis project worked towards extending the interactive capabilities of
the RoboTutor Nao, by enabling it to answers natural language questions about
topics such as computers, robots and the Nao itself. The main questions we
focused on were 1) what a system for answering questions in Dutch and intended for elementary school age children should look like, 2) how the interaction with users should be structured for a smooth interaction and 3) what type of experimental studies and metrics are best for evaluating the quality of the QA system and assess its impact on users.

We chose an existing English-language QA system and adapted it to work with
Dutch content. An interaction manager was developed for handling the
interaction between the system and users. Finally, the system was configured so that it could be interacted with via the Nao robot.

The system was then evaluated via two experimental studies. First a
quantitative evaluation without users was carried out, evaluating the
performance of the system using a collection of test questions. Secondly, a
field study with users was done. We took the robot to a day care center where
children could ask it questions and assessed the effects of these interactive
Q\&A sessions.

Results from the quantitative evaluation highlighted slow answering times as
the main shortcoming, while recall (70%) and precision (50%) scores were more competitive. The field study showed that even though the occasional repetitiveness and lack of speed were picked up on by users, this did little to curb their enthusiasm. Interaction was smooth, users were interested in many topics the system supported and they continued interacting with it despite the occasional mishap. Overall, the results indicate that, with some speedup, the delivered QA system would be a suitable choice for answering questions about the mentioned topics.