A conversational ai agent for industrial maintenance reporting

A conversational ai agent for industrial maintenance reporting

Kernan Freire, Samuel (TU Delft Industrial Design Engineering)

Rusak, Z. (mentor)
Aschenbrenner, D. (graduation committee)

Delft University of Technology

Integrated Product Design

DIAMOND

2020-07-23

This project is an exploration of potential applications of Conversational AI Agents (CAIA) for industrial maintenance. Specifically, it involves the scientific validation and development of a CAIA for a promising application: the automatic creation of information-rich maintenance reports by conversing with a technician while they perform industrial maintenance. The choice for this application was based on literature research, (in-situ) context analyses and a review of CAIA frameworks and design guidelines. These revealed that maintenance workers rely heavily on their own experience and intuition when solving problems but mechanisms for capturing and accessing this were non-existent. This knowledge is highly valuable and can represent a significant part of a company’s worth. Furthermore, maintenance technicians reported that the existing reporting mechanisms were a nuisance. Lastly, audits revealed that maintenance reports were frequently incomplete or of poor quality. In turn, CAIAs (Conversational AI Agents), have various affordances that make them well-suited to the context of industrial maintenance. They are (1) hands and gaze-free, (2) highly efficient (faster than writing or typing, facilitate multitasking and they provide faster access to specific information), (3) they can adapt to the skill level of the user and (4) impose a minimal cognitive load.
A between-subjects experiment with 24 participants, which involved changing a bicycle inner tube, was used to test three hypotheses regarding the potential value of the application. All three hypotheses compare using a CAIA for reporting whilst performing maintenance, as opposed to writing the reports on paper afterwards. They posited that using the CAIA would result in (1) reports of higher quality (more information relevant to the understanding of the task), (2) time saving, and (3) a lower perceived workload (NASA TLX). T-tests confirmed that all three hypotheses were true. These results indicate that using a CAIA for live-reporting has a clear value proposition for the industrial maintenance domain. Critically, it demonstrated that it could facilitate the capture of valuable “expert knowledge”. Future research could explore integrating multi-modal information capture (e.g. through smart-glasses), identify additional uses for the captured data (e.g. for prescriptive maintenance or providing tips to maintenance technicians) and improve the functionality and usability of the existing application. A prototype was built using the open-source frameworks, Rasa and Mycroft, to demonstrate the technical feasibility of the functional requirements. These requirements include (1) uttering “continuers” in response to the user describing their work, (2) tracking conversational context, (3) asking for clarifications when there is a lack of mutual understanding and (4) asking for status updates when the user is silent for more than a set amount of time. The main challenges for the future development of the prototype are (1) reducing the response time of the CAIA, (2) the accuracy of the intent classifier and entity extractor and (3) improving the handling of fragmented/lengthy user input. Some of the prototype’s features rely on inflexible, hard-coded logic, therefore, future work should collect more conversational data and explore the use of machine learning algorithms.

Conversational Agents
Industry 4.0
Maintenance reporting

http://resolver.tudelft.nl/uuid:9012abed-2736-4e7c-85e2-7fff8a309165

Student theses

master thesis

© 2020 Samuel Kernan Freire