Crowdsourced Knowledge Base Construction using Text-Based Conversational Agents

Abstract

Knowledge Base Construction (KBC) is a challenging and complex task involving several substeps and many experts of a knowledge domain. Crowdsourcing approach has been used to support KBC with promising scalability and output quality, but to enable even more people to participate in KBC, there is a need to broaden the pool of workers beyond the ones who are already familiar with existing crowdsourcing platforms. Meanwhile, the number of people who use messaging platforms has been increasing. There is also a renowned popularity of text-based conversational agents -- chatbots -- existing on these messaging platforms. By leveraging the fact that there is a large number of users who are familiar with conversational interfaces, we see an opportunity to broaden the participants of crowdsourced KBC by using chatbots to execute KBC tasks. In this thesis, we investigate the use of chatbots to enable crowdsourced construction of knowledge bases. We design a conversational crowdsourcing platform to support the execution of KBC tasks. An experiment involving 43 students using our system and interviews with 7 participants were conducted to evaluate the system within the context of constructing a knowledge base for the TU Delft campus. From the results, we show that the platform is suitable to be used for crowdsourced KBC with justifiable execution time, accuracy, and completeness. We also laid out the potential future work to improve and extend the functionalities of the chatbot system.