Unleashing the potential of Turkish chatbots

Abstract

The objective of this study is to adapt and evaluate the Turkish version of the Chatbot Usability Scale (BUS-11) through a confirmatory factor analysis method. The BUS-11 scale has been established in various languages except for Turkish; thus, its validation and dissemination could serve as a means to improve chatbot interaction satisfaction among the Turkish-speaking population and hence foster growth in Turkey’s conversational agent market. To achieve this aim, seven customer-oriented chatbots were rated on pre-designed tasks by participants. Data collection involved using the Turkish-adapted BUS11 (TBUS-11) to assess individuals’ experiences after interacting with Turkish-speaking chatbots, along with the Turkish version of the UMUX-LITE scale. Results show that TBUS-11 has been demonstrated to be highly reliable with a strong convergent validity with the UMUX-LITE already validated in Turkish. Moreover, the analysis demonstrated that the dataset supported the five-factor structure of the original version of the scale, thus confirming the psychometric properties of the TBUS. The study successfully adapted the BUS-11 into Turkish, providing a reliable and valid tool for assessing chatbot usability in the Turkish-speaking market. This can potentially enhance user satisfaction and promote the growth of conversational agents in Turkiye.