Large Language Models in Banking

Abstract

This thesis investigates the integration of Large Language Models (LLMs) within large organizations operating in low resource language (LRL) regions, with a particular focus on Íslandsbanki, one of the prominent banks in Iceland currently grappling with the adoption of LLM technology. The research aims to understand the frameworks governing LLM integration within the bank, explore stakeholder perspectives on the integration process, and identify blind spots and gaps in their understanding of associated risks in order to understand what a comprehensive approach would look like.

The methodology had mixed methods. It consisted of a comprehensive literature review followed by interviews with stakeholders involved in the LLM integration at Íslandsbanki. Subsequently, an Impact-Probability Assessment survey is conducted, where stakeholders are asked to evaluate the identified risks in terms of their potential impact and probability. This approach facilitates the identification and prioritization of risks, thereby enabling future integrators to navigate the integration process more effectively. Furthermore, data analysis techniques were applied to the survey results to provide quantitative insights into the identified gaps and perspectives.

Findings reveal that perspectives on risks associated with LLM integration vary significantly, particularly between individuals with technical backgrounds and decision-makers within the bank. Moreover, the concept of text inflation emerges as a novel risk factor, highlighting the importance of considering diverse perspectives in risk assessment and mitigation strategies. Emphasis is placed on adopting a sociotechnical perspective to comprehensively assess risks at an organizational level.

This research contributes to the understanding of the organizational impact of LLM integration, offers insights into diverse risk perspectives, and underscores the importance of considering sociotechnical factors in the integration process. By shedding light on the challenges and opportunities associated with LLM integration in the financial sector, this study aims to inform future research and practice in this evolving domain.