Large Language Models in Banking

Lessons from a Low Resource Language Region

Master thesis (2024)

Authors

S. Vignisson Technology, Policy and Management

Contributors

R.I.J. Dobbe (supervisor 1)
P.H.A.J.M. van Gelder Safety and Security Science - TPM (supervisor 2)
Faculty
Technology, Policy and Management
Large Language Models Low Resource Languges Emerging Risks
More Info
expand_more

Published Date

12-04-2024

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Technology, Policy and Management

Abstract

This thesis investigates the integration of Large Language Models (LLMs) within large organizations operating in low resource language (LRL) regions, with a particular focus on Íslandsbanki, one of the prominent banks in Iceland currently grappling with the adoption of LLM technology. The research aims to understand the frameworks governing LLM integration within the bank, explore stakeholder perspectives on the integration process, and identify blind spots and gaps in their understanding of associated risks in order to understand what a comprehensive approach would look like.

The methodology had mixed methods. It consisted of a comprehensive literature review followed by interviews with stakeholders involved in the LLM integration at Íslandsbanki. Subsequently, an Impact-Probability Assessment survey is conducted, where stakeholders are asked to evaluate the identified risks in terms of their potential impact and probability. This approach facilitates the identification and prioritization of risks, thereby enabling future integrators to navigate the integration process more effectively. Furthermore, data analysis techniques were applied to the survey results to provide quantitative insights into the identified gaps and perspectives.

Findings reveal that perspectives on risks associated with LLM integration vary significantly, particularly between individuals with technical backgrounds and decision-makers within the bank. Moreover, the concept of text inflation emerges as a novel risk factor, highlighting the importance of considering diverse perspectives in risk assessment and mitigation strategies. Emphasis is placed on adopting a sociotechnical perspective to comprehensively assess risks at an organizational level.

This research contributes to the understanding of the organizational impact of LLM integration, offers insights into diverse risk perspectives, and underscores the importance of considering sociotechnical factors in the integration process. By shedding light on the challenges and opportunities associated with LLM integration in the financial sector, this study aims to inform future research and practice in this evolving domain.