Print Email Facebook Twitter DALLMi: Domain Adaption for LLM-based Multi-label Classifier Title DALLMi: Domain Adaption for LLM-based Multi-label Classifier Author Beţianu, Miruna (TU Delft Electrical Engineering, Mathematics and Computer Science) Contributor Chen, Lydia Y. (mentor) Birke, Robert (mentor) Yang, J. (graduation committee) Degree granting institution Delft University of Technology Programme Computer Science Date 2023-12-06 Abstract Large language models (LLMs) increasingly serve as the backbone for classifying text associated with distinct domains and simultaneously several labels (classes). When encountering domain shifts, e.g., classifier of movie reviews from IMDb to Rotten Tomatoes, adapting such an LLM-based multi-label classifier is challenging due to incomplete label sets at the target domain and daunting training overhead. The existing domain adaptation methods address either image multi-label classifiers or text binary classifiers. In this paper, we design DALLMi, Domain Adaptation Large Language Model interpolator, a first-of-its-kind semi-supervised domain adaptation method for text data models based on LLMs, specifically BERT. The core of DALLMi is the novel variation loss and MixUp regularization, which jointly leverage the limited positively labeled and large quantity of unlabeled text and, importantly, their interpolation from the BERT word embeddings. DALLMi also introduces a label-balanced sampling strategy to overcome the imbalance between labeled and unlabeled data. We evaluate DALLMi against the partial-supervised and unsupervised approach on three datasets under different scenarios of label availability for the target domain. Our results show that DALLMi achieves higher mAP than unsupervised and partially-supervised approaches by 19.9% and 52.2%, respectively. Subject Domain AdaptationSemi-supervised LearningText Classification To reference this document use: http://resolver.tudelft.nl/uuid:1f6a9c7c-4f24-40bc-ab41-af57f83e19e7 Part of collection Student theses Document type master thesis Rights © 2023 Miruna Beţianu Files PDF DALLMi_M_Betianu_Thesis.pdf 4.23 MB Close viewer /islandora/object/uuid:1f6a9c7c-4f24-40bc-ab41-af57f83e19e7/datastream/OBJ/view