Evaluating the effectiveness of large language models in meeting summarization with transcript segmentation techniques

Evaluating the effectiveness of large language models in meeting summarization with transcript segmentation techniques: How well does gpt-3.5-turbo perform on meeting summarization with topic and context-length window segmentation?

Sándor, Kristóf (TU Delft Electrical Engineering, Mathematics and Computer Science)

Tarvirdians, M. (mentor)
Jonker, C.M. (mentor)
Molenaar, M.L. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-07-03

Large Language Models (LLM) have brought significant performance increase on many Natural Language Processing tasks. However LLMs have not been tested for meeting summarization. This research paper examines the effectiveness of the gpt-3.5-turbo model on the meeting summarization domain. However due to input length limitations, it cannot be applied directly to this task. Thus the paper investigates two segmentation methods: a simple context-length window approach and topic segmentation using Latent Dirichlet Allocation (LDA). The context-length window approach's performance is close to the Pointer Generator framework. The topic segmentation gives worse results. Overall gpt-3.5-turbo performs worse with both approaches than state-of-the-art models which use a transformer architecture adapted for long documents.

Natural Language Processing (NLP)
meeting summarization
meeting transcript
summarization
text summarization
dialogue summarization
Large Language Models
GPT-3

http://resolver.tudelft.nl/uuid:e2ce9fbb-435c-446e-9327-95f26d0336d5

Student theses

bachelor thesis

© 2023 Kristóf Sándor