Searched for: subject%3A%22Knowledge%255C+Distillation%22
(1 - 5 of 5)
document
Malmsten, Emil (author)
The application of large language models (LLMs) for programming tasks, such as automatic code completion, has seen a significant upswing in recent years. However, due to their computational demands, they have to operate on servers. This both requires users to have a steady internet connection and raises potential privacy concerns. Therefore,...
bachelor thesis 2023
document
de Moor, Aral (author)
Large language models are powerful because of their state-of-the-art language processing abilities. But, they come at the cost of being extremely resource-intensive, and are steadily growing in size. As a result, compressing such models for resource- constrained devices is an active and promising re- search area. In spite of their current...
bachelor thesis 2023
document
de Rijk, Philip (author)
Knowledge Distillation (KD) is a well-known training paradigm in deep neural networks where knowledge acquired by a large teacher model is transferred to a small student. KD has proven to be an effective technique to significantly improve the student's performance for various tasks including object detection. As such, KD techniques mostly rely...
master thesis 2022
document
Hafner, Frank M. (author), Bhuyian, Amran (author), Kooij, J.F.P. (author), Granger, Eric (author)
Person re-identification is a key challenge for surveillance across multiple sensors. Prompted by the advent of powerful deep learning models for visual recognition, and inexpensive RGB-D cameras and sensor-rich mobile robotic platforms, e.g. self-driving vehicles, we investigate the relatively unexplored problem of cross-modal re...
journal article 2022
document
Pintea, S. (author), Liu, Yue (author), van Gemert, J.C. (author)
Knowledge distillation compacts deep networks by letting a small student network learn from a large teacher network. The accuracy of knowledge distillation recently benefited from adding residual layers. We propose to reduce the size of the student network even further by recasting multiple residual layers in the teacher network into a single...
conference paper 2018
Searched for: subject%3A%22Knowledge%255C+Distillation%22
(1 - 5 of 5)