Explain Strange Learning Curves in Machine Learning

Bachelor Thesis (2022)
Author(s)

Z. Chen (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Contributor(s)

Tom Julian Viering – Mentor (TU Delft - Computer Science & Engineering-Teaching Team)

M. Loog – Mentor (TU Delft - Pattern Recognition and Bioinformatics)

G. Smaragdakis – Graduation committee member (TU Delft - Cyber Security)

Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright
© 2022 Zhiyi Chen
More Info
expand_more
Publication Year
2022
Language
English
Copyright
© 2022 Zhiyi Chen
Graduation Date
20-06-2022
Awarding Institution
Delft University of Technology
Project
['CSE3000 Research Project']
Programme
['Computer Science and Engineering']
Faculty
Electrical Engineering, Mathematics and Computer Science
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

The learning curve illustrates how the generalization performance of the learner evolves with more training data. It can predict the amount of data needed for decent accuracy and the highest achievable accuracy. However, the behavior of learning curves is not well understood. Many assume that the more training data provided, the better the learner performs. However, many counter-examples exist for both classical machine learning algorithms and deep neural networks. As presented in previous works, even the learning curves for simple problems using classical machine learning algorithms have unexpected behaviors. In this paper, we will explain what caused the odd learning curves generated while using ERM to solve two regression problems. Loog et al. [1] first proposed these two problems. As a result of our study, we conclude that the unexpected behaviors of the learning curves under these two problem settings are caused by incorrect modeling or the correlation between the expected risk and the output of the learner.

Files

Mandatory_cover_Page_5_.pdf
(pdf | 0.817 Mb)
License info not available