Filtering Knowledge: A Comparative Analysis of Information-Theoretical-Based Feature Selection Methods

Filtering Knowledge: A Comparative Analysis of Information-Theoretical-Based Feature Selection Methods

Vasilev, Kiril (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Web Information Systems)

Katsifodimos, A (mentor)
Ionescu, A. (mentor)
Isufi, E. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2023-06-28

The data used in machine learning algorithms strongly influences the algorithms' capabilities. Feature selection techniques can choose a set of columns that meet a certain learning goal. There is a wide variety of feature selection methods, however, the ones we cover in this comparative analysis are part of the information-theoretical-based family. We evaluate MIFS, MRMR, CIFE, and JMI using the machine learning algorithms Logistic Regression, XGBoost, and Support Vector Machines.
Multiple datasets with a variety of feature types are used during evaluation. We find that MIFS and MRMR are 2-4 times faster than CIFE and JMI. MRMR and JMI choose columns that lead to significantly higher accuracy and lower root mean squared error earlier. The results we present here can help data scientists pick the right feature selection method depending on the datasets used.

Data Augmentation
Feature Selection
Information Theory
Comparative analysis

http://resolver.tudelft.nl/uuid:fbcf96d8-3685-4838-85e6-ee6887c25e15

Student theses

bachelor thesis

© 2023 Kiril Vasilev