Bagging, Boosting and the Random Subspace Method for Linear Classifiers

Journal article (2002)

Authors

M Skurichina ImPhys/Quantitative Imaging
R.P.W. Duin ImPhys/Quantitative Imaging
Research Group
ImPhys/Quantitative Imaging
More Info
expand_more

Published Date

2002

Language

Undefined/Unknown

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Research Group

ImPhys/Quantitative Imaging

Abstract

Recently bagging, boosting and the random subspace method have become popular combining techniques for improving weak classifiers. These techniques are designed for, and usually applied to, decision trees. In this paper, in contrast to a common opinion, we demonstrate that they may also be useful in linear discriminant analysis. Simulation studies, carried out for several artificial and real data sets, show that the performance of the combining techniques is strongly affected by the small sample size properties of the base classifier: boosting is useful for large training sample sizes, while bagging and the random subspace method are useful for critical training sample sizes. Finally, a table describing the possible usefulness of the combining techniques for linear classifiers is presented.
Key words: Bagging; Boosting; Combining classifiers; Linear classifiers; Random subspaces; Training sample size