Robust Importance-Weighted Cross-Validation under Sample Selection Bias

None, None; None, None; None, None

Robust Importance-Weighted Cross-Validation under Sample Selection Bias

Conference Paper (2019)

Author(s)

Wouter M. Kouw (TU Delft - Electrical Engineering, Mathematics and Computer Science, Eindhoven University of Technology)

Jesse H. Krijthe (Radboud Universiteit Nijmegen)

Marco Loog (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Research Group

Pattern Recognition and Bioinformatics

Cross-validation Sample selection bias

DOI related publication

https://doi.org/10.1109/MLSP.2019.8918731 Final published version

To reference this document use

https://resolver.tudelft.nl/uuid:da88aa2f-659e-44e9-a9c3-9c1c625afdd7

More Info

expand_more

Publication Year

2019

Language

English

Research Group

Pattern Recognition and Bioinformatics

Article number

8918731

Pages (from-to)

1-6

ISBN (print)

978-1-7281-0825-4

ISBN (electronic)

978-1-7281-0824-7

Event

29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019 (2019-10-13 - 2019-10-16), Pittsburgh, United States

Downloads counter

183

Abstract

Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.