Robust Importance-Weighted Cross-Validation under Sample Selection Bias

Conference paper (2019)

Authors

W.M. Kouw Eindhoven University of Technology, Pattern Recognition and Bioinformatics - EEMCS
J.H. Krijthe Radboud Universiteit Nijmegen
M. Loog Pattern Recognition and Bioinformatics - EEMCS
Research Group
Pattern Recognition and Bioinformatics (EEMCS) (TU Delft)
Cross-validation Sample selection bias
More Info
expand_more

Published Date

2019

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Intelligent Systems

Research Group

Pattern Recognition and Bioinformatics

Abstract

Cross-validation under sample selection bias can, in principle, be done by importance-weighting the empirical risk. However, the importance-weighted risk estimator produces suboptimal hyperparameter estimates in problem settings where large weights arise with high probability. We study its sampling variance as a function of the training data distribution and introduce a control variate to increase its robustness to problematically large weights.

Files

08918731.pdf
(.pdf | 0.309 Mb)