Forward variable selection for random forest models

Forward variable selection for random forest models

Velthoen, J.J. (TU Delft Statistics)
Cai, Juan Juan (Vrije Universiteit Amsterdam)
Jongbloed, G. (TU Delft Statistics)

2022

Random forest is a popular prediction approach for handling high dimensional covariates. However, it often becomes infeasible to interpret the obtained high dimensional and non-parametric model. Aiming for an interpretable predictive model, we develop a forward variable selection method using the continuous ranked probability score (CRPS) as the loss function. eOur stepwise procedure selects at each step a variable that minimizes the CRPS risk and a stopping criterion for selection is designed based on an estimation of the CRPS risk difference of two consecutive steps. We provide mathematical motivation for our method by proving that in a population sense, the method attains the optimal set. In a simulation study, we compare the performance of our method with an existing variable selection method, for different sample sizes and correlation strength of covariates. Our method is observed to have a much lower false positive rate. We also demonstrate an application of our method to statistical post-processing of daily maximum temperature forecasts in the Netherlands. Our method selects about 10% covariates while retaining the same predictive power.

correlated covariates
CRPS
forward selection
Random forests
variable selection

http://resolver.tudelft.nl/uuid:823d6437-b58a-4d1b-8d3e-bffebdac8931

https://doi.org/10.1080/02664763.2022.2095362

0266-4763

Journal of Applied Statistics, 50 (2023) (13), 2836-2856

Institutional Repository

journal article

© 2022 J.J. Velthoen, Juan Juan Cai, G. Jongbloed