Using Nearest-Neighbors to Evaluate Overlap in Causal Inference
Rickard Carlsson – Mentor (Linnaeus University)
Jesse Krijthe – Mentor (TU Delft - Pattern Recognition and Bioinformatics)
Frans Oliehoek – Graduation committee member (TU Delft - Interactive Intelligence)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
To validate the results of a medical trial, there must be an overlap between the treatment and control groups. This implies the crucial need for good evaluation methods. This study, therefore, aimed to evaluate the overlap between causal classes using the Nearest Neighbours’ methods. Firstly, a case study was built around the common failures of those methods (i.e. dependencies on hyper-parameters and sensitivity to increasing features, samples, and outliers). Secondly, a comparison of the Nearest Neighbours to other already existing approaches was made, to determine if they vary from the standard solution. The results demonstrated that the methods can be used to assess overlap but had too much dependency on hyper-parameters, no drastic sensitivity to increasing sample and feature, and varied performance to outliers depending on their position. Additionally, the set of Nearest Neighbours methods predicted a smaller overlapping area compared to the established methods, emphasizing the caution with which the forecasts should be taken into account.