To validate the results of a medical trial, there must be an overlap between the treatment and control groups. This implies the crucial need for good evaluation methods. This study, therefore, aimed to evaluate the overlap between causal classes using the Nearest Neighbours’ meth
...
To validate the results of a medical trial, there must be an overlap between the treatment and control groups. This implies the crucial need for good evaluation methods. This study, therefore, aimed to evaluate the overlap between causal classes using the Nearest Neighbours’ methods. Firstly, a case study was built around the common failures of those methods (i.e. dependencies on hyper-parameters and sensitivity to increasing features, samples, and outliers). Secondly, a comparison of the Nearest Neighbours to other already existing approaches was made, to determine if they vary from the standard solution. The results demonstrated that the methods can be used to assess overlap but had too much dependency on hyper-parameters, no drastic sensitivity to increasing sample and feature, and varied performance to outliers depending on their position. Additionally, the set of Nearest Neighbours methods predicted a smaller overlapping area compared to the established methods, emphasizing the caution with which the forecasts should be taken into account.