The large amounts of observational data available nowadays have sparked considerable interest in learning causal relations from such data using machine learning methods. One recent method for doing this, which provided promising results, is the DragonNet (Shi et al., 2019), which utilises neural networks in order to estimate average treatment effects in populations. The performance of the model, however, was not tested on datasets which contain low amounts of overlap between the treated and non-treated subpopulations, which makes it harder to accurately estimate treatment effects. Therefore, the goal of this paper is to investigate the performance of the DragonNet when used on datasets with (near) overlap violations. This has been done by looking at the mean absolute errors and variances of the estimated treatment effects and comparing these to other models. The results showed that the performance of the DragonNet becomes significantly worse compared to other models when large portions of the population suffer from low overlap. Additionally, the variance of the results also increases in these cases, making the results less reliable. From the obtained results, it can be concluded that it is best to choose another model for treatment effect estimation if relatively large amounts of overlap violations are suspected.

Causal inference methods are often used for estimating the effects of an action on an outcome using observational data, which is a key task across various fields, such as medicine or economics. A number of methods make use of representation learning to try to obtain more informative feature representations, which are then used for effect estimation. However, such feature representations can introduce confounding bias in the results if they lose confounding information contained within the original features. In this work, we evaluate an existing metric for measuring confounding bias in representations and use this metric to study two representation learning methods in terms of their biases in settings with low overlap between treated and control populations. We show that the metric is a suitable measure for the amount of confounding bias in a representation and representation learning methods that minimise this bias lead to better average treatment effect estimation in our experiments.