Chapter 1 (Caliper Matching): Caliper matching is used to estimate causal effects of a binary treatment from observational data by comparing matched treated and control units. Units are matched when their propensity scores, the conditional probability of receiving treatment given pretreatment covariates, are within a certain distance called caliper. So far, theoretical results on caliper matching are lacking, leaving practitioners with ad-hoc caliper choices and inference procedures. We bridge this gap by proposing a caliper that balances the quality and the number of matches. We prove that the resulting estimator of the average treatment effect, and average treatment effect on the treated, is asymptotically unbiased and normal at parametric rate. We describe the conditions under which semiparametric efficiency is obtainable, and show that when the parametric propensity score is estimated, the variance is increased for both estimands. Finally, we construct asymptotic confidence intervals for the two estimands.

Chapter 2 (Combining Experimental And Observational Data: The APOLLO Trial): In the APOLLO trial (Tol et al., 2022, 2024), we inferred the causal effects of two hip fracture treatments, Posterolateral Approach (PLA) and Direct Lateral Approach (DLA), on health outcomes of patients in the Netherlands. The starting point of the inference was a Randomised Experiment (RE), where patients were randomly assigned to PLA or DLA, independently of their baseline characteristics. In addition, data from a `Natural Experiment' (NE, or observational data) were also collected, under the plausible assumption that therein the allocation to PLA or DLA can be considered as good as random conditional on the patients' baseline characteristics. We estimated the average treatment effects of DLA versus PLA in the RE and the NE data separately, using a flexible and asymptotically efficient estimation strategy. We found no significant difference between PLA and DLA in any of the RE or the NE datasets. To improve the precision of the inference by increasing the sample size, we tested whether the RE and the NE datasets can be combined. Having found no evidence against combination, we estimated the average treatment effect on the combined dataset as well. Despite the improved precision, there was still no significant difference between PLA and DLA. Our conclusions were weakened by missing data, but they proved to be robust to our approach in handling the missingness and to our estimation strategy.

Chapter 3 (Private Double Robust Inference): Privacy mechanisms preserve the privacy of individuals in a sample by injecting noise into their sensitive data in a controlled manner, revealing only the noisy, privatised data to the statistician for inference purposes. The inference of a parameter exhibits a rate double robustness property when the large-sample bias of an estimator of the parameter is characterised by the product of the estimation errors of two other, auxiliary (or nuisance), often infinite-dimensional, parameters. We propose a novel class of rate double robust parameters whose novelty lies in the potentially nonlinear but smooth dependence on a low-dimensional regression parameter. Among others, this includes average treatment effects. We show that the properties of the sensitive-data model carry over to the privatised-data model by a suitable choice of the privacy mechanism, which, in general, means a total-variationally private mechanism. In particular, the double robustness property is retained, enabling efficient estimation from the privatised sample. We also find that the estimation of the nuisance parameters is not harder, albeit possibly less efficient and computationally more demanding, from the privatised sample compared to the sensitive sample for a given, suitable privacy mechanism. Indeed, if the estimation is feasible from the sensitive sample by some procedure, we can directly transport that procedure to the privatised setting. Lastly, we develop a private method of moments estimator for parametric models. This shows that in the private setting a parametric assumption about one nuisance parameter affords more flexible modelling and slower estimation of the other one, as is the well-known case in the nonprivate setting.

Causal relationships are at the heart of the scientific method. The causal revolution of the 21st century has opened the doors for many new approaches to quantify such relationships. In this thesis, we study the novel framework of proximal causal inference, which enables estimation of causal parameters even in the presence of unmeasured confounders, overcoming the limitations imposed by the Conditional Exchangeability assumption of the classic causal framework. In particular, we shall focus on determining and estimating the Causal Exposure Response Function (CERF) under this new set of assumptions. First, we introduce the problem and present a literature review of existing approaches to estimate bridge functions; then, we show theoretical results for extensions of classic linear results and a novel quasi- Bayesian method. This is then completed by showcasing performance on many simulated numerical examples and two real-world problems: Sustainable Causal Investing, and the effect of exercise on sleep.

Suppose that we want to infer the effect of a treatment on a certain outcome, where both the treatment and outcome are influenced by other variables. It has been well-established that in the linear setting, in case we know beforehand which of these other variables are instrumental (for the effect of the treatment on the outcome), we can infer the treatment effect in a consis-tent sense. This thesis analyses 3 methods that deals with the issue of unknown instrumental variables (IVs) and functional relationships in different ways to infer the treatment effect. The first method, Causal Inference with Invalid Instruments (CIII), assumes that we have a linear setting and a set with potential instrumental variables for whom a majority or plurality rule holds to obtain a robust confidence interval for the treatment effect. The second method, Anchor Regression (AR), only assumes a linear setting. By mediating between different meth-ods, the AR-estimator turns out the be robust to changes in the distribution of the sampled data. Lastly, Two Stage Curvature Identification (TSCI), does not require a linear setting or information on the IVs. Instead, it relies on the difference in functional form between the effect of the variables on the treatment and the effect of the variables on the outcome for consistent estimation and asymptotic normality. TSCI also provides a test for IV presence in the non-linear setting. In this thesis, I will explain the workings of these 3 methods, analyse their theoretical foundation and do simulation studies. Based on these analyses, I make several additions and suggestions to expand the theoretical scope and improve practical efficacy.3

In this thesis, we study a particle method for Bayesian deep learning. In particular, we look at the estimation of the parameters of an ensemble of Bayesian neural networks by means of this particle method, called Stein variational gradient descent (SVGD). This method iteratively updates a collection of parameters and it has the property that its update directions are chosen such that they optimally decrease the Kullback-Leibler divergence. We also study gradient flows of probability measures and show how gradient flows corresponding to functionals on the space of probability measures can induce particle flows. We formulate SVGD as a method in this space. In the regime of infinite particles we show results about convergence of SVGD. An existing convergence result for SVGD can be extended by showing that the probability measures, governing the collection of SVGD particles, are uniformly tight. We give conditions under which this holds.

In problems with missing data, the data are often considered to be missing at random. This assumption can not be checked from the data. We need to assess the sensitivity of study conclusions to violations of non-identifiable assumptions. This thesis performs Bayesian sensitivity analysis for a missing data model with life time outcomes and covariate information. The outcome distribution is modelled through a Cox model, with a beta process prior on the cumulative hazard function. We run experiments in a simulation study to test the performance of the model in scenarios with simulated data of several sample sizes. We show the validity of the model in the context of Bayesian sensitivity analysis, and propose extensions.