Applying Informal Benchmarking to the f-Sensitivity Model

Abstract

Sensitivity analysis asks how much unobserved confounding would overturn a causal conclusion. Every framework leaves the analyst to choose how much confounding to allow for. For the marginal sensitivity model (MSM), informal benchmarking sets this choice from the data. Each observed covariate is dropped in turn, and the resulting shift in treatment odds is taken as a plausible value. We ask whether the same idea transfers to the f-sensitivity model, whose parameter ρ bounds confounding by an average within each covariate value rather than by a single worst case. We show that it does. The transfer relies on a single new quantity, a benchmark ρ bench. This is the symmetric-KL divergence that a dropped covariate induces between the treatment arms. We take the strongest covariate rather than the average, as informal benchmarking does for the MSM and as ρ requires. We compute ρ bench from the covariates. It is stable across seeds, and it separates covariates that the MSM treats as identical. As a rare confounding spike grows, ρ bench stays nearly flat while the MSM’s worst-case reading climbs, which behavior is to be expected of. On simulated data with a known hidden confounder, the benchmark recovers the divergence that the confounder induces, and it covers the true ρ in every scenario tested. It has its shortcomings as it can under-report confounding that is concentrated in a low-density region of a covariate’s range.