Sensitivity Analysis as a Causal Robustness Problem

Why Sensitivity Analysis Is a Causal Robustness Problem

An adjusted estimate from an observational study can be precise and still depend on an untestable claim: that the measured variables suffice to control confounding. Sensitivity analysis makes the consequences of relaxing that claim explicit. It asks how the causal conclusion would change if residual confounding produced a bias of a specified size, or within a specified range.

The first task is to keep three quantities separate: the target estimand (the causal effect we want to learn), the observed estimate (what the analysis calculates from the data under its identifying assumptions), and the sensitivity parameter (a value describing how far the target may differ from the quantity estimated under those assumptions). Sensitivity analysis does not make an untestable assumption disappear. It shows which conclusions depend on it.

Target Estimand and Identifying Assumptions

For example, an average effect on days alive and out of hospital over 30 days is not the same estimand as an effect on symptom score at six months. The units and direction of the outcome also matter: if a higher outcome is better, a positive \(\tau\) favors treatment; if a higher outcome is worse, it favors control.

In an observational study, one common identification strategy adjusts for measured pre-treatment covariates \(X\). The following assumptions state when that strategy identifies the average treatment effect:

• Consistency: for a person who receives treatment \(A=a\), the observed outcome equals that person's potential outcome \(Y(a)\). The treatment must be sufficiently well-defined for this correspondence to make sense.
• Conditional exchangeability: \(Y(a)\mathrel{\perp\!\!\!\perp} A\mid X\) for each \(a\in\{0,1\}\). Given the measured covariates, treatment assignment carries no further information about the potential outcomes.
• Positivity: within covariate values represented in the target population, each treatment option has a positive probability: \(0<P(A=a\mid X=x)\) for \(a=0,1\).
• No interference: one person's treatment does not change another person's potential outcome, unless the estimand and potential outcomes explicitly account for such effects.

With these assumptions, the causal mean under treatment level \(a\) is identified by averaging the observed conditional outcome mean over the target covariate distribution:

Here \(P_T(X=x)\) denotes the distribution of covariates in the target population. For continuous covariates, the sum is replaced by an integral. In practice, an analysis estimates the conditional means and averages the resulting contrasts over the target population. Correct implementation also requires that the estimator adequately estimate those conditional means and that the data support the target covariate distribution. These are estimation and data-support concerns, distinct from the causal identification assumptions above.

Representing Residual Confounding With a Sensitivity Parameter

To examine a possible failure of conditional exchangeability, define a discrepancy between the causal target and the quantity identified by the measured-covariate analysis. Let \(\theta_{\mathrm{ID}}\) be the observed-data estimand identified by that analysis if its identifying assumptions hold. Define the additive sensitivity parameter

The parameter \(b\) is expressed in the outcome's units. It is not generally identifiable from the observed data: different levels of unmeasured confounding can be compatible with the same observed distribution. A sensitivity analysis therefore specifies values or ranges for \(b\), explains why they are relevant, and reports the resulting causal estimates. The parameter is a transparent way to represent a departure from the identifying assumptions; it is not a correction automatically learned from the data.

This additive representation is useful when the effect and possible bias can be compared on a meaningful scale. It is a general sensitivity framework, not a claim that every form of confounding produces a constant additive shift in every population. More elaborate sensitivity models may define the departure differently. Whatever the parameterization, the analysis must say what the parameter means, its units or scale, and how it relates to the target estimand.

Sampling Uncertainty and Sensitivity Uncertainty

An estimate varies across samples even when the identifying assumptions hold. A confidence interval describes sampling uncertainty under the statistical model and assumptions used to construct it. Sensitivity analysis addresses a different uncertainty: whether the identifying assumptions are adequate, and how large the resulting discrepancy \(b\) might be. A narrow confidence interval does not, by itself, rule out substantial unmeasured confounding.

If \(b\) is treated as a fixed specified value, subtracting it from an estimate shifts the entire interval by the same amount. If the original interval is \([L,U]\), the shifted interval is \([L-b,U-b]\). This interval still reflects sampling uncertainty conditional on that value of \(b\); it does not incorporate uncertainty about whether that value is plausible. If \(b\) is allowed to vary over a range, reporting the results across that range shows how the conclusion changes, but the range itself needs justification.

Identification Assumptions Versus Robustness Assumptions

An identifying assumption is part of the argument that observed data determine a causal estimand. For the measured-covariate strategy above, conditional exchangeability is such an assumption. A sensitivity assumption instead specifies how the target might depart from the estimate if that identifying assumption fails. The two play different roles: the first supplies identification; the second evaluates dependence on that identification claim.

A useful sensitivity analysis keeps the estimand fixed while varying the assumed departure. Changing the target population, outcome, or treatment contrast changes the question, rather than merely testing robustness to confounding. It is also important to state which assumptions are held fixed across scenarios. For example, varying \(b\) does not automatically address positivity problems, an incorrectly defined treatment, or model misspecification.

A Short Derivation and a Practical Workflow

The additive adjustment follows directly from the definition \(b=\theta_{\mathrm{ID}}-\tau\). Rearranging gives \(\tau=\theta_{\mathrm{ID}}-b\). Replacing the unknown \(\theta_{\mathrm{ID}}\) with its observed estimate gives \(\hat{\tau}(b)=\hat{\theta}_{\mathrm{ID}}-b\). If a confidence interval for \(\theta_{\mathrm{ID}}\) is \([L,U]\) and \(b\) is fixed, subtracting the same constant from each possible endpoint gives \([L-b,U-b]\). If instead \(b\) is only known to lie between \(b_L\) and \(b_U\), the range of adjusted point estimates is \([\hat{\theta}_{\mathrm{ID}}-b_U,\hat{\theta}_{\mathrm{ID}}-b_L]\), because the adjusted estimate decreases as \(b\) increases.

Common Mistakes and Why This Matters

• Calling the observed estimate the causal effect without qualification. The estimate has a causal interpretation only under the stated identification assumptions and adequate estimation. Label it as an observed-data estimate under a specified strategy.
• Changing the estimand while claiming to test robustness. A different target population or outcome asks a different causal question. Keep the target fixed when varying a confounding sensitivity parameter.
• Treating a sensitivity value as an estimated fact. The observed data generally do not reveal the amount of bias from an unmeasured variable. Describe the value as a scenario, and justify any range.
• Confusing interval width with protection against confounding. A confidence interval can be narrow while the identifying assumptions remain questionable. Report sampling uncertainty and sensitivity results as distinct parts of the analysis.
• Interpreting a favorable sensitivity range too broadly. Robustness over a chosen range means robust over that range and parameterization. It does not prove robustness to every unmeasured-confounding structure.

Sensitivity analysis matters because causal conclusions are often conditional on assumptions that cannot be verified from the observed data alone. Making the target, estimate, and sensitivity parameter explicit lets readers see both what the analysis identifies under its assumptions and how much a stated violation would change the result.