Quantitative Bias Analysis for Unmeasured Confounding

Why Quantify Bias?

Adjustment for measured baseline covariates cannot remove confounding by a variable that was not measured or was measured inadequately. In a trial with an external control group, for example, a clinical severity factor might be more common in one source population and predict the outcome. The resulting treatment-effect estimate can then combine a causal treatment effect with an association attributable to that factor.

Quantitative bias analysis makes this concern explicit. Rather than merely stating that unmeasured confounding is possible, the analyst specifies plausible values for how common the confounder is in each group and how strongly it is associated with the outcome. The resulting calculation is a scenario-based correction, not a discovery of the unmeasured factor's true values. Its value is that readers can see how much those assumptions would change the estimated effect.

A Binary Confounder and Risk-Difference Bias

Let \(A=1\) denote treatment and \(A=0\) denote control. Let \(Y\) be a binary adverse outcome, and let \(U\) be a binary unmeasured baseline confounder, with \(U=1\) indicating the risk factor is present. Suppose the treatment effect is reported as a risk difference, treatment risk minus control risk, so negative values indicate fewer adverse outcomes under treatment.

The following quantities describe the sensitivity scenario after adjustment or standardization for the measured covariates:

• \(q_1=P(U=1\mid A=1)\): the prevalence of \(U\) in the treated group;
• \(q_0=P(U=1\mid A=0)\): the prevalence of \(U\) in the control group;
• \(\gamma\): the change in outcome risk associated with \(U=1\) rather than \(U=0\), on the risk-difference scale.

For this basic calculation, take \(\gamma\) to be the same across treatment groups. It is an outcome association conditional on treatment and measured covariates, not necessarily a causal effect of \(U\). Write \(\widehat{\theta}_{\mathrm{adj}}\) for the estimated, measured-covariate-adjusted risk difference before bias correction. The bias component attributable to \(U\), under the assumptions below, is:

This formula requires more than choosing two prevalences and an outcome association. It assumes that the groups are comparable with respect to measured covariates after the stated adjustment; that \(U\) is measured before treatment and its prevalence is meaningfully defined in each group; that the additive association \(\gamma\) is adequately represented across the relevant covariate and treatment strata; and that the outcome-risk model has no important treatment-by-\(U\) interaction. The simple calculation also treats the specified prevalence and association values as applicable to the target population of the estimated contrast. If these conditions fail, the formula may not capture the actual bias.

Why the Formula Has This Form

The basic result follows by comparing the contribution of \(U\) to the outcome risk in the two groups. Under the additive model, the risk for group \(a\), holding measured covariates appropriately standardized, can be written as a group-specific baseline risk plus \(\gamma q_a\). The treatment-versus-control contrast therefore contains the term \(\gamma q_1-\gamma q_0\).

Thus the unmeasured factor shifts the observed contrast by its outcome association multiplied by its prevalence imbalance. If \(U\) is equally prevalent in both groups, \(q_1-q_0=0\), and this particular source of bias is zero under the model even when \(\gamma\) is large. If \(U\) is more prevalent in the treated group and increases risk, then \(B\) is positive: the observed treatment risk difference is shifted upward, making a beneficial treatment look less beneficial or more harmful. The correction subtracts that positive bias.

With measured covariates \(X\), the same reasoning can be applied within covariate strata and averaged over a specified target distribution. A single product is justified when the outcome association is constant on the risk-difference scale and the prevalence difference is defined in a way that represents that standardized comparison. If \(\gamma\) varies substantially across strata, use stratum-specific values and average the resulting bias contributions using the target-population weights rather than multiplying unweighted averages.

Applying Scenarios to Bayesian Estimates

In a Bayesian analysis, the measured-covariate-adjusted treatment effect is represented by a posterior distribution. For a fixed bias scenario, apply the same correction to every posterior draw. If \(\theta_{\mathrm{adj}}^{(m)}\) is draw \(m\) from that posterior and \(B\) is the scenario's bias, define:

The corrected posterior mean is the original posterior mean minus \(B\), and the credible interval shifts by the same amount. The interval width is unchanged because a fixed shift does not add uncertainty. This is conditional on the scenario: it describes posterior uncertainty in the treatment effect given that particular bias value, not uncertainty about which bias scenario is true.

Comparing Several Plausible Scenarios

A single correction can make the analysis appear more certain about \(U\) than the evidence permits. A scenario table is often more informative: it shows how the treatment-effect estimate changes over a range of defensible prevalence imbalances and outcome associations. The values should be justified using clinical knowledge, related data, or explicit elicitation, and should include scenarios that could move the estimate in either direction when both are plausible.

The treatment effect and the bias parameters may also be treated probabilistically. In a probabilistic bias analysis, uncertainty distributions are assigned to \(q_1\), \(q_0\), and \(\gamma\), and bias is computed across draws. For example, for each draw \(m\), calculate \(B^{(m)}=(q_1^{(m)}-q_0^{(m)})\gamma^{(m)}\), then pair or otherwise combine it with a treatment-effect draw to obtain \(\theta_{\mathrm{corr}}^{(m)}=\theta_{\mathrm{adj}}^{(m)}-B^{(m)}\). The distributions and any dependence assumptions matter: drawing \(q_1\), \(q_0\), and \(\gamma\) independently is an assumption, not a neutral default. A scenario grid instead reports results conditional on selected values and does not require probability distributions for them.

Assumptions, Diagnostics, and Alternatives

The calculation is useful only if the scenario is interpretable. Before reporting it, check that \(q_1\) and \(q_0\) refer to the same binary definition of \(U\) and the same target population as the effect estimate. Check that the proposed \(\gamma\) is an adjusted risk difference, not a risk ratio, odds ratio, or unadjusted association mistakenly inserted into an additive formula. Confirm the signs and units: prevalences are proportions, \(\gamma\) is a risk difference, and \(B\) is also a risk difference.

The assumed risks must also be feasible. Under a constant additive model, if the baseline risk among \(U=0\) is \(r_0\), then the risk among \(U=1\) is \(r_0+\gamma\); both must lie between 0 and 1. If the model implies impossible risks in relevant strata, or if the risk difference clearly varies by treatment or patient subgroup, the simple product may be inappropriate. A further warning sign is a sensitivity result that changes sharply under small, clinically plausible changes in inputs: this indicates that the conclusion depends strongly on assumptions, not that the most convenient scenario has been validated.

Quantitative bias analysis complements, rather than replaces, adjustment for measured confounders. Direct adjustment is preferable when the confounder can be measured with adequate quality and there is sufficient overlap. The earlier material on sensitivity analysis for residual confounding addresses how to examine remaining confounding in a broader sensitivity framework; this tutorial's product formula is a transparent, parameterized calculation for a particular binary confounder on the risk-difference scale. An alternative is to specify a fuller outcome and treatment model that includes \(U\), then integrate over its assumed distribution. That can accommodate more structure but requires more assumptions and implementation. Neither approach identifies an unmeasured confounder's true prevalence or outcome association from the observed treatment and outcome data alone.

A common mistake is to say, “The severity factor is harmful, so the bias must make treatment look worse,” without checking its prevalence in each group. The direction depends on both components: \(B=(q_1-q_0)\gamma\). A harmful factor more common in controls gives a negative bias in a treatment-minus-control risk difference; subtracting it moves the estimate upward. A harmful factor more common in treated patients gives a positive bias and correction moves the estimate downward. The sign should always be calculated rather than inferred from the word “confounder.”