Calibrating Bias With Negative Control Outcomes

From a Control Association to a Bias Adjustment

A negative control outcome can reveal that an analysis retains an association between treatment and an outcome that treatment should not cause. That association is a diagnostic signal, not automatically a numerical estimate of bias in the outcome of interest. To use it for calibration, an analyst must specify how the bias affecting the control outcome relates to the bias affecting the target outcome.

This tutorial develops a simple additive calibration. It treats the estimated association for the target outcome as the sum of the target causal effect and bias. The control outcome has no causal effect of treatment, so its association represents bias under the control’s own assumptions. Calibration becomes possible only with a further transport assumption connecting those two biases.

For example, an additive contrast might be a difference in mean outcome between treatment groups after a stated adjustment procedure. The association and effect must refer to the same treatment contrast and target population. In an analysis using matching or another adjustment method, \(A_Y\) and \(A_C\) refer to the associations produced by that same analysis. The estimates are not made comparable merely by calling both of them “associations.”

The treatment effect on the control outcome must be absent for the chosen estimand. A control outcome caused by treatment, even indirectly, does not have \(\tau_C=0\) and cannot be treated as a pure bias indicator. In potential-outcome notation, a strong version of the no-effect condition is \(C(1)=C(0)\) for every unit; for a mean-effect analysis, the required condition is at least \(E[C(1)-C(0)]=0\) in the target population.

Deriving the Calibration

The key extra assumption is a model for how the bias in the target outcome relates to the bias in the control outcome. The simplest choice is equal additive bias. A more flexible choice allows a known proportionality factor \(\lambda\). This factor is a substantive calibration assumption, not something established just by observing a control association.

The result follows directly from the definitions. Since \(B_Y=A_Y-\tau_Y\), transport of bias gives \(A_Y-\tau_Y=\lambda B_C\). The negative control condition gives \(B_C=A_C-\tau_C=A_C\). Rearranging yields the calibrated causal effect:

In data, the population associations are unknown. Replacing them with their estimates gives the plug-in calibrated estimate \(\hat{\tau}_Y=\hat{A}_Y-\lambda\hat{A}_C\), when \(\lambda\) is treated as specified. This estimate corrects the target association only if the assumptions above are credible. A nonzero control association does not by itself show that the target bias equals the control bias, and a zero control association does not prove that the target analysis is unbiased.

What Must Transport From Control to Target?

The transport assumption is most plausible when the source of residual bias and the way it enters both outcomes are understood. If an unmeasured factor influences treatment and both outcomes, it may induce associations in each. But even a shared factor need not create equal biases: the outcomes can differ in their relationship to that factor, their measurement, or the population contrast being estimated. The proportionality factor \(\lambda\) represents this difference; it is not necessarily one.

A defensible calibration therefore needs more than a shared label such as “unmeasured confounding.” It needs a reason to believe that the control association tracks the target bias at the chosen value of \(\lambda\). In particular:

• The control outcome must not be caused by treatment for the causal estimand being calibrated.
• The target and control associations must use the same treatment contrast, population, and relevant adjustment procedure.
• The control association must reflect bias relevant to the target analysis, rather than a distinct source of association that does not affect the target.
• The assumed relationship \(B_Y=\lambda B_C\) must be substantively justified or explored as a sensitivity assumption.

These conditions can fail in different ways. A treatment effect on \(C\) makes \(A_C\) equal to bias plus a causal effect, so subtracting it as if it were all bias miscalibrates the target. A bias mechanism that affects \(Y\) but not \(C\) is not captured by the control. Conversely, a control may have a strong association due to a mechanism irrelevant to \(Y\). Biases can also cancel within the control association, leaving \(A_C\) near zero despite important bias elsewhere.

Sampling Uncertainty and Sensitivity

A calibrated estimate is computed from two estimated associations, which may be statistically dependent because they come from the same individuals or analysis. If \(\lambda\) is treated as fixed, the variance of their difference includes their covariance:

The covariance term cannot generally be omitted. A standard error based on this variance describes sampling uncertainty conditional on the chosen calibration model and fixed \(\lambda\). It does not account for uncertainty about whether the control is valid or whether the assumed bias relationship is correct. If \(\lambda\) is estimated rather than fixed, its uncertainty and its covariance with the association estimates also need to be included; treating it as known would understate uncertainty.

When the proportionality assumption is uncertain, report how the calibrated estimate changes across plausible values of \(\lambda\), rather than presenting one value as assumption-free. For example, with \(\hat{A}_Y=4.8\) days and \(\hat{A}_C=1.6\) days, allowing \(\lambda\) to range from 0.5 to 1.5 gives estimates from \(4.8-(1.5)(1.6)=2.4\) days to \(4.8-(0.5)(1.6)=4.0\) days. This range reflects uncertainty about the calibration factor, not a confidence interval for sampling uncertainty. If the control association is negative or the sign of \(\lambda\) is uncertain, calculate the endpoints rather than assuming the same ordering.

Diagnostics, Alternatives, and Limits

A useful diagnostic is to ask whether the proposed control is expected to share the target’s relevant bias mechanism, and then whether the assumed value of \(\lambda\) is credible. Multiple controls may help reveal that results vary across outcomes, but agreement does not prove that their biases transport to the target. A precise near-zero control association is also not a guarantee of no target bias: the control may be insensitive to the relevant mechanism, or different contributions to its association may cancel.

Calibration differs from simply reporting a negative control test. A falsification test asks whether the observed control association is inconsistent with a specified null expectation and can indicate problems with the analysis. Bias calibration goes further by imposing a quantitative relationship between control and target biases. Other approaches include reporting the unadjusted association alongside the control results, conducting sensitivity analyses over possible \(\lambda\) values, or using a different design or adjustment strategy. These approaches do not remove the need to state assumptions; they differ in how much they claim the control result can correct the target estimate.

The main limitation is structural: the data provide \(\hat{A}_Y\) and \(\hat{A}_C\), but do not by themselves identify the target bias-to-control bias ratio. Calibration is consequently assumption-dependent. It is best treated as a transparent quantitative analysis of a stated bias model, not as an automatic correction that converts an observational association into a causal effect.