Why a Negative Control Outcome Needs Formal Conditions
A treatment association with an outcome that the treatment cannot cause can be a warning that treatment groups differ in ways not adequately controlled. But the warning is informative only under specific conditions. The outcome must truly be unaffected by treatment, and the sources of residual treatment-outcome confounding must also be reflected in the control outcome. Without both features, an association can mislead, and a null association can offer false reassurance.
Let \(A\) denote treatment, \(Y\) the primary outcome, \(W\) a candidate negative control outcome, \(X\) measured pretreatment covariates, and \(U\) unmeasured factors. The target question concerns the causal effect of \(A\) on \(Y\); the control outcome is used as a diagnostic, not as a substitute outcome or a direct correction for bias.
The individual-level condition is stronger than saying that the average treatment effect on \(W\) is zero: positive effects for some individuals and negative effects for others could average to zero while treatment still changes the outcome distribution. For a diagnostic based on a mean association, the required null should at least ensure that treatment does not change the relevant conditional mean of \(W\). The stronger potential-outcome condition makes the logic especially clear.
When the Treatment Association Should Be Null
Suppose \(W\) is measured under a well-defined treatment assignment, consistency holds, and treatment is conditionally exchangeable with the potential control outcomes given \(X\): \(A \perp W(a)\mid X\) for \(a=0,1\). Consistency means the observed control outcome is \(W=W(A)\). If \(W(1)=W(0)=W\), then within levels of \(X\), the treated and untreated groups should have the same mean of \(W\).
The first equality follows from consistency, the second from conditional exchangeability, and the last from the no-effect condition. Thus the conditional mean contrast \(\Delta_W(x)=E[W\mid A=1,X=x]-E[W\mid A=0,X=x]\) is zero under these conditions. A nonzero contrast is inconsistent with at least one of the conditions or with ordinary sampling variation. It is not, by itself, proof that unmeasured confounding caused the result.
For a treatment association with \(W\) to diagnose residual confounding of the \(A\)-\(Y\) relationship, a further condition is needed: the relevant residual confounding must be shared. For example, after conditioning on \(X\), an unmeasured factor \(U\) may affect treatment assignment, predict the primary potential outcomes \(Y(a)\), and also affect \(W\). If \(U\) shifts the composition of treated and untreated groups and \(W\) reflects that shift, a treatment association with \(W\) can reveal imbalance that matters for the primary analysis.
This shared-confounding condition is substantive, not automatic. A factor that confounds treatment and \(Y\) may have no relationship with \(W\), in which case the control outcome cannot reveal it. Conversely, a treatment association with \(W\) can arise from residual imbalance in measured covariates, selection, measurement problems, or chance rather than from a confounder shared with \(Y\). The diagnostic interpretation therefore depends on design and subject-matter knowledge as well as the observed association.
- No causal effect: treatment does not change \(W\) in the relevant population and time frame.
- Comparable analysis: the treatment groups are compared in the same target population and with appropriate control for measured pretreatment covariates \(X\).
- Shared-confounding relevance: at least some residual factor that affects treatment assignment and the primary potential outcomes also predicts \(W\).
- Interpretable association: the association is not generated by a separate pathway such as selection or measurement artifacts, and it is not canceled by opposing imbalances.
Worked Examples
Worked Example: A Shared Confounder Creates a Control-Outcome Association
Consider a hypothetical population with binary \(U\), where \(P(U=1)=0.50\). Treatment probabilities are \(P(A=1\mid U=1)=0.80\) and \(P(A=1\mid U=0)=0.20\). Suppose \(W\) is binary, unaffected by treatment, and \(P(W=1\mid U=1)=0.70\), while \(P(W=1\mid U=0)=0.20\). Assume treatment and \(W\) are independent conditional on \(U\), and treatment is assigned independently of the primary potential outcomes conditional on \(U\).
First calculate the composition of treatment groups. The treated proportion with \(U=1\) is \[ P(U=1\mid A=1)=\frac{0.80(0.50)}{0.80(0.50)+0.20(0.50)}=0.80. \] Among untreated individuals, \[ P(U=1\mid A=0)=\frac{0.20(0.50)}{0.20(0.50)+0.80(0.50)}=0.20. \]
By the law of total probability, the control-outcome risks are \[ P(W=1\mid A=1)=0.80(0.70)+0.20(0.20)=0.60, \] and \[ P(W=1\mid A=0)=0.20(0.70)+0.80(0.20)=0.30. \] The observed risk difference is \(0.60-0.30=0.30\), or 30 percentage points. Because \(W\) is unaffected by treatment, this contrast is not a causal effect of treatment on \(W\). It reflects the different distributions of \(U\) in the two groups.
To verify that the example also represents confounding of the primary outcome, stipulate the potential-outcome risks \(P(Y(a)=1\mid U)=0.10+0.20U+0.05a\), with consistency, positivity, and \(A\perp\{Y(0),Y(1)\}\mid U\). The causal risk difference is \(0.05\) at either value of \(U\), and therefore also \(0.05\) on average. The observed treated risk is \(0.10+0.20(0.80)+0.05=0.31\); the observed untreated risk is \(0.10+0.20(0.20)=0.14\). Their difference is \(0.17\). The \(0.12\) difference between the observed contrast and the stipulated causal risk difference is attributable here to the groups’ different \(U\) composition under the stated model. The nonnull control association is a useful warning because this same \(U\) predicts \(Y(a)\) and treatment assignment.
Worked Example: A Null Control Association Can Miss Confounding
Now retain the same hypothetical treatment assignment and primary-outcome potential risks as above, so \(P(U=1)=0.50\), \(P(A=1\mid U=1)=0.80\), \(P(A=1\mid U=0)=0.20\), and \(P(Y(a)=1\mid U)=0.10+0.20U+0.05a\). Keep consistency, positivity, and conditional exchangeability of treatment given \(U\). Suppose instead that \(W\) is unaffected by treatment and independent of \(U\), with \(P(W=1\mid U)=0.40\) at either level of \(U\).
The treated and untreated groups still have \(U\) prevalences of \(0.80\) and \(0.20\), respectively. The primary-outcome risks are consequently \(0.31\) and \(0.14\), an observed risk difference of \(0.17\) compared with the causal risk difference of \(0.05\). There is confounding of the primary association under this stipulated model.
For the control outcome, however, \[ P(W=1\mid A=1)=0.80(0.40)+0.20(0.40)=0.40, \qquad P(W=1\mid A=0)=0.20(0.40)+0.80(0.40)=0.40. \] Thus the control risk difference is \(0.40-0.40=0\). The null occurs because \(W\) does not reflect \(U\), the factor driving the imbalance. This example shows why a null negative-control result cannot establish that residual confounding is absent.
Worked Example: Covariate Imbalance Produces a Crude Association
Suppose \(X\) is binary with \(P(X=1)=0.50\), and treatment is randomized within each \(X\) stratum, but with probabilities \(P(A=1\mid X=1)=0.80\) and \(P(A=1\mid X=0)=0.20\). Let \(W\) be unaffected by treatment, with \(P(W=1\mid X=1)=0.80\) and \(P(W=1\mid X=0)=0.20\). Assume treatment assignment within \(X\) is independent of \(W\).
The treated group has \(P(X=1\mid A=1)=0.80\), and the untreated group has \(P(X=1\mid A=0)=0.20\), by the same Bayes calculations used above. The crude control risks are \[ P(W=1\mid A=1)=0.80(0.80)+0.20(0.20)=0.68, \] and \[ P(W=1\mid A=0)=0.20(0.80)+0.80(0.20)=0.32. \] The crude difference is \(0.36\), or 36 percentage points, even though treatment has no effect on \(W\). Within each \(X\) stratum, the risk difference is zero: it is \(0.20-0.20=0\) when \(X=0\), and \(0.80-0.80=0\) when \(X=1\). Here the crude association is due to measured-covariate imbalance, not residual unmeasured confounding. The comparison must therefore condition on or standardize over the relevant \(X\) distribution before interpreting it as a residual-confounding diagnostic.
Interpretation, Diagnostics, and Limitations
A nonnull adjusted association between treatment and a credible negative control outcome is evidence that the observed data depart from the idealized conditions that would make the association null. If the outcome is genuinely unaffected by treatment and the analysis controls measured covariates appropriately, shared residual confounding is one plausible explanation. The size of the association does not, without additional assumptions, quantify the bias in the primary treatment effect or determine whether that effect is overestimated or underestimated.
A null association is also limited evidence. It may indicate little residual imbalance in factors represented by \(W\), but it can occur despite important confounding when the relevant confounder does not predict \(W\), when the control measurement is noisy, or when different sources of imbalance cancel. A nonsignificant test is not proof of exact equality; sampling variability may obscure a real association. The introductory material on falsification tests emphasizes the diagnostic role of such checks, not their ability to certify an analysis as unbiased.
Before interpreting a control-outcome result, check whether the treatment could plausibly affect the outcome directly or indirectly, whether the outcome is measured comparably across treatment groups, whether the adjustment set handles important pretreatment covariates, and whether the control outcome plausibly reflects factors relevant to the primary outcome. Consider selection into the analytic sample and missingness as possible sources of associations that do not represent shared confounding. These checks diagnose assumptions; they do not prove them.
Negative control outcomes complement rather than replace direct adjustment for measured confounders and design-based protections such as randomization. They differ from sensitivity analysis, including E-values and Rosenbaum bounds, which assess how strong unmeasured bias would need to be to alter conclusions under specified frameworks. A negative control probes a particular empirical implication of the assumed causal structure; it does not guarantee that all unmeasured confounding has been detected.
Specify why treatment cannot affect \(W\) in the population and period being analyzed; distinguish this claim from simply observing a small association.
State the target population and account for measured pretreatment covariates \(X\) so that crude imbalance is not mistaken for residual confounding.
Ask whether residual factors that influence treatment and the primary potential outcomes can also influence or predict \(W\).
Use nonnull results as warnings requiring explanation and null results as limited reassurance, not as proof for or against confounding.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- What does \(W(1)=W(0)\) state, and how is it different from observing equal treatment-group means of \(W\)?
- Under what assumptions does conditional exchangeability imply a zero treatment association with \(W\) within \(X\) strata?
- Why can a negative control outcome be null even when the primary treatment-outcome association is confounded?
- In the covariate-imbalance example, what are the crude control risks and their difference, and why is the within-\(X\) comparison zero?
- Does a nonnull negative-control association estimate the amount of bias in the primary treatment effect? Explain.
Solutions. (1) It states that treatment does not change the control outcome for any individual. Equal observed group means are a feature of the data and can occur by chance or cancellation; they do not establish the causal null. (2) Consistency, \(W(1)=W(0)\), and \(A\perp W(a)\mid X\) are sufficient for the displayed conditional mean equality; the comparison must also use the appropriate \(X\) strata or standardization. (3) The confounder may affect treatment and the primary potential outcomes but not \(W\), or its association with \(W\) may be weak or canceled. (4) The risks are \(0.68\) for treated and \(0.32\) for untreated, giving a crude difference of \(0.36\). Within each \(X\) stratum treatment is independent of \(W\), so each stratum-specific difference is zero. (5) No. The association is diagnostic evidence, but it need not share the magnitude or direction of bias in the primary estimate and may have explanations other than shared residual confounding.