Why Look Across More Than One Control?
A negative control is useful only through a causal argument: it should lack a particular causal connection while retaining a connection to the confounding or selection process under investigation. A single control can be uninformative if it fails to share that process, and a surprising association can arise because the control was not valid. Comparing several controls can reveal whether the evidence forms a coherent pattern or depends on one questionable choice.
Triangulation means examining the results from distinct controls together with their causal structures. It does not mean counting significant tests, averaging unlike associations, or treating agreement as proof that the primary estimate is biased. The central question is whether the controls, considered individually and jointly, behave as expected under plausible confounding pathways.
Write \(L\) for measured covariates, \(U\) for one or more unmeasured factors, \(A\) for the treatment, and \(Y\) for the primary outcome. A control outcome should not be \(Y\) itself: if \(A\) can affect \(Y\), then an association between \(A\) and \(Y\) is the effect association under study, not a negative-control result. Controls must also be chosen for the particular causal question. For example, an outcome measured before treatment cannot be caused by that later treatment, but it is informative about confounding only if it shares relevant causes of treatment assignment and outcome measurement or occurrence.
For a useful comparison, the controls should be analyzed in a way that makes their associations interpretable alongside the primary analysis: define the time ordering, adjust for relevant measured covariates, and state the association measure and target population. Their causal roles need not be identical, but the reason each is expected to share a pathway should be explicit. An association on one outcome’s scale cannot be compared numerically with an association on a different scale as if the units were interchangeable.
Why Shared Confounding Can Produce Control Associations
A short linear derivation illustrates the diagnostic logic. Consider variables after accounting for \(L\), so that they represent residual variation not explained by measured covariates. Suppose a valid control outcome \(C\) has no causal effect from \(A\), while an unmeasured factor \(U\) affects both \(A\) and \(C\). For illustration, let
where \(U\), \(\varepsilon_A\), and \(\varepsilon_C\) are centered, mutually independent variables. Since \(C\) contains no causal term for \(A\), any covariance in these equations comes from \(U\):
If both \(\alpha\) and \(\gamma\) are nonzero, this covariance is nonzero. The corresponding regression slope of \(C\) on \(A\) is
Thus an association can appear even though \(A\) cannot cause \(C\). In an actual analysis, variables may be binary, effects may be nonlinear, and there may be several unmeasured causes; this linear calculation is an explanation, not a universal model for all control estimates. It also shows why a null control result is not decisive: if \(\gamma=0\), or if the control is too imprecise to reveal the association, the result may be near zero despite confounding of the primary treatment effect.
Worked Example: Several Control Outcomes Before Treatment
Worked Example: Several Control Outcomes Before Treatment
Suppose a hypothetical observational study estimates the effect of enrollment in a care-management program \(A\) on hospital readmission within 90 days \(Y\). The adjusted primary estimate is a risk difference of \(-0.060\), or 6.0 percentage points fewer readmissions among enrollees. Investigators also examine two distinct outcomes recorded in the year before enrollment: prior emergency-department use for a minor injury, and prior receipt of a routine preventive service. Neither historical outcome can be caused by later program enrollment. Both could, however, reflect healthcare access or engagement factors that also influence enrollment and subsequent readmission.
The adjusted association between enrollment and prior minor-injury emergency use is a risk difference of \(0.060\), with \(SE=0.025\). Its approximate 95% confidence interval is
The estimate is 6.0 percentage points, and the interval runs from 1.1 to 10.9 percentage points. Because this outcome predates enrollment, this association is not an effect of enrollment on the historical outcome. It is consistent with a shared access or healthcare-use process, although it does not identify which process produced the association.
For prior preventive-service receipt, suppose the adjusted risk difference is \(0.040\), with \(SE=0.018\). Then
This second association is also positive, with an interval from about 0.5 to 7.5 percentage points. The two results are directionally concordant and fit a plausible shared-access explanation. They raise concern that measured adjustment may not have removed all relevant differences between enrollees and non-enrollees. They do not show that the primary readmission estimate is biased by 6 percentage points: the control outcomes differ from readmission and their associations are not direct measurements of bias in the primary estimate.
The causal checks matter. Both outcomes occurred before enrollment, so the program cannot cause them. The proposed diagnostic interpretation still depends on whether healthcare access or engagement plausibly influences both enrollment and those historical outcomes. If one control instead reflected a process unrelated to enrollment, a null association for that control would not contradict the other results.
Reading Concordance and Discordance
Concordance is most persuasive as a diagnostic when the controls are distinct, their causal conditions are credible, and the same plausible unmeasured process could explain their associations. Similar direction across controls can make a residual-confounding concern more coherent than a result from one control alone. Concordance does not guarantee that the controls are valid: several outcomes can share a direct causal relationship with treatment, a measurement artifact, or a selection mechanism.
Discordance is not a simple vote against bias. A control may be sensitive to one confounding pathway but not another; its measurement may be noisy; or its association may be estimated imprecisely. Conversely, a control may be invalid because treatment can affect it or because it does not share the pathway it was intended to probe. Investigators should ask which causal structure could generate the pattern before interpreting the estimates.
For a negative control exposure, the analogous reasoning focuses on the exposure-outcome association. If \(Z\) cannot cause \(Y\), but \(Z\) and \(A\) share causes such as healthcare access, an adjusted association between \(Z\) and \(Y\) can signal that those causes remain relevant. The value of this result depends on the claim that \(Z\) has no causal path to \(Y\), as well as on the claim that it shares the confounding process of concern.
Worked Example: A Negative Control Exposure With a Null Companion
In a second hypothetical study, the target treatment is a home-delivery pharmacy service \(A\), and the primary outcome \(Y\) is an asthma exacerbation during follow-up. The investigators propose two negative control exposures recorded before follow-up: an administrative enrollment indicator for an unrelated, nontherapeutic registry, \(Z_1\), and a historical record of a different administrative service, \(Z_2\). For this example, assume neither service has a causal pathway to asthma exacerbation during follow-up, and assume each may reflect healthcare access. Those assumptions must be defended in a real application; the labels “administrative” and “unrelated” do not establish validity by themselves.
Suppose the adjusted risk difference in follow-up asthma exacerbation for \(Z_1\) is \(0.050\), with \(SE=0.022\). The approximate 95% confidence interval is
The estimate is 5.0 percentage points, and its interval is about 0.7 to 9.3 percentage points. Under the stated no-causal-path assumption, this association is consistent with a shared access-related cause of the control exposure and asthma outcome. It raises a residual-bias concern for the treatment analysis if the same cause also influences \(A\).
For \(Z_2\), suppose the adjusted risk difference is \(0.000\), with \(SE=0.024\). Its interval is
This interval includes both negative and positive associations of meaningful size in this hypothetical setting. The point estimate is zero, but the result is not evidence that the association is exactly zero. Moreover, \(Z_2\) may be a poor probe if it captures little variation in access or is measured less accurately than \(Z_1\). The discordance therefore prompts a review of the controls’ pathways and precision; it does not erase the \(Z_1\) result or establish that the primary treatment estimate is unbiased.
The assumed causal conditions are essential to both interpretations. If either administrative service affects asthma care or treatment access, it may not be a valid negative control exposure. If it does not share the causes of the target treatment, its association may say little about confounding of \(A\).
Triangulating Distinct Confounding Pathways
A useful set of controls can probe different, specified pathways rather than repeatedly measuring the same feature. For instance, historical healthcare use might probe access and care-seeking, while a separate control could probe health status or a selection process. The interpretation should say which pathway each control is intended to reflect. If the results differ, this pathway map helps distinguish a genuinely different signal from a control that was never expected to respond to the same source of bias.
Worked Example: A Pattern That Points to More Than One Explanation
Consider a third hypothetical analysis of a treatment \(A\) and primary outcome \(Y\). Investigators examine two valid-by-assumption control outcomes intended to probe different unmeasured factors. One is a historical healthcare-use outcome, thought to share an access pathway with \(A\); the other is a historical health-status outcome, thought to share a severity pathway. Assume both precede treatment and neither can be affected by it. The adjusted risk-difference estimates are:
| Control | Estimated risk difference | SE | Approximate 95% confidence interval |
|---|---|---|---|
| Historical healthcare use | 0.040 | 0.015 | (0.0106, 0.0694) |
| Historical health status | -0.010 | 0.030 | (-0.0688, 0.0488) |
For the first control, the interval is calculated as
For the second, it is
The first result is a positive association whose interval excludes zero; the second is imprecise and compatible with associations in either direction. Since the controls were intended to probe different causes, this is not a contradiction that can be resolved by choosing the result with the smaller \(SE\). The pattern is consistent with an access-related residual process and gives weak information about the severity-related process. It does not rule out severity confounding, because the second estimate is uncertain, and it does not quantify the bias in the primary treatment effect.
The example also illustrates why control estimates should not be averaged merely because they appear in one analysis. The outcomes are different and their associations need not represent a common quantity. The causal role and uncertainty of each estimate must remain visible in the interpretation.
Diagnostics, Alternatives, and Limits
Before interpreting a pattern, check the study’s causal story and the analysis itself. Confirm that the control outcome is not affected by treatment and that the negative control exposure has no causal route to the primary outcome. Check timing, eligibility, measurement, and whether conditioning or selection could create an association. Then inspect estimates and confidence intervals, not only whether each test crossed a significance threshold. A wide interval may reflect limited diagnostic power rather than reassuring evidence.
Negative controls complement, rather than replace, adjustment for measured confounders. Propensity score methods can address measured covariate imbalance under their assumptions, but a balanced set of measured covariates does not establish that unmeasured confounding is absent. Negative controls are diagnostics for particular causal structures; they do not generally identify the primary causal effect or provide a correction automatically. Sensitivity analysis instead examines how conclusions might change under specified unmeasured-confounding scenarios. These approaches answer related but different questions.
A control association can arise from a violated negative-control condition, residual confounding, selection, measurement error, or chance. A null result may occur because the control does not share the relevant cause, because the effect is small, or because the estimate is imprecise. Several controls reduce dependence on any one diagnostic only when their limitations and causal roles are assessed individually. The overall conclusion should therefore be calibrated: describe which patterns support concern, which are inconclusive, and which assumptions would have to hold for the interpretation.
For each control, identify why treatment cannot cause the outcome or why the exposure cannot cause the primary outcome.
Specify the unmeasured cause, selection process, or measurement process that could connect the control and target analysis.
Report direction, units, and confidence intervals, and avoid treating a nonsignificant result as proof of no association.
Explain what concordance or discordance supports under the stated causal assumptions, and what remains uncertain.
Check Your Understanding
Answer each question using the causal rationale as well as the reported estimate.
- A control outcome occurred before treatment and is associated with treatment after adjustment. Can the treatment have caused this historical outcome, and what might the association indicate?
- A negative control exposure has an estimated risk difference of \(0.020\) with \(SE=0.015\). Calculate an approximate 95% confidence interval and interpret its uncertainty.
- Two negative control outcomes have estimates in opposite directions. Give two reasons, other than “one result must be wrong,” that could explain this discordance.
- Why does a null result for one negative control not establish that the primary treatment estimate is unbiased?
Solutions: (1) No: an outcome that occurred before treatment cannot be caused by that later treatment. The association may indicate shared causes, selection, or measurement processes, provided the control is valid and relevant. (2) \(0.020\pm1.96(0.015)=0.020\pm0.0294=(-0.0094,\ 0.0494)\). The interval includes zero and associations of either sign, so the estimate is uncertain; it does not establish no association. (3) The controls may probe different unmeasured pathways, or one may be less precise or measured differently; a control may also violate its assumed causal condition. (4) The control may not share the relevant confounding pathway, may have limited power, or may be imprecisely measured. It diagnoses only the pathways it can plausibly reveal.