A Clinical Example of Negative Control Diagnostics

Why Use Two Kinds of Clinical Control?

Suppose a clinical cohort compares patients who initiate a medication with similar patients who do not. Even after adjustment for measured covariates, the groups may differ in unmeasured factors such as disease severity, healthcare-seeking behavior, or clinicians’ treatment preferences. A positive treatment-outcome association could therefore reflect a medication effect, residual confounding, or both. Negative controls can probe whether the study design and adjustment leave patterns that a causal explanation alone cannot account for.

This tutorial follows a hypothetical study of a medication and a one-year cardiovascular event. It uses a control outcome that the medication is assumed not to cause, and a control exposure that is assumed not to cause the cardiovascular event. Each tests a different implication of the proposed causal interpretation. Neither is a substitute for establishing the assumptions needed for causal inference.

Definition: A negative-control outcome is an outcome that, under the scientific assumptions for the study, cannot be causally affected by the exposure, but may share causes or sources of bias with the primary outcome. A negative-control exposure is an exposure that cannot causally affect the outcome of interest, but may share relevant confounding or selection processes with the primary exposure. An association with a valid control can signal a problem with the design, measurement, or adjustment; a null association does not by itself establish that those problems are absent.

Study Setup and Assumptions

Let $A=1$ indicate initiation of the medication at the index date and $A=0$ indicate no initiation. Let $Y$ indicate a cardiovascular event in the following year, and let $L$ denote measured covariates recorded before treatment. The target is the average risk difference in the study population. For a binary outcome, define $Y^a$ as the outcome that would be observed under treatment level $a$. A causal interpretation of the adjusted estimate requires consistency, conditional exchangeability $Y^a \perp A \mid L$, and positivity: each covariate pattern in the target population must have a positive probability of each treatment level. It also requires adequate measurement and correct estimation of the adjusted risks.

For example, one can fit a model for the outcome conditional on treatment and $L$, predict each participant’s risk first setting $A=1$ and then setting $A=0$, and average those predictions over the same target population. The adjusted risk difference is the difference between these two averages. This standardization makes the target population explicit; it does not repair unmeasured confounding or lack of overlap.

\widehat{RD}_Y = \frac{1}{n}\sum_{i=1}^{n} \left\{\widehat{P}(Y=1\mid A=1,L_i) -\widehat{P}(Y=1\mid A=0,L_i)\right\}.

A valid negative-control outcome must have no causal pathway from $A$ to that outcome in the relevant time period. For a valid negative-control exposure, the control exposure must have no causal pathway to $Y$. These are substantive assumptions, not consequences of a nonsignificant test. The control must also be informative: it should share some bias-generating process with the primary analysis, and the analysis must not create a spurious association through conditioning, selection, or measurement error. The earlier material on treatment effects that invalidate control outcomes and confounding structures that make controls uninformative emphasizes why these conditions need clinical justification.

Worked Example: Choosing a Control Outcome

In the hypothetical medication study, investigators consider annual dental-care spending as a negative-control outcome. They reason that the cardiovascular medication has no plausible direct or indirect effect on dental-care spending during follow-up, while access to care and general healthcare use may influence both medication initiation and spending. What must they check before using it?

First, they must defend the no-effect claim: if the medication changes patients’ overall care use, or if adverse effects alter dental visits, the outcome may not be a valid control. Second, they must consider whether the outcome shares relevant confounding with cardiovascular risk. Shared healthcare-seeking behavior might make the control informative, but it does not guarantee that it captures confounding by cardiovascular disease severity. Third, they must apply a comparable adjustment strategy and check that the outcome is measured similarly across treatment groups.

If the adjusted association with dental spending is nonzero, that is evidence against the joint story that the control is unaffected and the design has eliminated all relevant bias. It does not identify which part of that story failed. If the estimate is near zero, its uncertainty and the control’s ability to detect shared bias still matter.

Adjusted Estimates and Their Uncertainty

For the numerical illustrations, assume estimates come from a hypothetical, adequately sized cohort after adjustment for prespecified pre-treatment covariates. Reported standard errors are robust standard errors for the adjusted estimates. For each estimate, use the approximate Wald statistic $z=\widehat{\theta}/SE$, a two-sided normal-reference p-value, and a 95% interval $\widehat{\theta}\pm1.96SE$. These calculations quantify sampling uncertainty under the fitted model; they do not account for an invalid control or unmeasured confounding.

Worked Example: The Primary Medication Association

Suppose the adjusted one-year cardiovascular risks are 0.084 for medication initiators and 0.066 for non-initiators, standardized to the same study population. The adjusted risk difference is $0.084-0.066=0.018$, or 18 events per 1,000 patients. Suppose its standard error is 0.006.

The Wald statistic is $z=0.018/0.006=3.00$, giving a two-sided p-value of about 0.0027. The 95% interval on the risk-difference scale is:

0.018 \pm 1.96(0.006) =0.018\pm0.01176 =(0.00624,\ 0.02976).

In context, the adjusted difference is 18 cardiovascular events per 1,000 patients, with a 95% interval from 6.24 to 29.76 more events per 1,000 patients. This is an adjusted association. Its causal interpretation additionally depends on exchangeability, positivity, consistency, and adequate model specification; the small p-value does not verify those assumptions.

For a negative-control outcome, the adjusted contrast may use a different scale. The example below reports the mean difference in annual dental-care spending in dollars per patient. Its interval can also be expressed as the corresponding difference per 1,000 patients by multiplying the per-patient values by 1,000. Keeping both the estimate and its units explicit prevents scale conversions from obscuring the interpretation.

Worked Example: A Noisy Control-Outcome Association

Suppose the adjusted difference in annual dental-care spending is $\$6.00$ per patient, with $SE=\$9.50$ per patient. The point estimate is small relative to its uncertainty. The statistic is $z=6.00/9.50=0.632$, giving a two-sided p-value of approximately 0.528.

The 95% interval per patient is:

6.00\pm1.96(9.50) =6.00\pm18.62 =(-12.62,\ 24.62)\ \text{dollars per patient}.

For 1,000 patients, the adjusted difference is $\$6.00\times1{,}000=\$6{,}000$, with standard error $\$9.50\times1{,}000=\$9{,}500$. Thus the interval on that scale is:

(6.00-1.96\times9.50)\times1{,}000 =-12{,}620,\qquad (6.00+1.96\times9.50)\times1{,}000 =24{,}620.

The 95% interval is therefore $-\$12{,}620$ to $\$24{,}620$ per 1,000 patients. The p-value does not provide evidence against a zero association at conventional significance levels, but this broad interval also does not rule out consequential differences in spending. This result is inconclusive as a diagnostic: it is not evidence that residual confounding is absent.

Adding a Control Exposure

A control exposure tests another implication of the study. Suppose investigators identify an unrelated topical medication used by some patients in the same healthcare system. They use its initiation as a control exposure for the cardiovascular outcome, provided there is a defensible clinical argument that it cannot affect cardiovascular events over the study’s follow-up. The control exposure should be analyzed with an adjustment strategy and target population comparable to those of the primary analysis. Shared determinants such as healthcare contact may make it informative, but its users need not have the same confounding structure as users of the primary medication.

Worked Example: A Positive Control-Exposure Association

Suppose the adjusted risk difference for the cardiovascular outcome comparing control-exposure initiators with non-initiators is 0.012, with $SE=0.005$. The statistic is $z=0.012/0.005=2.40$, giving a two-sided p-value of about 0.016.

The 95% interval is:

0.012\pm1.96(0.005) =0.012\pm0.0098 =(0.0022,\ 0.0218).

That is an adjusted association of 12 more cardiovascular events per 1,000 patients, with a 95% interval from 2.2 to 21.8 more events per 1,000 patients. If the control exposure truly cannot cause the outcome, this nonzero association warns that the analysis may retain confounding, selection bias, measurement problems, or model error. Another possibility is that the supposed control exposure is not actually causally inert.

The two controls now give different diagnostic messages: the control-outcome estimate is imprecise and compatible with zero as well as substantial differences, while the control-exposure estimate is positive and comparatively precise. These results do not cancel each other out. The positive association warrants investigation; the imprecise control-outcome result cannot reassure us that the primary estimate is unbiased. Nor does the control-exposure estimate tell us exactly how much bias to subtract from the primary estimate.

How to Interpret Conflicting Diagnostics

A control association is a diagnostic signal, not a direct measurement of the primary estimate’s bias. To use it responsibly, examine the causal justification and precision of each control, its overlap with the primary study’s bias mechanisms, and the possibility that the control itself has a causal effect. Then investigate whether the finding is sensitive to covariate definitions, model choice, outcome measurement, or the timing of exposure and outcome. A p-value alone does not distinguish these explanations.

When the primary and control estimates are on different scales, compare their diagnostic meaning rather than their raw magnitudes. A dollar difference in dental spending is not directly comparable to a risk difference in cardiovascular events. Even when associations share a scale, similar magnitudes do not establish that the controls and primary analysis share the same bias structure. The earlier material on comparing control associations on compatible scales provides the framework for making scale comparisons without treating them as automatic bias corrections.

Negative controls also differ from the main alternatives. Adjustment for measured covariates addresses confounding represented by those covariates, subject to adequate measurement and modeling. A negative control checks an implication of the overall design and adjustment but cannot guarantee control of unmeasured confounding. Sensitivity analysis asks how conclusions change under specified departures from assumptions; it can quantify robustness but depends on its chosen parameters. Triangulating these approaches can strengthen an assessment, but none turns a diagnostic into proof of causality.

State the causal claim.
Define the treatment contrast, outcome, follow-up, target population, and adjusted estimand.

Defend each control.
Explain why it cannot be causally affected in the relevant way and why it could share a relevant bias mechanism.

Estimate and quantify uncertainty.
Use the stated adjustment strategy, report the scale and units, and interpret intervals alongside p-values.

Investigate discordance.
Check control validity, measurement, selection, model specification, and whether one control is too imprecise or poorly matched to be informative.

Common Pitfalls and Limitations

Calling a nonsignificant control a clean bill of health. A large p-value can accompany a wide interval that includes consequential associations, as in the spending example.
Assuming every nonzero control association proves confounding. It may instead reflect a violated no-effect assumption, selection, measurement error, or model misspecification.
Subtracting the control estimate from the treatment estimate without justification. A control association does not generally identify the magnitude of bias in the primary estimate.
Ignoring clinical pathways. A medication may affect an apparent control outcome indirectly, or a control exposure may affect the primary outcome through an overlooked mechanism.
Overstating generalizability. A control that captures one source of bias may say little about other unmeasured causes or about bias that differs between the control and primary analyses.

The central limitation is that negative-control diagnostics depend on assumptions that cannot usually be verified from the observed data alone. Their greatest value is disciplined falsification: they can reveal inconsistencies in a causal story and motivate further checks, while a reassuring pattern remains limited evidence rather than a guarantee.

Key takeaway: Interpret each control association with its causal rationale, uncertainty, scale, and likely bias overlap in view. A precise nonzero result can challenge the analysis, while an imprecise null result cannot establish that the primary treatment estimate is unconfounded.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

Why does a null negative-control outcome association not prove that the primary treatment estimate is unbiased?
For the dental-spending result, what is the 95% interval per 1,000 patients, and what does its width imply?
The control-exposure risk difference is positive and its interval excludes zero. Name two explanations other than residual confounding.
Why should the dental-spending contrast not be directly compared in magnitude with the cardiovascular risk difference?
Can the control-exposure association be subtracted from the primary estimate to correct it? Explain.

Solutions. (1) The control may be imprecise, may not share the relevant bias mechanism, or may fail to reveal other sources of bias. (2) The interval is $-\$12{,}620$ to $\$24{,}620$ per 1,000 patients; it allows a broad range of differences and is not reassuring evidence of equivalence. (3) The control exposure may have an unrecognized causal effect, or selection, measurement error, or model misspecification may induce the association. (4) They use different outcome scales and units: dollars per patient versus events per patient. (5) No. That correction would require a justified model linking the control association quantitatively to bias in the primary estimate; the diagnostic result alone supplies no such link.