Positive Controls for Assay Sensitivity

Why Use a Positive Control?

A study can produce a null estimate for two very different reasons: the causal effect under study may be absent, or the design and analysis may be unable to detect an effect that is present. A positive control helps investigate the second possibility. It is an outcome or exposure with a causal relationship expected to be nonzero, chosen so that the same study process can be checked for its ability to recover a real signal.

For example, suppose an observational analysis estimates the effect of a medication on a clinical outcome. If that medication is also known to affect a routinely measured biomarker, the biomarker might serve as a positive-control outcome. If the analysis cannot detect the established effect on that biomarker, the result raises questions about power, measurement, or the analytic pipeline. If it does detect it, that is useful evidence of signal detection—but not proof that the primary estimate is unbiased.

“Known” must be interpreted carefully. The expected relationship should be supported by credible causal evidence and be relevant to the population, treatment version, outcome definition, and follow-up period in the current study. A relationship established in a different population may not transport unchanged. A positive control therefore has a benchmark and a rationale, not merely a plausible story.

Conditions for a Useful Positive Control

A positive control is informative only when the study has a sound reason to expect a nonzero causal effect and the analysis can estimate that effect meaningfully. Before looking at its result, specify the control, the causal contrast, the expected direction and approximate magnitude, and what counts as successful detection or acceptable recovery.

• Credible causal benchmark. The expected relationship should not be based only on a crude association that may itself be confounded. Ideally, it is supported by strong prior causal evidence.
• Relevant setting. The control effect should plausibly apply to the study population, exposure version, outcome measurement, and time window. Effect modification may make a previously established effect smaller or absent in the current setting.
• Comparable analytic process. The same design decisions, data source, confounder adjustment, matching or weighting strategy, and outcome measurement process should be applied as far as appropriate. Otherwise, success may not test the pipeline used for the main analysis.
• Identifiable and estimable contrast. The data must contain enough information to estimate the target contrast, including adequate overlap for the method and measurements that distinguish the relevant exposure and outcome values.
• Prespecified success criterion. A statistically significant result in the expected direction is one possible detection criterion, but it does not by itself show that the estimated magnitude is close to the benchmark. If magnitude recovery matters, define a tolerance in advance.

The positive control should not be selected after searching many outcomes or exposures for one that happens to produce a favorable result. Such selection can exaggerate apparent assay sensitivity. It is also important to distinguish a control’s causal effect from the estimate produced by a particular study: even a well-supported benchmark may have uncertainty, and an estimated effect will vary from sample to sample.

Detection and Power

A positive-control test usually asks whether the analysis can reject a null hypothesis of no effect in the expected direction or, for a two-sided test, whether it can detect a nonzero effect. The answer depends on both the effect size and the standard error. A nonsignificant result is not, by itself, evidence that the analysis lacks sensitivity; it may simply be imprecise.

For a simple approximation, suppose an estimator \(\hat{\theta}\) is approximately normally distributed with mean \(\theta\) and standard error \(SE\). For a two-sided test of \(H_0:\theta=0\) at significance level \(\alpha\), reject when \(|\hat{\theta}/SE|>z_{1-\alpha/2}\), where \(z_q\) is the \(q\)-quantile of the standard normal distribution. Under a specified nonzero effect \(\theta\), the test statistic has an approximate normal distribution with mean \(\theta/SE\) and variance 1. Thus the power is

This expression follows by writing \(\hat{\theta}/SE=\theta/SE+Z\) and calculating the probability that the statistic falls below the lower critical value or above the upper critical value. It makes the key point explicit: a genuine nonzero control effect can still be missed when the standard error is large relative to its magnitude. Power should therefore be assessed against a substantively justified effect size, not inferred from whether the observed p-value is below \(\alpha\).

When the Positive Control Is Not Detected

A missed positive-control signal is a warning that deserves investigation, not an automatic verdict that the method failed. The estimate, uncertainty, expected effect size, data quality, and relevance of the benchmark all matter. A wide interval that includes both zero and the expected effect is inconclusive; a precise interval excluding effects of the expected size is more concerning.

A more precise estimate that remains near zero and whose interval excludes effects close to the benchmark would be stronger evidence that something is wrong with the assumed benchmark or with the study’s ability to recover it. Potential explanations include inadequate exposure or outcome measurement, sparse data, poor overlap, inappropriate adjustment, an effect that differs in this population, or an analysis that does not target the same causal contrast as the benchmark.

Detection Is Not the Same as Benchmark Recovery

A positive control may be statistically detectable yet still be estimated at a magnitude that differs meaningfully from the expected effect. If the goal is to check recovery, define an acceptable range around a benchmark rather than treating a nonsignificant difference from that benchmark as proof of agreement. One formal approach is an equivalence assessment: specify a margin in advance and ask whether the uncertainty interval lies within that margin.

This distinction matters because “the control was significant” only addresses evidence against zero. It does not answer whether the estimated magnitude is compatible with a meaningful benchmark. Conversely, failure to establish equivalence does not necessarily mean a large discrepancy exists; the interval may simply be too wide.

Positive Controls Compared With Other Checks

Positive and negative controls address complementary questions. A positive control probes the ability to recover an expected signal; a negative control probes whether the analysis generates an unexpected one. Neither is a substitute for careful causal design, and neither alone certifies the primary estimate. Simulation or a separate benchmark study can also assess analytic performance, but these approaches depend on their own assumptions and may not reproduce the data-quality problems in the study at hand.

Diagnostics, Failure Modes, and Interpretation

If a positive control is not recovered, examine whether the expected causal relationship applies in the study setting before attributing the discrepancy to the estimator. Then inspect the analysis pipeline: exposure classification, outcome measurement, missingness, sample size, overlap, model specification, adjustment choices, and the causal contrast actually estimated. For a matching or weighting analysis, for example, poor overlap can limit which population is represented; a control analysis that targets a different population may not be comparable to its benchmark.

If the control is recovered, inspect the same features rather than treating success as a pass/fail stamp. A control effect can be easier to detect than the primary effect, measured with less error, or affected by different confounders. In an observational study, confounding can even contribute to an apparent positive-control association. A successful test therefore provides limited, context-specific evidence about assay sensitivity; it does not establish exchangeability, eliminate unmeasured confounding, or validate every modeling choice.

• Pitfall: “The positive control was significant, so the primary estimate is unbiased.” This is wrong because the control and primary outcome can have different confounding structures and measurement quality. The correction is to treat control success as evidence of signal detection for that control, not as proof of validity for another estimand.
• Pitfall: “The positive control was nonsignificant, so its causal effect is zero.” This is wrong because a test can miss a nonzero effect, especially with a large standard error. Examine the interval and power against the expected effect.
• Pitfall: “A positive estimate is enough to show recovery.” Direction alone may not establish detection or acceptable magnitude. Compare the estimate and uncertainty with prespecified detection criteria and, when relevant, a tolerance around the benchmark.
• Pitfall: using a control with a weak or nontransferable benchmark. If the effect is not expected in this population or time window, failure to detect it cannot meaningfully test the pipeline. Justify the benchmark and its applicability first.