Outcome Misclassification and Causal Effects

Why Outcome Misclassification Matters

A causal analysis can correctly handle treatment assignment and still give a misleading result if the outcome is recorded inaccurately. A diagnostic code, survey response, or screening test may fail to identify some true cases or may label some noncases as cases. The resulting outcome is not the true outcome of interest, so its risks and treatment contrast need not equal the causal risks and contrast for the true outcome.

The key distinction is whether the outcome classification process has the same accuracy in the treatment groups. With nondifferential misclassification, sensitivity and specificity are equal across treatment groups, subject to the conditioning set used to define that claim. With differential misclassification, at least one of these accuracy measures differs across groups. The distinction matters: common error has a simple effect on a risk difference, while group-specific error can generate an apparent effect even when the true causal effect is zero.

True Outcomes, Recorded Outcomes, and Classification Accuracy

Let \(A\) be a binary treatment, with \(A=1\) denoting treatment and \(A=0\) control. Let \(Y\) be the true binary outcome, where \(Y=1\) indicates the event, and let \(Y^*\) be the recorded, possibly misclassified outcome. For a causal comparison, write \(Y(a)\) for the true outcome that would occur under treatment level \(a\). The causal risk under \(a\) is \(p_a=P(Y(a)=1)\), and the causal risk difference is \(p_1-p_0\).

These probabilities describe the classification process, not treatment assignment. In an observational study, a causal interpretation also requires suitable identification conditions, such as consistency, positivity, and conditional exchangeability given measured confounders. Randomization can support identification without adjustment for confounding. Neither randomization nor adjustment makes a misclassified outcome accurate: they address different problems.

For clarity, first consider risks within treatment groups where the true risks are causally identified—for example, in a randomized study. The observed risk is \(q_a=P(Y^*=1\mid A=a)\). If the sensitivity and specificity in group \(a\) apply to that group’s true outcomes, then the observed positives consist of true positives plus false positives:

This expression separates two sources of recorded cases. Among the proportion \(p_a\) who are true cases, a proportion \(Se_a\) are recorded positive. Among the proportion \(1-p_a\) who are true noncases, a proportion \(1-Sp_a\) are incorrectly recorded positive. Rearranging gives a useful equivalent form:

Thus sensitivity and specificity affect the observed risk through both missed true cases and false positives. Even when classification accuracy is the same in both groups, the observed risk is generally not the true risk.

What Nondifferential Misclassification Does to a Risk Difference

When sensitivity and specificity are common across treatment groups, write them as \(Se\) and \(Sp\). Subtract the observed control risk from the observed treatment risk using the risk formula:

The false-positive component \(1-Sp\) cancels because it is common to both groups. The true risk difference is multiplied by \(J=Se+Sp-1\), sometimes called Youden’s index. This result concerns the risk difference; it does not say that every effect measure is attenuated in the same way.

In most applications, a classifier performs better than chance, so \(0<J\leq1\). Under that condition, a nonzero risk difference is attenuated toward zero. The conclusion depends on the assumptions: if sensitivity or specificity differs by treatment, the false-positive components need not cancel, and this result does not apply. Also, “nondifferential” must be understood relative to the variables being conditioned on. If accuracy is equal across treatment groups within covariate strata but varies between strata, the marginal relationship need not follow the simple formula with one common pair of accuracy values.

Why Differential Misclassification Can Create or Reverse an Effect

When classification accuracy varies by treatment group, the observed contrast is

Now the false-positive terms do not generally cancel, and the two true risks are multiplied by different factors. Consequently, the observed contrast can differ in magnitude or sign from the true causal contrast. In particular, group-specific errors can produce different observed risks even when the true risks are equal.

Interpreting the Causal Contrast Carefully

In a randomized study, the observed group risks \(q_a\) can estimate the risks of the recorded outcome under assignment to \(a\), subject to ordinary sampling and follow-up considerations. They do not automatically estimate \(p_a\), the causal risk of the true outcome. In an observational study, confounding adjustment may identify the causal risks for \(Y^*\), but that still does not make those risks the causal risks for \(Y\). Identification of treatment effects and accurate outcome measurement are separate requirements.

The classification properties should be relevant to the population and conditions in which the causal risks are being compared. If sensitivity and specificity vary with covariates, a single marginal pair may not describe both groups after adjustment or standardization. A sound interpretation therefore states the outcome definition, the treatment groups, and the assumed or estimated classification accuracy. Validation information can help assess accuracy, but the calculations here describe how accuracy relates to risks; they do not by themselves establish that the assumed accuracy values are correct.

Common Mistake

• Wrong reasoning: “The outcome error is nondifferential, so it cannot bias the causal effect.”
• Correction: Nondifferential error can change both observed risks and attenuate a risk difference. Under the specific common-\(Se\), common-\(Sp\) conditions, the observed risk difference equals \(J\) times the true difference; it equals the true difference only when \(J=1\), apart from a true difference of zero.
• Wrong reasoning: “Any outcome misclassification pulls the effect toward the null.”
• Correction: That claim is not valid for differential misclassification, and it is not a universal statement about every effect measure. Different sensitivities or specificities can create or reverse a risk difference.