Differential Outcome Misclassification

Why Classification Can Depend on Treatment

A binary outcome is often recorded through a test, a coded diagnosis, or a report rather than observed without error. If the probability of a missed true outcome or a false positive differs between treatment groups, the measured treatment contrast can be distorted even when the true causal contrast is clear. For example, treated patients might receive more intensive follow-up, making an outcome more likely to be detected among them.

The key issue is not merely that outcome labels are imperfect. It is whether the classification process varies with treatment or with covariates used in the causal analysis. A common error is to assume that misclassification must only weaken an association. When sensitivity or specificity differs across groups, the distortion can instead increase a contrast, shift it in either direction, or reverse its sign.

These probabilities describe classification conditional on the true outcome, treatment, and covariates. They do not describe the probability that a recorded positive is true; that probability also depends on the outcome prevalence in the group. A misclassification model must therefore be considered alongside the distribution of \(Y\).

From True Risk to Measured Risk

Within a treatment and covariate group, write the true outcome risk as \(p_a(l)=P(Y=1\mid A=a,L=l)\), and the recorded outcome risk as \(q_a(l)=P(Y^*=1\mid A=a,L=l)\). A recorded positive occurs either when a true case is classified positive or when a noncase is classified positive. The two events are disjoint, so

Define the Youden index \(J(a,l)=Se(a,l)+Sp(a,l)-1\). Rearranging the risk equation gives a compact expression for how classification changes risk:

The first term is the false-positive contribution among people without the outcome. The second term depends on the true risk and on the combined classification performance. Both can vary between treatment groups. Thus, comparing \(q_1(l)\) with \(q_0(l)\) need not give the same contrast as comparing \(p_1(l)\) with \(p_0(l)\).

This difference contains contributions from both specificity and sensitivity, through the Youden indices. In general, the bias is not a single attenuation factor multiplying the true contrast. The following derivation makes the source of each term explicit.

A Useful Contrast Derivation

Start with the measured risk difference and substitute the expression for each measured risk. Then subtract the true risk difference:

Here the covariate index \(l\) is omitted to keep the algebra readable; every quantity is still within the same covariate group. The derivation shows why differential classification can change the contrast in several ways at once: false-positive rates can differ, and the true risks can be weighted by different classification indices.

As a special case, if sensitivity and specificity are identical in the two treatment groups, then \(J_1=J_0=J\) and \(Sp_1=Sp_0\). The expression simplifies to \(q_1-q_0=J(p_1-p_0)\). When \(0<J<1\), this is attenuation toward zero. That familiar conclusion depends on equal classification properties across treatment groups; it does not generally apply under differential misclassification.

Covariates and Standardized Causal Contrasts

In an observational analysis, covariates may be needed to identify a causal contrast. Let \(Y(a)\) denote the potential outcome under treatment level \(a\). For a binary treatment, a standard causal risk difference is \(E[Y(1)]-E[Y(0)]\). Under consistency, conditional exchangeability \(Y(a)\perp A\mid L\), and positivity, the corresponding standardized risks can be written as

for discrete \(L\); with continuous covariates the sum is replaced by an integral. These conditions concern causal identification from treatment and covariate data. They do not, by themselves, ensure that the recorded outcome \(Y^*\) is classified correctly.

If measured conditional risks are standardized to a common covariate distribution, the resulting measured risk for treatment level \(a\) is \(\sum_l q_a(l)P(L=l)\). Its difference from the true standardized risk is

This expression makes two points. First, standardizing over covariates does not automatically correct outcome misclassification. Second, even if classification does not depend on treatment within each covariate group, its performance can differ across covariate groups. In that case, the distortion in a standardized contrast depends on both the group-specific classification properties and the true risks in those groups.

When the Apparent Direction Reverses

Differential misclassification can even reverse the sign of a contrast. The possibility follows directly from the risk equation: the observed risks include group-specific false-positive contributions and group-specific weights on the true risks. Unless those classification properties are constrained, there is no general guarantee that the measured contrast has the same sign as the true contrast.

Interpreting and Assessing Differential Misclassification

A defensible analysis distinguishes three questions. First, what causal contrast is being targeted? Second, how does the classification process operate conditional on true outcome, treatment, and relevant covariates? Third, what information supports the assumed sensitivity and specificity values? Earlier material on validation-based correction addresses how validation data can inform outcome classification. The key point here is that a single sensitivity and specificity may be inadequate when the classification process differs across treatment or covariate groups.

When sensitivity and specificity are known or credibly estimated within relevant groups, the risk equation shows how to translate between true and measured risks, subject to those inputs being appropriate for the study population. When they are unknown, the observed outcome distribution alone generally does not determine both the true risks and the classification probabilities. The observed contrast should then not be treated as a corrected causal contrast simply because treatment was randomized or confounders were adjusted.

A practical interpretation should keep the target outcome and the recorded outcome distinct. If the analysis compares recorded diagnoses, it estimates a contrast in recorded diagnoses unless an adequate measurement model connects those diagnoses to the true outcome. Conversely, a causal claim about the true outcome requires assumptions or external information that justify that connection. Reporting the assumed group-specific sensitivity and specificity, and showing how the contrast changes across plausible values, makes this dependence visible.