Why Identification Needs a Diagnostic
A natural direct or indirect effect is defined using counterfactual outcomes, but its definition alone does not say whether observed data can identify it. A proposed analysis can produce a precise estimate while relying on unsupported extrapolation, uncontrolled mediator–outcome confounding, or an untestable cross-world assumption. The key diagnostic question is therefore not simply whether a model can be fit; it is whether the causal structure and observed support justify the counterfactual quantity the model is meant to estimate.
Let \(A\) be a binary exposure, \(M\) a mediator, and \(Y\) an outcome. Let \(M(a)\) denote the mediator value under exposure \(a\), and \(Y(a,m)\) the outcome under exposure \(a\) and mediator set to \(m\). The following natural effects use the reference level \(A=0\) for the direct effect and \(A=1\) for the indirect effect.
On the additive scale, these two effects sum to the total effect \(E[Y(1,M(1))-Y(0,M(0))]\). That algebraic decomposition does not establish identification: each nested counterfactual still needs assumptions linking it to observed data. The audit below is for a proposed natural-effect analysis, not a claim that natural effects are always identified.
The Mediation Formula and Its Conditions
Suppose \(C\) is a set of measured pre-exposure covariates that includes all common causes of exposure and the relevant outcomes, and all baseline common causes of the mediator and outcome. In the simple setting with no exposure-induced mediator–outcome confounder, a common identification formula is
For continuous \(M\), the sum over \(m\) is replaced by an integral with respect to the conditional mediator distribution. The formula combines an outcome regression at exposure \(a\) with a mediator distribution at exposure \(a'\), then averages over the baseline covariate distribution. Subtracting the appropriate counterfactual means gives the natural direct or indirect effect.
This formula is justified only under substantive conditions. Consistency requires observed outcomes and mediators to agree with their corresponding counterfactuals under the treatment actually received, and the analysis assumes no interference between individuals. Treatment exchangeability requires no unmeasured confounding of exposure and the relevant counterfactuals given \(C\). Mediator–outcome exchangeability requires no unmeasured mediator–outcome confounding after conditioning on exposure and \(C\), with no such confounder caused by exposure. The formula also relies on a cross-world independence condition, such as \(Y(a,m)\perp M(a')\mid C\). Finally, the treatment and mediator distributions must have adequate positivity for the combinations used in the formula.
These assumptions are not interchangeable. Measuring baseline confounders does not solve a positivity failure; a fitted outcome model does not establish cross-world independence; and conditioning on an exposure-induced confounder does not, in general, restore the ordinary mediation formula.
Worked Example: Baseline Confounding and Standardization
Consider hypothetical data with a binary baseline severity variable \(C\), a binary mediator \(M\), and a binary outcome \(Y\). Suppose \(P(C=0)=P(C=1)=0.5\). Severity predicts both \(M\) and \(Y\), so comparing people with different mediator values without accounting for \(C\) would mix mediator differences with severity differences. Assume, for this example, that the stated conditional probabilities and outcome risks are correctly measured, that the other identification conditions hold, and that all required treatment–mediator combinations have support.
Under \(A=0\), suppose \(P(M=1\mid C=0)=0.2\) and \(P(M=1\mid C=1)=0.8\). The outcome risks \(E[Y\mid A,M,C]\) are:
| Severity \(C\) | \(E[Y\mid A=1,M=0,C]\) | \(E[Y\mid A=1,M=1,C]\) | \(E[Y\mid A=0,M=0,C]\) | \(E[Y\mid A=0,M=1,C]\) |
|---|---|---|---|---|
| 0 | 0.20 | 0.50 | 0.10 | 0.30 |
| 1 | 0.60 | 0.80 | 0.40 | 0.70 |
To obtain \(E[Y(1,M(0))]\), standardize the \(A=1\) outcome risks over the mediator distribution under \(A=0\), separately within each severity stratum:
For the other mean in the natural direct effect, use the \(A=0\) outcome risks with that same \(A=0\) mediator distribution:
Under the stated assumptions, the natural direct effect is a 0.12 increase in outcome risk, or 12 percentage points. The calculation is standardized within \(C\); it does not justify leaving severity out of the analysis. In a real study, the assumptions would need causal and design-based justification, not just agreement with these numbers.
Confounding and Exposure-Induced Confounding
The first confounding check is whether baseline common causes of \(M\) and \(Y\) have been addressed. A variable can be a confounder even when exposure was randomized: randomization of \(A\) does not randomize \(M\). If a baseline factor causes both \(M\) and \(Y\), a mediator–outcome comparison that omits it need not represent the effect of changing the mediator. The standard formula requires adequate adjustment for these baseline causes, along with the other conditions already stated.
A more difficult structure arises when exposure causes a variable \(L\) that in turn affects both \(M\) and \(Y\). Then \(L\) is an exposure-induced mediator–outcome confounder. It is not a baseline variable that can simply be placed in \(C\): its value can differ across exposure counterfactuals. Ignoring \(L\) leaves mediator–outcome confounding, while ordinary adjustment for observed \(L\) does not by itself supply the joint counterfactual information needed for a natural effect.
Worked Example: An Exposure-Induced Confounder
In a hypothetical study, \(A\) is a program, \(L\) is a post-program clinical response, \(M\) is a later adherence measure, and \(Y\) is a health outcome. Suppose \(A\) affects \(L\), and \(L\) affects both \(M\) and \(Y\). For illustration, let \(P(L=1\mid A=0)=0.2\) and \(P(L=1\mid A=1)=0.8\). Suppose the mediator probabilities are:
| Exposure \(A\) | \(P(M=1\mid A,L=0)\) | \(P(M=1\mid A,L=1)\) |
|---|---|---|
| 0 | 0.10 | 0.60 |
| 1 | 0.20 | 0.80 |
The observed mediator prevalence under \(A=0\) is \(0.8(0.10)+0.2(0.60)=0.20\). Under \(A=1\), it is \(0.2(0.20)+0.8(0.80)=0.68\). These calculations show that the mediator distribution changes both with exposure and with the exposure-shifted distribution of \(L\). Since \(L\) also predicts the outcome, a mediator analysis that omits \(L\) does not meet the no-unmeasured-mediator–outcome-confounding condition.
Adding observed \(L\) to an outcome regression is not a general repair. The nested mean \(E[Y(1,M(0))]\) combines an outcome under exposure 1 with a mediator generated under exposure 0; the corresponding \(L\) values may be \(L(1)\) and \(L(0)\), respectively. The ordinary formula conditions on a single observed \(L\) value and does not, without additional assumptions, identify the required cross-world relation between these values. The correct diagnosis is that the ordinary natural-effect formula is not justified by the displayed structure alone—not that every analysis involving \(L\) is impossible under every richer causal model.
Positivity and Cross-World Assumptions
Positivity asks whether the observed data include the treatment and mediator combinations needed to evaluate the formula. Treatment positivity requires a positive probability of each relevant exposure level within covariate strata being averaged over. Mediator positivity requires that mediator values drawn from the distribution under \(A=a'\) occur with support in the outcome regression under \(A=a\), within the relevant \(C\) strata. For continuous mediators, this is a support or absolute-continuity requirement rather than a positive probability at each point. Sparse support can make estimation unstable; a structural zero means the required comparison is absent and cannot be learned from the observed data without extra modeling assumptions.
Worked Example: A Structural Positivity Failure
Suppose a hypothetical study has one covariate stratum and the following mediator counts:
| Exposure | \(M=0\) | \(M=1\) |
|---|---|---|
| \(A=0\) | 30 | 20 |
| \(A=1\) | 0 | 50 |
Here, \(P(M=0\mid A=0)=30/50=0.60\), but \(P(M=0\mid A=1)=0/50=0\). To calculate \(E[Y(1,M(0))]\), the formula assigns positive weight to the \(A=1,M=0\) outcome regression: its weight from the mediator distribution under \(A=0\) is 0.60. Yet the study has no \(A=1,M=0\) observations, so the corresponding conditional outcome mean is unsupported. The natural-effect calculation cannot be completed from these data without extrapolation or additional assumptions. Reporting a model-based value does not remove the positivity failure.
Cross-world independence is a different issue. A condition such as \(Y(a,m)\perp M(a')\mid C\) links potential outcomes from different exposure worlds for the same individual. Observed data reveal outcomes and mediators under the exposure actually received, not the joint distribution of all these counterfactuals. Thus the condition is not directly testable by checking a regression fit or a balance table. It is a substantive assumption, sometimes justified within a specified causal model, and it should be stated rather than treated as an automatic consequence of covariate adjustment.
Worked Example: Why the Cross-World Joint Distribution Matters
Let \(M(0)\) and \(Y(1,0)\) each be binary with marginal probability 0.5 of equaling 1, and suppose \(Y(1,1)=0\) for everyone. These marginals do not determine \(E[Y(1,M(0))]\), because that nested outcome depends on how \(M(0)\) and \(Y(1,0)\) are paired across individuals:
If the two binary variables are independent, then \(E[Y(1,M(0))]=P(M(0)=0,Y(1,0)=1)=0.5(0.5)=0.25\). If instead \(Y(1,0)=1-M(0)\), the same two marginal probabilities hold, but \(E[Y(1,M(0))]=P(M(0)=0)=0.50\). The different results arise from different cross-world dependence, not from different marginal probabilities. This illustrates why observed marginal distributions alone do not establish the nested mean; an assumption governing the cross-world relationship is needed.
Recanting Witnesses and a Practical Audit
For a path-specific effect, a recanting witness is a variable that lies on paths assigned different treatment values: one set of outgoing paths is meant to carry the exposure effect, while another is meant not to carry it. The same variable would then need to take incompatible counterfactual values along its different outgoing edges. This graph feature obstructs identification of the requested path-specific effect from the observed data without stronger restrictions. It is particularly relevant when exposure-induced \(L\) affects both the mediator path and the outcome through a separate route: including its influence along one route while excluding it along another may require \(L\) to recant. The criterion is a warning about the target and graph, not a remedy supplied by adding another regression term.
A disciplined assessment keeps the estimand, graph, assumptions, and data support separate. Use the following sequence before interpreting a natural-effect estimate.
State the nested counterfactual contrast and its scale. Confirm that it is a natural effect, rather than an ordinary regression coefficient or a different mediator intervention.
Identify baseline common causes of exposure and outcome, and of mediator and outcome. Then check whether any mediator–outcome confounder is itself caused by exposure.
For a path-specific target, ask whether any variable must transmit exposure on one outgoing path but not another. If so, assess the recanting witness obstruction.
State the needed exchangeability and cross-world conditions. Verify that the exposure and mediator combinations required by the formula have support in the relevant covariate strata.
If a condition fails or is unsupported, say what prevents identification. A numerical estimate may still be computed, but it should not be presented as identified by the observed data under the stated assumptions.
A common mistake is to argue, “We adjusted for every measured variable, so the natural indirect effect is identified.” Adjustment cannot establish that all relevant confounders were measured, create observations in unsupported strata, or verify a cross-world independence condition. It also cannot make an exposure-induced confounder behave like a baseline covariate. The correction is to use the causal graph to identify the required conditions, assess support in the actual data, and state clearly which assumptions remain untestable.
Check Your Understanding
For each question, distinguish a definition of the target from evidence that the target is identified.
- What counterfactuals are contrasted in the natural indirect effect defined here?
- Why can a baseline common cause of \(M\) and \(Y\) bias a mediator–outcome comparison even when \(A\) was randomized?
- In the exposure-induced confounding example, why is omitting \(L\) problematic, and why does simply conditioning on observed \(L\) not generally establish the ordinary mediation formula?
- What mediator–outcome combination is unsupported in the positivity example, and which nested mean needs it?
- Why can two distributions with identical cross-world marginals yield different values of \(E[Y(1,M(0))]\)?
- In a path-specific analysis, what does it mean for a variable to be a recanting witness?