Why the Timing of an Adjustment Variable Matters
Adjustment is not just a way to improve a statistical comparison. It determines which groups are being compared and therefore can change the causal question. A variable measured after treatment assignment may itself be changed by treatment. If it lies on a pathway through which treatment affects the outcome, comparing treatment groups at the same value of that variable removes or alters part of the effect we may want to measure.
Consider a hypothetical randomized study of a program intended to improve health. Treatment is assignment to the program, \(A\), the mediator \(M\) is a measure of participants’ subsequent adherence, and \(Y\) is a later health outcome. The program might change adherence, which might then change health. The total treatment effect includes any effect operating through adherence as well as any effect operating through other pathways. Adjusting for adherence asks for a comparison that holds adherence fixed; that is not generally the total-effect question.
This definition assumes that the treatment and mediator potential outcomes are meaningful for the study question. The examples below also assume no interference between individuals, so one person’s outcome does not depend on other people’s treatment, and consistency, so observed treatment, mediator, and outcome values agree with the corresponding potential outcomes. In a randomized study, assignment makes treatment independent of potential outcomes before conditioning on later variables. That protection does not automatically apply after conditioning on a mediator.
What a Mediator-Adjusted Contrast Compares
Suppose treatment is randomized and the mediator is measured after treatment but before the outcome. A common adjusted comparison within mediator level \(m\) is \(E[Y\mid A=1,M=m]-E[Y\mid A=0,M=m]\). Under consistency, its two terms refer to different selected groups: treated people for whom \(M(1)=m\), and control people for whom \(M(0)=m\). Those groups need not represent the same people or the same mix of underlying characteristics.
The expression makes the selection explicit. Treatment can affect who appears in a mediator stratum, so conditioning on \(M\) can undo the balance created by random assignment. Also, even if the within-stratum contrast happened to equal a causal effect with the mediator held fixed, it would not in general include the part of the total effect operating through changes in the mediator.
A controlled direct effect at mediator value \(m\) is \(E[Y(1,m)-Y(0,m)]\): it compares treatment levels while setting the mediator to the same specified value. This is a distinct causal estimand. The observed within-mediator contrast is not automatically that controlled direct effect, because its two terms can average over different people. Identifying a direct or mediated effect requires additional assumptions and a clearly stated causal question; simply adding \(M\) to a regression does not supply them.
Worked Example: Adjustment Removes a Mediated Part of the Effect
Suppose a hypothetical randomized study has two treatment arms of 100 people each. The mediator is adherence, coded \(M=1\) for high adherence and \(M=0\) otherwise. The observed group counts and mean health-outcome scores are as follows. Higher scores are better.
| Assignment | Adherence | Number of people | Mean outcome score |
|---|---|---|---|
| Control | \(M=0\) | 75 | 0 |
| Control | \(M=1\) | 25 | 20 |
| Treatment | \(M=0\) | 25 | 0 |
| Treatment | \(M=1\) | 75 | 20 |
Assume, for this hypothetical illustration, that assignment was randomized, that the displayed scores are consistent with assignment and adherence, and that there is no interference. Both mediator levels occur in both arms, so a within-level comparison is possible. The adherence distribution differs by assignment, however, so the treatment has changed the mediator distribution.
The unadjusted mean in the control arm is \(0.75(0)+0.25(20)=5\) points. In the treatment arm it is \(0.25(0)+0.75(20)=15\) points. Thus the randomized total-effect contrast is \(15-5=10\) points.
Within \(M=0\), the treatment-control contrast is \(0-0=0\) points. Within \(M=1\), it is \(20-20=0\) points. A regression that adjusts for adherence in this simple table likewise attributes no difference to treatment at a fixed adherence level. But the total contrast is 10 points: in this constructed example, it arises because treatment increases the proportion with high adherence, which is associated with a higher outcome. The adjusted contrast answers a different question from the total effect.
The example’s simple outcome pattern makes the distinction especially clear; it does not establish that real-world mediator adjustment identifies a direct effect. The within-level comparisons involve different people, and more complex settings may also have treatment-induced selection or other sources of bias.
Conditioning Can Create Imbalance After Randomization
The issue is not limited to removing a pathway through the mediator. Suppose an underlying characteristic affects both the mediator and the outcome. Before conditioning, randomization balances that characteristic on average between treatment arms. But within a mediator stratum, treatment status can become associated with the characteristic: treatment helps determine who enters the stratum, and the characteristic also helps determine who enters it. The resulting treatment groups within that stratum may no longer be comparable.
Worked Example: A Spurious Within-Stratum Contrast
Consider a hypothetical randomized experiment with 100 people, equally divided between treatment and control. Let \(U\) be a binary baseline risk factor, with 25 people in each of the four \(A,U\) combinations. The mediator is \(M=1\) when \(A=U\), and \(M=0\) otherwise. The outcome is \(Y=10U\), in points: treatment has no effect on anyone’s outcome, while the risk factor determines the score. Assume assignment is randomized, the mediator and outcome are measured consistently, and there is no interference.
| Treatment \(A\) | Risk factor \(U\) | Mediator \(M\) | Mean outcome \(Y\) |
|---|---|---|---|
| 0 | 0 | 0 | 0 |
| 0 | 1 | 1 | 10 |
| 1 | 0 | 0 | 0 |
| 1 | 1 | 1 | 10 |
The unadjusted control mean is \((0+10)/2=5\) points, and the treatment mean is also \((0+10)/2=5\) points. The total-effect contrast is \(5-5=0\) points, as expected from the outcome rule \(Y=10U\).
Within \(M=1\), all control participants have \(U=1\) and mean outcome 10, while all treated participants have \(U=1\) and mean outcome 10; the contrast is 0. Within \(M=0\), both groups have \(U=0\) and mean outcome 0; again the contrast is 0. In this particular construction the within-stratum contrasts do not become spurious, because \(U\) is the same within each stratum. The table instead illustrates an important diagnostic: conditioning may produce strata whose membership is determined jointly by treatment and a baseline characteristic, and overlap can be fragile.
To see the selection problem more directly, modify the mediator rule to \(M=1\) only when \(A=1\) or \(U=1\). Then among \(M=0\), there are no treated participants; among \(M=1\), treated participants include both \(U=0\) and \(U=1\), whereas control participants have \(U=1\). A within-\(M=1\) contrast compares different risk-factor mixes, and no \(M=0\) treatment contrast is available. The total effect remains 0 because \(Y=10U\) does not depend on treatment. A mediator-adjusted comparison cannot recover a valid total effect from these strata merely because the original assignment was randomized.
This second rule also highlights a positivity problem: there is no treatment-control overlap within \(M=0\). Adjusted estimates relying on that missing comparison require extrapolation from a model rather than direct information in the data. A changed mediator distribution is not itself proof that an estimate is biased, but it is a signal to inspect what the adjusted contrast represents and whether it is supported by observed comparisons.
Derivation: Why the Total Contrast Includes Mediator Changes
The total effect compares \(Y(1,M(1))\) with \(Y(0,M(0))\). To see why holding the mediator fixed changes the contrast, add and subtract the intermediate potential outcome \(Y(1,M(0))\). This is an algebraic identity; it does not by itself identify either component from observed data.
The first term represents changing the mediator from its control value to its treatment value while holding treatment at 1. The second represents changing treatment while holding the mediator at its control value. This decomposition depends on the chosen intermediate outcome and should not be mistaken for an automatic empirical decomposition. Its purpose here is to show that a comparison at a fixed mediator value does not, in general, include the first term. If treatment changes the mediator and the outcome responds to the mediator, that omitted part may be consequential.
Worked Example: Calculating the Total Effect From Potential Outcomes
Suppose a hypothetical population has two equally common types. For type 1, adherence changes from \(M(0)=0\) to \(M(1)=1\); for type 2, it remains \(M(0)=M(1)=0\). The outcome rule is \(Y(a,m)=10a+20m\) points. This example assumes these potential outcomes are well-defined, consistency holds, there is no interference, and treatment is randomized so the population means can be learned from assignment groups.
For type 1, the treatment outcome is \(Y(1,M(1))=10(1)+20(1)=30\), and the control outcome is \(Y(0,M(0))=10(0)+20(0)=0\). Its individual total effect is 30 points. For type 2, the corresponding outcomes are \(Y(1,0)=10\) and \(Y(0,0)=0\), so its effect is 10 points.
The population total effect is \((30+10)/2=20\) points. In contrast, holding adherence at \(m=0\) gives \(Y(1,0)-Y(0,0)=10-0=10\) points for either type. The 10-point difference is the treatment contrast at fixed adherence in this model; the additional 10 points in the total effect arise because half the population’s adherence changes under treatment and the outcome increases by 20 points for that type. Reporting only the fixed-adherence contrast would therefore not report the total effect.
Practical Diagnostics and Alternatives
Before adjusting for a variable, establish when it was measured and whether treatment could have changed it. A variable collected after assignment is not automatically a mediator, but its timing makes causal reasoning essential. A mediator is a post-treatment variable on a pathway through which treatment may affect the outcome. Treating such a variable like a baseline confounder can change the estimand or introduce bias.
- Check temporal order. Distinguish variables measured before treatment from adherence, symptoms, or care processes measured after treatment begins.
- Compare mediator distributions. Large differences across treatment arms show that conditioning will change the composition of the groups being compared.
- Check within-stratum overlap. If one treatment arm is absent or nearly absent at a mediator value, adjusted contrasts there are unsupported or highly model-dependent.
- State the target contrast first. If the target is the total effect, keep post-treatment variables out of ordinary baseline adjustment and use an approach designed to estimate that marginal effect.
For a total effect, the main alternative is to compare outcomes marginally across randomized treatment groups, or, in observational data, to adjust for appropriate pretreatment covariates under the relevant exchangeability and positivity assumptions. The earlier material on covariate standardization and the g-formula establishes how baseline covariates can be used for that purpose. If the scientific question instead concerns pathways through a mediator or effects under a fixed mediator intervention, that is a separate mediation question requiring additional assumptions and methods.
A frequent mistake is to reason, “The mediator predicts the outcome, so adjusting for it controls confounding and must improve the treatment-effect estimate.” The correction is that predictive strength does not establish pretreatment confounding. If treatment can affect the variable, adjustment can remove part of the total effect and alter group comparability. Another mistake is to interpret the treatment coefficient from a regression containing a mediator as the total effect. Its interpretation depends on the model and assumptions; it is not automatically the marginal total-effect contrast.
Check Your Understanding
Answer each question before reading the brief solutions.
- A treatment increases exercise, which is measured after assignment, and exercise may affect a later outcome. Is exercise automatically a baseline confounder suitable for ordinary adjustment when estimating the total effect?
- In the adherence example with 100 people per arm, what is the unadjusted total-effect contrast in mean outcome points?
- Why can treatment groups within the same mediator level differ even when the original treatment was randomized?
- If one treatment arm has no observations at a mediator level, what problem does that create for a within-level comparison?
- In the potential-outcome example with two equally common types, what is the total effect and how does it differ from the treatment contrast holding \(M=0\)?
Solutions. 1. No. Exercise occurs after assignment and may be affected by treatment; adjusting for it can change the target contrast. 2. The treatment mean is 15 points and the control mean is 5 points, so the contrast is \(15-5=10\) points. 3. Conditioning selects people partly according to a variable treatment can change, so the mediator strata need not preserve the balance created by randomization. 4. There is no observed comparison in that stratum; an estimate may be unavailable or depend on extrapolation. 5. The total effect is \((30+10)/2=20\) points, whereas at \(M=0\) the contrast is \(10-0=10\) points; the latter excludes the effect from the mediator changing for type 1.