Why Timing Matters for Adjustment
In an analysis of covariance, adding a covariate can change an estimated treatment comparison. That change is not automatically a problem: adjustment for a suitable baseline covariate can improve precision, as discussed in the earlier material on precision adjustment versus causal adjustment. But a measurement taken after treatment begins may itself have been changed by treatment. Adjusting for it can remove part of the treatment effect that the study is meant to estimate, or create an association that was not present before adjustment.
The key question is not just whether a variable predicts the outcome. It is how that variable sits in the causal process. A post-treatment variable might transmit part of the treatment effect, or it might be a common effect of treatment and another cause of the outcome. These two roles have different consequences for adjustment.
Total Effects, Direct Effects, and Causal Diagrams
Let \(A\) indicate treatment, with \(A=1\) for treatment and \(A=0\) for control, and let \(Y\) be the outcome. The total treatment effect compares outcomes under the two treatment conditions, allowing any treatment-caused changes in intermediate variables to occur. In potential-outcomes notation, its average is \(E[Y(1)-Y(0)]\), where \(Y(a)\) is the outcome that would occur under treatment condition \(a\).
A controlled direct effect instead compares treatment and control while fixing a mediator \(M\) to the same specified value \(m\): \(E[Y(1,m)-Y(0,m)]\). This is a different causal question. In a simple additive model with no treatment-by-mediator interaction, the direct effect does not depend on the particular value \(m\). In more complex settings it can vary with \(m\).
A causal diagram is a compact way to state assumed causal relationships. An arrow \(A \rightarrow M\) means treatment may affect the mediator; \(M \rightarrow Y\) means the mediator may affect the outcome. These arrows represent causal assumptions, not correlations established by drawing the diagram. For a mediator, the key pattern is \(A \rightarrow M \rightarrow Y\), possibly alongside a direct arrow \(A \rightarrow Y\).
A common collider pattern is \(A \rightarrow C \leftarrow U \rightarrow Y\). Here \(C\) is affected by both treatment \(A\) and another variable \(U\), which also affects the outcome. Before conditioning on \(C\), randomization can make \(A\) independent of \(U\). Once we compare people at the same value of \(C\), however, treatment status can provide information about \(U\). The resulting treatment-outcome association through \(U\) can distort the comparison.
Worked Example: A Treatment Effect Transmitted Through a Mediator
Worked Example: A Treatment Effect Transmitted Through a Mediator
Suppose a hypothetical randomized trial studies a program \(A\) that affects a short-term activity measure \(M\), which in turn affects an outcome \(Y\). Consider the structural equations
where \(V\) and \(\varepsilon\) have mean zero, are independent of randomized treatment \(A\), and are independent of one another. Assume the equations adequately describe the causal relationships and that the same equations apply across treatment conditions.
Total effect. Under treatment \(a\), the mediator is \(M(a)=a+V\). Substituting it into the outcome equation gives
Therefore, for the same values of \(V\) and \(\varepsilon\), changing treatment from 0 to 1 changes the outcome by \(5\) units. The average total effect is \(E[Y(1)-Y(0)]=5\) outcome units.
Direct effect at a fixed mediator value. If \(M\) is held at the same value \(m\) under either treatment condition, the outcome contrast is
In this model, the direct effect is \(2\) outcome units. The remaining \(3\) units of the total effect arise because treatment increases \(M\) by one unit and each unit of \(M\) increases \(Y\) by three units. A regression that adjusts for \(M\) targets the conditional treatment coefficient \(2\) under this model; it does not recover the total effect of \(5\).
The example uses a particularly simple model. In general, a treatment coefficient from a regression that includes a mediator is not automatically a causal direct effect. A causal interpretation requires appropriate assumptions, including correct model specification, consistency and positivity, and no unmeasured confounding of the mediator-outcome relationship. It also requires care if there are treatment-induced variables that confound the mediator-outcome relationship. Randomizing treatment alone does not randomize the mediator.
Worked Example: The Size of the Change in an Adjusted Contrast
Worked Example: The Size of the Change in an Adjusted Contrast
Suppose a hypothetical randomized study measures an outcome in points and records a post-treatment engagement score \(M\). Assume a correctly specified additive model for the group means:
Suppose the mean engagement score is 3 in the control group and 5 in the treatment group. The unadjusted control mean predicted by the model is
The unadjusted treatment mean is
Thus, the unadjusted treatment-control contrast is \(62-50=12\) points. The coefficient of \(A\) in the model adjusted for \(M\) is \(4\) points: it compares treatment and control at the same engagement score. The difference between the two contrasts is also obtained directly:
In this model, 8 points of the difference in group means correspond to the groups’ 2-point difference in engagement multiplied by the 4-point outcome change per engagement point. If engagement is a mediator, adjustment removes this pathway from the comparison. The adjusted contrast is not “the total effect with extra precision”; it answers a different, conditional question. These model-based values illustrate the distinction and do not, by themselves, establish that the conditional contrast is a causal direct effect.
How Collider Adjustment Creates Bias
A collider can create a misleading comparison even when treatment is randomized and has no effect on the outcome. Consider the diagram \(A \rightarrow C \leftarrow U \rightarrow Y\). Treatment \(A\) and an unmeasured variable \(U\) both affect a post-treatment variable \(C\); \(U\) also affects outcome \(Y\). Conditioning on \(C\) can make \(A\) and \(U\) associated, opening a noncausal route from treatment to outcome.
Worked Example: Conditioning on a Treatment-Affected Collider
In a hypothetical randomized study, suppose \(U\) is independent of treatment \(A\), and let
where \(\varepsilon\) has mean zero and is independent of \(A\) and \(U\). Treatment has no causal effect on \(Y\): changing \(A\) while holding \(U\) fixed leaves \(Y\) unchanged. Since treatment is randomized, the unadjusted treatment contrast is zero in expectation.
Now consider adjusting for \(C\). From \(C=A+U\), we can write \(U=C-A\). Therefore, the conditional expected outcome is
A regression of \(Y\) on \(A\) and \(C\) consequently has treatment coefficient \(-1\), even though the true treatment effect is zero. At a fixed value of \(C\), the treated person must have \(U\) one unit lower than the untreated person in this model. Because \(U\) raises \(Y\), the adjusted comparison makes treatment appear to lower the outcome by one unit. That association is created by conditioning on the collider \(C\), not by a causal effect of treatment.
This calculation is an idealized illustration, but the mechanism is general. For example, treatment and underlying health could both affect whether a participant completes a follow-up assessment, while underlying health also predicts the outcome. Restricting the analysis to completers, or adjusting for completion status, can associate treatment with underlying health within the selected group. The direction and size of the resulting bias depend on the causal structure and data; they cannot be inferred from the word “post-treatment” alone.
A Short Derivation: Why Mediator Adjustment Changes the Contrast
The algebra behind the mediator example can be expressed generally. Suppose the conditional mean follows an additive model
and the average mediator values under treatment and control are \(\mu_{M,1}\) and \(\mu_{M,0}\). Substituting those values into the model gives the model-implied difference in group means:
The adjusted conditional contrast is \(\beta_A\). The marginal contrast additionally includes \(\beta_M(\mu_{M,1}-\mu_{M,0})\), the part associated with the difference in mediator means under this model. If treatment changes the mediator and the mediator predicts the outcome, the two contrasts will generally differ. This algebra describes the model-implied contrasts; interpreting them as causal effects requires the relevant causal assumptions.
A Practical Causal Check
Before including a covariate in an ANCOVA model, identify when it was measured and ask what could cause it. The following sequence helps separate an adjustment that addresses baseline imbalance from one that changes or distorts the target comparison.
Decide whether the scientific question concerns the total effect of treatment or an effect with a particular intermediate variable held fixed.
Determine whether the covariate was measured before treatment, after assignment, or after treatment began. Timing alone does not identify its causal role, but it signals what must be examined.
Ask whether treatment may affect the covariate, whether the covariate may affect the outcome, and whether other causes of the covariate also affect the outcome.
If adjustment blocks a mediated path, the coefficient need not represent the total effect. If it conditions on a collider, it can induce bias. Do not assign a causal interpretation without stating and defending the needed assumptions.
Common Mistakes and Why They Matter
- “The treatment was randomized, so adjusting for anything is safe.” Randomization protects the treatment comparison before conditioning, but a post-treatment variable can be caused by treatment. Conditioning on it can block a mediated pathway or open a collider pathway.
- “The adjusted treatment coefficient is always the direct effect.” It is a conditional model coefficient. A causal direct-effect interpretation needs additional assumptions about mediator-outcome confounding, model form, and the definition of the direct effect.
- “Adjustment is wrong whenever a variable is measured after treatment.” Not necessarily. Some analyses have a well-defined post-treatment estimand or need methods designed for treatment-affected confounding. The important point is to specify the target and justify the method rather than treating routine ANCOVA adjustment as automatically appropriate.
- “A variable that predicts the outcome must be adjusted for.” Predictive strength alone does not establish that adjustment is appropriate for a causal target. A variable’s place in the causal structure matters.
Check Your Understanding
Use the causal role of each variable, rather than its predictive value alone, to answer these questions.
- In the model \(M=A+V\) and \(Y=10+2A+3M+\varepsilon\), what is the total effect of changing \(A\) from 0 to 1, and what is the contrast when \(M\) is held fixed?
- Why can adjusting for a mediator be inappropriate when the target is the total treatment effect?
- In the diagram \(A \rightarrow C \leftarrow U \rightarrow Y\), which variable is the collider, and why can conditioning on it associate \(A\) with \(U\)?
- Does random assignment of treatment, by itself, guarantee that a regression coefficient adjusted for a post-treatment mediator is a causal direct effect? Explain briefly.
- A treatment and underlying health both affect follow-up completion, and underlying health predicts the outcome. What concern arises if the analysis adjusts for or restricts to completion status?