Why Timing Matters in Trial Regression
A randomized trial can collect many useful measurements: health status before treatment, treatment adherence during follow-up, and outcomes at the end of the study. But these measurements do not all play the same role in a regression model. In particular, a variable recorded after treatment begins may itself be affected by treatment. Adjusting for it can therefore change the treatment comparison being estimated.
The key question is not simply whether a variable predicts the outcome. Ask when it was measured, and could treatment have changed it? A measurement made before treatment begins can describe participants at baseline. A measurement made afterward may lie on a pathway from treatment to outcome, or may be influenced by other factors that also affect the outcome.
Let \(T\) denote randomized treatment assignment, with \(T=1\) for the treatment arm and \(T=0\) for the control arm. Let \(X\) be a baseline covariate, \(M\) a post-treatment variable, and \(Y\) the outcome. Randomization makes \(T\) independent of pre-existing participant characteristics on average. If outcomes are observed for the randomized participants, the difference in arm means estimates the overall treatment contrast: the difference in outcomes that includes any effect operating through changes in \(M\).
That overall contrast is often the question of interest. Including a baseline covariate in a regression may improve precision or account for chance imbalance, as appropriate to the analysis plan. Including \(M\), however, compares participants at the same value of a variable that treatment may have changed. It does not generally preserve the overall contrast.
Worked Example: Separating Baseline and Post-Treatment Measurements
Worked Example: Separating Baseline and Post-Treatment Measurements
Suppose a hypothetical randomized trial tests a coaching program intended to improve a health score. Investigators record a health score before the program, the number of coaching sessions attended during the program, and the health score after 12 weeks. The first score is \(X\), sessions attended is \(M\), and the final score is \(Y\).
The before-program score is measured before treatment begins, so it is a baseline covariate. Session attendance is measured after the program begins. The program could affect attendance, so attendance is a post-treatment variable, even if it is strongly associated with the final score.
In a small hypothetical summary, the baseline score averages 7 points in the treatment arm and 6 points in the control arm. The post-treatment outcome averages 18 points in treatment and 14 points in control. The raw outcome contrast is
The 4-point difference is the overall arm contrast in this summary. The baseline difference is \(7-6=1\) point, which could be considered when specifying a baseline-adjusted analysis. The attendance measure is not made a baseline covariate just because it predicts the outcome. Adjusting for attendance would compare outcomes at a common attendance level, which is a different comparison from the overall 4-point arm contrast.
How Adjustment Can Remove Part of the Treatment Effect
Consider a simple hypothetical mechanism in which treatment increases a post-treatment behavior \(M\), and that behavior improves the outcome \(Y\). Assume the following equations describe every participant in this simplified example:
Here \(U\) is a pre-existing factor that varies across participants, is not affected by treatment, and is equally distributed between randomized arms. The coefficient 2 represents a treatment contribution to the outcome not through \(M\); the coefficient 3 describes the outcome change associated with a one-unit increase in \(M\). These equations are an illustrative data-generating model, not a claim that real trial data follow this exact pattern.
Worked Example: A Post-Treatment Mediator Changes the Contrast
For easy arithmetic, suppose \(U\) takes values 0 and 1 equally often in each arm. For a participant with \(T=0\) and \(U=0\), \(M=0+0=0\), so \(Y=2(0)+3(0)=0\). With \(T=0\) and \(U=1\), \(M=1\) and \(Y=0+3(1)=3\).
For \(T=1\) and \(U=0\), \(M=1+0=1\), giving \(Y=2(1)+3(1)=5\). For \(T=1\) and \(U=1\), \(M=2\), giving \(Y=2(1)+3(2)=8\). Thus the arm means are
and the overall treatment contrast is \(6.5-1.5=5\) outcome units. It includes both the 2-unit contribution represented by \(2T\) and the 3-unit contribution from treatment increasing \(M\) by one unit.
If a regression includes both \(T\) and \(M\), then in this example it fits \(Y=2T+3M\) exactly. The coefficient of \(T\), holding \(M\) fixed, is 2, not 5. At the shared value \(M=1\), for instance, the outcome is 3 when \(T=0\) and 5 when \(T=1\), a conditional difference of \(5-3=2\). The model has separated the treatment contribution not through \(M\) from the contribution carried through \(M\). That 2-unit conditional contrast is not the overall 5-unit contrast.
The arithmetic illustrates the general issue. When treatment changes \(M\), and \(M\) in turn relates to \(Y\), adjusting for \(M\) holds part of the treatment pathway fixed. The resulting treatment coefficient answers a conditional question rather than preserving the total effect operating through all pathways. It should not automatically be interpreted as a causal direct effect: that interpretation requires additional assumptions and a clearly specified target.
How Adjustment Can Create a Misleading Association
There is another risk. A post-treatment variable can be affected both by treatment and by a separate factor that predicts the outcome. Conditioning on that variable can make treatment assignment associated with the separate factor, despite randomization. This can produce a treatment coefficient that suggests an effect where none exists.
Worked Example: Conditioning Creates a Treatment Association
Imagine a second hypothetical trial. Let \(U\) indicate high baseline motivation, with \(U=1\) for high motivation and \(U=0\) otherwise. Random assignment \(T\) and motivation \(U\) are independent, and each of the four \((T,U)\) combinations occurs equally often. Let \(M=1\) if either the participant was assigned treatment or has high motivation; otherwise let \(M=0\). Suppose the outcome is simply \(Y=U\), so treatment has no effect on the outcome.
The four equally likely cases are \((T,U,M,Y)=(0,0,0,0),(0,1,1,1),(1,0,1,0),(1,1,1,1)\). Without adjustment, each treatment arm has mean outcome \(0.5\), so the treatment contrast is
Now compare treatment groups at \(M=1\). Among controls with \(M=1\), the only case is \(U=1\), with outcome 1. Among treated participants with \(M=1\), both \(U=0\) and \(U=1\) occur, so their mean outcome is \((0+1)/2=0.5\). The conditional difference is \(0.5-1=-0.5\), despite the true overall contrast being zero.
The reason is selection: among participants with \(M=1\), a control participant must have high motivation, while a treated participant need not. Treatment and motivation have become associated within this selected group. In this example, a regression of \(Y\) on \(T\) and \(M\) gives a treatment coefficient of \(-0.5\). To verify it, the fitted values at \((T,M)=(0,0),(0,1),(1,1)\) must be \(0,1,0.5\), respectively. An additive model with intercept 0, coefficient 1 for \(M\), and coefficient \(-0.5\) for \(T\) gives exactly those values. The adjusted coefficient is therefore misleading about the randomized overall treatment contrast.
A Practical Rule for Choosing Adjustment Variables
For a primary regression intended to estimate the overall treatment contrast in a randomized trial, decide on adjustment variables using information available before treatment begins. Do not add a post-treatment variable merely because it is predictive, statistically significant, or imbalanced between the observed arms. Its timing means that treatment may have influenced it, and conditioning on it can change the question or introduce selection effects.
A practical review of a candidate variable should ask:
- Was it measured before treatment began?
- Could treatment assignment or exposure have changed it?
- Is the intended comparison the overall effect, including changes in this variable, or a different, explicitly defined contrast?
- Would conditioning on it select participants differently across treatment arms?
If the scientific aim is to study a pathway through a post-treatment variable, that is a separate analysis requiring a clearly stated target and assumptions. It should not be treated as an ordinary baseline-adjusted regression. For the primary overall contrast, keeping post-treatment variables out of the adjustment set is the safer default.
Common Mistake
Wrong reasoning: “Attendance predicts the final health score, so adding attendance to the regression must make the treatment estimate better.” Prediction alone does not justify adjustment. If the program changes attendance, controlling for it can block part of the program’s effect; if other causes of attendance also affect health, conditioning on attendance can create a spurious treatment association.
Correction: Determine measurement timing and the intended treatment contrast before selecting predictors. Baseline variables are measured before treatment begins; post-treatment variables may be consequences of treatment. For an overall randomized-arm contrast, do not adjust for the latter as if they were baseline covariates.
Check Your Understanding
Try these questions before reviewing the examples.
- A trial records participants’ age before randomization and medication adherence after the first month. Which is a baseline covariate, and which is post-treatment?
- In the model \(M=T+U\) and \(Y=2T+3M\), what is the overall treatment contrast when treatment increases \(M\) by one unit? What coefficient for \(T\) results when \(M\) is held fixed?
- Why can adjusting for a post-treatment mediator change the treatment question, even when the regression model is correctly calculated?
- In the motivation example, why do treated and control participants with \(M=1\) have different average motivation despite random assignment?
- A variable measured after treatment begins strongly predicts the outcome. Is that alone a reason to include it as an adjustment variable for the overall treatment contrast? Explain.