Why Test for Treatment Effect Modification?
In a randomized trial, the treatment contrast may differ across participants with different baseline outcomes. For example, an intervention might have a larger modeled effect for participants who begin with lower scores than for those who begin with higher scores. A regression model can represent this possibility by including a baseline-by-treatment interaction. Testing that interaction asks whether the modeled treatment contrast changes linearly with baseline outcome, rather than whether the treatment has an effect at one particular baseline value.
The test concerns a specified regression model and outcome scale. A conclusion about interaction on one scale, such as the mean difference in an outcome score, does not automatically imply interaction on another scale. The model and the meaning of its coefficients should therefore be stated before interpreting a test result.
The Interaction Model and Its Hypothesis
Let \(A_i\) indicate randomized assignment for participant \(i\), with \(A_i=1\) for the treatment group and \(A_i=0\) for the control group. Let \(X_i\) be the baseline outcome and \(Y_i\) the follow-up outcome. Write \(X_{ci}=X_i-c\), where \(c\) is a chosen centering value, such as the sample mean of the baseline outcome. The linear interaction model is
The model includes both main-effect terms, \(A_i\) and \(X_{ci}\), as well as their product. Including those main effects is important: the interaction is an additional term, not a substitute for either main effect. At a baseline value \(X_i=x\), the model’s treatment-versus-control contrast in expected follow-up outcome is \(\beta_1+\beta_3(x-c)\). Thus, \(\beta_3\) is the change in that treatment contrast for each one-unit increase in baseline outcome.
For the usual two-sided test, the hypotheses are \(H_0:\beta_3=0\) and \(H_a:\beta_3\ne 0\). The null says there is no linear change in the modeled treatment contrast across baseline values. It does not say the treatment contrast itself is zero: \(\beta_1\), the contrast at \(X_i=c\), could be nonzero even when \(\beta_3=0\).
Before using the standard model-based t test and confidence interval, assume the conditional mean is adequately represented by the stated linear model; the errors are independent with mean zero conditional on assignment and baseline; the errors have constant variance; and, for exact small-sample t inference, the errors are normally distributed conditional on assignment and baseline. There must also be sufficient baseline variation and observations in both assignment groups to estimate the model. Randomization supports comparisons by assignment, but it does not by itself guarantee that the linear form, constant variance, or other regression assumptions are appropriate.
In ordinary least squares with these assumptions, there are four fitted coefficients: the intercept, two main effects, and the interaction. For \(n\) complete observations, the residual degrees of freedom are \(df=n-4\). The interaction test statistic is
Under \(H_0\) and the stated assumptions, this statistic follows a t distribution with the residual degrees of freedom. The two-sided p-value is the probability, assuming \(H_0\), of a t statistic at least as far from zero as the observed statistic. At significance level \(\alpha\), reject \(H_0\) when the p-value is at most \(\alpha\).
A \(100(1-\alpha)\%\) confidence interval for the interaction coefficient is
where \(t_{1-\alpha/2,\,df}\) is the corresponding positive t critical value. The interval gives plausible values for the change in treatment contrast per one-unit increase in baseline outcome, under the model and its assumptions.
Why the Test and Interval Agree
For a two-sided test at level \(\alpha\), the decision from the t test matches the decision from the corresponding \(100(1-\alpha)\%\) confidence interval: reject \(H_0:\beta_3=0\) exactly when the interval excludes zero. This follows directly from the formulas. The test rejects when
Because \(SE(\hat{\beta}_3)>0\), this inequality is equivalent to \(|\hat{\beta}_3|>t_{1-\alpha/2,\,df}SE(\hat{\beta}_3)\). That is exactly the condition that zero falls outside the interval centered at \(\hat{\beta}_3\) with that margin of error. The p-value describes the test evidence relative to a chosen \(\alpha\); the interval also shows the estimated direction and range of interaction sizes compatible with the data.
Worked Example: Evidence of a Changing Treatment Contrast
Suppose a hypothetical randomized trial measures follow-up score in points and baseline score in baseline-score points. Baseline is centered at \(c=50\). There are \(n=40\) participants, so the interaction model has \(df=40-4=36\). The fitted product-term estimate is \(\hat{\beta}_3=-0.400\) follow-up points per baseline point, with \(SE(\hat{\beta}_3)=0.1667\) follow-up points per baseline point. Use a two-sided test at \(\alpha=0.05\).
Check the model basis. For this hypothetical analysis, assume the conditional mean is adequately linear in baseline and includes both main effects and the product term; observations are independent; residual variance is constant; and the conditional errors are approximately normal for the t inference. Also assume both assignment groups have adequate representation across the observed baseline range.
Test statistic. The hypotheses are \(H_0:\beta_3=0\) and \(H_a:\beta_3\ne 0\). Substitution gives
With \(df=36\), the two-sided p-value is approximately \(0.02169\). Since \(0.02169<0.05\), reject \(H_0\) at the 5% level.
Confidence interval. The 95% t critical value for 36 degrees of freedom is approximately \(2.0281\). Therefore,
The estimated treatment contrast decreases by 0.400 follow-up-score points for each one-point increase in baseline score. The 95% confidence interval suggests a decrease of about 0.0619 to 0.7381 follow-up-score points per baseline point. Because the interval excludes zero, it agrees with the test’s evidence of a linear baseline-by-treatment interaction. This describes the modeled contrast in this trial; it does not establish that baseline score itself causes a different response.
Worked Example: A Wide Interval That Includes No Modification
In another hypothetical trial, the estimated interaction is \(0.180\) outcome points per baseline point, with standard error \(0.200\). There are 50 complete observations, giving \(df=50-4=46\). Suppose the model assumptions are considered reasonable for this analysis.
The test statistic is
For \(df=46\), the two-sided p-value is approximately \(0.373\), which is greater than \(0.05\). The data do not provide sufficient evidence to reject \(H_0:\beta_3=0\) at the 5% level.
For a 95% interval, \(t_{0.975,46}\) is approximately \(2.0129\), so
This interval includes zero, but also includes both a negative interaction and a positive interaction of potentially meaningful size. The result is not proof that the treatment contrast is identical at every baseline value; it indicates that this analysis has not precisely established the direction or size of linear modification. To assess whether the interval rules out differences large enough to matter clinically, those differences must be considered in the outcome’s units and context.
Centering and the Interaction Test
Centering baseline changes the reference point for the treatment main-effect coefficient, but it does not change the interaction coefficient or its test when the model includes both main effects. This is useful when reporting the interaction: choosing a meaningful center can make \(\beta_1\) easier to interpret without changing the question answered by the product-term test.
To see why, write \(X_c=X-c\). The centered model’s treatment and interaction terms are \(\beta_1A+\beta_3A(X-c)\). Expanding them gives
The coefficient of \(AX\) remains \(\beta_3\); centering changes the coefficient of \(A\) to \(\beta_1-c\beta_3\). This is a re-expression of the same fitted mean model, not a change in the interaction question. In particular, the test of \(\beta_3=0\) is not a test of whether the treatment contrast at the centering value is zero.
Worked Example: Interpreting an Interaction After Centering
Suppose a hypothetical trial records baseline score in points and centers it at 60 points. The estimated product-term coefficient is \(0.300\) follow-up points per baseline point, with standard error \(0.120\), and \(df=76\). Assume the same linearity, independence, constant-variance, and conditional-normality conditions used for the standard t inference are reasonable.
The test statistic is
With 76 degrees of freedom, the two-sided p-value is approximately \(0.0146\). At \(\alpha=0.05\), this is evidence against \(H_0:\beta_3=0\). The 95% critical value is approximately \(1.9917\), giving
The interval is in follow-up points per baseline point. Under this model, each one-point increase in baseline score is associated with an increase of approximately 0.0610 to 0.5390 points in the treatment-versus-control contrast, as represented by the fitted interaction. The positive interval indicates that the modeled contrast increases with baseline score. Centering at 60 means the treatment main-effect coefficient describes the contrast at a baseline score of 60; it does not alter this product-term test.
Common Mistakes and Careful Reporting
A common mistake is to test whether the treatment contrast is zero at one baseline value and treat that as a test of effect modification. Those are different hypotheses. The contrast at \(X_i=c\) is \(\beta_1\), so its test concerns \(H_0:\beta_1=0\). The interaction test concerns \(H_0:\beta_3=0\), or whether the contrast changes as baseline changes. A treatment contrast can be nonzero at the center with no linear modification, or it can vary with baseline even when the contrast at the center is zero.
Another mistake is to call a nonsignificant interaction proof that treatment effects are the same across baseline values. A large p-value only says the observed result is not sufficiently inconsistent with the zero-interaction null under the model; it does not measure the probability that the null is true. Read the interval as well, because it shows which interaction sizes remain compatible with the data.
A clear report identifies the outcome scale, the baseline unit, the interaction estimate and standard error, the test result, and the interval. For example: “The estimated change in the treatment contrast was \(-0.400\) follow-up-score points per baseline-score point (SE \(0.1667\)); \(t(36)=-2.40\), \(p=0.02169\), with a 95% CI from \(-0.7381\) to \(-0.0619\).” This wording gives the direction and units without claiming that the interaction itself proves a mechanism.
Check Your Understanding
Use the model, test, and interpretation principles above to answer these questions.
- In the interaction model, what does \(\beta_3\) measure, and what hypotheses test for no linear modification?
- A trial reports \(\hat{\beta}_3=-0.25\) outcome points per baseline point with \(SE=0.10\). Calculate the t statistic and state what additional information is needed to obtain the p-value.
- Why does a 95% confidence interval for \(\beta_3\) that includes zero correspond to not rejecting the two-sided test at \(\alpha=0.05\)?
- Explain why testing whether the treatment contrast is zero at the centered baseline value is not the same as testing for treatment effect modification.
- A nonsignificant interaction estimate has a wide interval containing substantial positive and negative values. What can and cannot be concluded from that result?