Testing Conditional Effects at Chosen Values

Why Test a Conditional Effect?

An interaction model allows the relationship between a predictor and an outcome to vary with another variable. A fitted model can therefore imply different slopes or group differences at different moderator values. A confidence interval describes uncertainty about one such conditional effect; a hypothesis test asks whether that effect is consistent with a particular specified value.

For example, a researcher might ask whether the association between exercise and a well-being score is zero among people who sleep five hours per night. Or the question might be whether a tutoring group’s predicted test score differs from the comparison group by exactly zero points for students who study ten hours per week. These are tests about effects at chosen moderator settings, not necessarily tests about a single interaction coefficient.

Model and Test

Consider a continuous predictor \(X\) and a moderator \(M\), with the interaction model

Here, the conditional mean of \(Y\) at \(X=x\) and \(M=m\) is \(\beta_0+\beta_1x+\beta_2m+\beta_3xm\). Holding \(M=m\) fixed, the slope for \(X\) is \(\beta_1+\beta_3m\). Its estimated value at a chosen setting \(m_0\) is \(b_1+m_0b_3\), where \(b_j\) denotes the fitted estimate of \(\beta_j\).

A related model uses a binary group indicator \(G\), coded 0 or 1, and a continuous moderator \(X\):

At \(X=x_0\), the predicted mean for group 1 minus the predicted mean for group 0 is \(\beta_2+x_0\beta_3\). Its estimate is \(b_2+x_0b_3\). The same testing method applies to either conditional slope or group contrast.

The usual regression t test relies on the model being correctly specified for the conditional mean, independent errors, constant error variance, and normally distributed errors for exact small-sample inference. Under these conditions, the fitted coefficient estimates have a covariance matrix estimated from the regression. Residual plots and knowledge of the data collection can help assess whether the assumptions are plausible. If the conditions are seriously questionable, the usual standard error and t reference distribution may not be suitable.

The test statistic compares the estimated effect with the null value, measured in standard-error units. For a model with \(p\) fitted coefficients and \(n\) observations, the residual degrees of freedom are \(df=n-p\). Under the stated classical model assumptions, the statistic follows a t distribution with these degrees of freedom when \(H_0\) is true.

The covariance matrix matters because estimates of different regression coefficients can vary together. In the special case of the slope \(\beta_1+m_0\beta_3\), its estimated variance is

For a two-sided alternative, the p-value is the probability, under \(H_0\) and the assumed t distribution, of a statistic at least as far from zero as the observed one:

A small p-value is evidence against the stated null value under the model assumptions. It is not the probability that \(H_0\) is true, nor does it measure the size or practical importance of the effect.

Calculating and Interpreting Conditional Tests

Why the Covariance Term and the Null Value Matter

The standard error is not generally found by adding coefficient standard errors, and it is not generally correct to ignore covariance. The variance formula follows from the variance of a sum. For random estimates \(B_1\) and \(B_3\), and a fixed value \(m_0\),

To see the covariance contribution, subtract the means and expand the squared deviation:

The same reasoning gives \(L\widehat{\operatorname{Cov}}(\mathbf{b})L^{\mathsf T}\) for any linear combination. The null value \(c\) also matters: a test of whether a conditional slope equals 1.0 is not the same test as whether it equals zero.

Common Mistakes and Practical Interpretation

• Testing only the interaction coefficient. Testing \(H_0:\beta_3=0\) asks whether the slope changes with the moderator. It does not test whether the slope at a particular \(m_0\) equals zero or another value. The conditional-effect test uses the full combination, such as \(\beta_1+m_0\beta_3\).
• Leaving out covariance. Adding only the two variance terms can produce the wrong standard error. Include \(2m_0\widehat{\operatorname{Cov}}(b_1,b_3)\), or use the full coefficient covariance matrix.
• Comparing the estimate directly with the null. A difference such as \(1.00-0\) does not show whether that difference is large relative to its uncertainty. Divide by the standard error to obtain the t statistic.
• Reading a large p-value as proof of equality. A large p-value means the data do not provide strong evidence against the tested value; it does not show that the effect equals that value exactly.
• Ignoring the choice of moderator setting. State the chosen value, its units, and the conditional effect being tested. A result at one setting does not automatically describe the effect at other settings.

If several moderator values are tested and only the most favorable result is reported, the nominal p-value can give an overly optimistic picture of evidence. The chosen setting and hypothesis should be motivated and reported clearly. Always interpret the estimated effect and uncertainty in the outcome’s units, alongside the p-value.