Display Uncertainty for Interaction Predictions

Why an Interaction Plot Needs Uncertainty

A fitted interaction plot shows how predicted mean outcomes change across predictor values. Its lines are estimates, not known truths. A line may look steep, flat, or separated from another line partly because of sampling variation. Adding confidence intervals or a confidence band shows how uncertain the estimated mean outcomes are at the predictor values displayed.

For an interaction between two quantitative predictors, the plotted mean depends on both predictors. One common model is

Y_i=\beta_0+\beta_1x_{1i}+\beta_2x_{2i}+\beta_3x_{1i}x_{2i}+\varepsilon_i.

Here \(Y_i\) is the outcome for observation \(i\), \(x_{1i}\) and \(x_{2i}\) are its predictor values, and the product term allows the association between one predictor and the mean outcome to depend on the other. At a particular pair \((x_1,x_2)\), the fitted mean is \(\hat{Y}(x_1,x_2)\). The uncertainty attached to that fitted mean is not the same as the spread of individual outcomes around it.

Definition: A confidence interval for a fitted mean at \((x_1,x_2)\) estimates the mean outcome for the population represented by the model at those predictor values. A prediction interval estimates a range for one new individual outcome at those values, including both uncertainty in the fitted mean and individual outcome variation.

Model and Standard Error of a Fitted Mean

For the usual \(t\)-based intervals in this tutorial, assume the interaction model is correctly specified for the mean, observations are independent, the error terms have mean zero and a common variance, and the errors are normally distributed. The predictors are treated as fixed at the values where a prediction is requested, and the model matrix has full column rank. Under these assumptions, the estimated coefficients have a covariance matrix that quantifies their sampling uncertainty.

Write the coefficient estimates as \(\hat{\boldsymbol{\beta}}=(\hat{\beta}_0,\hat{\beta}_1,\hat{\beta}_2,\hat{\beta}_3)^\mathsf{T}\), and let \(\widehat{\operatorname{Var}}(\hat{\boldsymbol{\beta}})\) be their estimated covariance matrix. At the predictor pair \((x_1,x_2)\), define the row vector \(\mathbf{z}=(1,x_1,x_2,x_1x_2)\). Then

\hat{Y}(x_1,x_2)=\mathbf{z}\hat{\boldsymbol{\beta}}, \qquad SE\{\hat{Y}(x_1,x_2)\} = \sqrt{\mathbf{z}\widehat{\operatorname{Var}}(\hat{\boldsymbol{\beta}})\mathbf{z}^{\mathsf{T}}}.

The formula includes covariances between coefficient estimates. Those covariance terms can add to or subtract from the variance at a particular point; they should not be omitted unless the covariance matrix actually has zero entries there. The degrees of freedom for the usual four-coefficient model with \(n\) observations are \(df=n-4\).

A two-sided \(100(1-\alpha)\%\) pointwise confidence interval is

\hat{Y}(x_1,x_2)\ \pm\ t_{1-\alpha/2,\,df}\, SE\{\hat{Y}(x_1,x_2)\}.

“Pointwise” means the interval has the stated confidence level for the mean at that specified predictor pair. It does not automatically give the same confidence level that all intervals across a whole plotted line or surface will cover their means simultaneously.

Worked Example: Confidence Intervals at Two Points on an Interaction Plot

Suppose a hypothetical fit uses \(n=50\) observations and estimates \((\hat{\beta}_0,\hat{\beta}_1,\hat{\beta}_2,\hat{\beta}_3)=(8,1,2,0.5)\). For this example, suppose the estimated coefficient covariance matrix is diagonal, with diagonal entries \(0.18,\ 0.09,\ 0.04,\ 0.02\), in that order. These are assumed outputs from this hypothetical fit, not values inferred from the coefficient estimates alone. The residual standard deviation is \(s=3.00\) points, and \(df=50-4=46\). Use \(t_{0.975,46}\approx2.013\).

At \(x_1=1\) and \(x_2=2\), the fitted mean is

\hat{Y}=8+1(1)+2(2)+0.5(1)(2)=14.0\text{ points}.

Because the covariance matrix is diagonal, the variance at this point is the sum of each coefficient variance multiplied by the square of its corresponding predictor value:

SE(\hat{Y}) = \sqrt{0.18+1^2(0.09)+2^2(0.04)+(1\cdot2)^2(0.02)} = \sqrt{0.51} = 0.7141\text{ points}.

The margin of error is \(2.013(0.7141)=1.438\) points, so the 95% pointwise confidence interval is \(14.0\pm1.438\), or approximately \((12.56,\ 15.44)\) points.

At \(x_1=3\) and \(x_2=2\), the fitted mean is \(8+1(3)+2(2)+0.5(3)(2)=18.0\) points. Its standard error is

SE(\hat{Y}) = \sqrt{0.18+3^2(0.09)+2^2(0.04)+(3\cdot2)^2(0.02)} = \sqrt{1.87} = 1.3675\text{ points}.

The margin is \(2.013(1.3675)=2.753\) points. The 95% pointwise interval is therefore approximately \((15.25,\ 20.75)\) points. The larger standard error at this point follows from the predictor values multiplying the uncertain slope and interaction coefficients; it does not mean that the individual outcomes necessarily vary more there.

From Pointwise Intervals to a Displayed Band

To draw uncertainty on an interaction plot, calculate the fitted mean and its standard error at each plotted predictor value. For a fixed value of \(x_2\), these calculations trace a fitted line as \(x_1\) varies. At each \(x_1\), a pointwise interval can be displayed as a vertical bar, or its lower and upper endpoints can be joined to form a ribbon around the line. Repeat this process for each displayed value of \(x_2\).

A ribbon made from ordinary pointwise intervals is a visual summary of the pointwise intervals; it is not automatically a simultaneous confidence band. If a plot presents many intervals, the chance that they all cover their corresponding means can be lower than the nominal confidence level. For a finite set of \(m\) selected points, one conservative way to provide at least \(1-\alpha\) simultaneous coverage is the Bonferroni method: use a two-sided critical value \(t_{1-\alpha/(2m),df}\) at each point. This controls the probability of missing at least one of the \(m\) means by at most \(\alpha\), under the stated interval assumptions. It is conservative, and it applies to the chosen finite set; it does not by itself establish simultaneous coverage over every possible predictor value.

Worked Example: Adjusting Intervals for Two Displayed Means

Consider a separate hypothetical fit with \(df=46\), and two fitted means that will be highlighted on an interaction plot. At the first point, the estimated mean is \(10.4\) points with standard error \(1.039\) points. At the second, the estimated mean is \(15.2\) points with standard error \(1.311\) points. The aim is at least 95% simultaneous coverage for these two means, not just 95% coverage at each one separately.

There are \(m=2\) intervals, so Bonferroni uses \(\alpha/(2m)=0.05/4=0.0125\) in each tail. The critical value is \(t_{0.9875,46}\approx2.317\). For the first point, the margin is \(2.317(1.039)=2.407\) points, giving

10.4\pm2.407=(7.99,\ 12.81)\text{ points}.

For the second point, the margin is \(2.317(1.311)=3.038\) points, giving

15.2\pm3.038=(12.16,\ 18.24)\text{ points}.

These intervals are wider than the corresponding 95% pointwise intervals because the procedure accounts for making two interval statements together. The Bonferroni guarantee concerns these two selected fitted means; it does not mean that every point along a continuously drawn curve is simultaneously covered.

Why Mean Uncertainty Is Not Individual Variation

A confidence interval for the fitted mean concerns uncertainty about an average outcome at the specified predictor values. A new individual outcome also varies around that mean. Under the model assumptions, the standard error for predicting one new outcome includes both sources of uncertainty:

SE_{\mathrm{pred}} = \sqrt{s^2+SE\{\hat{Y}(x_1,x_2)\}^2}.

The \(s^2\) term represents the residual variation of individual outcomes around their model mean. Since it is added to the variance of the estimated mean, the prediction standard error is larger than the standard error of the fitted mean when \(s>0\). A prediction interval for one new outcome therefore tends to be wider than a confidence interval for the mean at the same predictor values.

Worked Example: Mean Confidence Interval Versus Individual Prediction Interval

In a separate hypothetical interaction model with \(df=46\), suppose a fitted mean at \(x_1=2\) hours and \(x_2=4\) hours is \(12.8\) points, its standard error is \(1.149\) points, and the residual standard deviation is \(s=3.00\) points. These values describe this separate hypothetical fit; no diagonal coefficient covariance assumption is being carried over from the first example. Use \(t_{0.975,46}\approx2.013\).

For the mean outcome, the 95% confidence interval has margin \(2.013(1.149)=2.313\) points:

12.8\pm2.313=(10.49,\ 15.11)\text{ points}.

For one new individual outcome at the same predictor values, the prediction standard error is

SE_{\mathrm{pred}} = \sqrt{(3.00)^2+(1.149)^2} = \sqrt{9.00+1.320201} \approx3.2125\text{ points}.

Using the displayed rounded value, the prediction margin is \(2.013(3.212)=6.466\) points. The 95% prediction interval is approximately \(12.8\pm6.466=(6.33,\ 19.27)\) points. It is much wider than the mean confidence interval because it includes individual outcome variation as well as uncertainty in the fitted mean. Neither interval says that there is a 95% probability that a fixed parameter or future outcome lies in this particular calculated interval; the 95% refers to the long-run coverage of the procedure under the model assumptions.

Common Mistakes When Displaying Uncertainty

Using only the plotted line. The fitted values alone do not show how precisely the mean is estimated. Calculate the standard error from the coefficient covariance matrix at each plotted point.
Dropping coefficient covariances without checking. The variance formula is a quadratic form. Unless the relevant covariance terms are zero, omitting them can give incorrect standard errors.
Calling pointwise ribbons simultaneous bands. Many pointwise intervals do not automatically provide the advertised coverage for all plotted means together. Use an appropriate simultaneous procedure when that is the goal, and state which predictor values it covers.
Confusing a mean interval with a prediction interval. A confidence interval describes uncertainty in a population mean. A prediction interval also accounts for the spread of individual outcomes and is wider.
Reading a wider interval as evidence of a larger effect. Interval width measures uncertainty, not the size or direction of the fitted mean or slope.

An uncertainty display makes an interaction plot more informative, but it does not repair a model whose assumptions are unsuitable. Interpret the intervals in the outcome’s units and at the predictor values actually used to calculate them. State whether the display is pointwise or simultaneous and whether it concerns a mean or a new individual outcome.

Check Your Understanding

Answer each question using the definitions and calculations in this tutorial.

In an interaction model, what quantities are needed to calculate the standard error of a fitted mean at a specified pair of predictor values?
Explain what “pointwise” means for a 95% confidence interval at one point on a fitted interaction line.
Why does a Bonferroni adjustment widen the two intervals in the second worked example?
At the same predictor values, why is a prediction interval for one new outcome generally wider than a confidence interval for the mean?
What is wrong with claiming that a ribbon formed from many ordinary 95% pointwise intervals necessarily covers the entire fitted line with 95% confidence?