Effect Uncertainty Versus Outcome Variability

Two Different Sources of Spread

A fitted regression model can be used to describe an average relationship and to predict individual outcomes. These uses involve two kinds of spread that are easy to confuse. Effect uncertainty is uncertainty about an estimated slope, contrast, or conditional mean. Outcome variability is the spread of individual responses around their conditional mean. A confidence interval for an effect or mean addresses the first; a prediction interval for an individual outcome must address both.

The distinction matters when interpreting an interval. A wide prediction interval does not necessarily mean that the estimated effect is imprecise: individual outcomes can vary substantially even when the fitted mean is estimated precisely. Conversely, a narrow residual spread does not guarantee a precise effect estimate, especially when the data provide limited information about the relevant predictor values.

A Model for Conditional Effects

Consider a continuous predictor \(X\), a categorical group indicator \(G\) coded 0 or 1, and an outcome \(Y\). A simple interaction model is

The model's conditional mean is \(E(Y_i\mid X_i,G_i)=\beta_0+\beta_1X_i+\beta_2G_i+\beta_3X_iG_i\). The slope for \(X\) when \(G=0\) is \(\beta_1\); when \(G=1\), it is \(\beta_1+\beta_3\). At a specified value \(x\), the conditional mean difference between group 1 and group 0 is \(\beta_2+x\beta_3\). These are model quantities, and their estimates have sampling uncertainty.

For the usual small-sample \(t\)-based intervals in this tutorial, assume the conditional mean has the stated linear form, errors are independent with mean zero and constant variance \(\sigma^2\), and errors are normally distributed. The predictor values are treated as fixed for the inference. Under these assumptions, the estimated coefficients are unbiased, their covariance matrix measures their sampling uncertainty, and the stated \(t\) intervals have their usual exact coverage in repeated samples. In applications, these assumptions need to be considered before relying on the intervals.

Let \(s^2\) estimate the error variance and let \(df\) denote the residual degrees of freedom. Any linear combination of estimated coefficients can be written as an estimate of a model quantity. Its standard error is the square root of its estimated sampling variance, calculated using coefficient variances and covariances. A confidence interval is the estimate plus or minus a \(t\) critical value times that standard error. It describes uncertainty in the model quantity, not the scatter of individual outcomes.

Intervals for Conditional Mean Contrasts

The same distinction applies to a conditional mean contrast. In the model above, the group difference at \(X=x\) is \(\beta_2+x\beta_3\), so the estimated difference is \(\hat{\beta}_2+x\hat{\beta}_3\). Its estimated variance is

The covariance term is essential: coefficient estimates from the same model are generally related. The standard error obtained from this variance describes how much the estimated contrast would vary across repeated samples under the model. It does not describe how far an individual response may be from its group-specific conditional mean.

Mean Response and Individual Prediction

At specified predictor values, a fitted regression model gives an estimated conditional mean, \(\hat{Y}_0\). A confidence interval for the true conditional mean accounts for the sampling uncertainty in \(\hat{Y}_0\). A prediction interval for one future individual outcome must also account for that individual's residual variation around the conditional mean.

In ordinary least squares with independent, constant-variance normal errors, write \(h_0\) for the design-based variance factor at the predictor values of interest. The estimated standard error for the fitted mean is \(s\sqrt{h_0}\). For one new individual at those same predictor values, the prediction standard error is \(s\sqrt{1+h_0}\). The extra 1 represents the new individual's error variance, measured in units of \(s^2\).

The derivation shows exactly why the intervals differ. For the fitted mean, the standard error is \(s\sqrt{h_0}\); there is no new individual's residual in the target. For an individual outcome, the additional residual term cannot be omitted. Since \(1+h_0>h_0\), a prediction interval is wider than the corresponding confidence interval for the mean, assuming both use the same \(t\) critical value.

Reading an Interval Before Interpreting It

An interval's numerical endpoints alone do not identify what it describes. The target quantity and the method used to calculate the interval matter. Before interpreting a reported interval, ask whether the target is a slope, a contrast, a conditional mean, or an individual outcome. Then check whether the calculation includes only coefficient-estimation uncertainty or also includes residual variation.

A useful summary is:

Common mistake: “The prediction interval is wide, so the regression effect must be estimated imprecisely.” This reasoning confuses outcome scatter with effect uncertainty. The correction is to identify the interval's target: a prediction interval is wide partly because individual outcomes vary around their means. To assess uncertainty in a slope or mean contrast, use the standard error and interval for that specific model quantity instead.

The reverse mistake is to use a confidence interval for a conditional mean as though it describes likely individual outcomes. Such an interval can be narrow while individual responses remain widely scattered. It does not promise that most individuals fall inside it; it concerns the conditional mean, not the residual distribution of individual observations.