Why the Mean Model Matters for Prediction
A prediction interval is built around a fitted mean, but the mean model must describe the relationship between the outcome and the predictors well enough for that center to be useful. If the true relationship bends with a predictor, or if the effect of one predictor depends on another, a model that omits those features can be systematically wrong. Increasing the interval width to reflect ordinary random error does not necessarily correct that error in the center.
The issue is especially important when predicting a new outcome at a particular predictor value or profile. A fitted model may have residuals that average to zero overall and still miss in a structured way: for instance, it may predict too low at both ends of a curved relationship and too high in the middle. Such patterns can make prediction intervals too narrow in some regions or simply misplaced relative to the outcomes they are intended to cover.
Residual Patterns as Clues
For an observed case \(i\), a residual is \(e_i=y_i-\hat{y}_i\), where \(y_i\) is the observed outcome and \(\hat{y}_i\) is the fitted mean. Under an adequate mean model, the conditional mean of the residuals should be zero across predictor values, so there should be no systematic pattern in residual means against fitted values or predictors. Patterns in residual spread, such as a fan shape, may indicate variance misspecification rather than mean-model misspecification. Random scatter around zero is consistent with an adequate model, though it does not prove one.
Curvature may appear as a U-shaped or inverted-U-shaped pattern when residuals are plotted against a continuous predictor or fitted values. A missing interaction may appear when residual patterns differ systematically across groups: for example, residuals rise with \(x\) in one group but fall with \(x\) in another. These are clues to investigate, not automatic proof that a particular alternative is correct. A small sample, unusual observations, or other model problems can also produce apparent patterns.
A targeted alternative model adds a term that represents the suspected structure. For curvature, one possibility is to add \(x^2\) to a model that already contains \(x\). For an interaction between a numerical predictor \(x\) and a group indicator \(G\), one possibility is to add \(xG\). The alternative should be motivated by the scientific setting and the residual pattern, rather than added simply because it improves a numerical fit in the observed data.
Worked Example: A Curved Relationship Hidden by a Straight Line
Consider hypothetical outcomes measured at five predictor values. Suppose the observed values are \(12, 7, 4, 7,\) and \(12\), respectively, at \(x=-2,-1,0,1,2\). Fit a straight-line mean model and inspect its residuals.
The mean of the predictor values is \(\bar{x}=0\), and the mean outcome is \(\bar{y}=(12+7+4+7+12)/5=8.4\). The least-squares slope is \(\hat{\beta}_1=\sum (x_i-\bar{x})(y_i-\bar{y})/\sum (x_i-\bar{x})^2\). The numerator is \((-2)(3.6)+(-1)(-1.4)+0(-4.4)+(1)(-1.4)+(2)(3.6)=0\), and the denominator is \(4+1+0+1+4=10\). Thus \(\hat{\beta}_1=0\), and the fitted line is \(\hat{y}=8.4\).
The residuals, in increasing order of \(x\), are \(12-8.4=3.6\), \(7-8.4=-1.4\), \(4-8.4=-4.4\), \(7-8.4=-1.4\), and \(12-8.4=3.6\). They are positive at both ends and negative near the center, a clear U-shaped pattern. A plausible targeted alternative is a quadratic mean model containing \(x^2\), although these values do not lie exactly on \(\hat{y}=4+2x^2\): that curve predicts 6 at \(x=\pm1\), rather than the observed 7.
The straight-line model's residuals sum to zero, as least-squares residuals do when an intercept is included, but that does not make the line adequate. The pattern shows why a zero overall residual average does not rule out local mean errors. In a real analysis, the added quadratic term would need to be assessed in context and the alternative model's assumptions checked; these five illustrative values alone do not establish a general relationship.
Omitted Interactions and Group-Specific Patterns
An additive model assumes that the change in mean outcome associated with one predictor is the same across levels of another predictor, unless the model explicitly allows otherwise. An interaction term lets that change depend on the other predictor. If the interaction is important but omitted, a model can fit group averages reasonably while still making systematically different errors across predictor profiles.
Worked Example: Detecting an Omitted Interaction
Suppose a hypothetical balanced \(2\times2\) data set has the following cell means. Here \(x\) is either 0 or 1, and \(G\) is a group indicator equal to 0 or 1.
| \(x\) | \(G\) | Mean outcome |
|---|---|---|
| 0 | 0 | 10 |
| 1 | 0 | 12 |
| 0 | 1 | 13 |
| 1 | 1 | 19 |
Fit the additive model \(\hat{y}=b_0+b_1x+b_2G\) to these four equally weighted cell means. The change across \(x\), averaged over the two groups, is \(b_1=[(12-10)+(19-13)]/2=(2+6)/2=4\). The group difference, averaged over the two values of \(x\), is \(b_2=[(13-10)+(19-12)]/2=(3+7)/2=5\). The grand mean is \((10+12+13+19)/4=13.5\), so \(b_0=13.5-(4)(0.5)-(5)(0.5)=9\). The fitted additive model is therefore \(\hat{y}=9+4x+5G\).
Its fitted cell means are 9, 13, 14, and 18, in the table's order. The residuals are \(10-9=1\), \(12-13=-1\), \(13-14=-1\), and \(19-18=1\). In group \(G=0\), the observed change across \(x\) is \(12-10=2\); in group \(G=1\), it is \(19-13=6\). The differing changes suggest an interaction. A targeted model adds \(b_3xG\), allowing the slope in group \(G=1\) to differ from the slope in group \(G=0\).
The fitted values here are based on cell means, not on individual outcomes, so this calculation illustrates the structure rather than providing a prediction interval or a significance test. In actual data, residual patterns and uncertainty in the fitted coefficients should be considered before concluding that an interaction is needed.
Why Misspecification Can Reduce Coverage
A conventional prediction interval for a new outcome is centered at the fitted mean for its predictor values. Its width accounts for uncertainty in estimating the mean and for variation of a new outcome around the true mean, under the model's assumptions. If the fitted mean is systematically offset from the true conditional mean at the prediction point, the interval is displaced. Its stated width does not automatically include that unmodeled bias.
The reason can be stated precisely. Suppose the new outcome is \(Y_{\mathrm{new}}=m+\varepsilon\), where \(\varepsilon\sim N(0,\sigma^2)\). Write the prediction interval as \([\hat{m}-w,\hat{m}+w]\), with half-width \(w>0\). The new outcome is covered exactly when \(\hat{m}-w\leq m+\varepsilon\leq\hat{m}+w\), or equivalently \(-w-\delta\leq\varepsilon\leq w-\delta\). Therefore its coverage probability is
where \(\Phi\) is the standard normal cumulative distribution function. If \(\delta=0\), this becomes the centered probability \(\Phi(w/\sigma)-\Phi(-w/\sigma)\). To see why a nonzero shift reduces coverage, set \(a=w/\sigma>0\) and \(d=\delta/\sigma\). The coverage as a function of \(d\) is \(C(d)=\Phi(a-d)-\Phi(-a-d)\). It is symmetric in \(d\). For \(d>0\), its derivative is \(C'(d)=-\phi(a-d)+\phi(a+d)<0\), because \(|a+d|>|a-d|\) and the standard normal density \(\phi\) decreases as the absolute value of its argument increases. Thus coverage is greatest at \(d=0\) and falls as the magnitude of the shift grows.
Worked Example: A Shifted Interval Has Lower Coverage
Suppose a hypothetical fitted model produces a prediction interval from 40 to 60 units at a particular predictor profile. Assume, for illustration, that the true conditional distribution of new outcomes at that profile is normal with mean 56 units and standard deviation 5 units. The interval's half-width is \(w=10\), but its center, 50, is 6 units below the true mean, so \(\delta=56-50=6\).
The actual coverage under the stated hypothetical distribution is \(P(40\leq Y_{\mathrm{new}}\leq60)=\Phi((60-56)/5)-\Phi((40-56)/5)=\Phi(0.8)-\Phi(-3.2)\). Using the unrounded CDF values, the coverage is approximately \(0.7875\), or \(78.75\%\). It is not 95%.
This calculation does not claim that every fitted interval with a 6-unit mean error has that coverage. The probability depends on the true error distribution and interval width as well as the shift. It demonstrates that a specified width alone cannot guarantee nominal coverage when the interval is centered at a biased mean.
Using the Diagnosis in Practice
When residuals suggest mean-model misspecification, the next step is to investigate the pattern with a focused alternative, not to assume that the first apparent pattern is definitive. For suspected curvature, compare a model containing \(x\) with one that also contains a suitable curvature term. For suspected effect variation across groups, compare an additive model with one containing the relevant interaction. Then examine whether the alternative removes the systematic residual pattern and whether its predictions make sense in the context of the problem.
Prediction intervals from the alternative model still rely on its assumptions. A quadratic term does not guarantee the true mean is quadratic, and an interaction in one pair of predictors does not address every possible omitted feature. The residual pattern can also be hard to detect when data are sparse, especially near the edge of the observed predictor range. Conclusions should therefore reflect the available data and the scope of the model.
It is useful to distinguish three issues: random variation in new outcomes, uncertainty from estimating the model, and systematic error from a poorly specified mean. Conventional interval width accounts for the first two under its assumptions. The third can shift predictions without being represented by the usual width calculation. Consequently, a wide interval is not automatically protected against mean-model bias, and a small overall residual average is not evidence that predictions are unbiased at every predictor profile.
Common Mistake
- Wrong reasoning: “The residuals average to zero, so the fitted mean model is correct and the prediction intervals have their advertised coverage.”
- Correction: With an intercept, least-squares residuals sum to zero overall, even when they follow a strong curved or group-specific pattern. Check residuals against predictors and fitted values, consider a scientifically justified alternative mean form, and remember that the nominal interval calibration depends on an adequate model at the prediction point.
Check Your Understanding
Answer each question by connecting the model structure to the prediction being made.
- What residual pattern might suggest that a straight-line mean model has omitted curvature?
- In an additive model with \(x\) and a group indicator \(G\), what feature of the relationship does adding \(xG\) allow the model to represent?
- Why does a zero sum of least-squares residuals not establish that the fitted mean is correct at every predictor value?
- In the coverage derivation, what does \(\delta=m-\hat{m}\) represent, and why does a nonzero value matter?
- Name one reason a residual pattern alone may not be enough to establish that a particular alternative model is correct.