Why Check Residuals Before Predicting?
A prediction interval is intended to cover a future individual outcome, not just the mean outcome at a predictor setting. Its coverage therefore depends on how future errors behave. A plot that reveals changing error spread, asymmetry, or unusually heavy tails can warn that the interval calculation may not achieve its intended coverage.
Residual checks are diagnostic: they show whether features of the observed data are consistent with the assumptions behind the interval. They do not prove those assumptions, nor can a plot guarantee that future errors will follow the same pattern as past errors.
For observed outcome \(y_i\), the fitted value is \(\hat{y}_i\), and the residual is \(e_i=y_i-\hat{y}_i\). A residual is the observed outcome minus the fitted mean. It is not the unobserved error itself, but residual patterns can provide evidence about whether the error assumptions are plausible.
Residuals Versus Fitted Values
A residual-versus-fitted plot places \(\hat{y}_i\) on the horizontal axis and \(e_i\) on the vertical axis. If the mean model and common-variance assumption are reasonably compatible with the data, the points should fluctuate around zero with roughly similar vertical spread across the range of fitted values. A funnel shape—narrow at one end and wide at the other—suggests that error variability may change with the fitted mean.
A band centered away from zero or a systematic curve can signal a pattern in the residuals that the fitted model has not accounted for. For prediction intervals, changing spread is especially important: a single common error scale may be too large in some parts of the fitted range and too small in others.
Worked Example: A Funnel in the Residual-versus-Fitted Plot
Suppose a hypothetical regression predicts an outcome measured in minutes. Five cases have fitted values from 10 to 14 minutes, with residuals \(-1, 0, 1, -1, 1\) minutes. Five other cases have fitted values from 20 to 24 minutes, with residuals \(-4, -2, 0, 2, 4\) minutes. Compare the residual spread in these two fitted-value groups.
For the first group, the mean squared residual is
Its root mean squared residual is \(\sqrt{0.8000}=0.8944\) minutes. For the second group,
Both groups have residuals that sum to zero, but the second group has much greater spread. On a residual-versus-fitted plot, this would look like a widening band. It raises concern that a common error variance may not describe the full fitted range. These five-point summaries are descriptive, not formal estimates of the true conditional variances.
The residual spread in a plot is not a perfect direct view of error spread. In ordinary least squares, residuals are affected by the fitting process; in particular, cases with high leverage can have less variable residuals even when the underlying errors have constant variance. A visible funnel is a warning to investigate, while a visually uniform band is not proof of constant variance.
Residuals Versus a Predictor
A residual-versus-predictor plot places residuals against one predictor variable at a time. It can help show where a pattern occurs in terms of an input that matters for prediction. For example, if the residual spread grows as a patient's age increases, plotting residuals against age may make that trend clearer than plotting against fitted values.
In multiple regression, examine the relationship with each predictor that is important to the prediction problem. A plot against one predictor does not isolate its effect from all the others, and absence of a pattern in that plot does not rule out a pattern involving several predictors together. As with any diagnostic plot, the question is whether there is a visible, interpretable structure that calls the model assumptions into question.
Worked Example: Spread Changes Across a Predictor
In a hypothetical multiple-regression analysis, consider residuals in minutes plotted against one predictor, hours of weekly activity, while the other predictors remain part of the fitted model. At 2 hours, the observed residuals are \(-0.5, 0.5\); at 6 hours, they are \(-1, 1\); and at 10 hours, they are \(-2, 2\). Compare the root mean squared residual at each setting.
At 2 hours, the mean squared residual is \((0.5^2+0.5^2)/2=0.2500\) minutes squared, so the root mean squared residual is \(0.5000\) minutes. At 6 hours, it is \((1^2+1^2)/2=1.000\) minutes squared, giving \(1.000\) minute. At 10 hours, it is \((2^2+2^2)/2=4.000\) minutes squared, giving \(2.000\) minutes.
The residuals are centered at zero at all three predictor values, but their spread increases as activity hours increase. The plot would suggest that error variability may depend on this predictor. Because these are residuals from a fitted model, the pattern is a diagnostic clue, not proof that the population error variance has exactly this form.
In simple regression, the fitted value is a linear function of the predictor, so residual-versus-fitted and residual-versus-predictor plots show the same horizontal ordering, apart from a change of scale and direction. In multiple regression, fitted values combine predictors, while a predictor plot shows one predictor at a time; the two views can therefore highlight different features.
Normal Quantile Plots: Shape and Tails
A normal quantile plot, also called a normal Q-Q plot, compares ordered residuals with the quantiles expected from a normal distribution. The horizontal axis gives theoretical normal quantiles, and the vertical axis gives ordered residuals, often standardized so their scale is comparable. If errors are approximately normal, the points should lie roughly along a straight line. Systematic bending or unusual departures at the ends suggest that the distribution may be skewed or may have tails unlike a normal distribution.
A one-sided departure, such as an unusually high point at the upper end, can indicate an unusually long upper tail or a potential outlier. If both endpoints depart from the line in opposite directions, the tails may be heavier than normal; if both endpoints bend inward, the tails may be lighter. These are visual descriptions rather than automatic classifications: sample size, isolated observations, and the particular plotting convention all affect the appearance.
Worked Example: A High Residual in a Normal Quantile Plot
Suppose diagnostic software reports eight ordered standardized residuals for a hypothetical regression: \(-2.0, -1.4, -1.0, -0.7, -0.3, 0.0, 0.4, 3.2\). For eight observations, approximate theoretical standard-normal quantiles at the plotting positions are \(-1.53, -0.89, -0.49, -0.16, 0.16, 0.49, 0.89, 1.53\). Compare the two ends of the plot.
At the lower end, the observed value \(-2.0\) is below its theoretical quantile of \(-1.53\), a difference of \(-0.47\). At the upper end, the observed value \(3.2\) is above its theoretical quantile of \(1.53\), a difference of \(1.67\). The remaining points should be assessed as a pattern, not individually against an exact equality; here, the most conspicuous departure is the high upper-tail value.
The plot gives reason to investigate whether the upper tail is heavier than normal or whether one observation is unusual. It does not establish which explanation is correct, and a sample of eight residuals is too small for a visual check to be decisive. A normal quantile plot assesses distributional shape; it does not check whether residuals are independent or whether their variance is constant.
Why Changing Spread and Tail Shape Matter
The connection between error spread and coverage can be seen by holding an interval's center and half-width fixed. Suppose, just to isolate the error contribution, that the future error at a predictor setting is normal with standard deviation \(\sigma(x)\), and the interval extends \(c s\) on either side of its center. Then its error-only coverage at that setting is
where \(\Phi\) is the standard normal cumulative distribution function. If \(\sigma(x)\) is larger than the scale \(s\) used to construct the interval, then \(cs/\sigma(x)\) is smaller, and the error-only coverage is lower. If \(\sigma(x)\) is smaller, that coverage is higher. The full prediction interval also accounts for uncertainty in the fitted mean, but this simplified calculation shows why a changing error spread can make one common scale misleading.
Tail shape matters as well. Even when the error variance is constant, a distribution with heavier tails than the assumed normal distribution can produce more extreme future errors than the normal model predicts. Consequently, a normal-based interval may miss more often than its nominal level suggests. A normal quantile plot can flag such a concern, but it cannot calculate the actual future coverage from the observed sample alone.
How to Use the Checks
Read the plots together rather than treating any one as a pass-or-fail test. A residual-versus-fitted plot gives an overview of spread across the fitted range. A residual-versus-predictor plot can locate a spread pattern in a specific input. A normal quantile plot focuses on whether residual distribution shape is compatible with normal errors, especially in the tails.
A sensible check is to ask what a visible pattern means for the new predictions of interest. If the intended predictions are concentrated in a region where residuals appear much more variable, a single interval width deserves particular caution. If the plot shows a strong upper-tail departure, an interval based on normal errors may not adequately represent the chance of unusually high outcomes. These checks do not by themselves select or construct an alternative interval.
Look for a band centered near zero and for roughly even vertical spread. Note any funnel or other systematic pattern.
Look for residual spread that changes across predictor values, remembering that each plot shows only one predictor at a time.
Look for systematic curvature, asymmetry, or endpoint departures that suggest skewness or unusual tails.
Describe the pattern as evidence about the fitted data, not proof of the future error distribution or of a particular coverage rate.
Common Mistake
Wrong reasoning: “The residuals are roughly centered at zero, so the prediction-interval assumptions are satisfied.” Centering alone says little about whether the spread is constant, whether the errors have normal tails, or whether observations are independent. Residuals can average near zero while forming a funnel or showing a large tail departure.
Correction: Check the vertical spread and distributional shape as well as the center. Then describe what the plots suggest and what they cannot establish. A plot without an obvious problem is reassuring, but it is not a guarantee of nominal coverage for future outcomes.
Check Your Understanding
Answer each question using the assumptions and diagnostic ideas in this tutorial.
- What pattern in a residual-versus-fitted plot would suggest that error variance changes across the fitted range?
- Why might a residual-versus-predictor plot be useful even when a residual-versus-fitted plot is available?
- In a normal quantile plot, what would an unusually high upper endpoint suggest, and why is that not conclusive proof of a heavy upper tail?
- Can residuals centered near zero establish that a normal-theory prediction interval has its nominal coverage? Explain.
- If the future error standard deviation is larger in one predictor region than the scale used to form an interval, what happens to error-only coverage for a fixed interval half-width?