Why Plot Residuals?
A regression model can describe the average outcome well while the variability around that average changes across observations. For example, prediction errors might be relatively small for low fitted values and much larger for high fitted values. This is conditional heteroskedasticity: the conditional variance of the error depends on the regressors. Residual plots help reveal such patterns and can show whether a concern is instead about the model’s conditional mean or a small number of unusual observations.
These displays are diagnostic, not decisive tests. A widening cloud of residuals is evidence to investigate, not proof of a particular variance model. Conversely, a plot with no obvious pattern cannot establish that the errors have constant variance.
Under the conditional-mean condition, the regression describes the conditional average outcome. A residual pattern that shifts systematically above and below zero can therefore signal that the mean relationship is not adequately represented. A variance pattern is different: the residuals remain roughly centered around zero, but their vertical spread changes with a predictor or fitted value. In practice, the two can occur together, and a mean pattern can make the apparent spread difficult to interpret.
Reading Residual-versus-Fitted and Residual-versus-Predictor Plots
A residual-versus-fitted plot puts \(\hat{y}_i\) on the horizontal axis and \(e_i\) on the vertical axis. It summarizes whether residual location or spread changes across the model’s predicted range. A residual-versus-predictor plot instead puts a selected regressor on the horizontal axis. This can make a relationship easier to see when a particular predictor is a plausible source of changing variance or mean structure.
A roughly horizontal band centered near zero, with similar vertical spread across the horizontal axis, is compatible with constant conditional variance. It does not prove that condition. A fan or funnel, in which the band becomes wider or narrower, suggests changing variance. Clusters with visibly different spreads may also be informative. A curved or otherwise systematic shift in the residual center points toward a possible conditional-mean problem. The plot’s main question is therefore not just “Does the cloud get wider?” but also “Does it stay centered around zero?”
Worked Example: Increasing Residual Spread Across Fitted Values
Suppose a hypothetical OLS fit uses an intercept and a single predictor \(x\). The following observations and fitted values are obtained:
| \(x_i\) | \(y_i\) | \(\hat{y}_i\) | \(e_i=y_i-\hat{y}_i\) |
|---|---|---|---|
| 1 | 9 | 10 | -1 |
| 1 | 11 | 10 | 1 |
| 2 | 18 | 20 | -2 |
| 2 | 22 | 20 | 2 |
| 3 | 27 | 30 | -3 |
| 3 | 33 | 30 | 3 |
The fitted line is \(\hat{y}_i=10x_i\). The residuals sum to zero, and their products with the centered predictor values also sum to zero: the two residuals at each \(x\) level cancel. These are the OLS residual conditions for an intercept and this predictor, so the table is consistent with the stated fit.
Group the residuals by fitted value. At \(\hat{y}=10\), their root mean square is \( \sqrt{((-1)^2+1^2)/2}=\sqrt{1}=1 \). At \(\hat{y}=20\), it is \( \sqrt{((-2)^2+2^2)/2}=\sqrt{4}=2 \). At \(\hat{y}=30\), it is \( \sqrt{((-3)^2+3^2)/2}=\sqrt{9}=3 \). The mean residual in each group is zero, while the root mean square residual increases from 1 to 3 outcome units. A residual-versus-fitted plot would show a centered band that widens as fitted values increase. This is a clear pattern to investigate as possible changing variance; this small hypothetical dataset alone does not establish the true conditional variance.
Scale-Location Displays and Groupwise Summaries
A scale-location display is designed to make changes in residual magnitude easier to see. A common version plots the square root of the absolute internally standardized residual against the fitted value. An internally standardized residual divides \(e_i\) by an estimate of its residual standard deviation that accounts for its leverage. Software conventions can differ, so the axis definition should be checked before interpreting the numerical scale. The diagnostic idea is consistent across versions: a roughly level band suggests no obvious change in residual scale, while a systematic rise or fall suggests that the scale changes with fitted values.
Groupwise summaries provide a compact numerical complement to the plot. For observations divided into ranges of a predictor or fitted value, useful summaries include the mean residual, mean absolute residual, and root mean square residual. The mean residual helps identify a shift in location; the latter two summarize magnitude. For a group \(G\) containing \(n_G\) observations, the root mean square residual is \( \sqrt{\sum_{i\in G}e_i^2/n_G} \). It measures residual magnitude in the outcome’s units, but it is not automatically an unbiased estimate of the conditional error standard deviation: fitted residuals are affected by estimating the regression parameters.
Worked Example: Separating a Location Pattern From a Scale Pattern
Consider a hypothetical regression with an intercept and predictor values \(x=-2,-1,0,1,2\). Suppose the fitted line is \(\hat{y}=10+2x\), with observed values and residuals as follows:
| \(x_i\) | \(\hat{y}_i\) | \(y_i\) | \(e_i\) |
|---|---|---|---|
| -2 | 6 | 8 | 2 |
| -1 | 8 | 7 | -1 |
| 0 | 10 | 8 | -2 |
| 1 | 12 | 11 | -1 |
| 2 | 14 | 16 | 2 |
The residuals sum to \(2-1-2-1+2=0\). Their products with \(x\) sum to \((-2)(2)+(-1)(-1)+0(-2)+1(-1)+2(2)=0\). Thus the values satisfy the OLS residual conditions for the stated line.
The residual-versus-predictor plot has positive residuals at both ends and negative residuals in the middle. Its pattern is U-shaped rather than a centered cloud whose spread simply increases or decreases. The mean residual across the two end observations is \((2+2)/2=2\); across the three middle observations it is \((-1-2-1)/3=-1.33\). These group means show a shift in residual location. The mean absolute residuals are 2 at the ends and \(4/3=1.33\) in the middle, but with so few observations this difference is not enough to distinguish changing variance from the visible location pattern. A reasonable diagnosis is that the linear conditional mean may be inadequate; the plot does not by itself establish a variance pattern.
Distinguishing Variance Patterns From Influential Observations
One unusual observation can make a residual plot look as if its spread is changing. This is especially important when an observation has high leverage: its predictor values are unusual relative to the rest of the design, so it can have substantial influence on the fitted regression. Residual size alone does not determine influence. A large residual can occur at an ordinary predictor value, while a high-leverage observation can affect the fitted line and leave a residual that does not look especially large.
When a plot’s apparent pattern seems to depend on one point, inspect that observation’s predictor values, residual, and leverage or influence diagnostics. It is also useful to see whether the overall pattern remains when that point is examined separately from the rest. Removing an observation solely to make a diagnostic plot look more regular is not justified: exclusion requires a defensible reason, and the analysis should make that decision transparent. A pattern that persists across many observations is more consistent with a general variance feature than one driven by a single influential case, although this distinction is not definitive.
Worked Example: A Single Influential Point Can Dominate a Plot
Suppose a hypothetical dataset has \(x=1,2,3,4,5\) and outcomes \(y=2,3,4,5,20\). For the OLS regression with an intercept, \(\bar{x}=3\), \(\bar{y}=6.8\), and \(S_{xx}=\sum_i(x_i-\bar{x})^2=10\). The slope is
The intercept is \(\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=6.8-(3.8)(3)=-4.6\). The fitted values are \(-0.8,3.0,6.8,10.6,14.4\), and the residuals are \(2.8,0,-2.8,-5.6,5.6\). Thus the last observation has a residual of 5.6 outcome units.
Its leverage in this intercept-and-one-predictor regression is \( h_{55}=1/5+(5-3)^2/10=0.6 \), larger than the leverage \(0.2\) for the observation at \(x=3\). The point is also influential: if it is omitted, the four remaining observations lie exactly on \(y=1+x\), whose slope is 1 rather than 3.8. The residual plot from the full fit can therefore reflect the way this one unusual, high-leverage observation changes the fitted line, not a general pattern of changing variance. This comparison is a reason to examine influence alongside the plot, not a reason to discard the point automatically.
A Practical Diagnostic Sequence
A useful sequence keeps the visual evidence tied to the model being assessed. It does not require a formal variance test, and it does not turn a plot into a decision rule.
Plot residuals against fitted values and relevant predictors. Look for curvature, clusters, or systematic movement above and below zero, which may indicate conditional-mean misspecification.
Look for a fan, funnel, or change in vertical spread. Use a scale-location display and groupwise mean absolute or root mean square residuals to clarify a pattern that is hard to see.
Inspect unusual predictor values and leverage or influence diagnostics. Ask whether the apparent variance pattern is broadly present or depends mainly on one or two observations.
Describe the observed location or spread pattern and its context. Treat it as evidence for further assessment, not as proof that the conditional variance has a particular form.
Why the Distinction Matters
Variance misspecification matters for inference because conventional OLS standard errors rely on assumptions about the error covariance, including constant conditional variance in the usual homoskedastic formulation. Earlier material on heteroskedasticity and OLS sampling variance explains why nonconstant variance can make those conventional standard errors unreliable, and why heteroskedasticity-robust covariance estimators are used under suitable conditions. Residual plots contribute by showing where a variance concern might arise; they do not determine which inference method is valid or replace careful modeling.
A common mistake is to say, “The residuals get larger as fitted values increase, so the model definitely has heteroskedasticity.” The correction is to describe what the plot shows: residual spread appears to increase over that range. Then check whether the residuals remain centered, whether a mean pattern is present, and whether a few influential observations drive the result. Visual diagnostics are most informative when these possibilities are considered together.
Check Your Understanding
Answer each question using the diagnostic distinctions developed above.
- In a residual-versus-fitted plot, the residuals remain centered near zero but the vertical spread increases. What does this pattern suggest, and what does it not prove?
- Why can a curved residual pattern point to a conditional-mean problem rather than a variance problem?
- A group contains residuals \(-2,2,-4,4\). Calculate its mean residual and root mean square residual, and state what each summary describes.
- How can a high-leverage observation affect a residual plot even if the plot is being used to assess variance?
- Why should residual plots be interpreted alongside, rather than instead of, checks for influential observations?