Why Read Residual Plots Together?
A residual plot is a check on how well a fitted regression model describes the data. No single plot diagnoses every assumption. A curved residual pattern, for example, concerns the model’s mean relationship; a widening band concerns the spread of errors. A histogram or normal Q-Q plot addresses a different feature, while a plot in observation order can reveal patterns that would be hidden if the rows were rearranged.
The goal is not to declare a model “valid” because a plot looks tidy. It is to identify patterns that would make the model’s assumptions doubtful, then consider whether the pattern is substantial, reproducible, and meaningful in context. Visual checks are especially useful for forming a diagnosis; they do not by themselves prove that an assumption holds or fails.
For a multiple linear regression, the working model is \(Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}+\varepsilon_i\). The standard assumptions considered here are that the conditional mean is represented by this linear predictor, the error variance is constant, errors are independent, and errors are normally distributed when exact finite-sample normal-theory inference is required. The first three assumptions matter for the behavior of least-squares estimates and their uncertainty; normality is particularly relevant to exact \(t\)- and \(F\)-based inference. A plot can suggest a problem, but the surrounding data-collection and modeling context also matters.
A Coordinated Reading of the Main Plots
| Plot | Pattern to look for | Defensible interpretation |
|---|---|---|
| Residuals versus fitted values | Curvature or a systematic trend in the residual center | The specified mean relationship may be inadequate |
| Residuals versus a predictor | A curve, step, or other pattern across predictor values | The predictor’s form, or an omitted relationship involving it, may be misspecified |
| Scale-location | Increasing or decreasing vertical spread | Error variance may change across fitted values |
| Histogram or normal Q-Q plot | Skew, heavy tails, or marked departures from a reference line | Normality of errors may be questionable; small samples make the visual assessment uncertain |
| Residuals in observation order | Long runs, trends, cycles, or clustering | Errors may be dependent, if the order has substantive meaning |
A useful distinction is center versus spread. A residual cloud can be centered near zero at every fitted value yet become much wider as fitted values increase. That points toward changing variance, not necessarily a wrong conditional mean. Conversely, a clear curve in the center of the cloud points toward a mean-structure concern even if the spread stays about the same.
Worked Example: Curvature Around the Fitted Values
Worked Example: Curvature Around the Fitted Values
A hypothetical insurer models annual service cost, in thousands of dollars, as a linear function of a risk score. The fitted values and observed costs for seven policies are shown below. Residuals are observed minus fitted. The working assumptions are a linear conditional mean, independent errors, constant error variance, and errors distributed as \(N(0,\sigma^2)\) for normal-theory inference. Does the residual plot support the linear mean form?
| Risk-score order | Fitted cost | Observed cost | Residual calculation | Residual |
|---|---|---|---|---|
| 1 | 10 | 12 | 12 − 10 | 2 |
| 2 | 20 | 20 | 20 − 20 | 0 |
| 3 | 30 | 29 | 29 − 30 | −1 |
| 4 | 40 | 38 | 38 − 40 | −2 |
| 5 | 50 | 49 | 49 − 50 | −1 |
| 6 | 60 | 60 | 60 − 60 | 0 |
| 7 | 70 | 72 | 72 − 70 | 2 |
Step 1: Read the residuals from left to right. They are \(2,0,-1,-2,-1,0,2\). The residuals are positive at both ends and negative through the middle. That is a U-shaped pattern, not a random-looking cloud around zero.
Step 2: Check the pattern is not just an overall offset. Their sum is \(2+0-1-2-1+0+2=0\). More importantly, their signs change systematically with fitted value: the model tends to underpredict at the ends and overpredict in the middle. A zero residual average does not rule out this structured error.
Conclusion and check. The plot suggests the linear conditional mean may be inadequate; a curved term or another justified mean structure could be considered. It does not establish which alternative is appropriate. The method is to inspect the residual center across the fitted range. The common mistake is to see an overall mean of zero and conclude that linearity is satisfactory; the ordered pattern shows why that conclusion is not justified.
Worked Example: Increasing Residual Spread
Worked Example: Increasing Residual Spread
A hypothetical model predicts monthly claim-handling expense. Its residual-versus-fitted display has eight points at fitted values 10, 20, through 80 (all in the same expense units). The residuals, in that order, are \(0.1,-0.1,0.2,-0.2,0.4,-0.4,-0.7,0.7\). For this illustrative scale-location calculation, take the residual scale \(s=1\). The working error model assumes a linear conditional mean, independent \(N(0,\sigma^2)\) errors with constant \(\sigma^2\). Is there evidence against constant variance?
Step 1: Calculate the absolute scaled residuals. Since \(s=1\), each scaled residual \(r_i=e_i/s\) equals \(e_i\). Thus the absolute scaled residuals are \(0.1,0.1,0.2,0.2,0.4,0.4,0.7,0.7\).
Step 2: Calculate the scale-location vertical coordinates. A scale-location plot commonly displays \(\sqrt{|r_i|}\). The values are \(\sqrt{0.1}=0.316\), \(\sqrt{0.1}=0.316\), \(\sqrt{0.2}=0.447\), \(\sqrt{0.2}=0.447\), \(\sqrt{0.4}=0.632\), \(\sqrt{0.4}=0.632\), \(\sqrt{0.7}=0.837\), and \(\sqrt{0.7}=0.837\), rounded to three decimals.
Conclusion and check. The vertical magnitudes generally rise across the fitted range, while the residuals themselves include both signs. The pattern suggests nonconstant variance rather than simply a residual center moving away from zero. This is visual evidence, not a formal test or proof. The method is to compare spread across fitted values, not just inspect whether positive and negative residuals both occur. The common mistake is to call any visible trend a linearity failure; here the diagnostic feature is widening spread.
Worked Example: Residuals in Observation Order
Worked Example: Residuals in Observation Order
A hypothetical analyst fits an intercept-only model to 12 consecutive daily measurements of a process. The residuals in collection order are \(-1,-1,-1,1,1,1,-1,-1,-1,1,1,1\). The working assumptions are independent errors with distribution \(N(0,\sigma^2)\) and a constant conditional mean. What does the order plot suggest?
Step 1: Check the residual balance. There are six values of \(-1\) and six of \(1\), so the residual sum is \(6(-1)+6(1)=0\). The overall residual average is \(0/12=0\), but that says nothing by itself about their order.
Step 2: Count adjacent pairs. There are 11 adjacent pairs. Within the four runs of three equal values, there are \(2+2+2+2=8\) same-sign pairs. Between runs, there are three sign changes. The sequence is therefore strongly grouped into runs rather than alternating irregularly.
Conclusion and check. Because the order is consecutive days, the long runs suggest possible serial dependence or an omitted time pattern. The plot cannot identify the cause, and a short sequence does not establish dependence conclusively. If the same observations had been placed in an arbitrary order, this interpretation would not be defensible. The common mistake is to treat an order plot as meaningful without checking whether row order represents time or another relevant sequence.
Worked Example: Checking the Shape of the Residual Distribution
Worked Example: Checking the Shape of the Residual Distribution
A hypothetical regression produces ten scaled residuals: \(-1.5,-1.2,-0.9,-0.6,-0.3,0,0.2,0.4,0.8,3.1\). Assume the intended error distribution is \(N(0,\sigma^2)\), with independent errors and constant variance. Use a simple binned summary to describe the distributional concern.
Step 1: Check the center. The sum of the five negative values is \(-1.5-1.2-0.9-0.6-0.3=-4.5\). The nonnegative values sum to \(0+0.2+0.4+0.8+3.1=4.5\). The total is zero and the mean is \(0/10=0\).
Step 2: Count values by interval. The intervals \([-2,-1)\), \([-1,0)\), \([0,1)\), \([1,2)\), and \([2,4)\) contain, respectively, \(2,3,4,0,\) and \(1\) residuals. The single value \(3.1\) creates a comparatively long right tail, despite the residual mean being zero.
Conclusion and check. The histogram would be asymmetric, so normality is questionable and a Q-Q plot could help assess the tail departure. With only ten values, the evidence is uncertain; the plot does not prove that the error distribution is nonnormal. The method is to inspect shape, not just the mean. The common mistake is to equate a zero average residual with a normal residual distribution.
Why These Patterns Arise—and What Plots Cannot Prove
The logic behind residual diagnostics is straightforward. If the conditional mean is correctly specified, the observed response can be written as its true mean plus an error, and the fitted response is an estimate of that mean. Subtracting the fitted response therefore leaves a quantity that should fluctuate around zero rather than trace a systematic curve. If the mean function is wrong, its omitted shape can remain visible in the residual center.
When the fitted mean \(\hat{Y}_i\) tracks the true conditional mean \(m(\mathbf{x}_i)\), the second term is small, so residuals mainly reflect error variation. If error variance is constant, that variation should be broadly similar across fitted values; systematic widening or narrowing is a warning. If errors are dependent over time, nearby errors can tend to share signs or follow cycles, making order plots useful.
Residuals are not identical to the unobserved errors: fitting the model imposes constraints on them. Thus one should not expect every plot to look perfectly random, especially in a small sample. A histogram’s irregularity may be sampling variation, and a pattern in an order plot may be caused by a changing process rather than dependence alone. Use substantive knowledge and, where appropriate, the formal diagnostic methods covered in the earlier material on regression diagnostics.
Normality deserves a separate qualification. It is the errors conditional on the predictors that are assumed normal for exact normal-theory inference, not the predictors themselves. A histogram or Q-Q plot of residuals is an indirect check. Strong skewness or tail departures can be important, but a visually imperfect plot does not automatically invalidate every regression estimate or prediction.
Look for structure in residuals versus fitted values and relevant predictors. Curvature or a changing center suggests a mean-model concern.
Use residual magnitude and a scale-location display to assess whether the vertical spread changes across fitted values.
Use a histogram or normal Q-Q plot to look for skew, heavy tails, and unusual departures, while accounting for sample size.
Inspect residuals in time or collection order when observations have a sequence that could make dependence relevant.
Describe what the plot suggests, name the assumption at issue, and avoid claiming that a visual check proves the assumption.
Common Interpretation Traps
- Reversing the residual sign. If the fitted value is 10 and the observed value is 12, the residual is \(12-10=2\), not \(-2\). Reversing the convention flips above-versus-below interpretations, even though absolute magnitudes are unchanged.
- Calling every pattern “nonlinearity.” A curve in the residual center concerns the mean form; a widening band concerns spread. Identify which feature changes.
- Reading independence from an arbitrary row order. Order plots are informative only when the order has meaning, such as time or sequence of collection.
- Treating normality as a visual pass/fail test. Small samples produce unstable histograms and Q-Q plots. Report the observed departure and its limitations.
- Confusing “no obvious pattern” with proof. A plot may lack power to reveal subtle departures. “No clear visual evidence” is more defensible than “the assumption is true.”
Check Your Understanding
For each situation, identify the most relevant diagnostic conclusion. Answers are included for checking.
- A residual-versus-fitted plot has a level center near zero, but its vertical spread grows from about 0.5 to about 3 units as fitted values increase. Which assumption is most directly in question? (Answer: Constant error variance; the pattern suggests increasing variance.)
- For an observation with fitted response 18 and observed response 15, calculate the residual and state whether the response lies above or below its fit. (Answer: \(15-18=-3\); it lies 3 units below the fitted value.)
- A residual plot against one predictor shows a smooth inverted U, while the overall residual average is zero. What is the defensible conclusion? (Answer: The mean relationship involving that predictor may be misspecified; a zero overall average does not remove the visible curvature.)
- Residuals from hourly observations alternate positive and negative for 20 hours. What should be checked before interpreting that pattern as evidence about independence? (Answer: Confirm that the row order is the true hourly order; if so, the regular alternation may indicate dependence or an omitted time pattern.)
- A residual histogram from eight observations has one extreme positive value, but the residual-versus-fitted plot has no obvious curve or widening. What can be concluded about normality? (Answer: The histogram raises a possible tail or outlier concern, but eight observations are too few for a definitive visual conclusion.)