Why Use a Fixed Diagnostic Sequence?
A fitted regression line can summarize a relationship well on average and still miss important features of the data. Residual diagnostics help assess whether the pattern left after fitting is consistent with a useful simple linear regression model. A fixed sequence makes the review more systematic: first look for a wrong mean pattern or changing spread, then assess normality, and finally examine the order in which observations were collected.
Suppose the working model is \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), for \(i=1,\ldots,n\). The simple linear regression assumptions used for inference are that the predictor values are treated as fixed (or inference is conditional on them), the errors have mean zero, the errors are independent, their common variance is \(\sigma^2\), and—when exact small-sample normal-theory inference is used—the errors are normally distributed. Write the fitted values as \(\hat{y}_i=b_0+b_1x_i\) and the residuals as \(e_i=y_i-\hat{y}_i\). A residual is an estimated error, not the unobserved error itself.
In each residual plot, look for a roughly horizontal band centered at zero. A smooth curve, funnel, isolated point, or sequence of runs is a reason to investigate—not, by itself, proof of a particular cause. Diagnostic plots are most useful when the plot is described precisely and the conclusion is kept proportional to the evidence.
Step 1: Residuals Versus Fitted Values
Plot \(e_i\) vertically against \(\hat{y}_i\) horizontally. A roughly patternless cloud centered near zero is compatible with a linear mean and constant error variance across the range of fitted responses. Curvature suggests that the fitted straight-line mean may be inadequate. A spread that grows or shrinks across fitted values suggests nonconstant variance. A single residual far from the rest is unusual and merits a closer look.
The fitted values summarize the model’s predicted response levels. This plot therefore asks whether residual behavior changes as the response predicted by the fitted line changes. It does not establish that the model assumptions hold: for example, independence is not determined by the horizontal coordinate.
Step 2: Residuals Versus the Predictor
Next plot \(e_i\) against \(x_i\), retaining the original predictor values. This makes it easier to see whether residual behavior changes across the predictor’s range. Curvature can indicate that a straight line does not capture the mean response as \(x\) changes. A widening vertical spread across \(x\) can indicate nonconstant variance. Gaps in the observed predictor values may also be easier to see here than in the fitted-value plot.
For simple regression, \(\hat{y}_i=b_0+b_1x_i\). If \(b_1\ne 0\), fitted values are a linear transformation of predictor values, so the two plots have the same horizontal ordering; the direction is reversed when \(b_1<0\). However, they are not interchangeable in every case. If \(b_1=0\), all fitted values are identical. The residual-versus-fitted plot then collapses every point to one horizontal coordinate, while the residual-versus-predictor plot can still show variation across predictor values.
Worked Example: Computing Residuals and Comparing the First Two Plots
A hypothetical analyst relates a response \(y\) to a predictor \(x\) for five observations. The data are \((1,2),(2,4),(3,5),(4,7),(5,12)\). The working model is simple linear regression with independent \(N(0,\sigma^2)\) errors, constant variance, and fitted mean \(\beta_0+\beta_1x\). No interest-rate or mortality model applies to this statistical example. Fit the line, compute the residuals, and describe what the first two diagnostic plots show.
Method: ordinary least squares. First calculate the means:
The predictor sum of squares is
The cross-product sum is
Thus \(b_1=S_{xy}/S_{xx}=23/10=2.3\), and \(b_0=\bar{y}-b_1\bar{x}=6-(2.3)(3)=-0.9\). The fitted line is \(\hat{y}=-0.9+2.3x\).
| \(x_i\) | \(y_i\) | \(\hat{y}_i=-0.9+2.3x_i\) | \(e_i=y_i-\hat{y}_i\) |
|---|---|---|---|
| 1 | 2 | 1.4 | 0.6 |
| 2 | 4 | 3.7 | 0.3 |
| 3 | 5 | 6.0 | -1.0 |
| 4 | 7 | 8.3 | -1.3 |
| 5 | 12 | 10.6 | 1.4 |
The residuals sum to \(0.6+0.3-1.0-1.3+1.4=0\), as expected for a fitted line with an intercept. On the residual-versus-fitted plot, the residuals are positive at the first two fitted values, negative in the middle, and positive at the largest fitted value. That sequence suggests possible curvature, although five points provide little evidence. Because \(b_1=2.3\ne0\), the predictor and fitted values have the same ordering; the residual-versus-predictor plot shows the same up-down-up sequence against \(x\). The diagnostic concern is a possible non-linear mean pattern, not a demonstrated violation. The common mistake avoided is to call this sequence proof of curvature from such a small sample.
Step 3: Residual Normal Q-Q Plot
A normal Q-Q plot compares ordered residuals with the corresponding quantiles of a normal distribution. If the errors are approximately normal, the plotted points should fall roughly along a straight line, allowing for ordinary sampling variation. Strong bending can indicate skewness; an S-shaped departure can indicate tail behavior that differs from normality; and one or two points far from the overall line can suggest unusually extreme residuals.
The residuals are estimated from the same fitted model, so they are not independent observations from the error distribution. A Q-Q plot is consequently a visual diagnostic, not a formal proof of normality. With a small sample, the plot has limited ability to distinguish a real departure from random variation. A point’s distance from the Q-Q line also should not be treated as an automatic rule for deleting that observation.
For a concrete scale comparison, let \(s=\sqrt{\sum e_i^2/(n-2)}\) be the residual standard error in simple regression. In Worked Example 1, \(\sum e_i^2=0.36+0.09+1+1.69+1.96=5.10\), so \(s=\sqrt{5.10/3}=\sqrt{1.70}\), approximately \(1.3038\). Dividing each residual by \(s\) gives approximately \(0.460, 0.230, -0.767, -0.997, 1.074\); ordered, these are \(-0.997,-0.767,0.230,0.460,1.074\). A Q-Q plot would compare that ordered sequence with increasing normal quantiles. Such a calculation sets a scale; it does not replace inspection of the plot or make a five-point normality assessment decisive.
Step 4: Residuals Versus Observation Order
Plot residuals vertically against the order in which the observations were collected. This plot is meaningful only if that order has a substantive interpretation, such as time, production sequence, or successive measurements. A trend, long runs of residuals with the same sign, or a repeating cycle can signal dependence or an omitted order-related feature. Random-looking scatter is reassuring but does not prove independence.
Do not rearrange observations by predictor value before making an order plot if the purpose is to inspect collection order. Doing so changes the question being asked. Also distinguish the order plot from the residual-versus-predictor plot: the first uses collection sequence; the second uses the predictor’s numerical value. They can look quite different when the predictor values were recorded in a different order.
Worked Example: A Zero Slope and a Collapsed Fitted-Value Plot
A hypothetical study records \((x,y)=(1,5),(2,7),(3,7),(4,5)\). Use a simple linear regression model with independent \(N(0,\sigma^2)\) errors as the working model, and compute the fitted line and residuals. No interest-rate or mortality assumption is relevant. Explain why the predictor plot retains information that the fitted-value plot loses.
Method: calculate the least-squares slope and intercept. The means are \(\bar{x}=(1+2+3+4)/4=2.5\) and \(\bar{y}=(5+7+7+5)/4=6\). The centered products for \(S_{xy}\) are
Also, \(S_{xx}=(-1.5)^2+(-0.5)^2+(0.5)^2+(1.5)^2=2.25+0.25+0.25+2.25=5\). Therefore \(b_1=S_{xy}/S_{xx}=0/5=0\), and \(b_0=\bar{y}-b_1\bar{x}=6\). Each fitted value is \(6\), and the residuals are \(5-6=-1,\ 7-6=1,\ 7-6=1,\ 5-6=-1\). They sum to \(-1+1+1-1=0\).
In the residual-versus-fitted plot, all four points have horizontal coordinate \(6\); it cannot display how residuals change across \(x=1,2,3,4\). The residual-versus-predictor plot retains that information: its residuals are \(-1,1,1,-1\) across the four predictor values, a non-flat pattern that merits investigation. The common mistake is to claim that the two plots always preserve the same horizontal information. They do so only when \(b_1\ne0\); here the fitted-value plot collapses, while the predictor plot does not.
Putting the Sequence Together
The four plots answer related but distinct questions. Review them in order, recording both the observed pattern and the concern it raises. Do not jump directly from a visual pattern to a model change: first check the data, collection process, and context. A suspicious point may reflect an error, an unusual but valid observation, or an inadequacy in the fitted model; the plot alone cannot decide which.
Look for curvature, changing spread, clusters, and isolated residuals across predicted response levels.
Check the mean pattern and spread across the predictor range. Remember that a zero fitted slope collapses the first plot but not necessarily this one.
Assess whether ordered residuals follow a roughly straight-line pattern, with particular care about tail departures and small sample size.
Look for trends, runs, and cycles in the actual collection sequence that could raise concern about independence.
The order of these steps helps keep the interpretations separate. For instance, curvature in the first two plots concerns the conditional mean, while bending in the Q-Q plot concerns the error distribution. A time-ordered run raises an independence concern; it should not be relabeled as non-normality merely because the residuals are not randomly interspersed in time.
Worked Example: Completing and Reporting the Workflow
A hypothetical response study has predictor values \(x=1,2,3,4,5,6\), with fitted line \(\hat{y}=5+0.5x\). Its residuals in increasing predictor order are \(1,-2,1,1,-2,1\). The observations were collected in the order of predictor values \(1,3,4,2,5,6\), so the chronological residual sequence is \(1,1,1,-2,-2,1\). Treat independent \(N(0,\sigma^2)\) errors as the working model; no interest-rate or mortality assumptions apply. Calculate the residual standard error and give a concise diagnostic summary, given that the first two plots show no clear funnel but a possible mean-pattern concern, the Q-Q plot has two low-tail points, and the order plot has runs of like signs.
Method: verify the fitted residuals, calculate their scale, and interpret each plot by its role. The residual sum is \(1-2+1+1-2+1=0\). Their product sum with \(x\) is
These checks are consistent with least-squares residuals for a line with an intercept. The residual sum of squares is
With \(n=6\) observations and two fitted coefficients, the residual standard error is \(s=\sqrt{\mathrm{SSE}/(n-2)}=\sqrt{12/4}=\sqrt{3}\), approximately \(1.7321\). Dividing by this value gives residuals \(0.5774,-1.1547,0.5774,0.5774,-1.1547,0.5774\). The two most negative residuals therefore form the low tail in the Q-Q plot, consistent with the stated visual concern, but six observations cannot establish non-normality.
The residual-versus-fitted and residual-versus-predictor plots have no clear funnel, so these plots do not show an obvious changing-spread pattern. Their possible mean-pattern concern should be reported for further review. In the actual collection order, the sequence is \(1,1,1,-2,-2,1\): three positive residuals occur consecutively, followed by two negative residuals. This is a run worth checking for an order-related feature; it is not proof of dependence. A measured summary is: “The plots show no clear funnel; a possible mean-pattern concern, two low-tail Q-Q points, and runs in the observation-order plot remain. The small sample limits conclusions, and the order pattern warrants checking against the data-collection process.” The common mistake avoided is to report a visual warning as a confirmed assumption failure.
Common Mistakes and How to Avoid Them
- Treating the first two plots as always equivalent. They retain the same horizontal ordering only when \(b_1\ne0\). If \(b_1=0\), every fitted value is the same, although predictor values can vary.
- Calling any curved-looking pattern conclusive. A small sample can produce irregular patterns by chance. Describe what is visible and acknowledge the limited evidence.
- Using observation order as a synonym for predictor order. Preserve the sequence in which the observations were collected when assessing possible order effects.
- Using a Q-Q plot as a test that proves normality. The plot can reveal departures, but it cannot establish that errors are normal, especially with few observations.
- Assuming an acceptable plot proves every regression assumption. Residual plots can reveal warning signs; they do not verify the data-collection design or prove independence.
Check Your Understanding
For each problem, apply the relevant part of the workflow and keep the conclusion proportionate to the evidence.
- A fitted simple regression has \(b_1=-1.2\). Do the fitted values preserve the predictor’s horizontal ordering in the residual-versus-fitted plot? (Answer: Yes; the ordering is reversed because \(b_1<0\).)
- A regression has \(b_1=0\), and the predictor takes four distinct values. Which plot can still show residual behavior across those predictor values: residuals versus fitted values or residuals versus predictor? (Answer: Residuals versus predictor; all fitted values are identical.)
- Ordered residuals in a normal Q-Q plot bend away from a straight line at both ends, while the middle points are approximately linear. What concern does this suggest, and what should not be concluded? (Answer: Possible tail behavior differing from normality; the plot does not prove non-normality.)
- Residuals versus fitted values show a widening vertical spread, while residuals versus predictor show a similar widening pattern. What is the main concern, and what does the pattern not establish? (Answer: Possible nonconstant error variance; it does not by itself prove the cause or establish other assumptions.)
- Measurements were collected in time order, and the order plot has six consecutive positive residuals followed by six consecutive negative residuals. What is the appropriate diagnostic concern and next step? (Answer: A possible order-related dependence or time-related feature; investigate the collection process and context rather than treating the run as proof.)