Why the Pattern Matters
A residual plot can contain one point far from the horizontal zero line, or it can show a pattern repeated across several observations. These are different kinds of evidence. One unusually large residual flags a particular observation for attention; a repeated arrangement of residuals may suggest that the model has missed a feature shared by a group of observations or by a range of predictor values.
Neither appearance is a verdict. An isolated residual does not automatically invalidate a regression model, and a visible pattern does not identify its cause by itself. The useful question is what the residuals do across observations: is one point exceptional, or do their sizes, signs, or locations recur in an organized way?
Residuals and the Model Being Checked
For a simple linear regression, write the model as \(Y_i=\beta_0+\beta_1x_i+\epsilon_i\). Here \(Y_i\) is the response for observation \(i\), \(x_i\) is its predictor value, and \(\epsilon_i\) is the unobserved error. The fitted line gives a predicted response \(\hat y_i=\hat\beta_0+\hat\beta_1x_i\). The residual is the observed response minus that fitted response.
Residual diagnostics are interpreted relative to a model and its assumptions. For the usual linear regression model, the conditional mean of the response is represented by the specified regression equation, errors have mean zero at each predictor setting, their variance is constant, and errors from distinct observations are independent. Normal errors are additionally used for exact small-sample regression inference. A residual plot can provide clues about some of these assumptions, but it cannot establish them with certainty.
An isolated unusually large residual is one whose distance from zero is conspicuous compared with the residuals for the other observations, in the response’s units and on the chosen plot scale. There is no universal raw-residual cutoff that defines “unusually large” in every data set. The response scale, the overall residual variability, the predictor settings, and the scientific context all matter.
A systematic pattern is a repeated arrangement rather than a one-point exception. Examples include residuals that trace a curve across a continuous predictor, a consistent difference between categories, or long runs of positive residuals followed by negative ones across observation order. The earlier material on residual plots considers these specific shapes; here the focus is on how to distinguish recurring structure from an isolated discrepancy.
Worked Examples
Worked Example: One Large Residual Among Otherwise Small Ones
Suppose a hypothetical linear model predicts weekly demand \(y\) in units from an advertising measure \(x\). Its fitted line is \(\hat y=20+2x\). For eleven observations, the predictor values are the integers from \(-5\) to \(5\). The observed values are shown below, along with the fitted values and residuals.
| \(x\) | \(\hat y=20+2x\) | Observed \(y\) | \(e=y-\hat y\) |
|---|---|---|---|
| -5 | 10 | 9.8 | -0.2 |
| -4 | 12 | 11.8 | -0.2 |
| -3 | 14 | 13.8 | -0.2 |
| -2 | 16 | 15.8 | -0.2 |
| -1 | 18 | 17.8 | -0.2 |
| 0 | 20 | 22.0 | 2.0 |
| 1 | 22 | 21.8 | -0.2 |
| 2 | 24 | 23.8 | -0.2 |
| 3 | 26 | 25.8 | -0.2 |
| 4 | 28 | 27.8 | -0.2 |
| 5 | 30 | 29.8 | -0.2 |
For instance, at \(x=0\), the fitted value is \(20+2(0)=20\), so the residual is \(22.0-20=2.0\) demand units. At \(x=3\), the fitted value is \(20+2(3)=26\), giving \(25.8-26=-0.2\) demand units. The value at \(x=0\) stands apart from the other residuals.
This set is consistent with the displayed least-squares line: the residuals sum to \(2.0+10(-0.2)=0\), and the predictor values are symmetric about zero, so \(\sum x_i e_i=0\). The residual plot therefore has one conspicuous point and no repeated curve across \(x\). The point deserves checking—for example, whether its response was recorded correctly and whether its context differs—but it does not, on its own, show that the entire linear mean pattern is wrong.
Worked Example: A Repeated Curved Pattern
Now consider hypothetical responses measured in seconds at predictor settings \(x=-3,-2,-1,0,1,2,3\). Suppose the fitted straight line is \(\hat y=10\) seconds. The observed responses are \(12,10,8.8,8.4,8.8,10,12\) seconds, respectively.
Subtract the fitted value from each observation. For example, at \(x=-3\), \(e=12-10=2\) seconds. At \(x=0\), \(e=8.4-10=-1.6\) seconds. The complete residual sequence in increasing \(x\) order is \(2,0,-1.2,-1.6,-1.2,0,2\) seconds.
The positive residuals at both ends and the negative residuals in the middle form a repeated U-shaped arrangement. This is not one point detached from otherwise unstructured residuals: observations at several predictor settings depart in related directions. In this hypothetical data set, the fitted line is indeed the least-squares line: the residuals sum to \(2+0-1.2-1.6-1.2+0+2=0\), and symmetry gives \(\sum x_i e_i=0\).
The recurring shape suggests that a straight-line conditional mean may be inadequate over these predictor values. The plot identifies a feature to investigate; it does not prove the cause, nor does it establish that curvature is the only possible explanation. The contrast with the first example is the repeated organization across \(x\), not simply whether any residual is large.
Worked Example: A Run of Residuals in Observation Order
A hypothetical process is monitored at eight consecutive times using an intercept-only model, which predicts the same response at every time. The recorded responses, in order, are \(49,49,49,49,51,51,51,51\) units. The fitted value is their mean:
The residuals are calculated as \(y_i-50\): the first four are \(49-50=-1\) unit each, and the last four are \(51-50=1\) unit each. Their sum is \(4(-1)+4(1)=0\), as expected for an intercept-only least-squares fit.
No single residual is unusually far from zero relative to the others, but the order matters: four negative residuals occur consecutively, followed by four positive residuals. That repeated run may indicate that the response changed over time in a way the constant-mean model does not represent, or that adjacent observations are related. The residual sequence is a reason to investigate the data collection and model assumptions, not proof of a particular time-related cause.
Why Repetition Is Different From Size
Least-squares fitting with an intercept imposes a zero-sum condition on the residuals. That condition can make a plot look reassuring if attention is limited to an overall average: positive and negative residuals cancel. But cancellation does not mean that their locations are random or that the model captures the structure in the data.
A short derivation shows how a clear pattern can coexist with that zero-sum condition. Consider the seven symmetric predictor values in the curved example, and let \(e_i=0.4(x_i^2-4)\). Because the mean of the seven squared predictor values is \(4\), the residuals sum to zero:
The residuals are also orthogonal to the predictor values: pairing \(x\) and \(-x\) gives equal residuals but opposite contributions to \(\sum x_i e_i\), while the \(x=0\) contribution is zero. Thus the straight least-squares line can have residuals that satisfy its fitting constraints and still display a U shape. The constraints describe averages and linear association; they do not guarantee the absence of repeated structure.
When reading a plot, consider the following questions together:
- How many observations are involved? A lone point differs from a band, curve, run, or group of points sharing a feature.
- Is the pattern organized by a variable? A curve across predictor values, a difference between categories, or a sequence in time gives structure a single isolated point does not.
- What is the scale? A residual is measured in response units. Changing the vertical scale can make an ordinary difference look dramatic or conceal a meaningful one.
- What else is known about the observations? Data-entry, measurement, and sampling context can help distinguish an unusual but valid observation from a recording or design problem.
What Each Pattern Can and Cannot Establish
An unusually large residual says that, for this fitted model, one observed response is far from its fitted value. It may prompt a check of the record, a search for relevant context, or a closer look at the model’s fit in that part of the predictor range. It does not by itself show that the observation is erroneous, that the errors are nonnormal, or that the model must be discarded. Nor does residual size alone tell whether an observation substantially changes the fitted line; those are separate questions.
Repeated structure is often more informative about model adequacy because multiple observations depart in related ways. A curved pattern can suggest that the modeled mean is too restrictive; category-specific departures can suggest that group differences remain; runs in observation order can suggest time-related structure or dependence. These are diagnostic clues, not automatic diagnoses. A plot may be noisy, a pattern may depend on a small number of points, and a pattern on one plotted dimension may have another explanation.
An important pitfall is to remove a point merely because its residual is large. The faulty reasoning is: “This point is far from zero, so it must be bad data and should be deleted.” The correction is to verify the observation and its context first. If it is a valid measurement, deleting it solely to make the plot look better can hide real variation and change the question being answered. Report and interpret diagnostic evidence in context rather than treating a residual plot as a deletion rule.
Check Your Understanding
Use the distinction between an isolated discrepancy and a recurring pattern to answer these questions.
- A residual plot has one point at \(5\) response units and all other residuals lie between \(-1\) and \(1\). What does the large residual show, and what can it not establish on its own?
- Residuals are mostly negative at low predictor values and mostly positive at high values. Why is this different from one unusually large residual?
- In an intercept-only model, explain how four negative residuals followed by four positive residuals could sum to zero and still be diagnostically useful.
- A researcher proposes deleting a valid observation because its residual is the largest in the data. What is the pitfall in that reasoning?
- Why does a recurring pattern provide a clue about model adequacy without proving the pattern’s cause?