Introduction
A fitted regression line can summarize an overall trend while still missing the shape of the relationship. One clue is a residual plot whose points tend to lie above the zero line at some predictor values and below it at others in an orderly curve. That pattern is different from the irregular scatter expected when the fitted mean describes the data adequately.
The diagnosis is not made by spotting one unusually large residual or by relying on a single plot. We will calculate residuals from a hypothetical data set, compare residual-versus-fitted and predictor-specific plots, and distinguish what the evidence supports from what it cannot establish.
The Model and the Diagnostic Question
The diagnostic question here is whether the model’s mean is adequately represented as linear in the included predictors. The usual regression framework also assumes independent observations and errors with conditional mean zero; constant conditional variance is another standard assumption for conventional uncertainty calculations. Normal errors are needed for some exact small-sample inference, but not for visually diagnosing a curved pattern. A plot can suggest that the conditional mean is misspecified; it does not by itself prove why.
If the mean model is suitable, residuals should fluctuate around zero without a persistent shape. A smooth bend, U-shape, or inverted U-shape can indicate that a straight-line mean is not capturing the way the response changes. In a model with several predictors, a residual-versus-fitted plot is a useful overall check, while plots against individual predictors can help locate which predictor’s relationship deserves closer examination.
Worked Example: A Clear U-Shape in a Simple Regression
Worked Example: A Clear U-Shape in a Simple Regression
Suppose a hypothetical calibration exercise records a response \(y\) at seven settings \(x=-3,-2,-1,0,1,2,3\). The values below are simulated, and follow \(y=6.8+2x+0.8x^2\). We fit a straight-line model, \(\hat{y}=b_0+b_1x\), by least squares.
| \(x\) | \(y\) | \(x^2\) | \(xy\) |
|---|---|---|---|
| -3 | 8.0 | 9 | -24.0 |
| -2 | 6.0 | 4 | -12.0 |
| -1 | 5.6 | 1 | -5.6 |
| 0 | 6.8 | 0 | 0.0 |
| 1 | 9.6 | 1 | 9.6 |
| 2 | 14.0 | 4 | 28.0 |
| 3 | 20.0 | 9 | 60.0 |
| Sum | 70.0 | 28 | 56.0 |
The predictor values sum to zero, and \(\sum x_i^2=28\). Thus the least-squares slope and intercept are
The fitted line is \(\hat{y}=10+2x\). Subtracting each fitted value from its observed response gives the following residuals.
| \(x\) | \(y\) | \(\hat{y}=10+2x\) | \(e=y-\hat{y}\) |
|---|---|---|---|
| -3 | 8.0 | 4 | 4.0 |
| -2 | 6.0 | 6 | 0.0 |
| -1 | 5.6 | 8 | -2.4 |
| 0 | 6.8 | 10 | -3.2 |
| 1 | 9.6 | 12 | -2.4 |
| 2 | 14.0 | 14 | 0.0 |
| 3 | 20.0 | 16 | 4.0 |
On a residual-versus-\(x\) plot, the residuals are positive at the low and high settings and negative through the middle. The orderly U-shape is evidence that the straight-line mean is missing curvature. It is not merely one isolated residual: the signs and sizes change systematically across the predictor range.
Here the residual-versus-fitted plot shows the same shape, because fitted values \(10+2x\) increase in direct order with \(x\). In this simple regression, plotting against fitted values or against \(x\) gives the same horizontal ordering, though the horizontal scales differ.
Why the Pattern Appears
The calculation in the example can be summarized without relying on visual inspection alone. The simulated response is \(y_i=6.8+2x_i+0.8x_i^2\). The fitted straight line is \(10+2x_i\), so
This identity explains the U-shape: the residual is positive when \(x_i^2>4\), negative when \(x_i^2<4\), and zero when \(x_i^2=4\). The positive and negative residuals do not mean the least-squares calculations failed. Least squares chooses a line that balances residuals overall; it does not require residuals to look patternless.
In this example, the seven residuals sum to \(4+0-2.4-3.2-2.4+0+4=0\). Their products with \(x\) also sum to zero: \((-3)(4)+(-1)(-2.4)+(1)(-2.4)+(3)(4)=0\). These checks agree with the least-squares constraints for a model with an intercept and \(x\) as its predictor. Those constraints do not rule out curvature; indeed, the curved pattern remains visible.
Worked Example: Comparing Plots With Two Predictors
Worked Example: Comparing Plots With Two Predictors
Consider six hypothetical observations with two predictors, \(x\) and \(z\), and simulated responses from \(y=50+3x+2z+x^2\). We fit a model that is linear in \(x\) and \(z\), omitting the squared term. The observations are balanced: each \(x\) value occurs once at each of \(z=0\) and \(z=1\).
For \(x=-2,0,2\), the average of \(x^2\) across these six observations is \((4+0+4)/3=8/3\). Consequently, the fitted linear model has intercept \(50+8/3=158/3\), coefficient 3 for \(x\), and coefficient 2 for \(z\). The fitted values and residuals are:
| \(x\) | \(z\) | \(y\) | \(\hat{y}=158/3+3x+2z\) | \(e=y-\hat{y}\) |
|---|---|---|---|---|
| -2 | 0 | 48 | 46.667 | 1.333 |
| -2 | 1 | 50 | 48.667 | 1.333 |
| 0 | 0 | 50 | 52.667 | -2.667 |
| 0 | 1 | 52 | 54.667 | -2.667 |
| 2 | 0 | 60 | 58.667 | 1.333 |
| 2 | 1 | 62 | 60.667 | 1.333 |
In the residual-versus-fitted plot, the residuals are positive at the lower and higher fitted values and negative in the middle. The predictor-specific residual-versus-\(x\) plot makes the same pattern easy to locate: at each \(x\), the two observations have the same residual, with positive values at \(-2\) and \(2\) and a negative value at 0. Plotting residuals against \(z\) would not reveal this particular pattern: at either value of \(z\), the residuals across \(x\) still show the same curve, so a plot against \(z\) alone does not identify \(z\) as the source.
The fitted-value plot is an overall diagnostic: its horizontal axis combines contributions from both predictors. A predictor-specific plot can help identify which direction in the data is associated with the pattern. In a real data set, however, a residual-versus-predictor plot is not automatically an isolated estimate of that predictor’s effect. If other predictors are unevenly distributed across its values, their structure can also influence the picture. Here the balanced construction lets us identify the simulated curvature in \(x\) clearly.
Worked Example: A Pattern That Does Not Clearly Support Curvature
Worked Example: A Pattern That Does Not Clearly Support Curvature
A different hypothetical simple regression has predictor values \(x=1,2,3,4,5,6\) and fitted line \(\hat{y}=10+2x\). Suppose the observed responses are \(13,12,17,17,22,21\), giving residuals \(1,-2,1,-1,2,-1\). The fitted values are \(12,14,16,18,20,22\), respectively.
The residuals sum to \(1-2+1-1+2-1=0\), and their products with \(x\) sum to \(1-4+3-4+10-6=0\). These checks are consistent with the reported least-squares line with an intercept. When plotted against \(x\) or the fitted values, the residuals alternate in sign rather than tracing a smooth bend across the range.
For these six observations, there is no clear visual evidence of a curved mean pattern. That conclusion is deliberately limited: a small, noisy sample can conceal curvature, and an irregular-looking plot does not prove the linear mean is correct. The diagnostic evidence is weaker for curvature here than in the previous examples, not a guarantee that no model feature has been missed.
How to State the Evidence
A useful diagnosis describes the visible pattern, identifies the plot or predictor where it appears, and states a measured conclusion. For example: “Residuals are mostly negative at intermediate \(x\) values and positive toward both ends, with the pattern visible against fitted values and against \(x\). This is evidence that the fitted linear mean may not capture curvature in the response’s relationship with \(x\).” The wording does not claim that a particular alternative model is certainly correct or that changing the model would establish causation.
Confirm that each plotted residual is \(y_i-\hat{y}_i\), and that the axes are identified correctly.
Ask whether residuals bend around zero across the range, rather than focusing on a single large value.
Use residuals versus fitted values for an overall view and residuals versus individual predictors to help localize a possible pattern.
Report that the pattern supports concern about the linear mean; do not treat a residual plot as proof of a unique explanation.
Common Mistake
Wrong reasoning: “The residuals sum to zero, so the model has no systematic problem.” In a least-squares fit with an intercept, residuals sum to zero by construction. That total can be zero even when residuals form a clear curve.
Correction: Examine how residuals vary across fitted values and predictors, not only their overall sum. A repeated change from positive to negative and back again can reveal a mean pattern that the zero-sum check cannot detect. Also avoid calling every large residual curvature: curvature is a systematic pattern across the predictor range, whereas one unusual point by itself is not such a pattern.
Check Your Understanding
Use the examples and definitions above to answer these questions.
- In the first worked example, why do positive residuals at both ends and negative residuals in the middle suggest curvature rather than one isolated unusual observation?
- For a simple regression with a positive slope, why will a residual-versus-fitted plot and a residual-versus-predictor plot have the same horizontal ordering?
- In the two-predictor example, what does the residual-versus-fitted plot show, and what does the plot against \(x\) add?
- Why does a zero sum of least-squares residuals not rule out a curved pattern?
- Write one cautious sentence describing what a U-shaped residual plot supports and what it does not prove.