Why Look for Curvature in Residuals?
A regression model summarizes how an outcome changes with its predictors. Even when the fitted line or curve seems plausible in a scatterplot, it may systematically miss the mean pattern in parts of the data. A residual-versus-fitted plot helps reveal such mismatches: if residuals tend to be positive in some ranges of fitted values and negative in others, the deviations may be too organized to look like random scatter.
This tutorial focuses on curvature: U-shaped, inverted-U, or wavelike patterns in residuals. These shapes suggest that the fitted mean relationship may not capture all of the curvature in the data. They are diagnostic clues, not proof that a particular alternative curve is correct.
A positive residual means the observed outcome is above the model’s prediction; a negative residual means it is below. The vertical scale has the same units as the outcome. For example, if the outcome is a concentration measured in milligrams per liter, a residual of \(2\) means the observation is 2 milligrams per liter above its fitted value.
Model and Diagnostic Logic
To interpret a residual pattern, first be clear about the model being checked. Suppose the observed outcome for each of \(n\) observations follows a regression model with a specified mean function:
Here \(x_i\) denotes the predictor information for observation \(i\), \(m(x_i)\) is the conditional mean of the outcome at that predictor information, and \(\varepsilon_i\) is the error. The usual conditions for interpreting a mean-function diagnostic include a correctly specified form for \(m\), errors with conditional mean zero, and independent observations. Constant error variance is also a standard regression condition, though the focus here is the shape of the residual pattern rather than changing spread. Normal errors are not required just to recognize curvature in a plot.
The model’s fitted values come from a chosen form for the mean. If that form misses systematic curvature, observed outcomes can tend to lie above the fitted values in some regions and below them in others. Since residuals are observed outcomes minus fitted values, that mismatch may appear as a curved pattern around the zero line.
For an ordinary least-squares fit, the residuals are constrained by the fitting procedure. With an intercept, for instance, their sum is zero. Thus, a visible residual pattern does not mean the residuals can have a nonzero overall average; it means their average can vary across fitted-value ranges. A curved pattern may remain even though residuals balance overall.
Recognizing the Main Shapes
Read the plot from left to right, following increasing fitted values. Focus on the general direction of the residuals rather than on any single point. Random scatter around zero has no persistent sequence of positive and negative regions. Curvature instead involves a systematic change in the residual center across the horizontal axis.
| Pattern | Typical residual sequence | What it suggests |
|---|---|---|
| U-shaped | Positive at lower and higher fitted values, negative in the middle | The fitted mean may be too high in the middle and too low at the ends |
| Inverted-U | Negative at lower and higher fitted values, positive in the middle | The fitted mean may be too low in the middle and too high at the ends |
| Wavelike | Residuals repeatedly alternate above and below zero | The mean mismatch may change direction more than once |
These descriptions use the sign of \(e_i=y_i-\hat{y}_i\). In a U-shaped pattern, positive residuals at both ends mean that the observed outcomes tend to exceed the fitted values at both ends. Negative residuals in the middle mean the model tends to predict too high there. An inverted-U pattern reverses those comparisons. A wavelike pattern has multiple changes in direction and may point to a more complicated mismatch, but it does not identify a unique model form.
Worked Example: A U-Shaped Residual Pattern
Suppose a hypothetical model predicts an outcome measured in units per liter. The fitted values and observed outcomes are shown below. The residual at each fitted value is calculated as observed outcome minus fitted value.
| \(\hat{y}_i\) | \(y_i\) | \(e_i=y_i-\hat{y}_i\) |
|---|---|---|
| 10 | 12.0 | 2.0 |
| 20 | 19.6 | -0.4 |
| 30 | 28.4 | -1.6 |
| 40 | 38.4 | -1.6 |
| 50 | 49.6 | -0.4 |
| 60 | 62.0 | 2.0 |
For example, when \(\hat{y}_i=30\), the residual is \(28.4-30=-1.6\) units per liter. At the lowest fitted value it is \(12.0-10=2.0\), and at the highest it is \(62.0-60=2.0\). The middle residuals are negative, while residuals at both ends are positive. Plotted against fitted value, these points form a U-shaped pattern.
The model tends to overpredict the outcome in the middle and underpredict it at the ends. This pattern suggests that the fitted mean relationship may be missing curvature. It does not, by itself, establish which alternative mean form is appropriate.
Worked Example: An Inverted-U Residual Pattern
For a second hypothetical outcome measured in points, suppose a fitted model produces the following values.
| \(\hat{y}_i\) | \(y_i\) | \(e_i=y_i-\hat{y}_i\) |
|---|---|---|
| 10 | 8.0 | -2.0 |
| 20 | 20.4 | 0.4 |
| 30 | 31.6 | 1.6 |
| 40 | 41.6 | 1.6 |
| 50 | 50.4 | 0.4 |
| 60 | 58.0 | -2.0 |
At \(\hat{y}_i=40\), the residual is \(41.6-40=1.6\) points; at \(\hat{y}_i=10\), it is \(8.0-10=-2.0\) points. Negative residuals at both ends and positive residuals in the middle create an inverted-U pattern. In this example, the fitted values tend to be too low in the middle and too high at the ends. The changing direction of the residuals is evidence to examine the adequacy of the fitted mean form, not a reason to claim that a specific curve has been proven.
Why a Mean-Function Mismatch Can Leave a Pattern
A short least-squares argument clarifies why a curved residual pattern can be informative. Let \(X\) be the design matrix for a linear regression model, with full column rank, and let \(H=X(X^\mathsf{T}X)^{-1}X^\mathsf{T}\) be the projection matrix that produces the fitted values. Suppose the true outcomes can be written as
Here \(X\beta\) is the part represented by the fitted model, \(\delta\) is a systematic mean component not represented by that model, and \(\varepsilon\) has conditional mean zero. Least squares gives fitted values \(Hy\), so the residual vector is \(e=y-Hy=(I-H)y\). Substituting the expression for \(y\) gives
Because \(HX=X\), the term \((I-H)X\beta\) is zero. Taking the conditional expectation given \(X\), and using \(E(\varepsilon\mid X)=0\), yields
Thus, a systematic component omitted from the fitted model can leave structured residuals. The projection also shows why the plotted pattern is not necessarily an exact picture of the omitted component: least squares absorbs the part that can be represented by the fitted model. A residual plot is therefore a useful diagnostic for lack of fit, not a direct graph of the true mean.
Worked Example: A Wavelike Pattern
Suppose a third hypothetical outcome, measured in milligrams, has the following fitted values and residuals from a regression fit. The observed values are calculated by adding each residual to its fitted value.
| \(\hat{y}_i\) | \(e_i\) | \(y_i=\hat{y}_i+e_i\) |
|---|---|---|
| 10 | 1.2 | 11.2 |
| 20 | -1.2 | 18.8 |
| 30 | -1.2 | 28.8 |
| 40 | 1.2 | 41.2 |
| 50 | 1.2 | 51.2 |
| 60 | -1.2 | 58.8 |
| 70 | -1.2 | 68.8 |
| 80 | 1.2 | 81.2 |
For example, the observation with fitted value 40 has observed value \(40+1.2=41.2\) milligrams, so its residual is \(41.2-40=1.2\) milligrams. Moving left to right, the residuals alternate in groups: positive, negative, positive, then negative, then positive. This repeated change of direction forms a wavelike pattern rather than one simple U or inverted U.
The pattern suggests that the fitted mean may be missing structure that changes direction more than once. It is not enough to identify the exact reason for that structure. The pattern could motivate further examination of the mean relationship and the data, while the plot alone cannot distinguish among all possible explanations.
Using the Plot Carefully
A residual-versus-fitted plot is most useful when the pattern is judged as a whole. Ask whether the center of the residuals appears to follow a curve as fitted values increase. A few isolated residuals do not establish curvature; a sustained sequence of positive and negative regions is more suggestive. The residuals need not trace a perfect geometric shape for the pattern to matter.
Fitted values summarize the predictions of the model, not necessarily one original predictor. In a model with several predictors, different observations can have similar fitted values for different combinations of predictor values. A pattern in this plot points to a possible problem with the fitted mean relationship overall, but it does not by itself reveal which predictor or combination is responsible.
The plot also does not establish causation, and it does not select a unique replacement model. Earlier material on choosing nonlinear forms provides ways to think about candidate mean relationships; here the key diagnostic conclusion is narrower: persistent curvature suggests that the current fitted mean may not adequately represent the data. A residual plot should be interpreted alongside the study context and other checks, rather than used as an automatic instruction to add a particular term.
Common Mistake
- Wrong reasoning: “The residual plot is U-shaped, so the true relationship must be quadratic.”
- Correction: A U-shaped pattern indicates that the current fitted mean may be systematically too high in one region and too low in others. It does not prove a quadratic relationship or identify a unique cause. Treat it as evidence to investigate the mean form, then assess any candidate form in context.
Another frequent error is reversing the residual signs. Since \(e_i=y_i-\hat{y}_i\), a positive residual means the observed outcome is greater than its prediction. Keeping that subtraction order in mind makes the interpretation of each curved pattern consistent.
Check Your Understanding
Use the residual definition and the direction of each pattern to answer these questions.
- If residuals are positive at low and high fitted values but negative in the middle, what shape is suggested, and where does the model tend to overpredict?
- An observation has \(y_i=47\) and \(\hat{y}_i=50\). Calculate its residual and interpret its sign in context.
- What does an inverted-U pattern suggest about the fitted values in the middle compared with the observed outcomes?
- Why does a wavelike pattern suggest a possible mean-function mismatch without identifying one unique alternative model?
- Why is a curved residual pattern still possible when the residuals from a least-squares fit with an intercept sum to zero?