When Residuals Trace a Curve
In “Reading a Residual Plot for Random Scatter,” you learned that residuals scattered above and below zero without a clear pattern support using a linear model. This tutorial looks at a clear reason for caution: the residuals form a curve as you move across the horizontal axis. Such a shape means the line’s errors are organized by the predictor or fitted response, rather than looking like unstructured scatter.
A common curved pattern is U-shaped: residuals tend to be positive at both ends of the horizontal range and negative in the middle. The reverse pattern, with negative residuals at both ends and positive residuals in the middle, looks like an upside-down U. The exact points need not make a perfect geometric curve. Look for a consistent change in residuals across the range.
The sign of a residual helps explain what the curve means. Recall from “Sign of a Residual: Over- and Underprediction” that a positive residual means the observed response is above the prediction, so the line underpredicted. A negative residual means the observed response is below the prediction, so the line overpredicted. When these signs follow a curved arrangement, the model’s misses change systematically as the predictor changes.
Read the Shape and Its Implications
For a U-shaped residual pattern, positive residuals near both ends indicate that the line underpredicts there. Negative residuals in the middle indicate that the line overpredicts there. The observed relationship bends upward relative to the fitted line: compared with the line, responses are lower in the middle and higher toward the ends. For an inverted-U residual pattern, the signs reverse: the line tends to overpredict at the ends and underpredict in the middle.
A systematic curve is different from a few residuals that happen to be positive or negative. The key is the order of the signs and values across the horizontal axis. If you see one region mostly above zero, another mostly below zero, and perhaps a return toward the first side, that organized shape is evidence that the straight-line fit leaves out curvature.
Use the horizontal reference line to identify positive and negative residuals.
Look for a progression such as positive, negative, then positive, or the reverse.
Positive residuals mean underprediction; negative residuals mean overprediction. Describe where each occurs in context.
State that the curve is systematic and that it makes a linear model inappropriate for describing the relationship over the observed range.
Do not confuse this conclusion with saying that the line was calculated incorrectly. A least-squares regression line can be calculated correctly and still be a poor choice for representing a curved relationship. As discussed in “Why Residuals Sum to Zero,” residuals from a least-squares line with an intercept balance overall. That balance does not prevent them from forming a curve.
Worked Example: U-Shaped Residuals for a Processing Task
A fictional materials lab records the time, \(y\) minutes, for a processing task at seven machine settings, \(x\). The times are \(23,18,15,14,15,18,23\) minutes for settings \(1\) through \(7\), respectively. A residual plot is made from the least-squares line.
| Setting \(x\) | Time \(y\) (minutes) | Predicted time \(\hat{y}\) (minutes) | Residual \(y-\hat{y}\) (minutes) |
|---|---|---|---|
| 1 | 23 | 18 | 5 |
| 2 | 18 | 18 | 0 |
| 3 | 15 | 18 | −3 |
| 4 | 14 | 18 | −4 |
| 5 | 15 | 18 | −3 |
| 6 | 18 | 18 | 0 |
| 7 | 23 | 18 | 5 |
State. Decide whether the residual plot supports describing processing time with a linear model across these settings.
Plan. Check that the predictions are from the least-squares line, then inspect the order of residuals across settings. A correct least-squares calculation alone does not establish that a line is an appropriate model.
Do. The mean setting is \(\bar{x}=4\), and the mean time is \(\bar{y}=126/7=18\) minutes. The data are symmetric around setting \(4\), so the products of deviations from the means cancel in pairs. In detail, the numerator for the slope is
The denominator is \(\sum (x-\bar{x})^2=9+4+1+0+1+4+9=28\), so the least-squares slope is \(b=0/28=0\). The intercept is \(a=\bar{y}-b\bar{x}=18-0(4)=18\), giving \(\hat{y}=18\). Subtracting this prediction from each observed time gives residuals \(5,0,-3,-4,-3,0,5\) minutes. They are positive near the ends and negative through the middle: a clear U-shaped pattern. As a check, the residuals sum to \(5+0-3-4-3+0+5=0\), and their products with the centered settings sum to \((-3)(5)+(-2)(0)+(-1)(-3)+0(-4)+(1)(-3)+(2)(0)+(3)(5)=0\), as expected for this least-squares fit.
Conclude. The residuals form a U-shaped pattern, not random scatter around zero. The line underpredicts processing time at the low and high settings and overpredicts it near the middle. A linear model is therefore inappropriate for describing processing time across the observed settings.
Curvature Does Not Have to Be Perfect
In real data, a curved residual plot may look less tidy than the first example. Some points can lie on the “wrong” side of the apparent curve, and residuals can vary in size. What matters is whether the overall arrangement still suggests a systematic bend. A gentle arch, bowl, or change in direction can be meaningful even when individual points do not land exactly on a smooth shape.
The residual plot shows how the line misses, not the exact equation of the relationship. A curve is evidence that a straight line is unsuitable over the observed range; it does not, by itself, prove that a particular curved equation is the best alternative. Do not automatically claim that a quadratic model is appropriate just because the residual plot looks U-shaped. That would require examining and assessing an alternative model.
Worked Example: An Inverted-U Pattern in a Sensor Score
A fictional lab records a sensor’s accuracy score, \(y\) points, at seven calibration settings, \(x=1\) through \(7\). The observed scores are \(21,26,29,30,29,26,21\). A residual plot from the least-squares line has the following values.
| Setting \(x\) | Observed score \(y\) (points) | Fitted value \(\hat{y}\) (points) | Residual (points) |
|---|---|---|---|
| 1 | 21 | 26 | −5 |
| 2 | 26 | 26 | 0 |
| 3 | 29 | 26 | 3 |
| 4 | 30 | 26 | 4 |
| 5 | 29 | 26 | 3 |
| 6 | 26 | 26 | 0 |
| 7 | 21 | 26 | −5 |
State. Determine what the residual pattern says about using a linear model for accuracy score and calibration setting.
Plan. Verify the fitted line from the data, then interpret how the residual signs change from the lowest to the highest settings.
Do. The mean score is \(182/7=26\) points, and the symmetry of the data around setting \(4\) makes the least-squares slope zero. More explicitly, the centered cross-products sum to \((-3)(-5)+(-2)(0)+(-1)(3)+0(4)+(1)(3)+(2)(0)+(3)(-5)=15-3+3-15=0\). The sum of squared setting deviations is \(28\), so \(b=0/28=0\), and \(a=26-0(4)=26\). Thus the fitted line predicts \(26\) points at every setting. The residuals are negative at both ends, positive in the middle, and zero at settings \(2\) and \(6\), forming an inverted U. Their sum is \(-5+0+3+4+3+0-5=0\), but the visible curve remains.
Conclude. At the lowest and highest settings, the line overpredicts the accuracy score; in the middle, it underpredicts. This systematic inverted-U pattern makes a linear model inappropriate for describing accuracy score across the observed settings. The plot indicates curvature but does not establish which alternative model should be used.
Use Residuals to Explain the Model’s Misses
A strong interpretation describes both the shape and its meaning in context. Saying only “the residual plot is curved” identifies a visual feature but leaves out why it matters. Explain where the line underpredicts and overpredicts, then connect those repeated errors to the suitability of a straight-line model.
For example, suppose a residual plot for task time against machine setting has residuals mostly positive at low and high settings and mostly negative at middle settings. A careful interpretation is: “The residuals form a U-shaped pattern: the line underpredicts task time at the lower and higher settings and overpredicts it at middle settings. This systematic pattern indicates that a linear model is not appropriate for describing task time across the settings observed.” That conclusion is more informative than “the points are not random.”
Worked Example: A Curved Pattern with a Nonzero Slope
A fictional equipment test records task time, \(y\) minutes, at five processor settings, \(x=1\) through \(5\). The observed times are \(55,55,57,61,67\) minutes. The fitted line is not horizontal, but its residual plot may still reveal curvature.
| Setting \(x\) | Observed time \(y\) (minutes) | Predicted time \(\hat{y}=50+3x\) (minutes) | Residual (minutes) |
|---|---|---|---|
| 1 | 55 | 53 | 2 |
| 2 | 55 | 56 | −1 |
| 3 | 57 | 59 | −2 |
| 4 | 61 | 62 | −1 |
| 5 | 67 | 65 | 2 |
State. Decide whether the residuals support a linear model for task time across these settings.
Plan. Verify the line from the data and then examine the residual sequence. A nonzero slope does not prevent the residual plot from having a curved pattern.
Do. The mean setting is \(\bar{x}=3\), and the mean time is \(\bar{y}=295/5=59\) minutes. The centered \(x\)-deviations are \(-2,-1,0,1,2\); the centered \(y\)-deviations are \(-4,-4,-2,2,8\). Thus the cross-products sum to \(8+4+0+2+16=30\), and the squared \(x\)-deviations sum to \(4+1+0+1+4=10\). The least-squares slope is \(b=30/10=3\) minutes per setting, and the intercept is \(a=59-3(3)=50\) minutes. The fitted line is \(\hat{y}=50+3x\). Subtracting predictions gives residuals \(2,-1,-2,-1,2\) minutes. These are positive at the two ends and negative in the middle, a U-shaped arrangement. Their sum is \(2-1-2-1+2=0\); their products with centered settings sum to \((-2)(2)+(-1)(-1)+0(-2)+(1)(-1)+(2)(2)=0\). The least-squares properties hold, but the residual shape is still systematic.
Conclude. The line underpredicts task time at the lowest and highest settings and overpredicts it at the settings in between. The U-shaped residual pattern shows that a linear model is not appropriate across the observed settings, even though the fitted line has a positive slope and is correctly calculated.
Common Mistakes and AP Exam Tips
- Calling a curve “random scatter.” A run of positive residuals, then negative residuals, then positive residuals is an organized pattern. Describe the shape and the sign changes.
- Reporting the shape without explaining the errors. State which regions have positive residuals (underprediction) and which have negative residuals (overprediction), in context.
- Assuming residuals must have a nonzero sum to show a problem. Least-squares residuals with an intercept sum to zero. They can still form a clear curve, so judge their arrangement across the horizontal axis.
- Claiming the line was calculated incorrectly. A least-squares line can be correctly calculated even when it is inappropriate for the relationship’s shape. Separate the calculation from the model assessment.
- Claiming a specific alternative is proven. A U-shaped pattern suggests curvature, but it does not by itself establish that a particular curved model is appropriate.
- Making an absolute statement beyond the data. Conclude that a linear model is inappropriate for describing the relationship over the observed range. Do not claim the plot proves how the variables behave beyond that range.
For a full-credit AP response, identify the systematic curve, connect the residual signs to overprediction and underprediction in context, and explain that the pattern makes a linear model inappropriate over the observed range. Avoid saying only that the residuals are “not random”; explain what the model is missing.
Check Your Understanding
Use the residual pattern and its context to answer the questions.
- A residual plot has positive residuals at both ends and negative residuals in the middle. What shape does this suggest?
- For that pattern, where does the line underpredict, and where does it overpredict?
- Why does a residual sum of zero not rule out a curved pattern?
- A plot shows negative residuals at both ends and positive residuals in the middle. What does the pattern imply about the line’s predictions across the range?
- What conclusion about a linear model is justified by a clear curved residual pattern, and what should you avoid claiming about an alternative model?