Three Questions, Three Kinds of Evidence
A regression diagnosis may involve several observations that look unusual, but “unusual” is not one single classification. A point can be far vertically from the fitted line, far out in its explanatory-variable value, or influential because it changes the regression results when removed. A residual plot asks a further question: do the line’s errors show a pattern?
In “Outlier, High-Leverage, and Influential Points Defined,” “Testing Influence by Removing a Point,” and “Reading a Residual Plot for Model Fit,” you learned the evidence used for each diagnosis. This practice set combines those ideas. Its organizing technique is to keep the evidence in separate columns: residual size for a y-direction outlier, x-position for leverage, with-and-without output for influence, and the overall residual pattern for model fit.
Use the classifications to describe what the evidence shows, not to decide automatically whether an observation should be removed. As discussed in “What to Do With an Unusual Point,” investigate an unusual observation and retain valid data in the primary analysis.
A Combined Diagnostic Routine
When a question presents a candidate point and a residual plot, it is tempting to jump to a single label. Instead, answer each question in sequence. The routine below helps prevent an extreme \(x\)-value from being mistaken for a large residual, or a patterned residual plot from being mistaken for proof that one observation is influential.
Use the residual \(y-\hat{y}\). A large residual compared with the typical vertical scatter supports calling the observation a y-direction outlier.
Compare the candidate’s explanatory-variable value with the other \(x\)-values. An unusual position suggests high leverage, but does not establish influence.
Compare regression results with and without the candidate point. A substantial change in the line or related regression summaries supports calling it influential.
Look for a curve, changing spread, or another systematic pattern across the \(x\)-values. Describe what the pattern suggests about the linear model’s errors.
These steps are related, but they do not collapse into one test. For example, a point with high leverage may have a small residual and little influence, while a point near the center of the \(x\)-values may have a large residual and affect the intercept or correlation.
Worked Example: A Large Residual Near the Center of the Data
Worked Example: Classify a Vertical Outlier and Check Its Influence
Original AP-style question. An invented sports-science exercise relates \(x\), training sessions completed, to \(y\), a performance score. The six observations are \((1,12),(2,14),(3,16),(4,18),(5,20),(3,40)\). Consider the observation \((3,40)\). Classify its position, and assess its influence by comparing the regression fits with and without it.
State. We will calculate the full-data line and the candidate’s residual, then compare the full-data regression with the fit that omits \((3,40)\).
Plan. Use observed minus predicted response to assess vertical distance. Compare the candidate’s \(x\)-value with the center of the other \(x\)-values to assess leverage. For influence, use the with-and-without comparison, not the \(x\)-position alone.
Do. For all six observations, \(\bar{x}=18/6=3\) and \(\bar{y}=120/6=20\). The centered sums are \(S_{xx}=10\) and \(S_{xy}=20\), so the slope is \(b=20/10=2\) score points per session and the intercept is \(a=20-(2)(3)=14\) score points. The full-data line is \(\hat{y}=14+2x\). At \(x=3\), it predicts \(\hat{y}=14+2(3)=20\), so the candidate’s residual is
This is a large positive residual: the observation lies 20 score points above the full-data line. Its \(x\)-value is not far from the center; it equals the mean \(x\)-value of 3. The candidate is therefore a y-direction outlier, but it is not high leverage based on its \(x\)-position.
To assess influence, compare the full fit with the fit omitting \((3,40)\). The remaining five observations lie exactly on \(\hat{y}=10+2x\), so their slope is 2 and their intercept is 10. For the full data, \(S_{yy}=520\). Thus
The full-data correlation is \(r=\sqrt{1/13}\approx0.2774\), positive because the slope is positive. Without the candidate, \(r=1\), \(r^2=1\), and \(s=0\), because the five remaining points lie exactly on a positive line.
Conclude. The candidate is a y-direction outlier because its residual is 20 score points, and it is not high leverage because its \(x\)-value is at the center of the observed values. The with-and-without comparison shows a substantial change in the intercept and in \(r\), \(r^2\), and \(s\), so the observation is influential for these data even though it does not change the slope. Influence is a description of its effect on the fit; it does not show that the recorded performance score is an error.
Worked Example: High Leverage Does Not Tell the Whole Story
Worked Example: An Extreme \(x\)-Value with a Small Residual
Original AP-style question. In a separate invented technology exercise, \(x\) is the number of hours a device is tested and \(y\) is a performance reading. The observations are \((1,7),(2,9),(3,11),(4,13),(5,15),(10,30)\). Assess the point \((10,30)\) for leverage, y-direction outlier status, and influence.
State. We will compare its \(x\)-position with the other observations, calculate its residual from the full-data line, and compare that line with the fit using the first five observations only.
Plan. The five ordinary observations lie on \(\hat{y}=5+2x\), which gives a useful with-and-without reference. For the full-data line, calculate the slope and intercept from the centered sums. Then compare the candidate’s residual with the full fit’s typical residual size before describing it as a y-direction outlier.
Do. The candidate’s \(x\)-value of 10 is well above the other five values, which run from 1 to 5, so it has high leverage. For all six observations, \(\bar{x}=25/6\), \(\bar{y}=85/6\), \(S_{xx}=305/6\), and \(S_{xy}=785/6\). Therefore
The full-data line is approximately \(\hat{y}=3.4426+2.5738x\). At \(x=10\), its predicted reading is \(3.4426+(2.5738)(10)\approx29.1803\), so the candidate’s residual is \(30-29.1803\approx0.8197\) reading units.
For a comparison of this residual with the full-data scatter, \(S_{yy}=2045/6\). The residual sum of squares is
The candidate’s residual of about 0.8197 reading units is not large compared with \(s\approx1.0122\). Thus, the point’s unusual \(x\)-position does not make it a y-direction outlier in this full fit. Without the candidate, the line is exactly \(\hat{y}=5+2x\). Adding the candidate changes the slope from 2 to about 2.5738 reading units per hour and changes the intercept from 5 to about 3.4426 reading units.
Conclude. The observation has high leverage because its \(x\)-value is far from the others, but its residual is not large relative to the full fit’s typical residual size. The slope and intercept change when it is included, supporting the conclusion that it is influential in this invented data set. The classifications are separate: high leverage describes its \(x\)-position, while influence is supported by the change in the fitted line.
Worked Example: Diagnose a Pattern in the Residuals
Worked Example: A Curved Residual Pattern
Original AP-style question. An invented school-maintenance exercise relates \(x\), weeks since a filter was installed, to \(y\), a measured airflow reading. The fitted line is \(\hat{y}=50+4x\). For weeks \(x=1,2,3,4,5,6\), the observed readings are \(58,58,58,62,70,78\). Calculate the residuals and explain what their pattern suggests.
State. We will calculate each residual as observed reading minus fitted reading, then describe the sequence across weeks.
Plan. A residual plot that shows random scatter in a reasonably even band around zero supports the linear fit. A systematic curve suggests that the line repeatedly overpredicts in some part of the range and underpredicts in another.
Do. The fitted readings for weeks 1 through 6 are \(54,58,62,66,70,74\). Subtracting fitted readings from observed readings gives
The residuals are \(4,0,-4,-4,0,4\) reading units. They begin positive, become negative in the middle, and return to positive at the high \(x\)-values. This is a curved pattern rather than a random band around zero: the line underpredicts at the ends and overpredicts in the middle.
Conclude. The residual plot provides evidence that the linear model misses a systematic pattern in these invented data. It suggests that a straight-line model may not describe the relationship adequately across the observed range. The conclusion comes from the sequence of residuals, not merely from the fact that some residuals are nonzero.
Putting the Evidence Together
The examples show why a mixed diagnosis is strongest when each claim is matched to its evidence. A residual’s sign tells whether an observation falls above or below the line; its magnitude helps assess vertical unusualness. The \(x\)-values show whether a point is far from the center. Only the with-and-without comparison directly establishes whether the candidate substantially changes the fitted results. The residual plot, in turn, is read for an overall pattern across the data.
Sometimes software provides a summary rather than all the original observations. For example, suppose a regression output for \(n=5\) cases reports \(SSE=156.8\). The residual standard deviation is calculated using \(n-2\) degrees of freedom:
This value is in response-variable units and describes the typical size of residuals around the fitted line. It can help put a residual’s size in perspective, but it does not replace checking the residual plot for patterns or comparing fits to assess influence.
- For a y-direction outlier, describe the residual’s size and direction.
- For high leverage, describe how unusual the candidate’s \(x\)-value is compared with the other \(x\)-values.
- For influence, compare the fitted results with and without the point and identify a meaningful change.
- For model fit, describe the residual plot’s overall pattern and explain what it suggests about the line’s errors.
- Do not infer that a point is invalid or should be removed simply because it is unusual or influential.
Common Mistakes and AP Exam Tip
- Using “outlier” without saying which direction. A large residual supports a y-direction outlier. An extreme \(x\)-value is evidence of high leverage, not a large residual.
- Calling every high-leverage point influential. Leverage is about \(x\)-position. Influence requires a with-and-without comparison showing a substantial effect on the regression.
- Calling every large residual influential. A point near the center of the \(x\)-values may have a large residual but little effect on the slope. Check the actual changes in the fits.
- Reading residuals one at a time and missing the pattern. A residual plot should be interpreted across \(x\). A curved sequence or fan shape matters because it shows systematic behavior in the errors.
- Confusing nonzero residuals with a poor fit. Residuals are commonly nonzero. Describe a systematic pattern or changing spread, rather than using “not all zero” as the diagnosis.
- Giving labels without evidence or context. Name the residual, \(x\)-position, or regression change that supports the label, and include units when interpreting a slope or residual.
For a strong AP-style response, keep the claims separate and specific: describe the residual pattern, classify the candidate using the appropriate evidence, and use the with-and-without comparison to evaluate influence. End with a qualified conclusion about what the data support.
Check Your Understanding
For each question, identify which evidence supports the conclusion and explain the reasoning.
- A point has a residual of \(-12\) response units, but its \(x\)-value is close to the center of the other observations. What classification might the residual support, and what does the \(x\)-position suggest?
- A point’s \(x\)-value is far from the others, but its residual is small. Which classification is supported by its \(x\)-position? What additional comparison is needed to assess influence?
- A point’s removal changes the slope from 3 to 4 response units per explanatory-variable unit. What evidence does this provide, and what should your conclusion still avoid claiming?
- A residual plot has positive residuals at low and high \(x\)-values and negative residuals in the middle. Describe the pattern and what it suggests about the linear fit.
- Why is “the residuals are not all zero” not enough to conclude that a linear model fits poorly?