Why Audit an Extreme Observation?
An unusual response can be a mistake, a genuine rare outcome, or a valid observation that strongly affects a regression fit. Those possibilities require different actions. A large residual is evidence that a response is unusual relative to the fitted model; it is not evidence, by itself, that the record is wrong or should be removed. A careful audit therefore asks several separate questions: Is the record valid? How unusual is its response under the fitted model? Does its predictor position give it leverage? How much does the fitted relationship change when the case is omitted? Finally, does that change alter the conclusion that matters?
This tutorial follows one hypothetical small dataset through those questions. The calculations use simple least-squares regression with an intercept, so the influence calculations can be checked directly. They illustrate an audit, not a universal threshold for declaring a case problematic.
Model and Audit Quantities
Let \(x_i\) be a predictor value and \(y_i\) the corresponding response for observation \(i\). The simple linear regression model is \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). Least squares chooses the intercept and slope to minimize the sum of squared residuals. The usual model conditions for interpreting regression uncertainty are a linear conditional mean, independent errors with mean zero and constant variance; normal errors are additionally used for exact small-sample \(t\)-inference. The audit below describes fitted values and sensitivity and does not calculate a test or confidence interval.
Leverage is determined by the predictor values, not by the observed response. Its values lie between zero and one, and their average in a regression with an intercept and one predictor is \(2/n\), because the leverages sum to the two fitted parameters. A leverage appreciably above that average merits attention, but no single cutoff decides whether a case is influential. In this audit, we will also compare the full-data fit directly with the fit after omitting the unusual response.
For a compact influence summary, Cook’s distance compares the full-data fitted values with the fitted values after deleting case \(i\). With \(p\) fitted parameters, including the intercept, and full-data mean squared error \(MSE\), it is
Here \(\hat{y}_{j(-i)}\) is the fitted value at predictor value \(x_j\) from the regression with case \(i\) deleted. Cook’s distance measures change in the fitted values relative to the residual variation in the original fit. It is a screening summary, not a rule that identifies a bad record or commands deletion.
Worked Example: Verify the Record Before Interpreting It
Suppose a hypothetical process audit records six pairs of predictor and response values. The predictor is a process setting in setting units, and the response is a measured output in output units.
| Case | Setting \(x\) | Measured output \(y\) |
|---|---|---|
| 1 | 1 | 5 |
| 2 | 2 | 8 |
| 3 | 3 | 11 |
| 4 | 4 | 14 |
| 5 | 5 | 17 |
| 6 | 6 | 32 |
The first five responses follow the simple pattern \(y=2+3x\); the sixth is substantially above that pattern. Before treating 32 as a genuine extreme response, an analyst would check the original record, units, instrument or measurement process, case eligibility, and whether the value was entered in the correct row. For this hypothetical audit, assume those checks confirm that case 6 is correctly recorded as 32 output units, measured under the same protocol and in the same units as the other responses.
That verification rules out the specific transcription or unit error checked; it does not show that the linear model is appropriate, nor does it establish why the response is large. Since the observation is verified, it should not be deleted solely for being unusual. The next steps quantify how unusual it is relative to the fitted line and how much it affects the analysis.
Residual and Leverage in the Full Fit
The residual is calculated from the regression fitted to all six observations. For this dataset, \(\bar{x}=3.5\), \(\bar{y}=14.5\), and \(S_{xx}=\sum_i(x_i-\bar{x})^2=17.5\). Also, \(S_{xy}=\sum_i(x_i-\bar{x})(y_i-\bar{y})=82.5\). The least-squares slope is \(S_{xy}/S_{xx}\), and the intercept is \(\bar{y}-\hat{\beta}_1\bar{x}\).
Worked Example: Calculate the Residual and Leverage
Using the full dataset, calculate the line and the residual for case 6. Then compare its leverage with the average leverage.
Full-data regression line. Substituting the summary values gives
Thus, \(\hat{y}=-2+\frac{33}{7}x\). At \(x_6=6\), the fitted output is \(\hat{y}_6=-2+\frac{33}{7}(6)=\frac{184}{7}\approx26.29\) output units. The residual is
The positive residual means the observed output is about 5.714 units above the full-data fitted value. It is not the difference from the fit based only on the first five cases; that distinction matters because the full-data line has already been pulled toward case 6.
Leverage. Here \(n=6\), \(\bar{x}=3.5\), and \(S_{xx}=17.5\), so
The average leverage is \(2/6=0.333\), so case 6 has leverage above the average: its setting is at the edge of the observed predictor range. Case 1, at the opposite edge, has the same leverage. Leverage alone does not distinguish an unusual response from an ordinary one; it only describes the predictor position.
Does the Case Change the Fit?
A residual is calculated after the case has helped determine the fitted line. Therefore, comparing its residual with the size of the original departure from a plausible pattern can be informative. Here, the first five responses lie exactly on \(2+3x\), while case 6 is 12 output units above that line: \(32-(2+3(6))=12\). Once all six cases are fitted together, the slope rises from 3 to \(33/7\), the intercept falls from 2 to \(-2\), and the case’s residual is only \(40/7\), or about 5.714 units. This is a concrete example of the fit moving toward an extreme response.
The change can also be summarized with Cook’s distance. In this dataset, the full-data residual sum of squares is \(480/7\), so with \(p=2\) parameters and \(n-p=4\) residual degrees of freedom, \(MSE=(480/7)/4=120/7\). Substitution into the formula gives
This value summarizes a substantial change relative to the full-data residual scale in this tiny example. It does not mean there is a 2.20 probability that the case is erroneous, and it does not prove that deletion is appropriate. Directly examining the refitted line makes the practical changes clearer.
Worked Example: Compare Fitted Values With and Without the Case
Delete case 6 temporarily and fit the line to cases 1 through 5. Then compare the slope and fitted outputs with the full-data fit.
Deletion fit. The first five observations satisfy \(y=2+3x\) exactly, so their least-squares line is \(\hat{y}_{(-6)}=2+3x\), with slope 3 output units per setting unit and intercept 2 output units. The slope with all cases is \(33/7\approx4.714\), so inclusion of case 6 increases the estimated slope by \(33/7-3=12/7\approx1.714\) output units per setting unit.
Fitted-value changes. At any of the six observed settings, the full-data prediction minus the deletion-fit prediction is
For example, at \(x=1\) the difference is \(-4+12/7=-16/7\approx-2.286\) output units. At \(x=6\) it is \(-4+72/7=44/7\approx6.286\) output units. Thus the case does not merely affect its own fitted value: it changes predictions across the observed range, with the direction of change depending on the setting.
Substantive interpretation. In context, the estimated average increase is 4.714 output units per setting unit with case 6 included, compared with 3 output units per setting unit in the deletion fit. Both slopes are positive, so the direction of the fitted association is unchanged. Its estimated magnitude is not: the full-data slope is about 57.1% larger than 3, since \((4.714-3)/3\approx0.571\). The decision-maker should report that sensitivity rather than presenting either estimate as unaffected by the case. These observational calculations alone do not show that changing the setting causes a change in output.
A Short Justification: Why Predictor Position and Residual Both Matter
The formulas help explain why a case’s influence depends on both its response departure and its predictor position. For simple linear regression with an intercept, the leverage at \(x_i\) is \(h_{ii}=1/n+(x_i-\bar{x})^2/S_{xx}\). A departure in response at a predictor value near \(\bar{x}\) has less ability to rotate the fitted line than the same departure far from \(\bar{x}\). But even high leverage does not guarantee a large change: if the response lies close to the line, the residual may be small.
The connection between a vertical response departure and the full-fit residual can be shown directly. Suppose a response at case \(i\) is increased by \(\delta\), leaving every predictor and other response unchanged. Least-squares fitted values are linear functions of the observed responses. The increase in the fitted value at case \(i\) is \(h_{ii}\delta\); therefore its residual changes by the original response increase minus that fitted increase:
In the example, the first five cases lie on a line and case 6 is 12 units above it. Its leverage is \(11/21\), so its residual after refitting is \((1-11/21)(12)=(10/21)(12)=40/7\), matching the direct calculation. This also explains why an extreme response can pull the line toward itself and leave a residual smaller than its departure from the pattern supported by the other cases. It is the combination of response discrepancy and predictor position that makes an audit useful.
Common Pitfall: Treating a Diagnostic as a Deletion Rule
A common mistake is: “The residual is large and Cook’s distance is high, so this observation must be an error and should be removed.” The reasoning confuses a statistical description with a data-quality judgment. Residuals, leverage, and influence identify features that deserve investigation; none establishes whether a value is valid. Deleting a genuine observation only because it changes the answer can hide important variation and create a result that does not represent the target population.
The correction is to record the audit trail and distinguish the decisions. First check the source and measurement process. If the record is erroneous, correct it when justified or exclude it under a stated, defensible data rule. If it is valid, retain it in the primary analysis and report how conclusions change in a clearly labelled sensitivity analysis. If it is valid but the linear model is not an adequate description, reconsider the model and scientific question rather than treating deletion as a substitute for modeling. In this example, the value is verified, and removing it materially changes the estimated slope and some fitted values; those facts belong in the substantive report.
Check Your Understanding
Use the audit principles and calculations above to answer these questions.
- Why does a large residual not, by itself, establish that a data value is wrong?
- What does leverage describe, and why can two cases at opposite ends of the predictor range have equal leverage?
- In the hypothetical dataset, how does deleting case 6 change the estimated slope, in output units per setting unit?
- What does Cook’s distance summarize, and why is it not a probability or an automatic deletion rule?
- If a verified observation changes the fitted slope but not its direction, what should a careful substantive report say?