Why a Case Study Helps
Influence diagnostics are most useful when considered together. A large residual indicates that an observation is poorly predicted by the fitted model; high leverage indicates that its predictor values are unusual relative to the rest of the data. Neither feature alone tells us how much the fitted results change. Influence is about that change, and a case-deletion comparison makes the practical meaning visible.
The hypothetical example below follows eight observations through a full-data fit, a calculation of influence summaries, and a sensitivity fit with one observation omitted. The deliberately exact pattern among seven observations makes one limitation especially clear: some scale-based diagnostics cannot be computed when the case-deletion fit has no residual variation.
The Hypothetical Data and Model
Suppose a researcher records study time \(x\), in hours, and a hypothetical quiz score \(y\), in points, for eight students. The proposed model is simple linear regression with an intercept. The table is invented for instruction and does not describe a real study.
| Case | Study time \(x\) (hours) | Score \(y\) (points) |
|---|---|---|
| 1 | 0 | 10 |
| 2 | 1 | 12 |
| 3 | 2 | 14 |
| 4 | 3 | 16 |
| 5 | 4 | 18 |
| 6 | 5 | 20 |
| 7 | 6 | 22 |
| 8 | 10 | 10 |
For the inferential interpretation of a linear regression model, the usual conditions include a correctly specified conditional mean, independent errors with mean zero and constant variance, and normally distributed errors when exact small-sample \(t\)-based inference is required. A data table alone cannot establish these conditions. Here, the regression and influence calculations are used to describe sensitivity; the numbers do not verify the assumptions or justify a causal interpretation. We also assume for the example that the entries have been recorded and matched to the correct students.
Worked Example: Fit the Full-Data Regression
Calculate the least-squares line using all eight cases. The sample means are \(\bar{x}=31/8=3.875\) hours and \(\bar{y}=122/8=15.25\) points. The centered sums of squares and cross-products are \(S_{xx}=70.875\) hours squared and \(S_{xy}=19.25\) hour-points.
For simple linear regression, the slope is \(b_1=S_{xy}/S_{xx}\), and the intercept is \(b_0=\bar{y}-b_1\bar{x}\). Substitution gives
Thus the full-data fitted line, with coefficients shown to four decimal places, is
At \(x=10\) hours, the fitted score is \(14.1975309+(0.271604938)(10)=16.9135803\), or 16.9136 points. Case 8 actually scored 10 points, so its residual, observed minus fitted, is \(10-16.9135803=-6.9135803\), or \(-6.9136\) points. This is a large discrepancy in this small, hypothetical data set, but the residual alone does not measure how much the fitted line changes when the case is omitted.
Calculating Influence Summaries
For a regression with an intercept and one predictor, there are \(p=2\) fitted coefficients. The leverage \(h_{ii}\) measures how unusual case \(i\)'s predictor value is relative to the predictor values in the fitted data. In simple linear regression, \(h_{ii}=1/n+(x_i-\bar{x})^2/S_{xx}\). Cook’s distance combines residual size and leverage to summarize the change in the fitted values when a case is deleted:
Here \(e_i\) is the ordinary residual from the full-data fit, and \(\mathrm{SSE}\) is the sum of squared residuals. Cook’s distance is a dimensionless summary, not a probability or a test statistic with a universal cutoff.
Worked Example: Leverage and Cook’s Distance for Case 8
For case 8, \(x_8=10\), \(n=8\), and \(x_8-\bar{x}=10-3.875=6.125\) hours. Its leverage is
This is high relative to the mean leverage \(p/n=2/8=0.25\): case 8 has a predictor value far from the center of the other observations. The full-data residual sum of squares can be calculated from \(S_{yy}-S_{xy}^2/S_{xx}\). In this example \(S_{yy}=143.5\) points squared, so
Using \(e_8=-6.9135803\) points, \(p=2\), and \(h_{88}=0.654321\), Cook’s distance is
The residual is large and the leverage is high, and the Cook’s distance indicates substantial sensitivity of the fitted values to this case. Common screening rules might flag a Cook’s distance above 1 or leverage above \(2p/n=0.5\), and both rules flag case 8 here. These are heuristics, not decision boundaries: the case still needs to be checked and interpreted in context.
Compare the Fits, Then Investigate
A case-deletion sensitivity analysis fits the same model after omitting a selected case, then compares relevant results with the full-data fit. This comparison answers a practical question: how much do the coefficients or predictions change under that particular deletion? It does not establish that either fit is the uniquely correct one.
Worked Example: Compare the Full-Data and Case-Deletion Fits
Delete case 8 temporarily and fit the line to cases 1 through 7. Those seven observations lie exactly on \(y=10+2x\), so the case-deletion fit is
The full-data line has intercept 14.1975309 and slope 0.271604938; the case-deletion line has intercept 10 and slope 2. The full-data fit minus the case-deletion fit therefore changes the intercept by \(4.1975309\) points and the slope by \(-1.728395062\) points per hour. These are substantial coefficient changes.
At 10 hours, the deleted-case line predicts \(10+2(10)=30\) points. The full-data line predicts 16.9135803 points. The difference, full-data prediction minus case-deletion prediction, is
This comparison shows that case 8 strongly changes the fitted line and the prediction at 10 hours. It does not show that the case is erroneous. A careful investigation would check the original record, measurement units, data-entry process, and whether a student with 10 hours of study and a score of 10 points is possible in the setting. If the record is verified and plausible, it should generally remain part of the analysis; the sensitivity comparison should be reported so readers can see its effect.
A Limitation Revealed by the Example
The seven retained observations fit their line perfectly, so their case-deletion residual sum of squares is zero. With \(n=8\) and \(p=2\), the case-deletion fit has \(8-2-1=5\) residual degrees of freedom, but its estimated residual variance is still zero. Consequently, standardized diagnostics that use the deleted-fit residual standard deviation in a denominator, such as externally studentized residuals or DFBETAS in their usual form, are undefined here. The zero denominator is a property of this deliberately exact example, not evidence that the case has no effect.
This is also a reason to report the actual sensitivity comparison rather than relying on one diagnostic number. Cook’s distance in the example uses the full-data MSE, which is positive; the direct coefficient and prediction comparisons are also available. In a less artificial data set, deleted-fit scale estimates may be positive and those additional diagnostics may be calculable. When a diagnostic cannot be computed, state why rather than substituting zero or treating it as reassuring.
How to Communicate the Case Study
A concise report should connect the diagnostic evidence to the investigation and the sensitivity analysis. It should identify the case and units, describe the checks made, and distinguish what the data show from what they cannot establish. Avoid reporting only that a case was “an outlier”: that label does not explain its leverage, its effect on the fit, or whether the observation is valid.
State which case is flagged and give the relevant summaries, such as its residual, leverage, or Cook’s distance, with units where applicable.
Verify values, units, identifiers, and measurement procedures. Explain whether the observation is a data error, a valid extreme case, or unresolved.
Compare the full-data and case-deletion coefficients or predictions, making clear which fit is the reference for each difference.
Retain a valid observation in the primary analysis unless there is a principled reason not to; if an alternative fit is informative, report it as a sensitivity analysis and explain the rationale.
For this hypothetical data set, an appropriate summary is that case 8 has high leverage, a large negative residual, and Cook’s distance of about 1.963; omitting it changes the slope from about 0.2716 to 2 points per hour and changes the prediction at 10 hours by about \(-13.086\) points when comparing full-data minus case-deletion predictions. The example’s deleted residual variance is zero, so certain scale-based diagnostics are undefined. These facts support investigating and transparently reporting case 8, not automatically deleting it.
Common Pitfall
- Wrong reasoning: “The case exceeds a screening cutoff, so it must be removed.”
- Correction: A cutoff identifies a case for review; it does not determine whether the record is erroneous or whether the observation belongs in the target population. Check the record and compare fits. If the observation is valid, report its influence and retain it unless a clear, defensible analysis rule says otherwise.
- Another error: Treating the case-deletion fit as the truth because its residuals are smaller. In this example, the seven retained cases fit perfectly by construction, but that does not make their line automatically preferable for the intended analysis.
Check Your Understanding
Use the case study and definitions above to answer these questions.
- Why does case 8 have high leverage in the full-data regression?
- What does the negative difference of about \(-13.086\) points represent when comparing predictions at 10 hours?
- Why does a Cook’s distance above a commonly used screening value not prove that a record is erroneous?
- What causes the usual externally studentized residual for case 8 to be undefined in this example?
- Suppose the original record confirms that case 8 is valid. What should be reported about its effect, and why is automatic deletion not justified?