Finding Response Outliers With Studentized Residuals

Why Residual Size Needs a Reference

A residual is the difference between an observed response and the response predicted by a fitted regression line. Its sign tells us whether the observation lies above or below the line, and its size is measured in the response variable’s units. But a residual of 8 points may be large in one analysis and ordinary in another. Even within the same regression, residuals do not all have the same expected variability. Studentized residuals address these issues by scaling a residual against an estimate of its variability.

A studentized residual helps identify a response outlier: an observation whose response is unusually far from the fitted value for its predictor value. It is a diagnostic flag, not a verdict. In particular, a large residual alone does not establish that the observation substantially changes the fitted line. That distinct question concerns influence.

Model and Definitions

For simple linear regression, consider the model \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), where \(x_i\) is the predictor value for case \(i\), \(Y_i\) is its response, and \(\varepsilon_i\) is its error. For the standard studentized-residual calculations, assume the errors are independent, have mean 0 and common variance \(\sigma^2\), and are normally distributed when using exact \(t\)-based inference. The predictor values may be treated as fixed, or the calculations may be understood conditionally on the observed predictor values.

The fitted response is \(\hat{y}_i\), and the raw residual is \(e_i=y_i-\hat{y}_i\). The design must have full rank, meaning the intercept and slope can be estimated separately; in simple linear regression this requires at least two distinct predictor values. The residual standard error \(s\) estimates \(\sigma\) using \(s=\sqrt{\mathrm{SSE}/(n-2)}\), where \(\mathrm{SSE}=\sum e_i^2\). Thus, \(n>2\) is required to estimate \(s\).

Both statistics are dimensionless: the residual and its estimated standard deviation are in the same response units, so those units cancel. A positive value means the observed response is above its fitted value; a negative value means it is below. The word “studentized” indicates that the residual has been scaled to account for estimated variability. In particular, \(s\sqrt{1-h_{ii}}\), not just \(s\), is the estimated standard deviation of residual \(e_i\).

Why the Scaling Includes the Fitted-Value Adjustment

The factor \(\sqrt{1-h_{ii}}\) arises because a residual is not simply the original error \(\varepsilon_i\). The fitted line was estimated from the data, including case \(i\), so the residual is partly constrained by that fitting process. In matrix notation, the vector of fitted values is \(H\mathbf{Y}\), where \(H\) is the hat matrix, and the vector of residuals is \((I-H)\mathbf{Y}\). Under the model, the residual vector is \((I-H)\boldsymbol{\varepsilon}\), since the fitted regression component is removed by \(I-H\).

The error covariance matrix is \(\sigma^2 I\) under the independence and equal-variance assumptions. The variance of the \(i\)th residual is therefore the \(i\)th diagonal element of \(\sigma^2(I-H)\), which gives:

Replacing the unknown \(\sigma\) with \(s\) gives the internal scaling. The external version instead uses an error-variation estimate that does not include case \(i\). This can be useful because an extreme residual can inflate the full-data estimate \(s\), making its own internally studentized value less extreme.

Calculating and Using Studentized Residuals

For simple linear regression, there are two fitted coefficients, the intercept and slope. The externally studentized statistic uses the residual variation from the regression fitted without the case being checked. If that deleted fit has full rank, it has \(n-1\) observations and \(n-3\) residual degrees of freedom. In particular, \(n>3\) is needed for a positive residual degrees of freedom. Under the normal-error model, the externally studentized residual has a \(t\) distribution with \(n-3\) degrees of freedom when the case is not an outlier under that model. This reference distribution is conditional on the predictor values and the stated assumptions.

A commonly used algebraic calculation for the deleted residual standard error is:

This formula requires that omitting case \(i\) still leaves a full-rank regression design. An externally studentized value can be compared with a \(t\) distribution to assess whether that individual response is unusually far from its fitted value. Such an assessment depends on the normal-error model being a reasonable description; residual plots and the data context still matter.

What a Large Studentized Residual Does—and Does Not—Show

A large absolute studentized residual points to an unusual response compared with the fitted response at that predictor value, relative to estimated residual variation. It can motivate checking for recording errors, unusual circumstances, or a feature of the relationship the model does not capture. It does not tell us which explanation is correct. Nor does it prove that the regression is useless: one unusual observation can coexist with a useful overall model.

Most importantly, response unusualness and influence are different questions. A studentized residual measures how far a response falls from its fitted value after scaling. Influence asks how much the fitted regression results would change if a case were omitted. That change depends on more than residual size. A case with an extreme response residual is not automatically influential, and an observation need not have a large residual to warrant a separate diagnostic check.

The size of a studentized residual is also not a substitute for checking model assumptions. Dependence, unequal error variance, or a systematic pattern in residuals can make the usual reference calculations misleading. A \(t\)-based p-value for an externally studentized residual relies on the stated model, including normal errors. For practical diagnostics, interpret the statistic alongside the residual patterns and the data-generating context.

Common Mistakes

• Using the raw residual alone. A raw residual is in response units and does not account for residual variability or the fitted-value adjustment. Compare a suitably studentized residual when screening for unusual responses.
• Calling every value beyond 2 a proven outlier. A threshold such as \(|r_i|>2\) is an informal flag, not a universal rule. A formal assessment must use an appropriate reference distribution and account for how many cases were examined.
• Calling a response outlier influential. A large studentized residual identifies distance from a fitted value, not the size of the change in the fitted regression when that case is removed.
• Using the external \(t\) reference without its conditions. The simple-regression reference uses \(n-3\) degrees of freedom, requires \(n>3\), and depends on the normal-error model and a full-rank deleted fit.