Checking Residual Normality for Regression Inference

Why Check Residual Normality?

A multiple regression model can describe the average response well and still give unreliable small-sample confidence intervals or tests if its errors are far from normal. This matters because the usual t and F procedures for regression coefficients rely on assumptions about the errors, not simply on the appearance of the observed response. Residual plots offer a practical way to look for conspicuous departures from those assumptions.

This check is one part of regression diagnostics. The earlier material on checking linearity and constant variance with residual plots addresses other model conditions. Here the focus is the shape of the residual distribution and what it suggests about inference. A plot can flag concerns, but it cannot prove that the underlying errors are normal.

Definition: In a fitted regression model, the residual for observation \(i\) is \(e_i=y_i-\hat{y}_i\), the observed response minus its fitted value. The model error \(\varepsilon_i\) is the unobserved difference between the response and its true conditional mean. Residuals estimate errors; they are not the errors themselves.

For the usual normal-theory regression inference, consider the model \(Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_{p-1}x_{i,p-1}+\varepsilon_i\), with \(p\) regression coefficients including the intercept. Conditional on the predictors, the standard assumptions include a correctly specified mean model, independent errors with mean zero and common variance \(\sigma^2\), and normally distributed errors. Normality is the assumption examined here; the other conditions still matter.

Key result: Under the normal-error regression model, the usual t statistics for individual coefficients and F statistics for specified coefficient restrictions have their stated t and F distributions. This gives their p-values and confidence intervals their usual exact calibration, conditional on the predictors and the model assumptions. If errors are not normal, these exact small-sample distributions need not apply.

Reading a Residual Histogram

A residual histogram groups residuals into bins and displays how many fall in each bin. For normality, look for a roughly unimodal, symmetric, bell-shaped pattern without isolated extreme values or unusually heavy tails. The histogram is most useful for spotting substantial departures, such as strong skewness or a second cluster. A modest irregularity is not surprising when the sample is small.

The scale is the response scale: if the outcome is measured in centimeters, the residuals and histogram are in centimeters. With an intercept in the model, the residuals sum to zero (apart from rounding), so their center is zero. This fact does not mean they must form a normal distribution. Histogram appearance also depends on the bin widths and cut points, so it should not be treated as a definitive test.

Worked Example: A Roughly Balanced Histogram

Suppose a hypothetical regression predicts a laboratory measurement in centimeters. Ten residuals, in centimeters, are \(-5,-4,-3,-2,-1,1,2,3,4,5\). For one observation, the recorded value is 52 cm and its fitted value is 50 cm, giving \(e_i=y_i-\hat{y}_i=52-50=2\) cm, as in the list.

Using the four bins \([-5,-2.5)\), \([-2.5,0)\), \([0,2.5)\), and \([2.5,5]\), the counts are 3, 2, 2, and 3. For example, the first bin contains \(-5,-4,-3\), and the final bin contains \(3,4,5\). The counts are balanced from left to right, with no isolated extreme value. This small histogram does not show an obvious strong departure from symmetry.

That is limited evidence, not confirmation of normality. Ten residuals are too few for the histogram to show the fine shape of a distribution reliably, and residuals are not an independent sample of the model errors. A normal probability plot gives a second view.

Reading a Normal Probability Plot

A normal probability plot, also called a normal Q–Q plot, compares the ordered residuals with the ordered quantiles expected from a normal distribution. Each point pairs one residual with the corresponding theoretical normal quantile. The plot may use residuals on the vertical axis and theoretical quantiles on the horizontal axis, or reverse the axes; the diagnostic idea is the same. If the errors are approximately normal, the points should lie roughly along a straight line. The line’s slope and intercept depend on the scale and center of the residuals.

Look for systematic patterns rather than requiring every point to fall exactly on a line. A sustained curve can indicate skewness; pronounced bending at one or both ends can indicate tail departures; and a point well separated from the general pattern can signal an unusually extreme residual. Small departures are common, particularly with limited data. The plot can reveal features that a histogram hides, but it cannot identify the cause of a pattern by itself.

Worked Example: An Extreme Upper-Tail Residual

In a hypothetical model for weekly demand, the ten residuals, in units, are \(-1,-1,-1,-1,-1,-1,-1,-1,-1,9\). Their sum is \((-1)\times9+9=0\), consistent with a fitted model that includes an intercept. A histogram has a large concentration at \(-1\) and one observation at 9, rather than a roughly balanced bell-shaped pattern.

For ten ordered observations, the most extreme theoretical normal quantiles are approximately \(-1.55\) and \(1.55\), depending slightly on the plotting convention. The ordered residuals at those ends are \(-1\) and 9. The upper-end point is far from the pattern of the other nine points, which all have the same residual. Thus, the normal probability plot would show a pronounced upper-tail departure rather than an approximately straight pattern.

This is a warning that the normal-error model may not describe these data well, especially in the upper tail. The plot alone cannot determine whether the cause is an unusual observation, a missing feature of the mean model, or another issue. It does justify caution about relying on exact normal-theory inference without further investigation.

Why Residuals Are Not the Errors

Residual plots are informative, but the plotted values have been calculated from the fitted model. Fitting constrains their behavior, so they do not form a fresh, independent sample from the error distribution. The following short derivation makes that distinction precise.

Write the model in matrix form as \(Y=X\beta+\varepsilon\), where \(X\) is the design matrix and \(\varepsilon\) is the vector of errors. For a full-rank design, the least-squares fitted values are \(HY\), where \(H=X(X'X)^{-1}X'\). Thus the residual vector is \(e=(I-H)Y\). Because \(HX=X\), we have \((I-H)X\beta=0\), and consequently:

e=(I-H)(X\beta+\varepsilon)=(I-H)\varepsilon.

If the errors are independent normal variables with variance \(\sigma^2\), then \(\varepsilon\) has covariance matrix \(\sigma^2I\). Since \(H\) is symmetric and satisfies \(H^2=H\), the residual covariance matrix is:

\operatorname{Cov}(e) =(I-H)\sigma^2I(I-H)' =\sigma^2(I-H).

In particular, residual \(e_i\) has variance \(\sigma^2(1-h_{ii})\), and different residuals generally have nonzero covariances. The derivation explains why normal-looking residuals are evidence about the errors, not a direct observation of independent errors. It also reinforces why a plot is a diagnostic rather than proof of the model assumptions.

What Normality Means for t and F Inference

In a multiple regression, a t test can assess an individual coefficient, while an F test can assess a set of restrictions, such as whether a group of coefficients is jointly zero. Under the stated regression assumptions, normally distributed errors provide the exact t and F reference distributions used by these procedures in finite samples. When the errors are substantially nonnormal, the calculated statistics can still be formed, but the usual small-sample p-values and confidence intervals may not have their claimed calibration.

A departure is not an automatic reason to discard all results. Its practical importance depends on how strong the pattern is and on the setting; a small visual deviation is different from a striking skew or extreme tail point. Do not make a universal claim that a particular sample size always makes inference safe. Use the plots to describe the evidence, and interpret inferential results with appropriate caution when a substantial departure is present.

Worked Example: Interpreting a Coefficient Test With a Tail Departure

Suppose a hypothetical multiple regression has \(n=24\) observations and \(p=4\) fitted coefficients, including the intercept. A coefficient for a predictor is estimated as 1.20 response units per predictor unit, with standard error 0.40 response units per predictor unit. A residual normal probability plot shows a noticeable departure in the upper tail.

For the null hypothesis that this coefficient is zero, the reported statistic is \(t=\hat{\beta}/SE=1.20/0.40=3.00\). The residual degrees of freedom are \(df=n-p=24-4=20\). Under the normal-error regression model, a two-sided t test with 20 degrees of freedom gives a p-value of about 0.007.

If the model assumptions hold, this p-value is evidence against a zero coefficient, conditional on the other predictors. But the upper-tail departure gives reason to question whether the exact t reference distribution is appropriate in this sample. The plot does not prove that the p-value is wrong; it indicates that its normal-theory calibration deserves caution. The same concern applies to F inference when it relies on the same normal-error assumption.

A Practical Diagnostic Sequence

Calculate and plot residuals.
Use residuals \(e_i=y_i-\hat{y}_i\), keeping their measurement units clear. A residual histogram shows the overall shape.

Inspect the normal probability plot.
Look for a reasonably straight pattern, paying particular attention to systematic curvature and the tails.

Describe the evidence, not a verdict.
State whether the plots show a substantial departure, a mild irregularity, or no conspicuous problem. Do not claim that a plot proves normality.

Connect the diagnostic to inference.
Explain that substantial nonnormality can undermine exact small-sample t and F calibration, while recognizing that the other regression assumptions must also be considered.

Common Mistakes

Calling residuals the errors. Residuals are fitted-model estimates of unobserved errors, and fitting creates dependence and constraints among them.
Expecting a perfect straight line. Random variation makes real plots irregular. Look for systematic, substantial patterns rather than judging every point separately.
Treating a histogram as a formal verdict. Bin choices and small samples can obscure shape. Use the normal probability plot as a complementary visual check.
Claiming that nonnormality makes every test invalid. It calls the exact normal-theory calibration into question, especially in a small sample; it does not by itself prove that a particular conclusion is false.

Key takeaway: Residual histograms and normal probability plots help assess whether the regression errors plausibly follow a normal distribution. A substantial departure matters because it can affect the exact small-sample t and F inference, but residual plots provide evidence rather than proof.

Check Your Understanding

Try these questions without looking back.

For an observation with \(y_i=18.4\) and \(\hat{y}_i=20.1\), calculate the residual and state its meaning in context if the response is measured in kilograms.
What broad shape in a residual histogram would be consistent with normal errors? Name two departures that would raise concern.
In a normal probability plot, what does a sustained curve suggest, and why should small deviations from a straight line not automatically be treated as a serious problem?
Why are fitted-model residuals not an independent sample of the regression errors?
Explain how a substantial residual tail departure could affect the interpretation of a small-sample t test or F test.