Tests of Residual Normality and Their Limitations

Why Test Residual Normality?

A regression model can produce residuals that look roughly symmetric in a histogram yet still depart from a normal shape in their tails. Formal tests provide a way to assess such departures, but their results need context: a small p-value is evidence against a specified null model, not a diagnosis of what went wrong or a complete verdict on the regression.

For the classical linear regression model, write the response for observation \(i\) as \(Y_i\), its fitted value as \(\hat{Y}_i\), and its residual as \(e_i=Y_i-\hat{Y}_i\). The normality condition concerns the errors conditional on the predictors: under the classical model, \(\varepsilon_i\mid X\sim N(0,\sigma^2)\). This condition is distinct from linearity and constant variance. It supports exact small-sample t and F inference when the other model assumptions also hold; it is not required merely for the OLS coefficients to be calculated.

That last distinction matters. If the design matrix is \(X\), the OLS residual vector is \((I-H)\boldsymbol{\varepsilon}\), where \(H\) is the projection matrix. Under normal errors its covariance matrix is \(\sigma^2(I-H)\), not \(\sigma^2 I\). Thus residuals can be correlated and can have different variances. Tests designed for an independent sample from a normal distribution are useful diagnostics, but their ordinary reference distributions may not be exact for fitted residuals. A model-specific calibration, such as simulation under the fitted model, can be useful when precise calibration is important.

Shapiro-Wilk Test

The Shapiro-Wilk test is designed to detect departures from normality, including departures in shape that may not be obvious in a small histogram. For an independent sample \(x_1,\ldots,x_n\), sort the values to obtain \(x_{(1)}\leq\cdots\leq x_{(n)}\). Its statistic has the form below. The coefficients \(a_i\) depend on the sample size and on expected order statistics from a standard normal distribution; they are supplied by statistical software or tables.

The numerator measures how closely the ordered observations align with the pattern expected under normality, while the denominator measures their total squared spread. Values of \(W\) closer to 1 are generally more consistent with normality. However, there is no universal rule such as “accept normality whenever \(W>0.95\)”: interpretation uses the p-value for the relevant sample size and test calibration.

Omnibus Skewness-Kurtosis Tests

An omnibus skewness-kurtosis test combines evidence from two features of distribution shape. Skewness measures asymmetry; kurtosis measures tail weight and peakedness relative to a normal distribution. The classic Jarque-Bera test uses sample skewness \(S\) and Pearson sample kurtosis \(K\), for which a normal distribution has population values 0 and 3, respectively. Equivalently, excess kurtosis is \(g_2=K-3\).

The classic formula uses skewness and kurtosis calculated with the stated convention. For hand calculations here, use moment estimates with divisor \(n\). If software provides a statistic directly, do not recompute it using a different finite-sample convention.

Why the Jarque-Bera Reference Distribution Has Two Degrees of Freedom

The large-sample justification follows from the behavior of sample skewness and excess kurtosis under normality. For an independent sample from a normal distribution, their approximate large-sample variances are \(6/n\) and \(24/n\), and they are asymptotically independent. Standardizing each gives two approximately standard normal quantities:

The sum of squares of two independent standard normal quantities has a chi-square distribution with 2 degrees of freedom. Substituting the expressions above gives

This explains both the formula and the reference distribution. It also shows why the result is asymptotic: the standardizations are only approximately normal in finite samples. For a chi-square distribution with 2 degrees of freedom, the upper-tail probability is particularly simple:

Why Significance Is Not a Complete Diagnostic Judgment

A p-value answers a limited question: if the null model and the test’s calibration were appropriate, how unusual would a statistic at least this extreme be? It does not measure the size or practical importance of the departure, establish the cause, or tell whether the regression’s predictions are useful. Sample size also matters. With a large sample, a modest departure can be detected; with a small sample, a test may lack power to reveal a meaningful departure.

For example, suppose a hypothetical set of \(n=400\) residuals has \(S=0.25\) and \(K=3.40\). The excess kurtosis is \(g_2=3.40-3=0.40\), so

The large-sample p-value is \(\exp(-6.8333\ldots/2)=\exp(-3.4167\ldots)\approx0.0328\). This rejects at 5%, but not at 1%. The test indicates evidence of a departure in skewness and/or kurtosis; it does not say that either departure is large enough to make a particular inference materially unreliable.

Conversely, a non-significant result is not proof of normality. In a small sample, the test may have limited ability to detect departures. A test statistic can also be sensitive to a few unusual observations, while a single omnibus p-value does not reveal whether the issue is asymmetry, tail behavior, or something else.

Interpret a test alongside the regression context and graphical checks such as the residual histogram or a normal probability plot. Also consider the other model conditions: a normality test does not test linearity, constant variance, or independence, and rejection does not identify which condition is responsible. The residual plots and the earlier material on regression diagnostics help organize that broader assessment. This careful separation of test evidence from diagnostic conclusions is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Practical Interpretation Checklist