Assessing Residual Normality With Q-Q Plots

Why Examine Residual Normality?

One-way ANOVA compares the means of several groups. Its usual model assumes that, within each group, responses vary around that group’s mean according to a normal error distribution, with errors independent across observations and a common variance. A normal Q-Q plot is a visual way to look for departures from the normal shape in the fitted model’s errors.

The relevant values are residuals, not the original responses pooled across groups. If groups genuinely have different means, pooling their raw responses combines distinct centers; even when each group is normal, that mixture need not look like a single normal distribution. A residual subtracts the fitted mean for its own group, so the diagnostic focuses on variation not accounted for by those group means.

How to Read a Normal Q-Q Plot

A Q-Q plot puts theoretical normal quantiles on the horizontal axis and ordered observed residuals on the vertical axis. The theoretical quantiles are locations expected from a standard normal distribution, such as approximately \(-1.645\) at cumulative probability \(0.05\) and \(1.645\) at cumulative probability \(0.95\). For \(m\) residuals, one common plotting convention assigns the \(i\)th ordered residual the probability \(p_i=(i-0.5)/m\), then uses the standard normal quantile at \(p_i\). Software may use a slightly different plotting convention.

Here \(e_{(i)}\) is the \(i\)th residual after sorting, \(x_i\) is its theoretical normal quantile, and \(\Phi^{-1}\) is the inverse cumulative distribution function of the standard normal distribution. If residuals are plotted in their original units, a straight reference line can have a nonzero intercept and a slope reflecting their scale. If residuals are divided by a common positive scale first, the line’s scale changes, but the overall shape does not.

• Approximately straight: the observed residual quantiles track the normal reference reasonably well. Small departures are common, especially in the tails of a modest data set.
• Systematic curvature: the residual distribution’s shape differs from a normal shape. Pronounced asymmetry, such as a much more extended upper tail than lower tail, suggests skewness.
• One conspicuous endpoint departure: an extreme residual may be separated from the general pattern. It is a signal to examine the observation and analysis carefully, not proof that the value is wrong.

Interpret the whole pattern, not just whether every point touches a line. With few residuals, the plot has limited ability to reveal subtle departures, and individual points can move noticeably. A mild bend at the ends is different from strong, coherent curvature across much of the plot.

What the Residuals Represent

The one-way ANOVA model can be written as \(y_{ij}=\mu_i+\varepsilon_{ij}\), where \(\mu_i\) is the mean for group \(i\) and \(\varepsilon_{ij}\) is its error. Under the normal-error assumption, the errors are normally distributed with mean zero and common variance \(\sigma^2\). The fitted group mean is \(\bar{y}_i\), so the residual is an error with its group’s average error removed.

This also explains a limitation of the plot: ANOVA residuals are not a collection of independent, exactly standard normal observations. Their values are constrained within each group, because \(\sum_j e_{ij}=0\). Under independent normal errors with variance \(\sigma^2\), each residual has mean zero and variance \(\sigma^2(1-1/n_i)\), where \(n_i\) is the group size. Residuals from the same group are also related. Therefore, a Q-Q plot is an approximate visual diagnostic of the model’s normal shape, not a test that proves the assumption true or false.

The justification for the plot is that, if the model’s errors have a normal shape, their ordered values should roughly track the corresponding ordered normal quantiles, apart from ordinary sample variation and the effects of estimating group means. A pronounced, systematic mismatch is more concerning than a small tail fluctuation. The plot does not assess whether observations are independent, and it is not a check of equal variances across groups; those questions require other information or diagnostics.

Common Mistakes and Practical Interpretation

• Plotting pooled raw responses. Different group means can create a nonnormal-looking mixture even when within-group errors are normal. Use residuals after subtracting each group’s fitted mean.
• Treating every deviation as a failure. Sample quantiles vary, and tail points are especially variable. Look for substantial, systematic curvature rather than demanding that all points lie exactly on the line.
• Calling a Q-Q plot a proof. A nearly straight plot is compatible with normality, but it cannot prove the assumption. A small sample may conceal a departure; a large sample may make small departures visible.
• Using this plot to check every ANOVA condition. A normal Q-Q plot concerns residual shape. It does not establish independence or equal variances, and those assumptions should not be inferred from its appearance.
• Declaring an extreme point an error. An endpoint departure is a prompt to investigate, not an automatic instruction to delete an observation or change the analysis.

The practical question is not whether residuals are perfectly normal, but whether the Q-Q plot shows departures substantial enough to raise concern about the normal-error model. Mild tail irregularities often call for measured interpretation, especially in small samples. Strong skewness or a single point far from the overall pattern calls for closer investigation and consideration of the full study context.