ANOVA Residuals Versus Raw Data for Diagnostics

Why the Choice of Data Matters

In one-way ANOVA, observations from different groups may have different means. A plot of all the raw observations therefore combines two features: variation among group means and variation of observations around their own group means. Those features answer different questions. Group means describe how the groups differ; residuals show how far individual observations lie from the means predicted by the model.

This distinction matters when checking whether an ANOVA model is a reasonable description of the data. A large gap between two groups can be a real feature of the fitted model, not evidence that the residuals have a pattern. Conversely, raw observations can make it difficult to see a pattern in the variation around the fitted group means. A diagnostic display should match the part of the model being examined.

The One-Way ANOVA Model and Its Residuals

Suppose there are \(k\) groups, with \(n_j\) observations in group \(j\). Write observation \(i\) in group \(j\) as \(Y_{ij}\). The one-way ANOVA model represents each observation as its group mean plus an error:

Here, \(\mu_j\) is the population mean for group \(j\), and \(\varepsilon_{ij}\) is the error for that observation: its departure from the group’s population mean. For the usual one-way ANOVA model, the errors have mean zero, a common variance \(\sigma^2\), are independent, and are normally distributed for the exact F-test. Equivalently, the errors are independent and each has distribution \(N(0,\sigma^2)\). These are model assumptions; the observed residuals are used to look for signs that the model may not describe the data well.

The population means are not known, so the model’s fitted value for an observation in group \(j\) is the sample mean of that group, \(\bar{x}_j\). The residual is the observed value minus its fitted value.

A positive residual means the observation is above its own group mean; a negative residual means it is below. Residuals have the same units as the response variable. The raw observation retains the group’s location on the response scale, while the residual expresses only the observation’s departure from its fitted group mean.

Why Residuals Are Centered Within Groups

Residuals from the fitted one-way ANOVA model sum to zero within each group. This is a mathematical consequence of using the sample group mean as the fitted value, not a diagnostic discovery.

because \(\bar{x}_j=(1/n_j)\sum_{i=1}^{n_j}Y_{ij}\). The fitted group mean also minimizes the sum of squared deviations from a single fitted value for that group. To see why, take any proposed fitted value \(c\). The squared deviations can be written as

To obtain this identity, write \(Y_{ij}-c=(Y_{ij}-\bar{x}_j)+(\bar{x}_j-c)\), square and sum. The cross-product is zero because \(\sum_i(Y_{ij}-\bar{x}_j)=0\). Since \(n_j(\bar{x}_j-c)^2\) cannot be negative, the squared deviations are smallest when \(c=\bar{x}_j\). Thus the ANOVA residuals are the departures from the group values that minimize within-group squared error. Their squared sum, \(\sum_j\sum_i e_{ij}^2\), is the within-group sum of squares, also called the residual sum of squares.

Because each group’s residuals are forced to sum to zero, they are not independent of one another in the way the model errors are assumed to be. Residuals are model-based estimates of errors, with constraints introduced by estimating group means. Their plots are useful diagnostic tools, but they should not be treated as direct displays of independent, observed model errors.

Raw-Data Displays and Residual Displays

Raw observations are essential for describing the actual responses. A dotplot or other suitable display organized by group can show where the groups are located, their observed ranges, and unusual values. These are useful features to see before or alongside the model. But if the question is whether deviations around the fitted group means show a systematic pattern, raw observations include group location as well as those deviations.

A residual display removes the fitted group means from the observations. For one-way ANOVA, useful basic displays include residuals plotted by group and residuals plotted in the order observations were collected. A plot against fitted values has one fitted value per group, so observations appear in vertical stacks. In such a plot, different horizontal positions correspond to different fitted group means; they do not, on their own, indicate a residual trend. The residual values within each stack are the deviations from that group’s mean.

Residual displays can suggest that the model leaves structure unexplained—for example, a systematic sequence in collection order or visibly different residual spreads among groups. The appropriate interpretation depends on the plot and on the study design. A visual display does not prove that an assumption is satisfied or violated, and it cannot establish independence without information about how the observations were collected. Detailed assessment of particular patterns uses dedicated diagnostic methods.

Common Mistake: Diagnosing the Raw Group Separation

A common error is to inspect raw observations from groups with different means, notice that the values form separate clusters, and conclude that the ANOVA residuals show a systematic pattern. That reasoning confuses the fitted group effects with the departures left after fitting those effects.

The correction is to identify what is plotted. If the vertical coordinate is the raw response \(Y_{ij}\), the values include both the group mean and the observation’s departure from it. For a residual diagnostic, calculate or plot \(e_{ij}=Y_{ij}-\bar{x}_j\), while retaining group labels or collection order as needed to interpret the display. At the same time, do not make the opposite mistake of assuming that residuals must look satisfactory because their group sums equal zero: that centering happens automatically and does not establish that the model assumptions hold.