Introduction
A one-way ANOVA analysis is not only a comparison of treatment means. It also gives estimates of those means, their differences, and the uncertainty in each estimate. Working through one complete example makes the connections among these quantities visible, especially when the treatment groups have different sample sizes.
Suppose a hypothetical experiment compares three treatments, A, B, and C, using a quantitative outcome measured in mg/L. The groups have unequal sizes. We will calculate their sample means, treatment effects, pooled residual variance, standard errors, and confidence intervals, then consider whether the assumptions supporting those calculations are plausible.
Model and Hypothetical Data
For a one-way fixed-effects model, let \(Y_{ij}\) be the outcome for experimental unit \(j\) in treatment group \(i\). The model is \(Y_{ij}=\mu_i+\epsilon_{ij}\), where \(\mu_i\) is the mean outcome for group \(i\). The standard assumptions are that errors are independent, normally distributed with mean zero, and have a common variance \(\sigma^2\) across groups. In this example, independence must be supported by the experimental design; the numerical calculations alone cannot establish it.
The following values are hypothetical. Imagine that 15 independent experimental units were randomly assigned to the three treatments, with one outcome recorded for each unit.
| Treatment | Observed outcomes (mg/L) | Group size |
|---|---|---|
| A | 18, 20, 19, 21 | 4 |
| B | 22, 24, 23, 25, 21 | 5 |
| C | 25, 27, 26, 28, 24, 26 | 6 |
Group Means and Treatment Effects
The sample mean in each treatment group estimates that group’s treatment mean. With unequal group sizes, it is important to distinguish the unweighted average of the group means from the overall sample mean. The unweighted average is useful for expressing treatment effects under a sum-to-zero constraint; the overall sample mean weights each observed unit equally.
Worked Example: Estimating Group Means and Treatment Effects
For treatment A, the sum of the four outcomes is \(18+20+19+21=78\), so \(\bar{x}_A=78/4=19.5\) mg/L. For B, \(\bar{x}_B=(22+24+23+25+21)/5=115/5=23.0\) mg/L. For C, \(\bar{x}_C=(25+27+26+28+24+26)/6=156/6=26.0\) mg/L.
The unweighted average of these three estimated group means is \[ \frac{19.5+23.0+26.0}{3}=\frac{68.5}{3}=22.8333\text{ mg/L}. \] Under the sum-to-zero treatment-effect convention, each estimated treatment effect is its group mean minus this unweighted average. Thus, \[ \hat{\tau}_A=19.5-22.8333=-3.3333,\quad \hat{\tau}_B=23.0-22.8333=0.1667,\quad \hat{\tau}_C=26.0-22.8333=3.1667\text{ mg/L}. \] The estimates sum to zero, apart from rounding. They describe each group mean’s deviation from the unweighted average of the three group means.
By contrast, the overall sample mean is \((78+115+156)/15=349/15=23.2667\) mg/L. It is a weighted average of the group means, with weights proportional to group sizes. It is not the center used in the sum-to-zero treatment-effect estimates here.
Pooled Residual Variance and Standard Errors
The pooled error variance \(s_p^2\) combines the within-group variation, not the variation among the group means. For each group, calculate the sum of squared deviations from that group’s own mean. The group-specific sample variances may differ by chance; pooling uses their combined within-group sums of squares and the combined degrees of freedom.
Worked Example: Pooling Within-Group Variation
For A, the deviations from 19.5 are \(-1.5, 0.5, -0.5, 1.5\), giving a within-group sum of squares of \(2.25+0.25+0.25+2.25=5\). For B, deviations from 23.0 give \(1+1+0+4+4=10\). For C, deviations from 26.0 give \(1+1+0+4+4+0=10\). Therefore, the total within-group sum of squares is \(5+10+10=25\).
There are \(N=4+5+6=15\) observations and \(a=3\) groups, so the residual degrees of freedom are \(N-a=12\). The pooled error variance and its square root are \[ s_p^2=\frac{25}{12}=2.0833\;(\text{mg/L})^2,\qquad s_p=\sqrt{2.0833}=1.443\text{ mg/L}. \]
For context, the separate sample variances are \(5/3=1.667\), \(10/4=2.500\), and \(10/5=2.000\;(\text{mg/L})^2\) for A, B, and C. The pooled estimate is a common-variance estimate based on all three groups; it does not say that the observed group variances are identical.
A group mean averages \(n_i\) independent observations. Under the model, its variance is \(\sigma^2/n_i\), so replacing \(\sigma^2\) with the pooled estimate gives \(SE(\bar{x}_i)=\sqrt{s_p^2/n_i}\). For a difference between two independent group means, the variances add, giving \(SE(\bar{x}_i-\bar{x}_j)=\sqrt{s_p^2(1/n_i+1/n_j)}\). These formulas show why unequal group sizes lead to different standard errors.
Confidence Intervals for Means and Differences
For these calculations, use a 95% \(t\) critical value with 12 residual degrees of freedom, \(t^*=2.179\). Each interval has the form estimate \(\pm t^*\times SE\). Under the model assumptions, this method has 95% coverage in repeated sampling for the corresponding parameter. The intervals below are individual, not simultaneous, intervals.
Worked Example: A Group Mean and a Pairwise Difference
For group A, the standard error is \[ SE(\bar{x}_A)=\sqrt{\frac{2.0833}{4}}=\sqrt{0.5208}=0.7217\text{ mg/L}. \] The 95% confidence interval for the mean outcome in group A is \[ 19.5\pm 2.179(0.7217)=19.5\pm1.573, \] or approximately 17.93 to 21.07 mg/L.
For the difference between A and B, the estimated difference is \(19.5-23.0=-3.5\) mg/L. Its standard error is \[ SE(\bar{x}_A-\bar{x}_B)=\sqrt{2.0833\left(\frac{1}{4}+\frac{1}{5}\right)} =\sqrt{0.9375}=0.9682\text{ mg/L}. \] The 95% interval is \[ -3.5\pm2.179(0.9682)=-3.5\pm2.110, \] or approximately \(-5.61\) to \(-1.39\) mg/L. In context, the interval estimates the population mean outcome for A minus the population mean outcome for B. Its negative values indicate that the estimated mean for A is lower; the interval does not imply that treatment assignment caused an outcome change unless the study design supports that conclusion.
For comparison, the standard errors of the B and C means are \(\sqrt{2.0833/5}=0.6455\) mg/L and \(\sqrt{2.0833/6}=0.5893\) mg/L. Their respective 95% intervals are \(23.0\pm2.179(0.6455)\), or 21.59 to 24.41 mg/L, and \(26.0\pm2.179(0.5893)\), or 24.72 to 27.28 mg/L. The larger group C has the smaller standard error, because its mean averages more observations.
Uncertainty for Sum-to-Zero Treatment Effects
Treatment effects under the sum-to-zero constraint are estimated using all three sample means, so their standard errors are not simply the standard errors of individual group means. For \(\hat{\tau}_i=\bar{x}_i-\frac{1}{a}\sum_j\bar{x}_j\), independence of the group means gives \[ SE(\hat{\tau}_i)=s_p\sqrt{\frac{(1-1/a)^2}{n_i}+\frac{1}{a^2}\sum_{j\ne i}\frac{1}{n_j}}. \] This expression accounts for the uncertainty in both the group mean and the estimated unweighted center.
Worked Example: A Confidence Interval for a Treatment Effect
For treatment A, \(a=3\), \(n_A=4\), and the other group sizes are 5 and 6. The estimated variance factor is \[ \frac{(1-1/3)^2}{4}+\frac{1}{9}\left(\frac{1}{5}+\frac{1}{6}\right) =\frac{1}{9}+\frac{11}{270} =\frac{41}{270}=0.1519. \] Thus \(SE(\hat{\tau}_A)=\sqrt{2.0833(0.1519)}=0.5625\) mg/L. Using \(\hat{\tau}_A=-3.3333\) mg/L, its 95% confidence interval is \[ -3.3333\pm2.179(0.5625)=-3.3333\pm1.226, \] or approximately \(-4.56\) to \(-2.11\) mg/L. This interval estimates treatment A’s mean deviation from the unweighted average of the three treatment means, not its difference from treatment B.
Checking the Assumptions in Context
The calculations are only as reliable as the model assumptions. In the hypothetical experiment, random assignment of independent units and one measurement per unit would support independence. If several measurements came from the same unit, or if units were grouped by a shared source, treating all observations as independent could understate uncertainty. Independence is principally a design question, not something established by a residual plot.
Normality concerns the errors within groups, not whether the combined raw outcomes look normal. A residual is an observed outcome minus its group mean. Here the residuals in each group are small and show no obvious extreme value, but with only four to six observations per group, a normal Q-Q plot has little power to reveal departures. The data alone do not prove normality.
Equal variance means the error variance is the same for all treatments. The observed group variances, 1.667, 2.500, and 2.000 \((\text{mg/L})^2\), are not identical, as is expected in samples. Their largest-to-smallest ratio is \(2.500/1.667=1.50\). That comparison is not proof that the population variances are equal; it is one diagnostic to consider alongside residual plots, sample sizes, and subject-matter knowledge. The pooled estimate and the usual intervals rely on the common-variance model being reasonable.
Confirm that experimental units are independent and that each unit contributes one observation to one treatment group.
Look for unusual outliers or strong departures from a roughly normal pattern; small groups limit what a diagnostic plot can establish.
Use group residual plots and sample variances to judge whether a common error variance is plausible, rather than expecting the sample variances to match exactly.
Common Mistakes and Why the Estimates Matter
- Using the wrong center for treatment effects. With unequal group sizes, the overall sample mean is generally not the unweighted average of the group means. Under the sum-to-zero constraint, subtract the unweighted average of group means.
- Subtracting means incorrectly. The A-minus-B estimate is \(19.5-23.0=-3.5\) mg/L, not \(-3.0\) mg/L. Keep the order of subtraction explicit, because reversing it changes the sign.
- Pooling between-group differences into the error variance. The pooled error variance uses deviations from each group’s own mean. Variation among the group means reflects treatment differences as well as sampling variation, so it is not the within-group error estimate.
- Claiming an assumption has been proven. Similar sample variances or a roughly straight Q-Q plot do not establish the assumptions, especially with small groups. State what the design and diagnostics support, and what remains uncertain.
Estimation gives a practical account of both the size of the group differences and their precision. Unequal group sizes matter because they affect standard errors; the pooled variance matters because it supplies the common estimate of residual variation. Interpreting those estimates alongside the design and diagnostics is essential: a numerical interval cannot repair dependence or severe model misspecification.
Check Your Understanding
Use the calculations and distinctions in this example to answer the following questions.
- Why is the unweighted average of the three sample means different from the overall sample mean in this dataset?
- Calculate the standard error of the estimated mean for treatment B using the pooled error variance.
- What parameter is estimated by the A-minus-B difference, and why does its sign matter?
- Why are the within-group sums of squares used to calculate the pooled error variance?
- What evidence in this example can inform an equal-variance assessment, and why does it not prove that the population variances are equal?