ANOVA Decomposition For Random Groups

Why Partition Variation?

In a one-way random-effects study, observations come from several sampled groups or levels of a factor. Some observations differ because they belong to different groups; others differ even within the same group. ANOVA separates these two sources of observed variation. This partition is useful for describing the data and for connecting the observed group pattern to the random-effects model.

The partition is an exact identity for any observed data, not a test of whether group effects are present. Its two parts are sums of squared deviations, measured in squared units of the response. To interpret them, first specify the model that describes how group effects and observation-level errors arise.

The Random-Effects Model and the ANOVA Quantities

Suppose $a$ groups are sampled from a population of groups. Group $i$ contains $n_i$ observations, and the total number of observations is $N=\sum_{i=1}^{a}n_i$. Write $Y_{ij}$ for the response of observation $j$ in group $i$.

Model: The one-way random-effects model is $$ Y_{ij}=\mu+A_i+\varepsilon_{ij}. $$ Here $\mu$ is the population mean, $A_i$ is the random deviation for sampled group $i$, and $\varepsilon_{ij}$ is the observation-level error. The standard model assumes $A_i$ are independent with mean 0 and variance $\sigma_A^2$, and $\varepsilon_{ij}$ are independent with mean 0 and variance $\sigma^2$. The errors are independent of the group effects. For the usual normal-model analysis, assume $A_i\sim N(0,\sigma_A^2)$ and $\varepsilon_{ij}\sim N(0,\sigma^2)$. The group effects are shared by observations in the same group; observations from different groups have independent group effects.

These assumptions describe a population of possible groups and the variation of observations around their group values. The algebraic decomposition below does not require normality: it holds for every set of observed responses. Normality is an assumption of the usual model-based inference, not a condition needed to calculate the sums of squares.

Let $\bar{Y}_{i\cdot}$ be the sample mean in group $i$, and let $\bar{Y}_{\cdot\cdot}$ be the mean of all $N$ observations. The dot indicates that the corresponding index has been averaged over. When group sizes differ, the overall mean is a size-weighted average of group means.

Definition: The total sum of squares measures variation of observations around the overall mean. The within-group sum of squares measures variation of observations around their own group means. The between-group sum of squares measures variation of group means around the overall mean, weighted by group size. $$ SS_{\text{Total}}=\sum_{i=1}^{a}\sum_{j=1}^{n_i}(Y_{ij}-\bar{Y}_{\cdot\cdot})^2,\qquad SS_{\text{Within}}=\sum_{i=1}^{a}\sum_{j=1}^{n_i}(Y_{ij}-\bar{Y}_{i\cdot})^2,\qquad SS_{\text{Between}}=\sum_{i=1}^{a}n_i(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})^2. $$

The Exact Decomposition

Each observation’s deviation from the overall mean can be split into two deviations: its deviation from its group mean, and the group mean’s deviation from the overall mean. Squaring and summing these pieces gives the ANOVA decomposition.

Theorem: For any observed data from $a$ groups, $$ SS_{\text{Total}}=SS_{\text{Within}}+SS_{\text{Between}}. $$

To see why, write $Y_{ij}-\bar{Y}_{\cdot\cdot}=(Y_{ij}-\bar{Y}_{i\cdot})+(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})$. After squaring and summing, the cross-product is zero because the deviations from each group mean add to zero.

\begin{aligned} SS_{\text{Total}} &=\sum_i\sum_j\left[(Y_{ij}-\bar{Y}_{i\cdot})+(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})\right]^2\\ &=SS_{\text{Within}}+SS_{\text{Between}} +2\sum_i(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot}) \sum_j(Y_{ij}-\bar{Y}_{i\cdot})\\ &=SS_{\text{Within}}+SS_{\text{Between}}, \end{aligned}

The last equality follows from $\sum_{j=1}^{n_i}(Y_{ij}-\bar{Y}_{i\cdot})=0$ for every group $i$. The decomposition therefore does not depend on whether the groups have equal sample sizes.

Degrees of Freedom

Degrees of freedom count the independent pieces of information available for each sum of squares. The total deviations from the overall mean sum to zero, so only $N-1$ of the $N$ deviations can vary independently. Thus $SS_{\text{Total}}$ has $N-1$ degrees of freedom.

Within group $i$, the $n_i$ deviations from that group’s mean sum to zero, leaving $n_i-1$ degrees of freedom. Across all groups, the within-group degrees of freedom sum to $\sum_i(n_i-1)=N-a$. The $a$ group means have one weighted constraint: their deviations from the overall mean satisfy $\sum_i n_i(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})=0$. Consequently, the between-group degrees of freedom are $a-1$, and $(a-1)+(N-a)=N-1$, as required by the decomposition.

Source	Sum of squares	Degrees of freedom
Between groups	$\sum_i n_i(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})^2$	$a-1$
Within groups	$\sum_i\sum_j(Y_{ij}-\bar{Y}_{i\cdot})^2$	$N-a$
Total	$\sum_i\sum_j(Y_{ij}-\bar{Y}_{\cdot\cdot})^2$	$N-1$

Dividing a sum of squares by its degrees of freedom gives its mean square: $MS_{\text{Between}}=SS_{\text{Between}}/(a-1)$ and $MS_{\text{Within}}=SS_{\text{Within}}/(N-a)$. These are convenient scale-adjusted summaries. Their relationship to the model’s variance components requires additional results; the sums of squares alone should not be treated as variance-component estimates.

Worked Decompositions

Worked Example: A Balanced Set of Groups

Suppose a hypothetical experiment records a response in three randomly sampled groups, with three observations in each group. The observations are $4,5,6$ in group 1, $7,8,9$ in group 2, and $3,4,5$ in group 3. The response is measured in points. Calculate the decomposition and degrees of freedom.

The group means are $5$, $8$, and $4$. The overall mean is $\bar{Y}_{\cdot\cdot}=(15+24+12)/9=51/9=17/3$ points.

Within each group, the squared deviations from its mean are $1,0,1$, so each group contributes $2$ points squared. Therefore, $SS_{\text{Within}}=2+2+2=6$ points squared.

For the between-group sum of squares, each group has size 3: $SS_{\text{Between}}=3[(5-17/3)^2+(8-17/3)^2+(4-17/3)^2] =3[(4+49+25)/9]=26$ points squared.

Thus $SS_{\text{Total}}=6+26=32$ points squared. The degrees of freedom are $a-1=2$ between groups, $N-a=9-3=6$ within groups, and $N-1=8$ total; $2+6=8$. The mean squares are $26/2=13$ and $6/6=1$ points squared, respectively. These calculations describe the observed partition; by themselves, they do not establish that group effects vary in the population.

Worked Example: Unequal Group Sizes

Consider hypothetical observations of a response measured in minutes: group A has values $1,3$, and group B has values $4,5,6$. The group sizes differ, so use the weighted overall mean and the group-size weights in the between-group sum of squares.

The means are $\bar{Y}_{A\cdot}=2$ and $\bar{Y}_{B\cdot}=5$ minutes. The overall mean is $(1+3+4+5+6)/5=19/5=3.8$ minutes. The within-group sum of squares is $(1-2)^2+(3-2)^2+(4-5)^2+(5-5)^2+(6-5)^2=2+2=4$ minutes squared.

The between-group sum of squares uses each group’s sample size: $SS_{\text{Between}}=2(2-3.8)^2+3(5-3.8)^2=2(3.24)+3(1.44)=6.48+4.32=10.8$ minutes squared.

The total is $SS_{\text{Total}}=4+10.8=14.8$ minutes squared. Here $a=2$ and $N=5$, so the degrees of freedom are 1 between groups, 3 within groups, and 4 total. The counts add correctly: $1+3=4$. Weighting the group means is essential: treating the two group means as equally representative would not give the overall mean of the five observations.

What the Partition Means Under the Model

Under the random-effects model, observations in the same group share $A_i$, while their errors $\varepsilon_{ij}$ differ. The within-group deviations remove the shared group effect exactly: $Y_{ij}-\bar{Y}_{i\cdot}=\varepsilon_{ij}-\bar{\varepsilon}_{i\cdot}$, where $\bar{\varepsilon}_{i\cdot}$ is the average error in group $i$. Thus the within-group sum of squares describes variation among observations after centering each group separately.

The between-group sum of squares describes variation in the observed group means. Those means reflect both the sampled group effects and the errors averaged within each group. In particular, a group mean is not just its random effect: $\bar{Y}_{i\cdot}=\mu+A_i+\bar{\varepsilon}_{i\cdot}$. The difference between a group mean and the overall mean therefore contains sampling noise as well as group-to-group variation.

Worked Example: Group Differences Include Sampling Error

Suppose three randomly sampled groups each have two hypothetical responses, in arbitrary response units: group 1 has $2,4$, group 2 has $5,7$, and group 3 has $3,5$. Calculate the partition, then interpret what the observed group-mean differences do and do not show.

The group means are $3,6,4$, and the overall mean is $(6+12+8)/6=26/6=13/3$. Each group has within-group squared deviations $1+1=2$, so $SS_{\text{Within}}=6$ squared response units.

The between-group sum of squares is $SS_{\text{Between}}=2[(3-13/3)^2+(6-13/3)^2+(4-13/3)^2] =2[(16+25+1)/9]=28/3$, or about $9.333$ squared response units. Hence $SS_{\text{Total}}=6+28/3=46/3$, or about $15.33$ squared response units. The degrees of freedom are 2 between, 3 within, and 5 total.

The group means are visibly different in this sample, and that pattern contributes $28/3$ to the between-group sum of squares. Under the model, however, each observed mean also includes an average of its group’s errors. The decomposition alone cannot separate those error averages from the realized group effects or establish a population-level conclusion.

Common Mistake and Key Takeaway

Mistake: “The between-group sum of squares measures only the random group effects, so a large value proves that the population groups differ.”
Correction: The between-group sum of squares is calculated from observed group means. Those means contain both random group effects and averaged observation errors. The decomposition is an exact accounting identity, not by itself an inference about the variance components.
Mistake: “For unequal group sizes, each group mean gets the same weight in the between-group sum of squares.”
Correction: The correct contribution is $n_i(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})^2$. The weight reflects how many observations contribute to that group mean.

Key takeaway: Total observed variation divides exactly into within-group and between-group sums of squares. Their degrees of freedom are $N-a$ and $a-1$, which add to the total $N-1$. Under random effects, these pieces describe different patterns in the data, but neither sum of squares is itself a pure measure of a single variance component.

Check Your Understanding

Try these questions using the definitions and identities above.

There are four groups with five observations each. Find the degrees of freedom for between groups, within groups, and total.
Why does the identity $SS_{\text{Total}}=SS_{\text{Within}}+SS_{\text{Between}}$ hold even when group sizes differ?
For unequal group sizes, what weight is applied to group $i$’s squared deviation from the overall mean in $SS_{\text{Between}}$?
Under the random-effects model, why is an observed group mean not identical to its group effect plus the population mean?
Does a large observed between-group sum of squares, by itself, prove that the population variance of group effects is large? Explain.

Source	Sum of squares	Degrees of freedom
Between groups	\(\sum_i n_i(\bar{Y}_{i\cdot}-\bar{Y}_{\cdot\cdot})^2\)	\(a-1\)
Within groups	\(\sum_i\sum_j(Y_{ij}-\bar{Y}_{i\cdot})^2\)	\(N-a\)
Total	\(\sum_i\sum_j(Y_{ij}-\bar{Y}_{\cdot\cdot})^2\)	\(N-1\)