Between- and Within-Group Decomposition in One-Way Layouts

Why One-Way Layouts Have Two Sources of Variation

In a one-way layout, each observation belongs to exactly one of several groups. The variation around the overall sample mean can be separated into two parts: variation among the group means, and variation among observations around their own group means. This distinction is useful because the two parts correspond to different subspaces of the observation vector.

The key fact is geometric. Vectors that are constant within each group form a subspace. The projection of the data vector onto that subspace replaces each observation by its group mean. The part left over consists of within-group deviations. Separating the constant-vector direction from the group-constant subspace then isolates the between-group component.

Definition: Suppose there are $a$ nonempty groups, with $n_j$ observations in group $j$, for $j=1,\ldots,a$, and let $n=\sum_{j=1}^a n_j$. Write the observations as $y_{ij}$, their group means as $\bar y_j=n_j^{-1}\sum_{i=1}^{n_j}y_{ij}$, and the overall mean as $\bar y=n^{-1}\sum_{j=1}^a\sum_{i=1}^{n_j}y_{ij}$. The within-group sum of squares is $\sum_{j=1}^a\sum_{i=1}^{n_j}(y_{ij}-\bar y_j)^2$; the between-group sum of squares is $\sum_{j=1}^a n_j(\bar y_j-\bar y)^2$; and the corrected total sum of squares is $\sum_{j=1}^a\sum_{i=1}^{n_j}(y_{ij}-\bar y)^2$.

The factor $n_j$ in the between-group sum is essential: the deviation of group $j$'s mean from the overall mean represents that deviation for each of the $n_j$ observations in the group. Thus, groups of different sizes contribute different amounts even when their means have the same distance from $\bar y$.

The Sums-of-Squares Identity

For every observation, add and subtract its group mean:

y_{ij}-\bar y=(y_{ij}-\bar y_j)+(\bar y_j-\bar y).

Squaring and summing suggests a within part, a between part, and a cross term. The cross term vanishes because deviations from a group mean sum to zero inside each group.

Theorem: For any observed values in $a$ nonempty groups, $$ \sum_{j=1}^a\sum_{i=1}^{n_j}(y_{ij}-\bar y)^2 = \sum_{j=1}^a n_j(\bar y_j-\bar y)^2 + \sum_{j=1}^a\sum_{i=1}^{n_j}(y_{ij}-\bar y_j)^2. $$ This is an algebraic identity; it requires no distributional assumptions.

Proof. Expand the square after adding and subtracting $\bar y_j$, then sum over all groups and observations:

\begin{aligned} \sum_{j=1}^a\sum_{i=1}^{n_j}(y_{ij}-\bar y)^2 &=\sum_{j=1}^a\sum_{i=1}^{n_j} \left[(y_{ij}-\bar y_j)^2 +2(y_{ij}-\bar y_j)(\bar y_j-\bar y) +(\bar y_j-\bar y)^2\right]\\ &=\sum_{j=1}^a\sum_{i=1}^{n_j}(y_{ij}-\bar y_j)^2 +2\sum_{j=1}^a(\bar y_j-\bar y)\sum_{i=1}^{n_j}(y_{ij}-\bar y_j) +\sum_{j=1}^a n_j(\bar y_j-\bar y)^2. \end{aligned}

For each group $j$, $\sum_{i=1}^{n_j}(y_{ij}-\bar y_j)=\sum_i y_{ij}-n_j\bar y_j=0$, by the definition of $\bar y_j$. Therefore the middle term is zero, leaving exactly the stated identity. $\square$

Worked Example: Verify the Identity With Unequal Group Sizes

Group A has observations $2,4$, group B has $5,7,9$, and group C has $1,3$. Find the within-group, between-group, and corrected total sums of squares, and check that they add up.

The group sizes are $2,3,2$, and the group means are $3,7,2$. The total is $31$, so $\bar y=31/7$.

For the within-group component, the squared deviations are $1+1=2$ in A, $4+0+4=8$ in B, and $1+1=2$ in C. Thus the within-group sum of squares is $2+8+2=12$.

For the between-group component, weight each squared difference by its group size:

\begin{aligned} \sum_j n_j(\bar y_j-\bar y)^2 &=2\left(3-\frac{31}{7}\right)^2 +3\left(7-\frac{31}{7}\right)^2 +2\left(2-\frac{31}{7}\right)^2\\ &=\frac{200+972+578}{49} =\frac{1750}{49} =\frac{250}{7}. \end{aligned}

Independently, the sum of the squared observations is $185$, so the corrected total is $185-31^2/7=334/7$. The components agree: $12+250/7=84/7+250/7=334/7$.

Projection Representation and Ranks

Order the $n$ observations in any fixed way and collect them in the vector $y\in\mathbb R^n$. Let $Z$ be the $n\times a$ group-indicator matrix: its $j$th column has entry $1$ for observations in group $j$ and $0$ otherwise. Because each group is nonempty, the columns are linearly independent, and $Z'Z=\operatorname{diag}(n_1,\ldots,n_a)$.

The orthogonal projection onto the space of vectors constant within each group is $P_G=Z(Z'Z)^{-1}Z'$. Its action replaces each observation by the mean of its group. Let $\mathbf 1$ be the $n$-vector of ones, and let $P_0=\mathbf 1\mathbf 1'/n$ be the projection onto the space of constant vectors. Since every constant vector is also group-constant, the constant-vector space is contained in the group-constant space.

Definition: The between-group projection is $P_B=P_G-P_0$, and the within-group projection is $P_W=I-P_G$. The corrected-total projection is $P_T=I-P_0$. Their quadratic forms give the three sums of squares: $y'P_B y$ is between-group, $y'P_W y$ is within-group, and $y'P_T y$ is corrected total.

These matrices are orthogonal projections. In particular, $P_G$ and $P_0$ are symmetric and idempotent, and the nesting of their ranges gives $P_GP_0=P_0P_G=P_0$. Consequently,

P_B'=P_B,\qquad P_B^2=(P_G-P_0)^2=P_G-P_0=P_B, \qquad P_W'=P_W,\qquad P_W^2=P_W.

The ranges of $P_B$ and $P_W$ are orthogonal: $P_BP_W=(P_G-P_0)(I-P_G)=0$, using $P_G^2=P_G$ and $P_0P_G=P_0$. Also $P_B+P_W=I-P_0=P_T$. Taking quadratic forms therefore yields the projection version of the decomposition:

$$ y'P_Ty=y'P_By+y'P_Wy. $$

To identify the terms explicitly, $P_Gy$ contains $\bar y_j$ repeated $n_j$ times, while $P_0y$ contains $\bar y$ repeated $n$ times. Thus $P_By=P_Gy-P_0y$ has entries $\bar y_j-\bar y$ in group $j$, and $y'P_By=\sum_j n_j(\bar y_j-\bar y)^2$. The residual vector $P_Wy=(I-P_G)y$ has entries $y_{ij}-\bar y_j$, so $y'P_Wy=\sum_j\sum_i(y_{ij}-\bar y_j)^2$. Similarly, $y'P_Ty=\sum_{ij}(y_{ij}-\bar y)^2$.

Theorem: With $a$ nonempty groups and $n$ observations, the ranks of the corrected-total, between-group, and within-group projections are respectively $n-1$, $a-1$, and $n-a$.

Proof. The range of $P_G$ has dimension $a$, since its $a$ indicator columns are linearly independent. The range of $P_0$ has dimension $1$, and it is a subspace of the range of $P_G$. The range of $P_B=P_G-P_0$ is the part of the group-constant space orthogonal to the constant-vector space, so its dimension is $a-1$. The range of $P_W=I-P_G$ is the orthogonal complement of the group-constant space in $\mathbb R^n$, so its dimension is $n-a$. Finally, the range of $P_T=I-P_0$ is the orthogonal complement of the one-dimensional constant-vector space, with dimension $n-1$. These dimensions are the respective ranks. $\square$

The ranks also add as the subspaces do: $(a-1)+(n-a)=n-1$. These are dimensions of the projection spaces, regardless of the observed values. If a particular data vector happens to lie in one of the spaces, its corresponding sum of squares can still be zero even though the projection itself has positive rank.

Worked Example: Read the Projections in a Balanced Layout

Take three groups of size two, with observations $(1,3)$, $(2,4)$, and $(5,7)$, in that order. The group means are $2,3,6$, and the overall mean is $22/6=11/3$.

The projection onto group-constant vectors gives $P_Gy=(2,2,3,3,6,6)'$, while $P_0y=(11/3,\ldots,11/3)'$. Hence $P_By$ has entries $-5/3,-5/3,-2/3,-2/3,7/3,7/3$, and

y'P_By=2\left(\frac{25}{9}+\frac{4}{9}+\frac{49}{9}\right)=\frac{52}{3}.

The within-group residual vector is $P_Wy=(-1,1,-1,1,-1,1)'$, so $y'P_Wy=6$. The corrected total is therefore $52/3+6=70/3$. Checking directly, $\sum y_k^2=104$ and $104-22^2/6=104-242/3=70/3$. Here the ranks are $a-1=2$ for between groups, $n-a=3$ for within groups, and $n-1=5$ for corrected total; indeed $2+3=5$.

Worked Example: A Direct Check With Two Unequal Groups

Suppose group A contains $2,6$, and group B contains $1,3,5$. The means are $4$ and $3$, and the overall mean is $17/5$.

Within A, the squared deviations from $4$ sum to $4+4=8$. Within B, the squared deviations from $3$ sum to $4+0+4=8$. Thus the within-group sum of squares is $16$. The between-group sum is $2(4-17/5)^2+3(3-17/5)^2=2(9/25)+3(4/25)=30/25=6/5$.

Their sum is $16+6/5=86/5$. Directly, $\sum y_k^2=75$, and subtracting $5\bar y^2=5(17/5)^2=289/5$ gives $75-289/5=86/5$, as required. The between and within projection ranks are $1$ and $3$, and the corrected-total rank is $4$.

What the Decomposition Does—and Does Not—Say

The decomposition matters because it separates two distinct kinds of observed variation while keeping an exact accounting of the total. Its projection form makes the geometry explicit, and its ranks describe the dimensions available to each component. In particular, the number of within-group dimensions is $n-a$, not $n-1$: estimating one mean in each of $a$ groups uses $a$ dimensions.

A common mistake is to use the unweighted quantity $\sum_j(\bar y_j-\bar y)^2$ as the between-group sum of squares. That gives each group equal weight, regardless of how many observations it contains, and generally breaks the identity. The correct contribution is $n_j(\bar y_j-\bar y)^2$ for group $j$. Another common mistake is to infer independence merely from the algebraic decomposition. The identity and orthogonality are deterministic statements about projections; distributional assumptions are needed for probabilistic conclusions.

For example, the identity holds for any numerical observations, including data that are not normally distributed. It does not, by itself, say how either component is distributed or whether the components are independent. Those are separate questions. The appropriate conclusion here is exact and limited: the corrected-total projection splits into two orthogonal projections, with ranks $n-1$, $a-1$, and $n-a$.

Check Your Understanding

Try to answer each question by identifying the relevant means, projection spaces, or ranks.

Why is each squared group-mean deviation multiplied by $n_j$ in the between-group sum of squares?
For four nonempty groups and $n=18$ observations, what are the ranks of the between-group, within-group, and corrected-total projections?
Which property of the within-group deviations makes the cross term in the direct sums-of-squares proof vanish?
What does $P_Gy$ contain, and what does $(I-P_G)y$ contain?
Does the sums-of-squares identity require normality, and does it establish independence? Explain the distinction.

Between- and Within-Group Decomposition in One-Way Layouts

What You'll Learn