Additive Sums-of-Squares Decomposition in Balanced Two-Way Layouts

Why Balance Gives a Clean Decomposition

In a two-way layout, each observation is indexed by a row level and a column level. When every row-column combination has the same number of replicates, the grand mean, row differences, and column differences can be separated by orthogonal projections. The remaining component is what the additive row-plus-column fit does not explain.

This is a geometric decomposition: it holds for any observed data array, without a normality assumption. Balance is essential to the simple formulas below because it gives every row and every column the same replication weight. The residual component is defined relative to the additive fit; its interpretation in factorial designs is a separate question.

Layout, Means, and Additive Fit

Suppose there are $a$ row levels, $b$ column levels, and $n$ observations in each row-column combination. Write the observations as $y_{ijk}$, where $i=1,\ldots,a$, $j=1,\ldots,b$, and $k=1,\ldots,n$. The total number of observations is $N=abn$.

Definition: The cell mean, row mean, column mean, and grand mean are $$ \bar y_{ij\cdot}=\frac{1}{n}\sum_{k=1}^n y_{ijk},\qquad \bar y_{i\cdot\cdot}=\frac{1}{bn}\sum_{j=1}^b\sum_{k=1}^n y_{ijk}, $$ $$ \bar y_{\cdot j\cdot}=\frac{1}{an}\sum_{i=1}^a\sum_{k=1}^n y_{ijk},\qquad \bar y_{\cdot\cdot\cdot}=\frac{1}{abn}\sum_{i=1}^a\sum_{j=1}^b\sum_{k=1}^n y_{ijk}. $$ The additive fitted value for observation $ijk$ is $\hat y_{ijk}=\bar y_{i\cdot\cdot}+\bar y_{\cdot j\cdot}-\bar y_{\cdot\cdot\cdot}$, and its residual is $e_{ijk}=y_{ijk}-\hat y_{ijk}$.

Subtracting the grand mean from the row mean gives the estimated row deviation; subtracting it from the column mean gives the estimated column deviation. The fitted value adds these deviations to the grand mean, counting that mean only once. Balance ensures this fit is the orthogonal projection onto the space of additive fitted arrays.

The Four Orthogonal Components

View the array of observations as a vector in $\mathbb R^N$, with inner product equal to the sum of entrywise products. Let $U$ be the subspace of constant arrays. Let $R$ contain arrays that are constant across columns and replicates within each row, with row values summing to zero. Let $C$ contain arrays that are constant across rows and replicates within each column, with column values summing to zero. The additive model space is $U+R+C$.

The grand-mean projection has entries $\bar y_{\cdot\cdot\cdot}$; the row projection has entries $\bar y_{i\cdot\cdot}-\bar y_{\cdot\cdot\cdot}$; and the column projection has entries $\bar y_{\cdot j\cdot}-\bar y_{\cdot\cdot\cdot}$. The fourth component is the residual array $e$. These components are mutually orthogonal.

Theorem: For a balanced $a$-by-$b$ layout with $n$ replicates per cell, the observation vector has the orthogonal decomposition into grand-mean, row, column, and additive residual components. Their ranks are respectively $1$, $a-1$, $b-1$, and $N-a-b+1$. Consequently, $$ \sum_{i,j,k}(y_{ijk}-\bar y_{\cdot\cdot\cdot})^2 =SS_{\mathrm{row}}+SS_{\mathrm{column}}+SS_{\mathrm{res}}, $$ where $$ SS_{\mathrm{row}}=bn\sum_{i=1}^a(\bar y_{i\cdot\cdot}-\bar y_{\cdot\cdot\cdot})^2,\qquad SS_{\mathrm{column}}=an\sum_{j=1}^b(\bar y_{\cdot j\cdot}-\bar y_{\cdot\cdot\cdot})^2, $$ $$ SS_{\mathrm{res}}=\sum_{i,j,k}e_{ijk}^2. $$

Proof of orthogonality and ranks. A constant array is orthogonal to every array in $R$ or $C$, because the row or column deviations sum to zero. For $r\in R$ with row values $r_i$, and $c\in C$ with column values $c_j$, their inner product is $$ \langle r,c\rangle=n\sum_{i=1}^a\sum_{j=1}^b r_i c_j =n\left(\sum_i r_i\right)\left(\sum_j c_j\right)=0. $$ Thus $U$, $R$, and $C$ are mutually orthogonal. The row residuals satisfy $\sum_{j,k}e_{ijk}=0$ for each $i$: the row sum of $y$ is $bn\bar y_{i\cdot\cdot}$, the row sum of the fitted values is $bn(\bar y_{i\cdot\cdot}-\bar y_{\cdot\cdot\cdot})+bn\bar y_{\cdot\cdot\cdot}$, and these are equal. Similarly, $\sum_{i,k}e_{ijk}=0$ for each $j$. These zero sums make $e$ orthogonal to $U$, $R$, and $C$.

There is one free dimension in $U$. The $a$ row values in $R$ obey one sum-to-zero constraint, so $\dim(R)=a-1$; likewise, $\dim(C)=b-1$. Their orthogonal sum therefore has dimension $1+(a-1)+(b-1)=a+b-1$. Its orthogonal complement in $\mathbb R^N$ has dimension $N-a-b+1$, which is the residual rank. Orthogonality implies that squared lengths add. Repeating each row deviation across $bn$ observations and each column deviation across $an$ observations gives the stated formulas. $\square$

The displayed identity is corrected: it measures variation around the grand mean. If the uncorrected sum of squares $\sum y_{ijk}^2$ is needed, the grand-mean component must also be included: $$ \sum_{i,j,k} y_{ijk}^2 =N\bar y_{\cdot\cdot\cdot}^{\,2} +SS_{\mathrm{row}}+SS_{\mathrm{column}}+SS_{\mathrm{res}}. $$

Worked Decompositions

Worked Example: A Replicated Two-by-Two Layout

Consider two rows and two columns, with two observations per cell: row 1 has $(8,10)$ and $(12,14)$; row 2 has $(10,12)$ and $(14,16)$. The cell means are $9,13,11,15$. The row means are $11,13$, the column means are $10,14$, and the grand mean is $12$.

The row sum of squares is $SS_{\mathrm{row}}=bn[(-1)^2+(1)^2]=4(2)=8$. The column sum of squares is $SS_{\mathrm{column}}=an[(-2)^2+(2)^2]=4(8)=32$. The additive fitted cell values are $9,13$ in row 1 and $11,15$ in row 2. Each of the eight observations differs from its fitted value by $1$ or $-1$, so $SS_{\mathrm{res}}=8$.

The corrected total is $8+32+8=48$. Directly, the deviations from 12 are $-4,-2,0,2,-2,0,2,4$, whose squared sum is also $48$. Here $N=8$, and the ranks are $1,1,1,5$, which sum to 8.

Worked Example: An Unreplicated Three-by-Two Layout

Take the array with rows $(4,6)$, $(5,9)$, and $(9,10)$. Here $a=3$, $b=2$, and $n=1$. The row means are $5,7,9.5$, the column means are $6,25/3$, and the grand mean is $43/6$.

The row deviations from the grand mean are $-13/6,-1/6,14/6$. Thus $SS_{\mathrm{row}}=2[(169+1+196)/36]=61/3$. The column deviations are $-7/6,7/6$, so $SS_{\mathrm{column}}=3[(49+49)/36]=49/6$.

For a direct residual check, the fitted values, row by row, are $(23/6,37/6)$, $(35/6,49/6)$, and $(50/6,64/6)$. The residuals are $(1/6,-1/6)$, $(-5/6,5/6)$, and $(4/6,-4/6)$. Therefore $SS_{\mathrm{res}}=2(1+25+16)/36=7/3$. The corrected total is $185/6$: the raw squared sum is $339$, and subtracting $6(43/6)^2=1849/6$ gives $185/6$. The components sum to $122/6+49/6+14/6=185/6$. The ranks are $1,2,1,2$, summing to $N=6$.

Worked Example: Checking an Additive Fit and Its Degrees of Freedom

For the unreplicated array with rows $(1,2,3)$ and $(4,6,8)$, the row means are $2,6$, the column means are $5/2,4,11/2$, and the grand mean is $4$. The row sum of squares is $3[(-2)^2+2^2]=24$, and the column sum of squares is $2[(-3/2)^2+0^2+(3/2)^2]=9$.

The additive fitted values are $(1/2,2,7/2)$ and $(9/2,6,15/2)$. Subtracting these from the observations gives residuals $(1/2,0,-1/2)$ and $(-1/2,0,1/2)$, so $SS_{\mathrm{res}}=1$. The corrected total is $34$: the raw squared sum is $130$, and $6(4)^2=96$, leaving $130-96=34=24+9+1$. The component ranks are $1,1,2,2$, totaling 6. In particular, the residual component has two dimensions even though there is only one observation per cell.

Common Pitfall and Why the Decomposition Matters

A frequent error is to treat the residual degrees of freedom as $N-ab$, the number of within-cell degrees of freedom. That count applies to variation around separate cell means, not to residuals from the additive row-plus-column fit. The additive model uses a space of dimension $a+b-1$, so its residual rank is $N-(a+b-1)=N-a-b+1$. With one observation per cell, this rank is generally positive.

Another error is to add row and column sums of squares to the uncorrected total without accounting for the grand mean. The corrected total is centered at $\bar y_{\cdot\cdot\cdot}$; the uncorrected total includes the additional squared length $N\bar y_{\cdot\cdot\cdot}^{\,2}$. Keeping the centering explicit prevents the two identities from being confused.

The projection structure matters because ranks are the dimensions of the spaces being measured, and orthogonality is what permits the squared lengths to add. Under Gaussian spherical errors, orthogonal projection components also connect to the distribution and independence results developed through Cochran’s theorem. The algebraic decomposition itself, however, requires no distributional assumptions.

Check Your Understanding

Use the definitions and projection argument above to answer these questions.

For a balanced layout with $a=4$, $b=3$, and $n=2$, find $N$ and the ranks of all four components.
Why are the row and column subspaces orthogonal when both sets of deviations sum to zero?
Explain why the additive residual rank is $N-a-b+1$, rather than $N-ab$.
What term must be added to the corrected decomposition to obtain $\sum_{i,j,k}y_{ijk}^2$?
For the data in the first worked example, verify the corrected total directly from the squared deviations about the grand mean.