Why the Treatment Test Must Adjust for Blocks
In an incomplete-block design, each block contains only some of the treatments. Comparing raw treatment averages can therefore mix treatment differences with block differences: the treatments may have been observed in different sets of blocks. The treatment test must account for blocks before measuring how much variation is attributable to treatments.
For a balanced incomplete-block design, every treatment occurs equally often, every block contains the same number of treatments, and every pair of distinct treatments occurs together equally often. These balance properties make the adjustment especially simple: after removing block effects, all treatment contrasts have the same information. This tutorial develops the resulting treatment-adjusted ANOVA test.
Let \(y_{ij}\) denote an observed response for treatment \(i\) in block \(j\), when that treatment is present. The additive incomplete-block model is
Here \(\mu\) is a general level, \(\tau_i\) is the effect of treatment \(i\), and \(\beta_j\) is the effect of block \(j\). Use constraints such as \(\sum_i\tau_i=0\) and \(\sum_j\beta_j=0\) to identify the parameters. The errors \(\varepsilon_{ij}\) are assumed independent and normally distributed with mean zero and common variance \(\sigma^2\). The response model is additive: there is no treatment-by-block interaction.
The treatment test concerns \(H_0:\tau_1=\cdots=\tau_v=0\) against \(H_a:\) at least one treatment effect differs from zero. Under the stated constraint, the null says all treatment means are equal after allowing for block effects. The blocks are nuisance effects: the question is whether treatments explain additional variation once blocks have been accounted for.
Adjusted Treatment Sum of Squares
Write \(T_i\) for the total of the observations assigned to treatment \(i\), \(B_j\) for the total in block \(j\), and \(G\) for the grand total. Let \(N=bk=vr\) be the number of observations. To adjust a treatment total for blocks, subtract from it the block totals in which that treatment appears, each divided by the block size:
These adjusted totals are within-block comparison scores. They satisfy \(\sum_iQ_i=0\), because each block total is distributed across its \(k\) treatments in equal shares.
In a balanced design, the treatment-adjusted sum of squares is
This is the extra sum of squares explained by treatments after accounting for blocks. The notation \(\text{treatment}\mid\text{blocks}\) emphasizes that this is not the unadjusted sum of squares from comparing raw treatment totals.
Why the Balance Formula Works
A brief derivation shows why one common multiplier applies to every treatment contrast. Let \(N\) be the treatment-by-block incidence matrix: its \((i,j)\) entry is 1 if treatment \(i\) occurs in block \(j\), and 0 otherwise. Each block has size \(k\). Eliminating the block effects from the normal equations leaves the treatment information matrix
The diagonal entries of \(NN'\) are \(r\), since each treatment occurs in \(r\) blocks. Its off-diagonal entries are \(\lambda\), since each treatment pair occurs together in \(\lambda\) blocks. Thus \(NN'=(r-\lambda)I+\lambda J\), where \(J\) is the all-ones matrix. For treatment contrasts, vectors whose entries sum to zero, multiplication by \(J\) gives zero. Consequently \(C\) acts on every treatment contrast as multiplication by
where the final equality follows from \(r(k-1)=\lambda(v-1)\). The adjusted treatment score vector is \(Q=(Q_1,\ldots,Q_v)'\). Since \(\sum_iQ_i=0\), the fitted treatment contrast vector is \(Q/(\lambda v/k)\). Its explained sum of squares is the score vector dotted with the fitted vector:
This establishes the formula. The balance conditions matter: for a nonbalanced design, the treatment information generally does not act as one common scalar on all contrasts, so this shortcut need not apply.
Block, Residual, and Degrees-of-Freedom Calculations
Because each block contains \(k\) observations, the block sum of squares adjusted for the grand mean is
The corrected total sum of squares is \(SS_{\text{total}}=\sum y_{ij}^2-G^2/N\). After removing the block and adjusted treatment components, the residual sum of squares is
The degrees of freedom follow from the number of independent observations and fitted effects. Total corrected degrees of freedom are \(N-1\), blocks use \(b-1\), and treatments use \(v-1\). Therefore
Provided \(df_E>0\), the treatment statistic is
Under \(H_0\), the additive model, and the independent normal errors with common variance, this statistic has an \(F\) distribution with \(v-1\) numerator and \(N-b-v+1\) denominator degrees of freedom. The p-value is the upper-tail probability \(P(F_{v-1,N-b-v+1}\geq F_{\text{observed}})\). Reject \(H_0\) at significance level \(\alpha\) when this p-value is at most \(\alpha\).
Worked Example: A Four-Treatment Balanced Design
Suppose a hypothetical design has four treatments and four blocks; each block omits a different treatment. Thus \(v=4\), \(b=4\), \(k=3\), \(r=3\), and \(\lambda=2\). Suppose the responses are as follows; the treatment labels in each row identify the three observations in that block.
| Block | Observed responses | Block total |
|---|---|---|
| 1 | Treatment 2: 12; Treatment 3: 14; Treatment 4: 15 | 41 |
| 2 | Treatment 1: 10; Treatment 3: 14; Treatment 4: 14 | 38 |
| 3 | Treatment 1: 11; Treatment 2: 12; Treatment 4: 16 | 39 |
| 4 | Treatment 1: 12; Treatment 2: 13; Treatment 3: 15 | 40 |
Each treatment occurs three times and each pair occurs together twice, so the balance conditions hold. The treatment totals are \(T_1=33\), \(T_2=37\), \(T_3=43\), and \(T_4=45\); the grand total is \(G=158\), with \(N=12\). The adjusted totals are
They sum to zero, as expected. Since \(k/(\lambda v)=3/8\),
The sum of squared responses is 2116, and the sum of squared block totals is 6246. Hence
Therefore \(SS_E=35.6667-1.6667-33.0833=0.9167\). The error degrees of freedom are \(12-4-4+1=5\), so \(MS_E=0.9167/5=0.1833\). The treatment mean square is \(33.0833/3=11.0278\), giving \(F=11.0278/0.1833=60.15\). Wait: using the exact values, \(MS_E=(11/12)/5=11/60\), so the ratio is \((397/36)/(11/60)=1985/33=60.15\). The upper-tail p-value for \(F_{3,5}=60.15\) is less than 0.001. At \(\alpha=0.05\), reject \(H_0\): the data provide evidence that treatment effects are not all equal after accounting for blocks.
The hypothetical design and response values alone do not establish the model assumptions. The analysis presumes independent errors with common variance and an additive treatment-plus-block mean structure; in practice, the design and residual diagnostics should be considered when judging whether those assumptions are reasonable.
Why the Denominator Degrees of Freedom Matter
A large treatment mean square relative to the residual mean square is evidence against equal treatment effects, but the p-value also depends on both degrees of freedom. In particular, a design with few residual degrees of freedom estimates the error variance imprecisely. The next example illustrates why the size of \(F\) alone is not enough.
Worked Example: A Large F Statistic With Little Error Information
Consider a hypothetical balanced design with three treatments and three blocks, each containing two treatments. Each treatment occurs twice and each pair occurs together once, so \(v=3\), \(b=3\), \(k=2\), \(r=2\), and \(\lambda=1\). Suppose the six responses are: block 1, treatments A and B, 8 and 10; block 2, B and C, 11 and 13; block 3, A and C, 9 and 14. The treatment totals are \(17,21,27\), and block totals are \(18,24,23\), with \(G=65\) and \(N=6\).
The adjustment multiplier is \(k/(\lambda v)=2/3\). The adjusted totals are
Thus \(SS_{\text{treatment}\mid\text{blocks}}=(2/3)(12.25+0+12.25)=49/3=16.3333\). The corrected total sum of squares is \(731-65^2/6=161/6\), and the block sum of squares is \((18^2+24^2+23^2)/2-65^2/6=31/3\). So \(SS_E=161/6-31/3-49/3=1/6\). Here \(df_E=6-3-3+1=1\), hence \(MS_E=1/6\), \(MS_{\text{treatment}\mid\text{blocks}}=49/6\), and \(F=49\).
For \(F_{2,1}\), the upper-tail probability at 49 is approximately 0.1414. Thus at \(\alpha=0.05\), this example does not reject \(H_0\), despite the large-looking statistic. With only one residual degree of freedom, the denominator variance estimate is highly uncertain. The additive model and error assumptions are also difficult to assess reliably from so little residual information.
Using Summary Quantities to Complete the Test
Once the balance conditions hold and the adjusted totals have been calculated, the remaining steps require only the corrected total and block sums of squares. This example emphasizes that the treatment adjustment and the error calculation must be used together.
Worked Example: Testing From Adjusted Summaries
Suppose a hypothetical balanced design has \(v=5\) treatments, \(b=10\) blocks, and \(k=2\) treatments per block. Each treatment occurs \(r=4\) times, and each pair occurs together once, so \(\lambda=1\). These values satisfy \(bk=20=vr\) and \(\lambda(v-1)=4=r(k-1)\). Suppose the adjusted totals calculated from the responses are \((-4,-2,0,2,4)\), while \(SS_{\text{total}}=100\) and \(SS_{\text{blocks}}=50\).
Here \(k/(\lambda v)=2/5\), and the squared adjusted totals sum to \(16+4+0+4+16=40\). Therefore
The treatment and error degrees of freedom are \(v-1=4\) and \(N-b-v+1=20-10-5+1=6\), respectively. Consequently \(MS_{\text{treatment}\mid\text{blocks}}=16/4=4\), \(MS_E=34/6=5.6667\), and \(F=4/5.6667=0.7059\). The p-value is about 0.61, so at \(\alpha=0.05\) the test does not reject equal treatment effects. This conclusion is a failure to find sufficient evidence of a treatment difference, not proof that all five effects are exactly equal.
As in the other examples, an F test is justified only under the additive model and independent, normally distributed errors with a common variance. The summary values are hypothetical; for actual data, residual patterns and the experimental process should be considered before relying on the test.
Common Mistake and Practical Interpretation
A frequent error is to use the ordinary one-way treatment sum of squares, computed from treatment totals or means without adjustment, and then divide by a residual mean square that accounts for blocks. That combines two different decompositions. In an incomplete-block design, the treatment groups need not occur in identical blocks, so raw averages may reflect block composition as well as treatment. Use the adjusted scores \(Q_i\) and the balanced-design multiplier for the treatment component.
A second error is to use \(N-v\) for the residual degrees of freedom. The block effects also use degrees of freedom, so the correct count is \(N-b-v+1\). Finally, a small p-value does not give the probability that \(H_0\) is true, and a nonsignificant result does not establish that treatments are identical. The p-value measures how unusual an F statistic at least as large as the observed one would be if \(H_0\) and the model assumptions were true.
Check Your Understanding
Use the definitions and formulas above to answer these questions.
- In a balanced incomplete-block design, what does \(\lambda\) count, and what equation relates it to \(r\), \(k\), and \(v\)?
- Why must the treatment sum of squares be adjusted for block totals rather than calculated from raw treatment totals alone?
- A design has \(v=6\), \(b=9\), \(k=4\), and \(N=36\). What are the treatment and residual degrees of freedom for the F test?
- If \(SS_{\text{treatment}\mid\text{blocks}}=24\), \(v=4\), and \(SS_E=30\) with \(df_E=6\), calculate both mean squares and the F statistic.
- What does a p-value of 0.03 mean for a test conducted at \(\alpha=0.05\), and what does it not mean?