Comparing Blocked and Unblocked Analyses

Why Compare the Two Analyses?

In a randomized complete block design, each block receives every treatment, with treatment assignment randomized within the block. Blocking is intended to account for differences among blocks when comparing treatments. To see what this changes in practice, we can analyze the same balanced data in two ways: include block in the model, or ignore block and use a one-way analysis.

When every treatment occurs once in every block, both analyses give the same treatment means and therefore the same estimated treatment differences. Their estimated standard errors can differ substantially, however, because the analyses use different residual variation and different degrees of freedom. The comparison below is about those standard errors—not a claim that ignoring the design is appropriate.

Definition: In a balanced randomized complete block design with \(t\) treatments and \(b\) blocks, there is one response \(y_{ij}\) for each treatment \(i\) in each block \(j\). The additive blocked model is \(y_{ij}=\mu+\tau_i+\beta_j+\epsilon_{ij}\), where \(\mu\) is an overall mean, \(\tau_i\) is a treatment effect, \(\beta_j\) is a block effect, and the errors \(\epsilon_{ij}\) are independent with mean 0 and common variance \(\sigma^2\). For the usual \(F\)- and \(t\)-based analysis, the errors are also assumed normally distributed. The model assumes treatment effects are additive across blocks: there is no treatment-by-block interaction.

Standard Errors Under the Two Analyses

For the blocked analysis, the error sum of squares measures variation left after accounting for both treatment and block. Its degrees of freedom are \((t-1)(b-1)\), so the blocked mean square error is \(MSE_{\text{blocked}}=SSE_{\text{blocked}}/[(t-1)(b-1)]\). In a balanced design, the estimated difference between treatments \(i\) and \(i'\) is \(\bar{y}_{i.}-\bar{y}_{i'.}\), where each mean averages over the \(b\) blocks. Its estimated standard error is:

SE_{\text{blocked}}(\bar{y}_{i.}-\bar{y}_{i'.}) =\sqrt{\frac{2MSE_{\text{blocked}}}{b}}.

The factor 2 arises because the contrast compares two treatment means, each based on \(b\) observations. In the unblocked one-way analysis, block is left out of the model. The residual sum of squares is calculated around each treatment mean, and its degrees of freedom are \(t(b-1)\). The resulting estimated standard error is:

SE_{\text{unblocked}}(\bar{y}_{i.}-\bar{y}_{i'.}) =\sqrt{\frac{2MSE_{\text{unblocked}}}{b}}.

These formulas use the same balanced treatment means. The main difference is the estimate of residual variance: \(MSE_{\text{blocked}}\) measures variation not explained by treatment or block, whereas \(MSE_{\text{unblocked}}\) also includes variation attributable to blocks.

Why the Residual Sums of Squares Differ

In a balanced table, the unblocked residual sum of squares separates exactly into the blocked error sum of squares plus the block sum of squares. Write \(\bar{y}_{.j}\) for the mean in block \(j\), and \(\bar{y}_{..}\) for the grand mean. For each observation, split its deviation from its treatment mean into a blocked residual and a block-mean deviation:

y_{ij}-\bar{y}_{i.} =\bigl(y_{ij}-\bar{y}_{i.}-\bar{y}_{.j}+\bar{y}_{..}\bigr) +\bigl(\bar{y}_{.j}-\bar{y}_{..}\bigr).

For each block, the blocked residuals in parentheses sum to zero across treatments. Therefore the cross-product terms vanish when the squared deviations are summed. There are \(t\) observations contributing the block-mean deviation for each block, giving:

SSE_{\text{unblocked}} =SSE_{\text{blocked}}+ t\sum_{j=1}^{b}(\bar{y}_{.j}-\bar{y}_{..})^2 =SSE_{\text{blocked}}+SS_{\text{blocks}}.

This identity explains why ignoring blocks can leave block-to-block variation in the one-way residuals. But the two analyses also divide their sums of squares by different degrees of freedom, so the identity alone does not imply that the unblocked mean square error will be larger in every realized sample.

Worked Example: Comparing Both Analyses for Three Treatments

Suppose a hypothetical experiment compares treatments A, B, and C in four blocks, with one response per treatment in each block. The response is measured in points.

Block	Treatment A	Treatment B	Treatment C
1	11	11	14
2	12	15	18
3	16	19	19
4	19	21	23

Assume the responses come from distinct experimental units, treatments were randomized within each block, and the additive model with independent, normally distributed errors of common variance is appropriate. With one observation per treatment-block cell, the data do not separately estimate treatment-by-block interaction; additivity is a model assumption here, not something these data can establish.

The treatment means are \(\bar{y}_{A.}=58/4=14.5\), \(\bar{y}_{B.}=66/4=16.5\), and \(\bar{y}_{C.}=74/4=18.5\) points. Thus the estimated difference between A and C is \(14.5-18.5=-4.0\) points in either analysis.

For the blocked analysis, the block means are 12, 15, 18, and 21 points, with grand mean 16.5. The block sum of squares is:

SS_{\text{blocks}} =3\bigl[(12-16.5)^2+(15-16.5)^2+(18-16.5)^2+(21-16.5)^2\bigr] =3(45)=135.

Using the fitted treatment and block means, the blocked residuals are \((1,-1,0)\) in block 1, \((-1,0,1)\) in block 2, \((0,1,-1)\) in block 3, and \((0,0,0)\) in block 4. Their squared values sum to \(SSE_{\text{blocked}}=6\). There are \((3-1)(4-1)=6\) error degrees of freedom, so \(MSE_{\text{blocked}}=6/6=1\) point squared. The standard error for A minus C is:

SE_{\text{blocked}} =\sqrt{\frac{2(1)}{4}} =\sqrt{0.5} =0.7071\text{ points}.

For the unblocked analysis, the decomposition gives \(SSE_{\text{unblocked}}=6+135=141\). Its error degrees of freedom are \(3(4-1)=9\), so \(MSE_{\text{unblocked}}=141/9=15.67\) points squared. The corresponding standard error is:

SE_{\text{unblocked}} =\sqrt{\frac{2(15.67)}{4}} =2.799\text{ points}.

Here the blocked standard error is much smaller because the large block-to-block differences are accounted for instead of being included in the unblocked residual variation. This calculation compares uncertainty estimates; it does not change the estimated A-versus-C difference of \(-4.0\) points.

When the Estimated Standard Errors Do Not Favor Blocking

Blocking removes block variation from the error term under the additive model. Still, an estimated standard error depends on the residuals actually observed in the experiment. In a particular sample, the blocked residual mean square can be relatively large, while the unblocked mean square can be smaller after division by its larger number of degrees of freedom. The next example shows why a smaller unblocked standard error in one data set is not proof that ignoring the design is generally better.

Worked Example: A Sample Where the Unblocked Estimate Is Smaller

Consider a hypothetical two-treatment experiment in three complete blocks. Responses are in milligrams.

Block	Treatment A	Treatment B
1	9	11
2	10	10
3	11	9

Suppose there is one independent experimental unit per treatment in each block, allocation was randomized within blocks, and the additive model assumptions are used for the blocked calculation. Each treatment mean is 10 mg, so the estimated difference is 0 mg. The block means are also all 10 mg. The blocked residual sum of squares is \(1+1+0+0+1+1=4\), with \((2-1)(3-1)=2\) degrees of freedom. Thus \(MSE_{\text{blocked}}=4/2=2\) mg squared, and:

SE_{\text{blocked}} =\sqrt{\frac{2(2)}{3}} =\sqrt{\frac{4}{3}} =1.155\text{ mg}.

The unblocked analysis has the same sum of squared deviations around the treatment means, \(SSE_{\text{unblocked}}=4\), but \(2(3-1)=4\) residual degrees of freedom. Thus \(MSE_{\text{unblocked}}=4/4=1\) mg squared, giving:

SE_{\text{unblocked}} =\sqrt{\frac{2(1)}{3}} =\sqrt{\frac{2}{3}} =0.8165\text{ mg}.

In this sample the unblocked estimate is smaller. There is no observed block-mean variation to remove, while the blocked analysis estimates its residual variance using fewer degrees of freedom. This is a sample-specific outcome, not a reason to ignore the randomized block design or a guarantee about future experiments.

What the Comparison Means

For a balanced design, the two treatment-difference estimates coincide because every treatment is observed in every block. Their standard errors differ because their variance estimates use different models. The blocked analysis is appropriate when the design and additive model justify comparing treatments within blocks; the unblocked analysis treats block-related variation as part of its residual variation and does not represent the randomization structure.

Under a model with substantial block variation and no treatment-by-block interaction, within-block comparisons can remove common block shifts from the treatment contrast. If those shifts are small relative to the remaining error, the standard-error advantage may be modest. The observed standard error is also affected by sampling variation, as the previous calculation illustrates. A comparison of standard errors alone does not establish that a treatment effect is important, nor does it describe the probability of detecting an effect.

Common Mistake

Wrong reasoning: “The blocked estimate was larger in this sample, so blocking made the treatment comparison less precise.”
Correction: The sample estimate can move either way because the mean square errors and their degrees of freedom differ. Blocking should be assessed in light of the design and its assumptions, not selected after inspecting which analysis gives the smaller standard error.
Wrong reasoning: “The treatment means differ between the two analyses, so one calculation must be wrong.”
Correction: In a balanced complete block design, the treatment means—and hence their differences—are the same. It is the estimated residual variance, and therefore the estimated standard error, that changes.

Key takeaway: In a balanced randomized complete block design, blocked and unblocked analyses estimate the same treatment differences but use different residual mean squares. Blocking can reduce the standard error when it accounts for substantial block variation, although a particular sample need not show a smaller blocked estimate.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

In a balanced complete block design with four treatments and five blocks, what are the error degrees of freedom for the blocked analysis and for the unblocked one-way analysis?
For a treatment difference with \(b=6\) and \(MSE_{\text{blocked}}=3\), calculate the blocked standard error. State its units if the response is measured in grams.
Why do the balanced blocked and unblocked analyses give the same treatment means?
What part of the unblocked residual sum of squares corresponds to block-to-block variation in the balanced design?
Why can an unblocked standard error be smaller in a particular sample without showing that the blocked analysis is inappropriate?